Information Theory

I have now moved on to computers and information theory.

I did try to do some research on accelerated charge in a field .. and got one piece of info that hadn't struck me till now ... All electromagnetic phenomena on earth is happening in the earth's magnetic field and every charge that accelerates on earth accelerates in the omnipresent magnetic field of earth so would always generate an EM wave.

Now the next obvious question would be would an accelerated charge in a absolute vacuum devoid any field still radiate EM wave out of interaction with its own field? Couldn't find an answer for that yet.

Anyways moving on to the current topic : Computers and Information Theory.

Today's "Computer" stands for much more than what the name implies. Today's computer is more of a "simulator" than just a calculator. That idea was first conceived by Alan Turing, british scientist who said there could be a machine that would generate the solution of any problem that can be solved provide all the required steps are pre determined ... An algorithm is predefined.

This machine could be built, today we call them computers, only because of technological advancement of semiconductor physics, and the encoding of "rules of logic" into logic circuits.
Transistors, after its invention in bell labs, underwent a major revolution in terms of application when logic circuits were created, they were now capable of "processing" a set of input signals, converting them into a predefined set of output signals, and then these signals could be interpreted and rendered in the form the user required. A graph. A table. A equation. Anything basically. So computer could now simulate the entire process, that was defined in the algorithm, and give us the results instantly. Now that is immense power if you really think about it.

Personally this is what I have liked about computers and software programming the best. The power to create anything without any rules. If you can write a code for it, you can practically make anything. :)

With the advent of computers, information became even more readily available. All sorts of information. And that became a problem in itself. But the one I'm concerned about is the Engineering problem. And as we all know, Information theory means Shannon's thoery.

Again, Shannon's theory is something I have always hated during my engineering. It just never made sense to me. If we already know something its information content is nil, and if we get an unknown signal, that has information. Its totally counter intuitive. Until I saw it from an engineers perspective. From a Receiver's perspective. If I receive a signal, which I already knew was coming, communicating it was pretty useless to me isn't it? On the other hand, if I receive a unknown signal, now my job would be to find out whether or not its a valid signal. Its like jigsaw puzzle solving. If I already know where a piece should be and what the piece is, I would put it there anyway, but for every new signal I receive, I have to first find out whether its a valid piece of the puzzle, and if it is, then I try to put it somewhere where it would help form the whole content or the picture. So the new signal is a clue, full of some information. That is the way I understand it as of now.

Another interesting thing about Shannon is that he was the one who came up with the idea of logic circuits. He first researched into how the telephone switching relays worked and said they could be designed even more efficiently incorporating Bool's "laws of human thought". And that they infact embodied that exact idea. Now if systems displayed use of logic, then vice versa logic "circuits" could be used to design systems. That is where all of digital electronics began.

Shannon also showed that the digital/binary encoded bit stream was the most efficient way of data transfer that allowed self correction.

As electronics engineer I can really appreciate the far reaching applications of his theory... And the reason it prevailed was simply because he took out the subjectivity. He said, I don't care what the content is, I don't care if you are sending a word doc, a music file, a image or a radio transmission, my theory applies to all, he generalized it and reduced it down to the most basic level so much that it could be applied practically to anything. Genius.