PrefacePrefaceThese notes, and the related freshman-level course, are about information. Although you may havea general idea of what information is, you may not realize that the information you deal with can bequantified. That’s right, you can often measure the amount of information and use general principles abouthow information behaves. We will consider applications to computation and to communications, and we willalso lo ok at general laws in other fields of science and engineering.One of these general laws is the Second Law of Thermodynamics. Although thermodynamics, a branchof physics, deals with physical systems, the Second Law is approached here as an example of informationprocessing in natural and engineered systems. The Second Law as traditionally stated in thermodynamicsgoverns a physical quantity known as “entropy.” Everybody has heard of entropy, but few really understandit. The Second Law is surely one of science’s most glorious achievements, but as usually taught, throughphysical systems and models such as ideal gases, it is difficult to appreciate at an elementary level. On theother hand, the form of the Second Law that applies to computation and communications is more easilyunderstood, especially to day as the information revolution is getting under way.These notes and the course based on them are intended for first-year university students. Although theywere developed at the Massachusetts Institute of Technology, a university that specializes in science andengineering, and one at which a full year of calculus is required, calculus is not used in these notes. Mostexamples are from discrete systems, not continuous systems, so that sums and differences c an be used insteadof integrals and derivatives. In other words, we can use algebra instead of calculus. This course should beaccessible to university students who are not studying engineering or science, and even to well preparedhigh-school students. In fact, when combined with a similar one about energy, this course could provideexcellent science background for lib e ral-arts majors.The analogy between information and energy is interesting. In both high-school and first-year collegephysics courses, students learn that there is a physical quantity known as energy which is conserved (it cannotbe created or des troyed), but which can appear in various forms (potential, kinetic, electric, chemical, etc.).Energy can exist in one region of space or another, can flow from one place to another, can be stored forlater use, and can be converted from one form to another. In many ways the industrial revolution was allabout harnessing energy for useful purposes. Energy is conserved — despite its being moved, stored, andconverted, at the end of the day there is still exactly the same total amount of energy.This conservation of energy principle is sometimes known as the First Law of Thermodynamics. It hasproven to be so important and fundamental that whenever a “leak” was found, the theory was rescued bydefining a new form of energy. One example of this occurred in 1905 when Albert Einstein recognized thatmass is a form of energy, as expressed by his famous formula E = mc2. That understanding later enabledthe development of devices (atomic bombs and nuclear power plants) that convert energy from its form asmass to other forms.The corresponding unified theory of information and entropy is not as well developed, for several historicalreasons. Yet such a theory is exactly what is needed to simplify the teaching of the concepts. The basicAuthor: Paul Penfield, Jr.This document: http://www.mtl.mit.edu/Courses/6.050/2007/notes/preface.pdfVersion 1.4, December 13, 2006. Copyrightc 2007 Massachusetts Institute of TechnologyStart of notes · back · next | 6.050J/2.110J home page | Site map | Search | About this document | Comments and inquiriesiiinotions can be appreciated, even at the freshman level, and applied to several domains.These notes present such a unified view of information, in which entropy is one kind of information, butthere are other kinds as well. Like energy, information can reside in one place or another, it can be transmittedthrough space, and it can be stored for later use. But unlike energy, information is inherently subjective,because it deals with what you know and what you don’t know (entropy, as one form of information, is alsosubjective — this point makes some physicists uneasy). Also, information is not conserved as is energy; theSecond Law states that entropy never decreases as time goes on — it generally increases, but may in specialcases remain constant.The unified framework presented here has never before been developed specifically for freshmen. It isnot entirely consistent with conventional thinking in various disciplines, which were developed separately. Infact, we may not yet have it right. One consequence of teaching at an elementary level is that unansweredquestions close at hand are disturbing, and their resolution demands new rese arch into the fundamentals.Trying to explain things rigorously but simply often requires new organizing principles and new approaches.In the present case, the new approach is to start with information and work from there to entropy, and thenew organizing principle is the unified theory of information.This will be an exciting journey. Welcome ab
View Full Document
Unlocking...