Information Theory Fundamentals – Part I

Set theory and the axioms of probability; the pervasiveness of information.

In this course, students will see that information is found in the stock market, in Brownian motion of pollen, in neural firing patterns, in the quantum realm of particle creation and annihilation, and also in intersection with many of the basic sciences. Information intersects with quantum physics, to yield quantum information theory, with neuroscience to yield computational neuroscience, and with economics to yield investment theory. This broad overview of information is accompanied by a discussion of the philosophical issues that information touches upon. We present an outline for a complete course on information theory of which the present course is the zeroth offering.

What you’ll learn

  • The meaning of probability.
  • The axiomatic approach to probability.
  • The classical approach to probability.
  • Basics of Set theory.
  • The pervasiveness of information.
  • Information theory and the basic sciences intersected.

Course Content

  • Introduction to Information –> 1 lecture • 15min.
  • Introduction to Probability –> 4 lectures • 22min.

Information Theory Fundamentals - Part I

Requirements

In this course, students will see that information is found in the stock market, in Brownian motion of pollen, in neural firing patterns, in the quantum realm of particle creation and annihilation, and also in intersection with many of the basic sciences. Information intersects with quantum physics, to yield quantum information theory, with neuroscience to yield computational neuroscience, and with economics to yield investment theory. This broad overview of information is accompanied by a discussion of the philosophical issues that information touches upon. We present an outline for a complete course on information theory of which the present course is the zeroth offering.

 

In the lecture on probability, we start with a brief historical introduction to probability. We go on to define the meaning of probability via probability axioms, we introduce the classical definition, and the relative frequency definition of probability. We provide examples to illustrate these definitions. These are simple examples not requiring a calculator.

 

Finally, we introduce the student to set theory fundamentals, explaining the notion of belonging, exclusion, disjointedness, intersection, union, complements. We also introduce partitions, De Morgan’s laws and the properties of unions and intersections.

 

All of the above makes this a strong foundation on which other course work can build.

Get Tutorial