.

Click here to return to August Report

For a distribution of events there is a probability for each outcome; the mth outcome has probability . Then Shannon's formula for the uncertainty is:

In the coin toss case this reduces to:

Shannon's formula has the satisfying property that for uncorrelated multiple events uncertainties are additive. One expects that the uncertainty in the result of TWO coin tosses should be two bits. That is just what his formula yields. For THREE coin tosses his formula gives three bits.

The most stunning thing about Shannon's formula is that it is the same as the formula for entropy, a general property of any physical system. The formula for entropy in statistical mechanics represents the amount of disorder or randomness in the system. Hence uncertainty and randomness are connected.

where

August 1996
Marvin Chester
e-mail: chester@physics.ucla.edu

Return to Concept Exchange Society

© m chester 1996 Occidental CA