A Source taking values in a discrete set where all are identically distributed and independent is called a Discrete Memoryless Source. Sometimes it is also called a Bernoulli Source.

Theorem

A DMS satisfies Asymptotic Equipartition Property with entropy equal to Mathematical Entropy

Proof

Let be the distribution of and distribution of Then:

Set , the Information Content of and use Weak Law of Large Numbers for to find:

Hence satisfies Asymptotic Equipartition Property with entropy