Interpretation of Asymptotic Equipartition Property

Using notation from Asymptotic Equipartition Property definition.

Each string has probability of occurring

The result of Random Probability of Random Variable is:
Choose one of the probabilities at random
(following the distribution of )
i.e. choose with probability

AEP says that this randomly chosen number is close to
(with high probability).

In other words, most strings
should have probability close to .
These are called Typical Strings.

There will usually be a lot of strings that are not typical.
We make no claims on whether they are more or less probable than
However, seeing anything from the non typical set is improbable.
For example, consider
and are iid with and .
We can find .
The string is the most probable individual string,
but its not very “typical” (we expect to see a ratio)
Indeed, it’s probability is which gives:

and that is not close to .