The result of Random Probability of Random Variable is:
Choose one of the probabilities at random
(following the distribution of )
i.e. choose with probability
AEP says that this randomly chosen number is close to
(with high probability).
In other words, most strings
should have probability close to .
These are called Typical Strings.
There will usually be a lot of strings that are not typical.
We make no claims on whether they are more or less probable than
However, seeing anything from the non typical set is improbable.
For example, consider
and are iid with and .
We can find .
The string is the most probable individual string,
but its not very “typical” (we expect to see a ratio)
Indeed, it’s probability is which gives: