Distribution Type

Let be a finite alphabet and a Probability Simplex on it.
Given a string , the empirical distribution it induces is its type.
The set of all -types is

Proposition

Let be a distribution on .
Then for any :

where is Mathematical Entropy and is Relative Entropy.