Asymptotic Equipartition Property Relative Entropy Hypothesis Test Fano’s Inequality Conditional Entropy Joint Entropy Entropy Chain Rule Data Processing Property of the Entropy Data Processing Property of the Relative Entropy Total Variation Distance Pinsker’s Inequality Poisson Approximation Mutual Information Data Processing Property of the Mutual Information Synergy Ruzsa Triangle Inequality Submodularity for Sums inequality Entropy Rate Stationary Source Ergodic Birkhoff Ergodic Theorem Probability Simplex Distribution Type Distribution Type Class Sanov’s Theorem