Let and be two distributions on the same discrete alphabet . The relative entropy is: Additionally, we define Lemma For any and we have with Proof Using Log-Sum Inequality: with equality if and only if . But then this constant has to be .