Gibbs' inequality

Let and be probability distributions.
Then

with equality iff .

Proof

We can replace by if we multiply through by .
Note, with equality iff .

Now multiply through and sum over

Equality holds iff