Log-Sum Inequality

Let and be nonnegative
Then

Moreover, holds if and only if is constant over

Proof

Let and
Assume WLOG that and for all
Define a Random Variable

Let .
Then everywhere and

Jensen’s Inequality gives

with equality if and only if is constant.