Suppose and on .
Let be an arbitrary function
and let and define the distributions of and .
Then Relative Entropy satisfies

Proof

For each let
Then