Let and be probability distributions. Then with equality iff . Proof We can replace by if we multiply through by . Note, with equality iff . Now multiply through and sum over Equality holds iff