Neumann-Pearson Regions

Let and be two distributions on which we want to do a Hypothesis Test given data samples . Consider the regions:

for some with error probabilities

Then the decision region is optimal: For any other if is as good as :

then necessarily:

Proof

Observe the inequality that always holds:

Multiplying out we find

Proposition

Let be the empirical distribution from samples . The Neumann-Pearson decision region can be expressed in terms of Relative Entropy as:

with .

Proof

By definition:

then

so

Substitute this back and we get exactly the result.