Neumann-Pearson Regions

Let and be two distributions on which we want to do a Hypothesis Test
given data samples .
Consider the regions:

for some with error probabilities

Then the decision region is optimal:
For any other if is as good as :

then necessarily:

Proof

Observe the inequality that always holds:

Multiplying out we find

Proposition

Let be the empirical distribution from samples .
The Neumann-Pearson decision region can be expressed
in terms of Relative Entropy as:

with .

Proof

By definition:

then

so

Substitute this back and we get exactly the result.