Lagrange multipliers

Suppose we are minimizing a function subject to .
This can (sometimes) be done by instead minimizing the function:

Note that the partial derivative in will give us the condition ,
so we just need to solve and .
Lagrange Multipliers for Functionals

General method

We are minimizing subject to and .
The following method works as long as we have The Necessity Theorem.

1. Introduce the slack variable s.t. .
2. Introduce the Lagrangian

3. Find the set of Feasible Lagrange Multipliers

4. Note that has to be because of term.
5. Now find and
   which minimize for every
6. Note that ,
   this is called Complimentary Slackness
7. Hence, determine such that and are feasible
   (this might FAIL in which case the method doesn’t work)
8. Use Lagrange sufficiency theorem to finalize.

Example 1 (no slack variable)

Minimize
s.t. and

Rewrite

Now this cannot go to so we get:

Differentiate w.r.t. each of ,
and set to zero to find extreme values:

Find s.t. the conditions are satisfied:

This gives , and
Now apply Lagrange sufficiency theorem to this.

Example 2 (slack variable)

Minimize
s.t. and

Add slack variables:
and

Write
Terms and give that

Differentiating w.r.t. :


If , this system is inconsistent, so . This means in order to minimize

NOTE: Here we are minimizing as if are all INDEPENDENT!
That is why we can say the previous line.
Later we should check if there is a such that the minimum is feasible.

Now we have two cases: and . First case does not yield a feasible minimum. In the second case we are just solving:

This should yield , ,
That is a feasible solution so by Lagrange sufficiency theorem we are done.