Optimization: Critical Points and Lagrange Multipliers

Unconstrained Optimization

Finding extrema of $f(x,y)$ starts by computing the gradient vector $\nabla f = (f_x, f_y)$ and setting it to zero.

A critical point is any $(a,b)$ where $\nabla f = \vec{0}$. However, flat points also include saddles: a maximum in one direction and a minimum in another.

To classify critical points, use the second derivative test via the Hessian determinant:

$$D = f_{xx}f_{yy} - (f_{xy})^2$$

Common pitfall: $\nabla f = 0$ alone only finds flat spots. The determinant $D$ is required to separate peaks, pits, and passes.

Constrained Optimization

To optimise $f(x,y)$ subject to a constraint $g(x,y) = c$, use the method of Lagrange multipliers:

$$\nabla f = \lambda\,\nabla g \quad\text{and}\quad g(x,y) = c$$

Geometric meaning: At the optimum, the level curves of $f$ and $g$ are tangent, meaning their gradients are parallel.

The scalar $\lambda$ is the Lagrange multiplier. It measures how sensitive the optimum value is to small changes in the constraint value $c$.

Tip: Lagrange multipliers only find interior constrained points. Always check boundary behavior too, as global extrema often occur at domain endpoints.