The Series *Natural Extrema* — 8. Extremum Analysis

《自然极值》系列——8.极值分析 苏剑林 · 2010-12-26 · Mathematics 2010 extremum analysis variational 2011

Nonlinear Functional Analysis and Its Applications, Volume III: Variational Methods and Optimization

This article is the last in the Natural Extrema series, and probably also the last post of 2010. Over this wonderful year of 2010, I'm sure everyone has learned a great deal — BoJone has certainly grown a lot too. As 2010 draws to a close, BoJone and Scientific Spaces wish everyone a happier new year, and a faster stride forward on the path of science.

In this article, BoJone will discuss with you the most fundamental principle of finding extrema. This line of thought was inspired by the genius Feynman's Feynman Lectures on Physics. We will give a brief analysis of extrema of functions (via differentiation) and extrema of functionals (via the calculus of variations) separately.

I. Extrema of Functions

For a function $y=f(x)$, suppose it attains its maximum at $x=x_0$. Then clearly, for a very small increment $\Delta x$, we have

$$f(x_0+\Delta x) \leq f(x_0)\tag{3}$$By Taylor series, we have

$f(x_0+\Delta x)=f(x_0)+f'(x_0)\Delta x$————(4)more

Here we have dropped the quadratic and higher-order terms, because the mean value theorem tells us that the sum of the remaining terms is still only a second-order term (a second-order infinitesimal) — that is, it cannot "unseat the dominance of $f'(x_0)\Delta x$." Substituting (4) into (3) gives

$$f'(x_0)\Delta x \leq 0$$

Note that $f'(x_0)$ is a fixed value, while $\Delta x$ is a variable that can be either positive or negative. So we get

$$f'(x_0) \leq 0,f'(x_0) \geq 0$$

which gives us $f'(x_0)=0$.

We can also replace the word "minimum" above with "maximum," swapping $\leq$ and $\geq$, and carry out a similar discussion — the result is the same. Thus we conclude: $f'(x)=0$ is a necessary condition for $f(x)$ to attain a maximum (minimum) value.

II. Extrema of Functionals

Our discussion of the brachistochrone and the catenary problems both ultimately reduced to the following problem:

Find a function $y=f(x)$ passing through $(x_1,y_1),(x_2,y_2)$, such that the integral $\int_{x_1}^{x_2} F(x,y,\dot{y})dx$ attains a maximum (minimum) value.

Suppose the function $y=y(x)$ is the desired function. Then for a small increment function of y, $\varepsilon=\varepsilon(x)$, where $\varepsilon(x_1)=\varepsilon(x_2)=0$, $y=y(x)+\varepsilon(x)$ is likewise a function passing through $(x_1,y_1),(x_2,y_2)$. Then

$$\int_{x_1}^{x_2} F(x,y+\varepsilon,\dot{y}+\dot{\varepsilon})dx \leq \int_{x_1}^{x_2} F(x,y,\dot{y})dx\tag{5}$$

Expanding $F(x,y+\varepsilon,\dot{y}+\dot{\varepsilon})$ using the multivariate Taylor series, we get

$$F(x,y+\varepsilon,\dot{y}+\dot{\varepsilon})=F(x,y,\dot{y})+\frac{\partial F}{\partial y}\varepsilon+\frac{\partial F}{\partial \dot{y}}\dot{\varepsilon}$$

Here again we drop the quadratic and higher-order terms. Substituting into equation (5) gives

$$\int_{x_1}^{x_2}(\frac{\partial F}{\partial y}\varepsilon+\frac{\partial F}{\partial \dot{y}} \dot{\varepsilon})dx \leq 0\tag{6}$$

There is a trick here for handling $\int(\frac{\partial F}{\partial \dot{y}} \dot{\varepsilon})dx$, using "integration by parts" from mathematical analysis, namely

$$\int(\frac{\partial F}{\partial \dot{y}} \dot{\varepsilon})dx=\int(\frac{\partial F}{\partial \dot{y}}d\varepsilon)=\frac{\partial F}{\partial \dot{y}}\varepsilon-\int[\frac{d(\frac{\partial F}{\partial \dot{y}})}{dx}\varepsilon] dx$$

Substituting into equation (6) gives

$$(\frac{\partial F}{\partial \dot{y}}\varepsilon)|_{x_1}^{x_2}+\int_{x_1}^{x_2}[\frac{\partial F}{\partial y}-\frac{d(\frac{\partial F}{\partial \dot{y}})}{dx}]\varepsilon dx \leq 0$$

Since $\varepsilon(x_1)=\varepsilon(x_2)=0$, we have $(\frac{\partial F}{\partial \dot{y}}\varepsilon)|_{x_1}^{x_2}=0$, and likewise $\varepsilon$ can be either positive or negative, so it must be the case that

$$\int_{x_1}^{x_2}[\frac{\partial F}{\partial y}-\frac{d(\frac{\partial F}{\partial \dot{y}})}{dx}]\varepsilon dx =0$$

This equation must hold for all $\varepsilon=\varepsilon(x)$, so the value inside the parentheses can only be 0, giving

$$\frac{\partial F}{\partial y}-\frac{d(\frac{\partial F}{\partial \dot{y}})}{dx}=0\tag{7}$$

Swapping maximum and minimum, and swapping $\leq$ and $\geq$, we can carry out a similar discussion, and the result is again the same. So we conclude: equation (7) is a necessary condition for the integral $\int_{x_1}^{x_2} F(x,y,\dot{y})dx$ to attain an extreme value.

Equation (7) is the famous (two-dimensional form of the) Euler-Lagrange equation.

Using a similar approach, the equation can be extended to more dimensions, and to higher orders (for instance, when F contains a term like $\ddot{y}$). As you can see, the underlying idea is the same throughout: assume an extremum → introduce an increment → expand to first order → compare with the original value → analyze and simplify → derive an equation → solve the equation. Although the details of the treatment differ from case to case, the underlying principle never changes. We can therefore consider this the most fundamental approach to extremum problems.

Since this article is meant to guide intuition rather than serve as a rigorous tutorial, our discussion of this topic ends here. For further details, please consult the relevant entries on Wikipedia.

Calculus of variations:
http://zh.wikipedia.org/zh/%E5%8F%98%E5%88%86%E6%B3%95
Euler-Lagrange equation:
http://zh.wikipedia.org/zh-sg/%E6%AD%90%E6%8B%89%EF%BC%8D%E6%8B%89%E6%A0%BC%E6%9C%97%E6%97%A5%E6%96%B9%E7%A8%8B

And so the Natural Extrema series draws to a close, just as 2010 itself draws to a close. Even with much reluctance and a few regrets, we've gained so much and grown so much this year. May we all carry our brightest hopes into the coming 2011, growing slowly, advancing gradually, through sunshine and through storm, savoring science and glimpsing truth! On the road of science, may we continue forward together with fellow lovers of science!

End of the Natural Extrema series.