MATH 209, Lab 5

But setting the gradient to 0 is actually a system of two equations:
f x (x, y) = 0, (7)
f y (x, y) = 0, (8)
with two unknowns x and y. Fortunately, we can form the tangent plane approximation (the generalization
of the linear approximation) to both equations:
T fx (x, y) = f x (x i , y i ) + (x − x i )f xx (x i , y i ) + (y − y i )f xy (x i , y i ), (9)
T fy (x, y) = f y (x i , y i ) + (x − x i )f xy (x i , y i ) + (y − y i )f yy (x i , y i ), (10)
Setting both equations simultaneously to 0 at the new iterate (x i+1 , y i+1 ), we have:
[
] [ ] [ ]
f xx (x i , y i ) f xy (x i , y i ) x i+1 − x i f x (x i , y i )
= −
. (11)
f xy (x i , y i ) f yy (x i , y i ) y i+1 − y i f y (x i , y i )
This is a familiar system of two equations for two unknowns. We can solve this!
[
] [
x i+1
=
y i+1
] [
x i
−
y i
f xx (x i , y i ) f xy (x i , y i )
f xy (x i , y i ) f yy (x i , y i )
] −1 [
f x (x i , y i )
f y (x i , y i )
]
. (12)
References
[1] F. Bornemann et al. Think Globally, Act Locally in The SIAM 100-Digit Challenge, SIAM, 4:77–100,
2004.
[2] M. Beekman et al. Biological Foundations of Swarm Intelligence in Swarm Intelligence, Springer-Verlag,
1:3–41, 2008.
[3] J. Kennedy and R.C. Eberhart Particle swarm optimization, Proceedings of the IEEE international
conference on neural networks IV, 1942–1948, 1995.
8