Linear Programming Lecture Notes - Penn State Personal Web Server

Theorem 1.15. If z : R n → R is differentiable, then all directional derivatives exist.
Furthermore, the directional derivative of z at x0 in the direction of v is given by:
(1.15) ∇z(x0) · v
where · denotes the dot product of two vectors.
Proof. Let l(t) = x0 + vt. Then l(t) = (l1(t), . . . , ln(t)); that is, l(t) is a vector function
+ vit.
Apply the chain rule:
whose i th component is given by li(t) = x0i
(1.16)
Thus:
(1.17)
dz(l(t))
dt
= ∂z dl1 ∂z dln
+ · · · +
∂l1 dt ∂ln dt
d
dl
z(l(t)) = ∇z ·
dt dt
Clearly dl/dt = v. We have l(0) = x0. Thus:
d
(1.18)
dt z(x0
+ tv) = ∇z(x0) · v
t=0
We now come to the two most important results about gradients, (i) the fact that they
always point in the direction of steepest ascent with respect to the level curves of a function
and (ii) that they are perpendicular (normal) to the level curves of a function. We can
exploit this fact as we seek to maximize (or minimize) functions.
Theorem 1.16. Let z : R n → R be differentiable, x0 ∈ R n . If ∇z(x0) = 0, then ∇z(x0)
points in the direction in which z is increasing fastest.
Proof. Recall ∇z(x0) · v is the directional derivative of z in direction v at x0. Assume
that v is a unit vector. We know that:
(1.19) ∇z(x0) · v = ||∇z(x0)|| cos θ
(because we assumed v was a unit vector) where θ is the angle between the vectors ∇z(x0)
and v. The function cos θ is largest when θ = 0, that is when v and ∇z(x0) are parallel
vectors. (If ∇z(x0) = 0, then the directional derivative is zero in all directions.)
Theorem 1.17. Let z : R n → R be differentiable and let x0 lie in the level set S defined
by z(x) = k for fixed k ∈ R. Then ∇z(x0) is normal to the set S in the sense that if v
is a tangent vector at t = 0 of a path c(t) contained entirely in S with c(0) = x0, then
∇z(x0) · v = 0.
Remark 1.18. Before giving the proof, we illustrate this theorem in Figure 1.5. The
function is z(x, y) = x 4 +y 2 +2xy and x0 = (1, 1). At this point ∇z(x0) = (6, 4). We include
the tangent line to the level set at the point (1,1) to illustrate the normality of the gradient
to the level curve at the point.
7