Stochastic Programming - Index of

226 STOCHASTIC PROGRAMMING
By assumption (9.3 i), ϕ(x) :=E˜ξF (x, ˜ξ) isconvexinx. If this function is
also differentiable with respect to x at any arbitrary point z with the gradient
g := ∇ϕ(z) =∇ x E˜ξF
(z, ˜ξ), then −g is the direction of steepest descent of
ϕ(x) = E˜ξF
(x, ˜ξ) inz, and we should probably like to choose −g as the
search direction to decrease our objective. However, this does not seem to be
a practical approach, since, as we know already, evaluating ϕ(x) =E˜ξF
(x, ˜ξ),
as well as ∇ϕ(z) =∇ x E˜ξF
(z, ˜ξ), is a rather cumbersome task.
In the differentiable case we know from Proposition 1.21 on page 81 that,
for a convex function ϕ,
(x − z) T ∇ϕ(z) ≤ ϕ(x) − ϕ(z) (9.6)
has to hold ∀x, z (see Figure 27 in Chapter 1). But, even if the convex function
ϕ is not differentiable at some point z, e.g.ifithasakinkthere,itisshown
in convex analysis that there exists at least one vector g such that
(x − z) T g ≤ ϕ(x) − ϕ(z) ∀x. (9.7)
Any vector g satisfying (9.7) is called a subgradient of ϕ at z, and the set of all
vectors satisfying (9.7) is called the subdifferential of ϕ at z and is denoted by
∂ϕ(z). If ϕ is differentiable at z then ∂ϕ(z) ={∇ϕ(z)}; otherwise,i.e.inthe
nondifferentiable case, ∂ϕ(z) may contain more than one element as shown
for instance in Figure 32. Furthermore, in view of (9.7), it is easily seen that
∂ϕ(z) is a convex set.
If ϕ is convex and g ≠ 0 is a subgradient of ϕ at z then, by (9.7) for λ>0,
it follows that
ϕ(z + λg) ≥ ϕ(z)+g T (x − z)
= ϕ(z)+g T (λg)
= ϕ(z)+λ‖g‖ 2
>ϕ(z).
Hence any subgradient, g ∈ ∂ϕ, such that g ≠ 0 is a direction of ascent,
although not necessarily the direction of steepest ascent as the gradient would
be if ϕ were differentiable in z. However, in contrast to the differentiable case,
−g need not be a direction of strict descent for ϕ in z. Consider for example
the convex function in two variables
ψ(u, v) :=|u| + |v|.
Then for ẑ =(0, 3) T we have g =(1, 1) T ∈ ∂ψ(ẑ), since for all ε>0the
gradient ∇ψ(ε, 3) exists and is equal to g. Hence, by (9.6), we have, for all
(u, v),
[( ) ( )] T ( ) T ( )
u ε u − ε 1
− g =
v 3
v − 3 1
= u − ε + v − 3
≤|u| + |v|−|ε|−|3|,