Stochastic Programming - Index of

116 STOCHASTIC PROGRAMMING
Proof The unique meaning of “max” implies that we have that
max ϕ 1(r 1 (z 1 ,x 1 ),ψ 2 (r 2 (z 2 ,x 2 ), ···,r T (z T ,x T )))
{x t∈X t,t≥1}
≥ ϕ 1 (r 1 (z 1 ,x 1 ), max [ψ 2(r 2 (z 2 ,x 2 ), ···,r T (z T ,x T ))]),
{x t∈X t,t≥2}
for all x 1 . Therefore this also holds when the right-hand side of this inequality
is maximized with respect to x 1 .
On the other hand, it is also obvious that
max ψ 2(r 2 (z 2 ,x 2 ), ···,r T (z T ,x T ))
{x t∈X t,t≥2}
≥ ψ 2 (r 2 (z 2 ,x 2 ), ···,r T (z T ,x T )) ∀x t ∈ X t ,t≥ 2.
Hence, by the assumed monotonicity of ϕ 1 with respect to ψ 2 ,wehavethat
ϕ 1 (r 1 (z 1 ,x 1 ), max ψ 2(r 2 (z 2 ,x 2 ), ···,r T (z T ,x T )))
{x t∈X t,t≥2}
≥ ϕ 1 (r 1 (z 1 ,x 1 ),ψ 2 (r 2 (z 2 ,x 2 ), ···,r T (z T ,x T ))) ∀x t ∈ X t ,t≥ 1.
Taking the maximum with respect to x t ,t≥ 2, on the right-hand side of this
inequality and maximizing afterwards both sides with respect to x 1 ∈ X 1
shows that the optimality principle (1.2) holds.
✷
Needless to say, all problems considered by Bellman in his first book on
dynamic programming satisfied this proposition. In case (b) of our example,
however, the monotonicity does not hold. The reason is that when “⊕”involves
multiplication of possibly negative factors (i.e. negative immediate returns),
the required monotonicity is lost. On the other hand, when “⊕” is summation,
the required monotonicity is always satisfied.
Let us add that the optimality principle applies to a much wider class of
problems than might seem to be the case from this brief sketch. For instance,
if for finitely many states we denote by ρ t the vector having as ith component
the immediate return r t (z t = i), and if we define the composition operation
“⊕” such that, with a positive matrix S (i.e. all elements of S nonnegative),
ρ t ⊕ ρ t+1 = ρ t + Sρ t+1 , t =1, ···,T − 1,
then the monotonicity assumed for Proposition 2.2 follows immediately. This
case is quite common in applications. Then S is the so-called transition matrix,
which means that an element s ij represents the probability of entering state
j at stage t + 1, given that the system is in state i at stage t. Iterating the
above composition for T − 1,T − 2, ···, 1wegetthatF (ρ 1 , ···,ρ T )isthe
vector of the expected total returns. The ith component gives the expected
overall return if the system starts from state i at stage 1. Putting it this way
we see that multistage stochastic programs with recourse (formula (4.13) in
Chapter 1) belong to this class.