Stochastic Programming - Index of

DYNAMIC SYSTEMS 115
Figure 3 Dynamic program: multiplicative composition. Solid lines show the
result of the backward recursion (with z 1 = 4), whereas the dotted line shows
the optimal sequence of decisions.
max{F (r 1 (z 1 ,x 1 ), ···,r T (z T ,x T )) | x t ∈ X t ,t=1, ···,T} (1.2)
= max [ϕ 1 (r 1 (z 1 ,x 1 ), max ψ 2 (r 2 (z 2 ,x 2 ), ···,r T (z T ,x T )))].
x 1∈X 1 x 2∈X 2,···,x T ∈X T
This relation is the formal equivalent of the well-known optimality principle,
which was expressed by Bellman as follows (quote).
Proposition 2.1 “An optimal policy has the property that whatever the
initial state and initial decision are, the remaining decisions must constitute
an optimal policy with regard to the state resulting from the first decision.”
As we have seen in Example 2.1, this principle, applied repeatedly in
the backward recursion, gave the optimal solution for case (a) but not for
case (b). The reason for this is that, although the composition operation
“⊕” is separable in the sense of (1.1), this is not enough to guarantee that
the repeated application of the optimality principle (i.e. through backward
recursion) will yield an optimal policy. A sufficient condition under which the
optimality principle holds involves a certain monotonicity of our composition
operation “⊕”. More precisely, we have the following.
Proposition 2.2 If F satisfies the separability condition (1.1) and if ϕ 1
is monotonically nondecreasing in ψ 2 for every r 1 then the optimality
principle (1.2) holds.