Stochastic Programming - Index of

130 STOCHASTIC PROGRAMMING
2.5 Stochastic Dynamic Programming
Looking back at Section 2.2 on dynamic programming, we observe two major
properties of the solution and solution procedure. First, the procedure (i.e.
dynamic programming) produces one solution per possible state in each stage.
These solutions are not stored, since they are not needed in the procedure, but
the extra cost incurred by doing so would be minimal. Secondly, if there is only
one given value for the initial state z 0 , we can use these decisions to produce
a series of optimal solutions—one for each stage. In other words, given an
initial state, we can make plans for all later periods. In our small investment
Example 2.2 (to which we added randomness in the interest rates in Section 2.4
we found, in the deterministic case, that with S 0 > 1000, x 0 = x 1 = A and
x 2 = B was the optimal solution. That is, we put the money in account A for
the two periods, before we send the money to account B as required at the
end of the time horizon.
When we now move into the area of stochastic dynamic programming, we
shall keep one property of the dynamic programming algorithm, namely that
there will be one decision for each state in each stage, but it will no longer be
possible to plan for the whole period ahead of time. Decisions for all but the
first period will depend on what happens in the mean time. This is the same
as we observed for stochastic decision trees.
Let us turn to the small investment example, keeping the extra requirement
that the money must stay in one account, and using the utility function
u(s) =ln(s − 1000).
Stage 2 As for the deterministic case, we find that
f ∗ 2 (A, S 2 )=ln(S 2 − 1010),
f ∗ 2 (B,S 2)=ln(S 2 − 1000),
since we must move the money into account B at the end of the
second year.
Stage 1 We have to consider the two accounts separately.
Account A If we keep the money in account A, we get the following
expected return:
f 1 (A, S 1 ,A)=0.5[f ∗ 2 (A, S 1 × 1.05 − 20) + f ∗ 2 (A, S 1 × 1.09 − 20)]
=0.5ln[(S 1 × 1.05 − 1030) (S 1 × 1.09 − 1030)].
If we move the money to account B, weget
f 1 (A, S 1 ,B)=f ∗ 2 (B,(S 1 − 10) × 1.05)
=ln(S 1 × 1.05 − 1010.5).
To find the best possible solution, we compare these two possibilities
by calculating