STOCHASTIC

Appendix A.
An Intuitive Outline of Stochastic Differential
Equations and Stochastic Optimal Control
R. G. Vickson
UNIVERSITY OF WATERLOO
In many modeling situations, continuous-time formulations may be preferable
to discrete time formulations, for precisely the same reasons that derivatives
are simpler than finite differences and integrals are simpler than finite sums.
However, when the models possess stochastic elements, the interpretation of
continuous-time processes becomes somewhat delicate. In this outline we treat
the continuous-time cases as limits of discrete time cases, when the length of
the discrete intervals tends to zero. Conceptually there is no loss of rigor in
this procedure: This is precisely the manner in which many mathematically
impeccable presentations define the properties of continuous-time processes.
[see, e.g., Breiman (1968)]. What is missing in this outline is any form of proof.
We remain throughout at the level of plausibility arguments.
I. ltd Processes
A Brownian motion is a stochastic process {z{t)} having stationary, independent
increments and satisfying a continuity property:
lim(l/5)-P[|z(r+«)-z(0|^fc] = 0 for all k > 0.
It can be shown (Breiman, 1968) that for any such process {z(t)}, t S: 0, with
z(0) = 0, z(t) is normally distributed, with Ez(t) = [it and varz(?) = a 2 t for
some real \i and a.
Consider now a stochastic process {X(t)} whose dynamics is that of a
deterministic, memoryless law (first-order ordinary differential equation),
perturbed by a random disturbance at each point in time. In particular,
suppose that for sufficiently small 5 > 0 we have
SX(t) = X(t + S) - X(t) = f(X(t\ t)-8+ g(X(t), t) • z(S) + o(S), (i)
where {z(t)} is a Brownian motion having n = 0, a 2 = 1. Recall that o(d)
stands for terms which tend to zero faster than