Applications of state space models in finance

3.4 Maximum likelihood estimation 31
score vector with respect to problems formulated in state space, the reader is referred to
Durbin and Koopman (2001, ¢ 7.3.3). In order to avoid numerical or analytical computation,
the Hessian can usually be approximated. In Ox, H(ψ) is approximated using the
quasi-Newton method according to Broyden-Fletcher-Goldfarb-Shanno (BFGS), which
ensures negative definiteness of the Hessian; details on this nonlinear optimization techniques
are discussed by Greene (2003, ¢ E.6.2).
3.4.3 The EM algorithm
The unknown elements of the system matrices can alternatively be estimated via the
expectation-maximization (EM) algorithm by Dempster et al. (1977). The EM algorithm,
named after its two steps of maximizing the expectation of the loglikelihood, is
an iterative algorithm for maximum likelihood estimation. It has originally been developed
for dealing with missing observations and can be employed for maximizing the
loglikelihood of many state space models (cf. Shumway and Stoffer 1982; Watson and
Engle 1983).
As in this thesis the likelihood will be generally maximized using numerical maximization
procedures, both in the context of Kalman filter based state space models as
well as the basic hidden Markov model to be introduced below, the reader is referred
to the well-established literature for details on the EM algorithm. For an introductory
outline, see, for example, Bulla (2006, Appendix A) and the references provided therein.
A general demonstration of how to use the EM algorithm to compute the ML estimates
in some common statistical applications is provided by McLachlan and Krishnan (1997);
a Bayesian treatment is given by Tanner (1993).
3.4.4 Parameter restrictions
Sometimes the parameters of a model are not allowed to take any value in R, the set of
real numbers. In this case, it may become necessary to impose parameter constraints.
For example, the elements of the covariance matrices Q t and Ht in (3.1) and (3.2)
are restricted to positive values. While it is theoretically possible to introduce such constraints
into the numerical maximization procedure directly, it is not practically feasible.
In order to employ Newton-type maximization routines implemented in standard statistical
software packages, it is recommendable to perform any maximization with respect
to unconstrained quantities. This can be achieved by transforming the parameters in an
appropriate way. In case of a positive variance term σ 2 , the constraint will be imposed
by defining
ψσ = log σ 2 , −∞ < ψσ < ∞. (3.63)
Once the loglikelihood is maximized using the transformed but unconstrained parameter
ψσ, the constrained parameter can be calculated by back-transformation:
ˆσ 2 = exp( ˆ ψσ), ˆσ 2 > 0. (3.64)