Applications of state space models in finance

56 4 Markov regime switching
4.5 Model selection and validation
A problem that naturally arises in connection with HMMs is the determination of the
number of discrete states. A common approach to select an appropriate model is based
on the information criteria presented in ¢ 3.6.2.3. According to the proposed AIC and
BIC, the specification that yields the smallest relative information criterion is chosen.
While an information criterion allows for the selection of the best specification among
various fitted models, it cannot guarantee appropriateness of the selected model. This
can only be checked by assessing the model’s fit. However, while it is straightforward
to derive model diagnostics based on generalized recursive and generalized least squares
residuals in the context of continuous state space models (cf. ¢ 3.6), assessing the fit of
HMMs is more complex. As under a hidden Markov model the observations that are related
to different states are produced by different distributions, the residuals should also
be modeled by different distributions. The assumption of independently and identically
distributed errors, which is commonly made in diagnostic testing procedures, does not
even hold approximately for the residuals of a hidden Markov model (cf. Zucchini et al.
2006, ¢ 6.2). A possible solution is to employ so-called pseudo residuals, which allow for a
comparison of observations induced by different distributions. As the concept of pseudo
residuals is beyond the scope of this thesis, it will not find any consideration hereafter;
for a comprehensive introduction to the subject, see, for example, Stadie (2002). Rather
than relying on standard diagnostic tests, in the empirical part below the relative appropriateness
of a hidden Markov model will be formally assessed based on its respective
in- and out-of-sample forecast performance.