Applications of state space models in finance

6.4 Concluding remarks 119
of 150 weekly observations is considered in the second step of the proposed evaluation
procedure.
6.3.3.2 Step II: Out-of-sample period of ten years to identify the overall best modeling
approach
Table 6.9 summarizes the calculated mean absolute and squared errors for the RW,
GRW and MMR model, respectively, by reporting their averages across all sectors; for
a comprehensive sectoral breakdown, see Table C.6 in the appendix. According to the
considered average mean errors, the out-of-sample forecasting performance of all three
models over the last ten years of the given sample is very similar. Even though the
average MAE is slightly lower for the MMR model and the average MSE is lowest for
the RW model, based on these results neither approach significantly stands out. This
is also confirmed by looking at the relative ranks, which indicate that the different
modeling approaches yield an average rank of around two.
Table 6.9: Average out-of-sample MAE and MSE across sectors (520 samples).
Mean absolute error (×10 2 ) Mean squared error (×10 4 )
RW GRW MMR RW GRW MMR
Average error 1.311 1.310 1.309 3.584 3.600 3.589
Average rank 2.33 2.11 1.94 1.94 2.22 2.17
When Spearman’s rank correlation coefficient is employed to evaluate the forecasting
performance in a cross-sectional context, the RW model has a small advantage over
its competitors. While Figure 6.14 illustrates that the realized rank correlations are
similarly distributed for all three models, the median rank correlation as well as the
information criteria of the RW model are slightly higher.
Overall, neither the less parsimonious moving mean reverting nor the generalized random
walk model, which are both motivated by their respective capability to capture
volatility clusters and outliers, yield a forecasting advantage over the random walk specification.
This result suggests that heteroskedasticity and outliers can be considered as
being “third-order” problems in the context of applying the Kalman filter to model the
time-varying behavior of systematic risk for pan-European sector indices.
6.4 Concluding remarks
Despite the considerable empirical evidence that systematic risk is not constant over
time, only few studies deal with the explicit modeling of the time-varying behavior of
betas. Previous studies with a focus on Australia, India, New Zealand, the U.S. and
the UK primarily employed Kalman filter and GARCH based techniques. The empirical
analysis presented in this chapter contributes an investigation of time-varying betas for
pan-European industry portfolios. The spectrum of modeling techniques is extended by
(i) incorporating two Markov switching approaches, whose capabilities to model time-