"Frontmatter". In: Analysis of Financial Time Series

FORECASTING 163in m 11 and m 22 indicate better forecasts. The test statisticχ 2 =2∑2∑i=1 j=1(m ij − m i0m 0 jm )2m i0 m 0 jmcan then be used to evaluate the performance of the model. A large χ 2 signifies thatthe model outperforms the chance of random choice. Under some mild conditions,χ 2 has an asymptotic chi-squared distribution with 1 degree of freedom. For furtherdiscussion of this measure, see Dahl and Hylleberg (1999).For illustration of the directional measure, consider the 1-step ahead probabilityforecasts of the 8-4-1 feed-forward neural network shown in Figure 4.6. The 2 × 2table of “hits” and “misses” of the network isPredictedActual up downup 12 2 14down 8 2 1020 4 24The table shows that the network predicts the upward movement well, but farespoorly in forecasting the downward movement of the stock. The chi-squared statisticof the table is 0.137 with p value 0.71. Consequently, the network does not significantlyoutperform a random walk model with equal probabilities for “upward” and“downward” movements.4.4.2.2 Magnitude MeasureThree statistics are commonly used to measure performance of point forecasts. Theyare the mean squared error (MSE), mean absolute deviation (MAD), and mean absolutepercentage error (MAPE). For l-step ahead forecasts, these measures are definedasMSE(l) = 1 m−1 ∑[x T +l+ j − x T + j (l)] 2 (4.47)mj=0MAD(l) = 1 m−1 ∑| x T +l+ j − x T + j (l) | (4.48)mj=0MAPE(l) = 1 m−1∑x T + j (l)m ∣ − 1x ∣ , (4.49)T + j+lj=0where m is the number of l-step ahead forecasts available in the forecasting subsample.In application, one often chooses one of the above three measures, and the