statisticalrethinkin..

200 6. MODEL SELECTION, COMPARISON, AND AVERAGING(a)(b)deviance48 52 5624.1N = 204.97.29.2deviance22 26 302.2 4N = 105.3 79.21 2 3 4 5parameters1 2 3 4 5parametersFIGURE 6.8. Accuracy of the DIC estimate. Black points are D train and openpoints D test . e blue trend shows DIC for each model. Data simulatedidentically to FIGURE 6.6, but all models now use a regularizing prior. Seemain text for details. Each point is the mean of 10-thousand simulations.(a) N = 20. (b) N = 10.look like this:y i ∼ Normal(µ i , 1)µ i = α + β 1 x i + β 2 z iα ∼ Normal(0, 100)β 1 ∼ Normal(0, 1)β 2 ∼ Normal(0, 1)Priors of this kind—zero mean and applied to regression coefficients—are oen called REGU-LARIZING priors. ey are designed to produce less overfitting. Let’s see how DIC anticipatesthis.FIGURE 6.8 displays plots analogous to panels c and d in FIGURE 6.6, but now with DICshown by the blue trend. Black points still show D train and open points D test . In panel (a),note that models with 3, 4, and 5 parameters overfit by less than twice the number of parameters.DIC also correctly anticipates this fact, although it was a little too pessimistic aboutthe model with 3 parameters.In panel (b), the sample size is cut in half to N = 10. Compare to FIGURE 6.6, panel(d). With flat priors, the more complex models overfit more than AIC expects. But nowusing the regularizing priors, DIC manages to get much closer to the target. A little bit ofregularization can go a long way towards taming overfit models. e reason is that overfitmodels are fitting parameters on the basis of very little evidence per parameter. is resultsin wide posterior densities, indicating broad uncertainty about the parameters. Just a littlebit of prior information on the parameters can have a big effect on unreliable estimates, whilehaving little impact on the parameters we can be confident about.