statisticalrethinkin..

6.4. DEVIANCE INFORMATION CRITERION 197But some well-known Bayesian statisticians dislike the Bayes factor approach, 88 and all admitthat there are technical obstacles to its use. Foremost for our purposes, computing average likelihoodis hard. AIC, DIC, and the like have their own computational problems, but nothing like averagelikelihood. And even when priors are weak and have no influence on estimates within models, priorscan have a huge impact on comparisons between models.So while the approach is tremendously valuable, and learning an alternative to information criterianecessarily helps one to understand information criteria even better, a robust treatment of Bayesfactors is just beyond the scope of this book. It’s important to realize, though, that the choice ofBayesian or not does not also decide between information criteria or Bayes factors. Moreover, there’sno need to choose, really. We can always use both and learn from the ways they agree and disagree.6.4.1. Defining DIC. First some notation. ere are two elements in calculating DIC: theaverage deviance and the deviance of the average of the parameters.(1) e average deviance arises from noting that, since deviance is computed from parameters,and since parameters have a distribution, then the deviance must alsohave a distribution. Compute the posterior density of the deviance, and then takeits mean. at’s the average deviance, ¯D.(2) e deviance of the average of the parameters implies finding the mean of the parametersand then computing a single deviance value for this posterior mean. at’sthe deviance of the average of the parameters, ˆD.en DIC is defined as:DIC = ˆD + 2(¯D − ˆD) ≈ E D testIn English: the overfitting penalty is twice the difference between the average deviance andthe deviance of the average of the parameters. e difference ¯D − ˆD is called the EFFECTIVENUMBER OF PARAMETERS, oen labeled p D . It takes the place of the parameter count k inAIC’s formula.Overthinking: Computing DIC using posterior variance. An alternative formula for DIC is:DIC = ˆD + var(D)where var(D) is the variance of the posterior distribution of the deviance. is implies that var(D)/2is an estimate of the effective number of parameters, p D . is formula is sometimes more accurate ormore convenient than the original difference formula. Much of the time, it’s almost exactly the same.Whichever you use, just be sure to compute DIC the same way on all models you wish to compare.Or better yet, you could compute DIC both ways, to be sure inference is not sensitive to the details ofcomputation. It doesn’t matter so much that the two formulas for DIC give slightly different answers.What matters is the relative positions of the models remain the same, using the different formulas.6.4.2. Simulating DIC. To get a grasp on DIC, let’s do an experiment that reveals both howto compute it and why it behaves as it does. Suppose we observe a single measurement, andthis measurement has value 1. Also suppose we know the measurement has a standard errorof σ = 1. So our inference problem is just the mean value of the measurement. Using a flatprior, we fit the model and compute ¯D and ˆD: