View - Statistics - University of Washington

36The error of the approximation in equation 3.10 is O(N −1/2 ).At this point, we could drop terms which do not increase with N from equation3.10 and obtain equation 3.11. The error of our approximation would then be O(1).However, if we consider the prior on θ more closely, we find that the approximationis actually better for a certain prior.Suppose p(θ|M i ) is multivariate normal with mean ˜θ and covariance matrixequal to I −1 . On average, this would give the prior about the same impact onlog(p(X|M i )) as a single observation. We can compute p(˜θ|M i ) by finding the densityof this multivariate normal evaluated at its mean; this is (2π) −Di/2 |I −1 | −1/2 .Recall that for any nonsingular matrix A, |A −1 | = |A| −1 . Substituting into equation3.10, we see that several terms cancel.log(p(X|M i )) ≈ log(p(X|˜θ, M i )) − D i2log(N) (3.11)In going from equation 3.10 to equation 3.11, we have simply chosen a certainprior which conveniently cancels a few other terms. Thus, the error of theapproximation is still O(N −1/2 ).We now multiply equation 3.11 by 2 and substitute the MLE of θ for theposterior mode. Denote the maximized loglikelihood of the data given model i byL(X|M i ). We arrive at the usual formulation of BIC in equation 3.12.BIC(M i ) = 2L(X|M i ) − D i log(N) (3.12)Returning to the Bayes factor idea of equation 3.2, we now have a relativelyeasy way to compute an approximate Bayes factor.2 log(B 21 ) = 2 log(p(X|M 2 )) − 2 log(p(X|M 1 )) (3.13)≈ BIC(M 2 ) − BIC(M 1 ) (3.14)