Ivancevic_Applied-Diff-Geom

Applied Manifold Geometry 331pretations. Given the metric I ij (w) = g ij (w) on the space of parameters,the infinitesimal volume element on the parameter manifold isdV = dw √ det I(w) =k∏dw l√ det I(w).The Riemannian volume of the parameter manifold is obtained by integratingdV over the space of parameters:∫ ∫V M = dV = dw √ det I(w).In other words, the third term (functional complexity) in MDL penalizesmodels that occupy a large volume in the space of distributions.In fact, the volume measure V M is related to the number of ‘distinguishable’probability distributions indexed by the model. Because of theway the model family is embedded in the space of distributions, two differentparameter values can index very similar distributions. If complexityis related to volumes occupied by model manifolds, the measure of volumeshould count only different, or distinguishable, distributions, and not theartificial coordinate volume. It is shown in [Myung et. al. (2000a)] that thevolume V M achieves this goal.l=1Selecting Among Qualitative ModelsApplication of any of the preceding selection methods requires that themodels are quantitative models, each defined as a parametric family of probabilitydistributions.Pseudo–probabilistic MDL ApproachThe ‘pseudo–probabilistic’ approach [Grunwald (1999)] for selectingamong qualitative models derives a selection criterion that is similar tothe MDL criterion for quantitative models, but it is a formulation that iscloser to the original spirit of the MDL principle, which states:‘Given a data set D and a model M, the description length of the data,DL M (D), is given by the sum of (a) the description length of the data whenencoded with help of the model, DL(D|M), and (b) the description lengthof the model itself, DL(M) : DL M (D) = DL(D|M)+DL(M). Among a setof competing models, the best model is the one that minimizes DLM(D).’The above MDL principle is broad enough to include the MDL criterionfor quantitative models as a specific instantiation. The first, lack–of–fit