a la physique de l'information - Lisa - Université d'Angers

Recommendations

Info

F. CHAPEAU-BLONDEAU, D. ROUSSEAU, The minimum description length principle for probability density estimation by regular histograms, Physica A, vol. 388, 3969–3984, 2009. 149/197
Author's personal copy Physica A 388 (2009) 3969–3984 Contents lists available at ScienceDirect Physica A journal homepage: www.elsevier.com/locate/physa The minimum description length principle for probability density estimation by regular histograms François Chapeau-Blondeau ∗ , David Rousseau Laboratoire d’Ingénierie des Systèmes Automatisés (LISA), Université d’Angers, 62 avenue Notre Dame du Lac, 49000 Angers, France a r t i c l e i n f o Article history: Received 17 December 2008 Received in revised form 15 May 2009 Available online 16 June 2009 PACS: 05.45.Tp 05.40.-a Keywords: Statistical information processing Data analysis Random signal analysis Probability density estimation Minimum description length Statistical information theory 1. Introduction a b s t r a c t The minimum description length principle is a general methodology for statistical modeling and inference that selects the best explanation for observed data as the one allowing the shortest description of them. Application of this principle to the important task of probability density estimation by histograms was previously proposed. We review this approach and provide additional illustrative examples and an application to realworld data, with a presentation emphasizing intuition and concrete arguments. We also consider alternative ways of measuring the description lengths, that can be found to be more suited in this context. We explicitly exhibit, analyze and compare, the complete forms of the description lengths with formulas involving the information entropy and redundancy of the data, and not given elsewhere. Histogram estimation as performed here naturally extends to multidimensional data, and offers for them flexible and optimal subquantization schemes. The framework can be very useful for modeling and reduction of complexity of observed data, based on a general principle from statistical information theory, and placed within a unifying informational perspective. © 2009 Elsevier B.V. All rights reserved. In statistical information processing, probability density estimation is a ubiquitous and very useful process. For probability density estimation from observed data, a much common approach proceeds through the construction of an empirical histogram with regular (equal width) bins. When a fixed number of bins is imposed, the construction of a histogram is a rather straightforward operation. However, the number of bins in itself has a major impact on the quality of the estimation realized by the histogram for the underlying probability density. For a given number N of observed data points, if the number of bins is too small, the resolution of the histogram is poor and leads to a very raw estimate of the probability density. On the contrary, if the number of bins is too large, the counts of data points in the bins fluctuate strongly to yield a very jerky histogram as a poor estimate for the probability density. This points to an optimal number of bins between these two extremes that will lead to an optimal histogram for estimating the probability density. Any approach aiming at determining an optimal number of bins needs necessarily to rely on a definite criterion to measure optimality in this context with histograms. A specially interesting approach of this type is based on the principle of minimum description length. The minimum description length (MDL) principle provides a general approach for statistical modeling and inference from observed data [1–3]. Briefly stated, this principle amounts to choosing for data, among a class of possible models, the model that allows the shortest description of the data. The MDL approach is rooted in the Kolmogorov theory of complexity [4]. Since its formal introduction some thirty years ago [5], the MDL principle has developed along both theoretical and practical directions. The theoretical foundations of the MDL principle have been investigated to great depth in statistics, and new theoretical aspects are still being explored [1,3,6]. At the same time, the MDL principle has been considered to provide solutions to a large variety of problems, including nonlinear time series modeling [7,8], Markov- ∗ Corresponding author. E-mail address: chapeau@univ-angers.fr (F. Chapeau-Blondeau). 0378-4371/$ – see front matter © 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.physa.2009.05.044 150/197
Page 1 and 2:
Université d’Angers Laboratoire
Page 3 and 4:
Ce mémoire est dédié à mon épo
Page 5 and 6:
4 Physique de l’information 39 4.
Page 7 and 8:
1.2 Organisation du document L’es
Page 9 and 10:
4/197
Page 11 and 12:
de Rennes 1. Licence et Maîtrise
Page 13 and 14:
2.5 Activités de recherche 2.5.1 B
Page 15 and 16:
2.5.2 Encadrement Thèses Thèse so
Page 17 and 18:
2.5.3 Responsabilités Management d
Page 19 and 20:
[A20] S. BLANCHARD, D. ROUSSEAU, D.
Page 21 and 22:
In 8th Euro-American Workshop on In
Page 23 and 24:
études visaient alors à analyser
Page 25 and 26:
Fig. 3.1 réside dans le fait qu’
Page 27 and 28:
également permis de réaliser qu
Page 29 and 30:
caracteristique effective g eff (u)
Page 31 and 32:
des architectures neuronales contr
Page 33 and 34:
Dans l’ Éq.(3.10), ∆t représe
Page 35 and 36:
quantifieur, le niveau de saturatio
Page 37 and 38:
exemples présentés dans [39], l
Page 39 and 40:
• le type de validation des résu
Page 41 and 42:
présenté de façon condensée dan
Page 43 and 44:
38/197
Page 45 and 46:
pour l’étude de la turbulence pa
Page 47 and 48:
A B C Figure 4.1 : Images de coupe
Page 49 and 50:
atteint la capacité informationnel
Page 51 and 52:
* Figure 4.5 : Échelle optimale d
Page 53 and 54:
• L’“intégrale de corrélati
Page 55 and 56:
4.2.3 Analyse multifractale en imag
Page 57 and 58:
fonction de partition Z 10 30 10 25
Page 59 and 60:
exposant τ(q) 15 10 5 0 −5 −10
Page 61 and 62:
complexité colorimétrique des ima
Page 63 and 64:
dimension fractale généralisée D
Page 65 and 66:
est la moyenne du signal calculée
Page 67 and 68:
A B C Figure 4.17 : Observations in
Page 69 and 70:
A B C Figure 4.20 : Nombre moyen de
Page 71 and 72:
[14] S. Blanchard, D. Rousseau et F
Page 73 and 74:
[43] J. Chauveau, D. Rousseau et F.
Page 75 and 76:
[76] A. Humeau, B. Buard, F. Chapea
Page 77 and 78:
[100] M. D. McDonnell, N. G. Stocks
Page 79 and 80:
[131] D. Rousseau, F. Duan, J. Roja
Page 81 and 82:
76/197
Page 83 and 84:
D. ROUSSEAU and F. CHAPEAU-BLONDEAU
Page 85 and 86:
ROUSSEAU AND CHAPEAU-BLONDEAU: BAYE
Page 87 and 88:
ROUSSEAU AND CHAPEAU-BLONDEAU: BAYE
Page 89 and 90:
D. ROUSSEAU, G. V. ANAND, and F. CH
Page 91 and 92:
quantizers. Classically, the design
Page 93 and 94:
Fig. 2. Nonlinear array used as pre
Page 95 and 96:
probability of error P er 10 0 10 -
Page 97 and 98:
probability of error (expected as s
Page 99 and 100:
[3] N.H. Lu, B.A. Einstein, Detecti
Page 101 and 102:
Abstract Neurocomputing 71 (2007) 3
Page 103 and 104: 3. Assessing nonlinear transmission
Page 105 and 106: output SNR 250 200 150 100 50 0 N=3
Page 107 and 108: precisely the scope of the present
Page 109 and 110: [4] F. Chapeau-Blondeau, G. Chauvet
Page 111 and 112: Nonlinear SNR amplification of harm
Page 113 and 114: P.R. BHAT and D. ROUSSEAU and G. V.
Page 115 and 116: It is assumed that the signal s is
Page 117 and 118: Let us de£ne the improvement in pe
Page 119 and 120: F. CHAPEAU-BLONDEAU and D. ROUSSEAU
Page 121 and 122: Contents Raising the noise to impro
Page 123 and 124: Raising the noise to improve perfor
Page 135 and 136: S. BLANCHARD, D. ROUSSEAU, D. GINDR
Page 137 and 138: 1984 OPTICS LETTERS / Vol. 32, No.
Page 139 and 140: F. CHAPEAU-BLONDEAU, D. ROUSSEAU, S
Page 141 and 142: 1288 J. Opt. Soc. Am. A/Vol. 25, No
Page 147 and 148: 36 IEEE SIGNAL PROCESSING LETTERS,
Page 149 and 150: 38 IEEE SIGNAL PROCESSING LETTERS,
Page 151 and 152: A. HISTACE and D. ROUSSEAU. Constru
Page 153: noises Z i in (3) are chosen Gaussi
Page 157 and 158: Author's personal copy F. Chapeau-B
Page 161 and 162: edundancy Author's personal copy F.
Page 163 and 164: model description length Author's p
Page 165 and 166: total description length x 10 4 1.6
Page 171 and 172: J. CHAUVEAU and D. ROUSSEAU and F.
Page 173 and 174: 2 for natural images, and carry rel
Page 175 and 176: 4 Fig. 2 Random RGB color image I2(
Page 177 and 178: 6 log2[ number of boxes N(a) ] 24 2
Page 179 and 180: 8 Fig. 7 Color histogram in the RGB
Page 181 and 182: 10 log2[ number of boxes N(a) ] 8 7
Page 183 and 184: 12 17. Landgrebe, D.: Hyperspectral
Page 185 and 186: Numerical simulation of laser Doppl
Page 187 and 188: esults are meaningful to better app
Page 189 and 190: A. HUMEAU, B. BUARD, F. CHAPEAU-BLO
Page 191 and 192: 618 A Humeau et al 1. Introduction
Page 193 and 194: 620 A Humeau et al (a) (b) (c) Figu
Page 195 and 196: 622 A Humeau et al (a) (b) (c) Figu
Page 197 and 198: 624 A Humeau et al Figure 5. Averag
Page 199 and 200: 626 A Humeau et al Therefore, the m
Page 201 and 202: 628 A Humeau et al peripheral cardi
show all

a la physique de l'information - Lisa - Université d'Angers

Create successful ePaper yourself

Delete template?

Save as template?