Thesis - Instituto de Telecomunicações

5.4. UNCERTAINTY BASED CLASSIFICATION AND CLASSIFIER FUSION 107p(w i |x) = p(x|w i)p(w i )∑j p(x|w j)p(w j ) . (5.16)We will assume p(x|w i ) as being normal with the parameters µ, σ: X ∼ N(µ, σ 2 ). Now,parameters µ and σ are estimated from a training population of size n, producing theestimates ˆµ and ˆσ. We would like to study the effect of the training vector size on p(ˆx|w i )in a particular point x for some class w i . The maximum likelihood estimators for µ and σare given by:ˆσ 2 = 1 nˆµ = 1 n∑x i , (5.17)ni=1n∑(x i − ˆµ) 2 . (5.18)i=1The estimate of µ is a normally distributed random variable: ˆµ ∼ N(µ, σ2n ). ˆσ2 followsa chi-square distribution: ˆσ 2 ∼σ2(n−1) χ2 n−1The distribution ˆp(x|w i )=p(x|w i , ˆµ, ˆσ) = 1(x−ˆµ)2e− 2ˆσ 2 , for a constant x is a combinationof three components (where (x − ˆµ) 2 ∼ χˆσ √ 2π2n.c. 1chi-square distribution with one degree of freedom).(λ = N µ σ), with χ being the non-centraln.c.1This will form a complex operation over distributions that we cannot use to obtain adirect closed form.The uncertainty of the error probability will then be assessed empirically. Figure 5.8presents a normal model with zero mean and unitary standard deviation. When estimatingthe parameters (both µ and σ unknown) from a data set with 100 samples we can see theempirical distribution of ˆp(x|w i ) for each x. The three lines at the points [−3, −2, −1, 0]represent cuts where we observe the distribution at fixed x points. The histograms of ˆp(x|w i )for each of these points are depicted in figure 5.9. We can verify that for distinct values ofx the distribution has different models. We obtained the empirical distribution repeatingthe estimation procedure 100 times, using random selection of the training patterns. Thevariance of ˆp(x|w i ) is depicted in figure 5.10 presenting a function that we could only observeempirically.