Thesis - Instituto de Telecomunicações

106 CHAPTER 5. FEATURE SELECTION AND CLASSIFICATION5.4 Uncertainty Based Classification and Classifier FusionIn this section we address a novel classification technique proposed in this thesis, to beapplied on data with low separability.We can assess the error probability of a MAP classification usingpe(x) = 1 − max g i (x), (5.14)iwhere the discriminant function g i (x) =p(w i |x) for the selected class is used to inferthe classification error probability of sample x in class i.However instead of g i (x), an estimate is used, g i (x) = ˆp(w i |x), based on a parametric ornonparametric model, and learned from a training set. Two problems can occur: (1) we havetoo few data available and therefore poor estimates of the model; (2) the model does not fitthe data well (incorrect model). These two problems are not reflected in the discriminantfunction; in spite of this, the discriminant function is used both for the classification decisionand for the rejection decision.We propose a new classification scheme based on the classification uncertainty, by evaluatingĝ i (x) as a random variable for a fixed x ∗ . Our intention is to be able to use informationfrom the classification error probability, even when the data is not well modeled.We decided to use the term uncertainty that is related to the measurement error [232],defined by the International Organization for Standardization (ISO) as the dispersion ofthe values that could reasonably be attributed to the measurand [107].5.4.1 Uncertainty ModelingWe start by considering a normal distribution estimation problem.maximum a posteriori (MAP) classifier, defined by the decision rule:Consider again thedecide w i if i = argmax i (p(w i |x)). (5.15)We want to study the behavior of the error estimate of the MAP classifier ˆp ewi (x) =1 − ˆp(w i |x). We start by studying p(x|w i ), since p(w i |x) is given by