CSE555: Introduction to Pattern Recognition Midterm ... - CEDAR

More documents

Recommendations

Info

wherep(x|ω i ) = √ 1 [exp − 1 ( ) ] x − µi 22πσi 2 σ iIf we assume the prior probabilities are equal, i.e. P(ω ′ 1) = P(ω ′ 2) = 0.5, thenthe threshold will be about −0.4933. That is, we decide ω 1 if w t x > −0.4933,otherwise decide ω 2 .3. (20pts) Suppose p(x|w 1 ) and p(x|w 2 ) are defined as follows:p(x|w 1 ) = 1 √2πe − x22 , ∀xp(x|w 2 ) = 1 4, −2 < x < 2(a) (7pts) Find the minimum error classification rule g(x) for this two-class problem,assuming P(w 1 ) = P(w 2 ) = 0.5.Answer:(i) In case of −2 < x < 2, because P(ω 1 ) = P(ω 2 ) = 0.5, we have the discriminantfunction g(x) asg(x) = ln p(x|ω 1)p(x|ω 2 ) = ln √ 4 − x22π 2The Bayes rule for classification will beorDecide ω 1 if g(x) > 0; otherwise decide ω 2Decide ω 1 if − 0.9668 < x < 0.9668; otherwise decide ω 2(ii) In case of x ≥ 2 or x ≤ −2, we always decide ω 1 .(b) (10pts) There is a prior probability of class 1, designated as π ∗ 1, so that if P(w 1 ) >π ∗ 1, the minimum error classification rule is <strong>to</strong> always decide w 1 regardless of x.Find π ∗ 1.Answer:According <strong>to</strong> the question, π ∗ 1 will satisfy the following equationp(x|ω 1 )π ∗ 1 = p(x|ω 2 )(1 − π ∗ 1)when x = 2 or x = −2Therefore, we have1√2πe −4 2 π∗1 = 1 4 (1 − π∗ 1)π ∗ 1 ≈ 0.8224(c) (3pts) There is no π ∗ 2 so that if P(w 2 ) > π ∗ 2, we would always decide w 2 . Whynot?Answer:Because p(x|ω 2 ) is only defined for −2 < x < 2, therefore we would always decidew 1 for x ≥ 2 or x ≤ −2, no matter what is the prior probability p(w 2 ).4
4. (20pts) Let samples be drawn by successive, independent selections of a state of naturew i with unknown probability P(w i ). Let z ik = 1 if the state of nature for the kthsample is w i and z ik = 0 otherwise.(a) (7pts) Show thatn∏P(z i1 , · · · ,z in |P(w i )) = P(w i ) z ik(1 − P(w i )) 1−z ikk=1Answer:We are given that{1 if the state of nature for the kz ik =th sample is ω i0 otherwiseThe samples are drawn by successive independent selection of a state of naturew i with probability P(w i ). We have thenandThese two equations can be unified asPr[z ik = 1|P(w i )] = P(w i )Pr[z ik = 0|P(w i )] = 1 − P(w i )P(z ik |P(w i )) = [P(w i )] z ik[1 − P(w i )] 1−z ikBy the independence of the successive selection, we haveP(z i1 , · · · ,z in |P(w i )) ==n∏P(z ik |P(w i ))k=1n∏[P(w i )] z ik[1 − P(w i )] 1−z ikk=1(b) (10pts) Given the equation above, show that the maximum likelihood estimatefor P(w i ) isˆP(w i ) = 1 n∑z iknAnswer:The log-likelihood as a function of P(w i ) isk=1l(P(w i )) = lnP(z i1 , · · · ,z in |P(w i ))[ n]∏= ln [P(w i )] z ik[1 − P(w i )] 1−z ikk=1n∑= [z ik ln P(w i ) + (1 − z ik ) ln(1 − P(w i ))]k=1Therefore, the maximum-likelihood values for the P(w i ) must satisfy∇ P(wi )l(P(w i )) = 1P(w i )n∑z ik −k=1511 − P(w i )n∑(1 − z ik ) = 0k=1
Page 1 and 2: CSE555: Introduction to Pattern Rec
Page 3: Thus, we first compute the sample m
Page 7 and 8: (b) (10pts) Suppose the initial hid

CSE555: Introduction to Pattern Recognition Midterm ... - CEDAR

Create successful ePaper yourself

Delete template?

Save as template?