Learning binary relations using weighted majority voting

More documents

Recommendations

Info

262 GOLDMAN AND WARMUTH 0 0 0 -.. 0 1 1 1 --- 1 [~I-I ... [~I-1 I Figure 5. FirSt/zi elements of the sequence (0) m {1) m (2) m .... = ([--~1- 1)(#i- 2[--~] ) P~ #4 (1 - d)dm - + 2m 2 2 > #2 #i - 2m 2' [ 1-2 .-- [ 1-2 • t where d = I~] - ~m and the last inequality follows from the observation that 0 _< d < 1. This completes the proof of the lemma. • Observe that the simple linear bound is a better lower bound only for m < #i < 2m. Next, we capture the relationship between Ji and the noise within cluster i of the partition to obtain an upper bound for Ji. LEMMA 5 For 1 majority of the entries in S~, are 1. Thus 6~,j is exactly the number of 0's in Sj. Observe that for every known 0 entry in @, the quantity Ji is increased by one whenever the learner makes a prediction error when predicting the value of an entry in S} that is a 1. Thus, in the worst case, each of the 6i,j entries in S~ that are 0 could cause Ji to be incremented for each of the n~ - 5i,j entries in S~ that are 1. Thus, J~ < E 6i,i (ni - ~i,i) = 6ini - 5? -- 7.,, 3 • j=l j=l
LEARNING BINARY RELATIONS 263 m V Since x 2 is convex, it follows that Ej=I 52~,3" -- > ~" This completes the proof of the lemma. • Next we obtain an upper bound on the sum of the logarithms of the weights of a set of edges from .4. A key observation used to prove this upper bound is that the overall weight in the system never increases. Therefore, since the initial weight in the system is n(n - 1)/2 we obtain the following lemma. LEMMA 6 Throughout the learning session for any `4~ C .4, E w(e) < n(n- 1____~) - 2 eE.A' Proof: In trials where no mistake occurs the total weight of all edges ~~e,a w(e) clearly does not increase. Assume that a mistake occurs in the current trial. Ignore all weights that are not updated. Of the remaining total weight W that participates in the update let c be the fraction that was placed on the correct bit and 1 - c be the fraction placed on the incorrect bit. The weight placed on the correct bit is multiplied by 2 - 3' and the weight placed on the incorrect bit by 3". Thus the total weight of all edges that participated in the update is (c(2 - 3") + (1 - c)~/)W at the end of the trial. Since the latter is increasing in c and c < 1/2 whenever a mistake occurs, we have that the total of all weights updated in the current trial is at most (-7- 2-~ + ~)W "y = W at the end of the trial. We conclude that the total weight of all edges also does not increase in trials where a mistakes occurs. Finally since A I C `4, the result follows. LEMMA 7 Throughout the learning session for any `4i C_ `4, eEA' tgw(e) _< IA'llg 21A'------~ Proof- This result immediately follows from Lemma 6 and the concavity of the 10g function. • We are now ready to prove our main result. THEOREM 2 For all/3 E [0, 1), Algorithm Learn-Relation when using the parameter 28 7 = T~ makes at most T-~~) lg 3rnn21gkp+2C~p(mn-ap)lg-~ min kpm + min ~ lg e + -- -"2" - ~ 2 lg ~ lg mistakes in learning a binary-relation where the outside minimum is taken over all partitions p, and kp denotes the size and ap the noise of partition p.
Page 1 and 2: Machine Leaming, 20, 245-271 (1995)
Page 3 and 4: LEARNING BINARY RELATIONS 247 When
Page 5 and 6: LEARNING BINARY RELATIONS 249 ~,~,.
Page 7 and 8: LEARNING BINARY RELATIONS 251 l J i
Page 9 and 10: LEARNING BINARY RELATIONS 253 Learn
Page 11 and 12: LEARNING BINARY RELATIONS 255 We no
Page 13 and 14: LEARNING BINARY RELATIONS 257 Solvi
Page 15 and 16: LEARNING BINARY RELATIONS 259 In th
Page 17: LEARNING BINARY RELATIONS 261 depen
Page 21 and 22: LEARNING BINARY RELATIONS 265 Final
Page 23 and 24: LEARNING BINARY RELATIONS 267 32500
Page 25 and 26: LEARNING BINARY RELATIONS 269 mance
Page 27: LEARNING BINARY RELATIONS 271 Kivin

Learning binary relations using weighted majority voting

Create successful ePaper yourself

Delete template?

Save as template?