Machine Learning 3. Nearest Neighbor and Kernel Methods - ISMLL

More documents

Recommendations

Info

Machine Learning / 1. Distance Measures Distances for Nominal Variables For binary variables there is only one reasonable distance measure: { 1 if x = y d(x, y) := 1 − I(x = y) with I(x = y) := 0 otherwise This coincides with the L ∞ distance for the indicator/dummy variables. The same distance measure is useful for nominal variables with more than two possible values. For hierarchical variables, i.e., a nominal variable with levels arranged in a hierarchy, there are more advanced distance measures (not covered here). Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), Institute BW/WI & Institute for Computer Science, University of Hildesheim Course on Machine Learning, winter term 2007 9/48 Machine Learning / 1. Distance Measures Distances for Set-valued Variables For set-valued variables (which values are subsets of a set A) the Hamming distance often is used: d(x, y) := |(x \ y) ∪ (y \ x)| = |{a ∈ A | I(a ∈ x) ≠ I(a ∈ y)}| (the number of elements contained in only one of the two sets). Example: d({a, e, p, l}, {a, b, n}) = 5, d({a, e, p, l}, {a, e, g, n, o, r}) = 6 Also often used is the similarity measure Jaccard coefficient: Example: sim(x, y) := |x ∩ y| |x ∪ y| sim({a, e, p, l}, {a, b, n}) = 1 6 , sim({a, e, p, l}, {a, e, g, n, o, r}) = 2 8 Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), Institute BW/WI & Institute for Computer Science, University of Hildesheim Course on Machine Learning, winter term 2007 10/48
Machine Learning / 1. Distance Measures Distances for Strings / Sequences edit distance / Levenshtein distance: d(x, y) := minimal number of deletions, insertions or substitions to transform x in y Examples: d(man, men) =1 d(house, spouse) =2 d(order, express order) =8 Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), Institute BW/WI & Institute for Computer Science, University of Hildesheim Course on Machine Learning, winter term 2007 11/48 Machine Learning / 1. Distance Measures Distances for Strings / Sequences The edit distance is computed recursively. With x 1:i := (x i ′) i ′ =1,...,i = (x 1 , x 2 , . . . , x i ), i ∈ N we compute the number of operations to transform x 1:i into y 1:j as c(x 1:i , y 1:j ) := min{ c(x 1:i−1 , y 1:j ) + 1, // delete x i , x 1:i−1 y 1:j c(x 1:i , y 1:j−1 ) + 1, // x 1:i y 1:j−1 , insert y j c(x 1:i−1 , y 1:j−1 ) + I(x i ≠ y j )} // x 1:i−1 y 1:j−1 , substitute y j for x i starting from c(x 1:0 , y 1:j ) = c(∅, y 1:j ) := j // insert y 1 , . . . , y j c(x 1:i , y 1:0 ) = c(x 1:i , ∅) := i // delete x 1 , . . . , x i Such a recursive computing scheme is called dynamic programming. Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), Institute BW/WI & Institute for Computer Science, University of Hildesheim Course on Machine Learning, winter term 2007 12/48
Page 1 and 2: Machine Learning Machine Learning 3
Page 3 and 4: Machine Learning / 1. Distance Meas
Page 5: Machine Learning / 1. Distance Meas
Page 9 and 10: Machine Learning / 1. Distance Meas
Page 11 and 12: Machine Learning / 2. k-Nearest Nei
Page 21 and 22: Machine Learning 1. Distance Measur
Page 23 and 24: Machine Learning / 3. Parzen Window
Page 29: Machine Learning / 3. Parzen Window

Machine Learning 3. Nearest Neighbor and Kernel Methods - ISMLL

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?