Data Mining: Practical Machine Learning Tools and ... - LIDeCC

128 CHAPTER 4 | ALGORITHMS: THE BASIC METHODSare working with binary data. Weights are unchanged if the attribute value is 0,because then they do not participate in the decision. Otherwise, the multiplieris a if that attribute helps to make a correct decision and 1/a if it does not.Another difference is that the threshold in the linear function is also a userspecifiedparameter. We call this threshold q and classify an instance as belongingto class 1 if and only ifwa + wa+ wa + ... + wa > q.0 0 1 1 2 2The multiplier a needs to be greater than one. The w i are set to a constant atthe start.The algorithm we have described doesn’t allow negative weights, which—depending on the domain—can be a drawback. However, there is a version,called Balanced Winnow, which does allow them. This version maintains twoweight vectors, one for each class. An instance is classified as belonging to class1 if:( 0 0) 0+ ( 1 - 1 ) 1+ + -( k k ) k >+ - + - + -w - w a w w a ... w w a qkkFigure 4.11(b) shows the balanced algorithm.Winnow is very effective in homing in on the relevant features in a dataset—therefore it is called an attribute-efficient learner. That means that it may be agood candidate algorithm if a dataset has many (binary) features and most ofthem are irrelevant. Both winnow and the perceptron algorithm can be used inan online setting in which new instances arrive continuously, because they canincrementally update their hypotheses as new instances arrive.4.7 Instance-based learningIn instance-based learning the training examples are stored verbatim, and a distancefunction is used to determine which member of the training set is closestto an unknown test instance. Once the nearest training instance has beenlocated, its class is predicted for the test instance. The only remaining problemis defining the distance function, and that is not very difficult to do, particularlyif the attributes are numeric.The distance functionAlthough there are other possible choices, most instance-based learners useEuclidean distance. The distance between an instance with attribute values a 1(1),a 2(1),...,a k(1)(where k is the number of attributes) and one with values a 1(2),a 2(2),...,a k(2)is defined as( ) + ( - ) + + ( - )( ) ( 2) 2 ( ) ( 2) 2 ( 1) ( 2)21 12 2k k11a - a a a ... a a .