View - Universidad de Almería

Proceedings of GO 2005, pp. 67 – 69.Globally optimal prototypesin kNN classification methods ∗E. Carrizosa, 1 B. Martín-Barragán, 1 F. Plastria, 2 and D. Romero-Morales 31 Universidad de Sevilla, Spain{ecarrizosa,belmart}@us.es2 Vrije Universiteit Brussel, BelgiumFrank.Plastria@vub.ac.be3 University of Oxford, United Kingdomdolores.romero-morales@sbs.ox.ac.ukAbstractThe Nearest Neighbor classifier has shown to be a powerful tool for multiclass classification. Inorder to alleviate its main drawbacks (high storage requirements and time-consuming queries), aseries of variants, such as the Condensed or the Reduced Nearest Neighbor, have been suggested inthe last four decades.In this note we explore both theoretical properties and empirical behavior of another such variant,in which the Nearest Neighbor rule is applied after selecting a set of so-called prototypes, whosecardinality is fixed in advance, by minimizing the empirical misclassification cost.The problem is shown to be N P-Hard. Mixed Integer Programming (MIP) programs are formulated,theoretically compared and solved by a standard MIP solver for problems of small size.Large sized problem instances are solved by a variable neighborhood metaheuristic yielding goodclassification rules in reasonable time.Keywords:Data Mining, Classification, Optimal Prototype Subset, Nearest Neighbor, Integer Programming.1. IntroductionIn a Classification problem, one has a database with individuals of |C| different classes, andone wants to derive a classification rule, i.e., a procedure which labels every future entry v asmember of one of the |C| existing classes.Roughly speaking, classification procedures can be divided into two types: parametric andnon-parametric. Parametric procedures assume that each individual from class c ∈ C is associatedwith a random vector with known distribution, perhaps up to some parameters, to beestimated, (e.g. data are multivariate normal vectors, with unknown mean µ c and covariancematrix Σ c ), and use the machinery of Statistics as main technique, see e.g. [21].For complex databases, with no evident distributional assumptions on the data (typicallythe case of databases with a mixture of quantitative and qualitative variables), non-parametricmethods, as the one described in this talk, are needed.In recent years there has been an increasing interest in deriving (non-parametric) classificationrules via Mathematical Programming. Most of such methods require, for each individuali, a vector v i of n numerical variables. In particular this assumes variables to be ratio-scaled,∗ Partially supported by projects BFM2002-04525-CO2-02, Ministerio de Ciencia y Tecnología, Spain, and FQM-329, Junta deAndalucía, Spain