The Generation of Fuzzy Rules from Decision Trees - Department of ...

More documents

Recommendations

Info

created by the well known C4.5 learning system [20, 2]. FCG generates fuzzy CLIPS rules [17]. Thegenerated rules can then be used as a fuzzy controller or a fuzzy function model and applied to real data.There are several difficulties with FCG. One of our goals for the generation of fuzzy rules is to havea relatively small and understandable set of rules. FCG can, when applied to some domains, generatemore rules than are easy to understand. Also, it would be ideal to use the decision tree generated ruleswithout any tuning and the smoothness of the control or a fuzzy model provided by the initial rules canbe improved for some domains.In this paper a discussion of the problems and some potential remedies [9] is given. A small artificialexample will be used to illustrate an improved approach. In Section 2, we describe creating decisiontrees when all data, including the outputs, is continuous. In Section 3, FCG is described and Section4 discusses the areas in which the system might be improved and suggests improvements. Results froma commonly used data set, the Box-Jenkins gas furnace [5], and a function approximation problem areused for illustration. Section 5 is a summary of the work.2. Decision treesConsider the problem of determining whether to play tennis based on the weather: (Sunny, Cloudy),Wind: (Windy, Quiet) and Temperature (0,100 o F). There are two outcomes (play, Don’t Play). Since weare only interested in continuous inputs, assume the level of windiness and cloudiness are from 0 (no windor clouds) to 10 (windy or cloudy). Given the training examples shown in Table 1 to C4.5, the decisiontree in Figure 1 would be produced. The decision tree allows the classification of examples into n classes(n=2 here) by choosing an attribute whose values may split the examples up into more homogeneousgroups. For continuous valued attributes a value in the data set is chosen as the “split point”. When acontinuous valued attribute is chosen as the attribute to partition a set of examples at a given node inthe tree, the examples will be split into 2 subsets, i.e. a binary split is done.Table 1. A training set from the domain of tennis.Weather Wind Temperature Class0 0 95 no play0 0 80 play10 10 75 no play10 4 65 play10 10 100 no play10 10 80 playThe attribute values of a continuous valued attribute are each examined as a possible attribute forsplitting the example set at a node in a decision tree into subsets. The selection of a specific value isbased upon the information gain ratio associated with choosing that attribute [20]. The attribute, whichhas the highest information gain associated with it is chosen as the attribute for splitting the examplesat a node. The attribute’s information gain will be that of its value in the set of examples at the nodefor which the information gain is highest.In our example tree at the top level the attribute weather has only 1 possibility for a split and theinformation gain is 0. The temperature attribute has 4 possible choices (65, 75, 80 and 95; note everyexample is
Wind 75PlayFigure 1. A simple decision tree created by C4.5 from the examples in Table 1.2.1. Creating class labelsThe examples from which fuzzy rules will be created have continuous valued outputs. The C4.5decision tree algorithm requires crisp class assignments for all objects. It is necessary to partition thecontinuous output values into an effective set of discrete output classes. We use a modification of theapproach given in [21] to create output fuzzy sets. Fuzzy c-means (FCM) clustering [6] (with m=2) isdone on just the set of output values with variable numbers of clusters, denoted c in this paper. Thepartition validity metric [1]n∑ c∑S(c) = (u ik ) 2 (‖x k − v i ‖ 2 − ‖v i − x‖ 2 ) (1)k=1 i=1is used to determine the number of clusters c to use. The number of clusters, c, will correspond directlyto the number of classes in the FCG. In (1), u ik denotes the membership of the k th feature vector in thei th class or cluster, x k is the k th feature vector, v i is the i th cluster center and x is the average vectorcreated from the full set of feature vectors. The u ik ’s and v i ’s are obtained from first applying FCM(searching for c clusters) to the x k ’s.Clustering begins with c 0 = I with c 1 = c 0 + 1, c 2 = c 1 + 1, . . . and ends when S(c n ) > S(c n−1 ) withc n−1 ≥ c 0 . The usual case is I = 2, but to force finer grained partitions it is possible to set I > 2 andthis was done in some experiments.From our set of one dimensional feature vectors, c n fuzzy clusters are created within FCM. Theyconsist of cluster centers v i , 1 ≤ i ≤ c n and membership values for each x k in the clusters representedby the v i ’s, which are contained in u ik . It is possible to create c n triangular fuzzy sets with each setcentered at one of the cluster centers v i . The spread of each symmetric fuzzy set is equal to twice thesmallest distance between any 2 cluster centers. Figure 2, shows the results of clustering 5 values into 2classes by showing the cluster centers and the associated triangular fuzzy sets. The largest 2 clusteredvalues (64, and 70) belong to centroid v2 with the smallest 3 belonging to v1.Since the clustering is one dimensional, we limit, each feature vector to membership in at most 2 classesor clusters. C4.5 requires that each training example presented to it belong to only 1 class. Therefore, ourinitial approach is to create 5 possible classes to which a feature vector with some membership in class Aand class B may belong. If the membership of a feature vector is greater than 0.875 in A, it is assignedto class A. It belongs to class-A-B-25-75 if its membership in A is in [.125, .375), to class-A-B-50-50 ifits membership in A is in [.375, .625), and to class-A-B-75-25 if its membership in A is in [.625, .875). Ineach of our example instances the remaining membership would be in class B. These joint classes can bedirectly implemented in terms of their component fuzzy sets representing class A and class B by usingfuzzy truth values in fuzzy CLIPS [17]. Joint classes will be further discussed in the next section.3
Page 1: The Generation of Fuzzy Rules from
Page 5 and 6: experimentation showed that creatin
Page 7 and 8: is often called the Box-Jenkins [5]
Page 9 and 10: 5. SummaryA method for creating fuz

The Generation of Fuzzy Rules from Decision Trees - Department of ...

Create successful ePaper yourself

Delete template?

Save as template?