The Generation of Fuzzy Rules from Decision Trees - Department of ...

More documents

Recommendations

Info

y summing the memberships of all the examples in all classes and the numerator is the sum of thememberships of all the examples that (partially) belong to Class 1. That is, fuzzy cardinalities are usedto determine all class counts [4]. The numerator and denominator are no longer necessarily integers. 14.2x015.1(a)class12/1675-25class16x011x2 7.612.9MostlyClass5MostlyClass9MostlyClass12MostlyClass16(b)Figure 5. Fuzzy decision trees (a) from FCG and (b) with the use of fuzzy entropy.In Figure 5 two trees created from an artificial 32 element data set 1 are shown. Clustering was appliedand produced 4 output classes. The tree in Figure 5 (a) resulted from the application of FCG, while thetree in Figure 5 (b) resulted from allowing examples to belong to two classes. The train examples wereallowed to belong to the two classes in which they had the most fuzzy membership after clustering. Asit is likely that most leaves will consist of mixed classes, a stopping criterion other than all examplesbelonging to one class was needed. The stopping criterion used for this example was that the information1in the leaves was required to be less thanC−1 or 1 3of the total information of the entire training set,where C is the number of classes obtained from clustering. It is clear that the decision tree producedwith the use of fuzzy entropy is smaller (7 nodes vs. 13 nodes) and will result in less rules.There is a third approach (which we have not applied) to reducing the number of rules. After rulesare generated from the decision tree, classic approaches [20] can be adapted to prune conditions fromthe rules and rules that are ineffective. Genetic search can be applied to the initial set of rules to find amore compact set of fuzzy rules with equivalent accuracy [16, 18]. Size reduction of the rule base throughinterpolation [12] may be attempted. In each approach it is important that the final set of rules havemaximum interpretability.1 Available from the author.8
5. SummaryA method for creating fuzzy rules from decision trees has been presented and analyzed. The methodtends to produce too many rules. The rules are generated directly from data. The control resulting fromthe rules suggests they can benefit from post generation tuning. An example using fuzzy entropy wasshown which can reduce the size of the generated decision tree and hence reduce the number of fuzzyrules created from the decision tree. This paper did not address how to create the actual rules fromthe decision tree that was obtained with the use of the fuzzy entropy measure. It is possible to createproportionally weighted classes based on the mixture of examples at the node or to create a new modifiedset of outputs sets based upon the final tree, for example. Both of these approaches are being pursued.Acknowledgements: This research was partially supported by the United States Department of Energythrough the Sandia National Laboratories LDRD program, contract number DE-AC04-76DO00789. Thanks toHamid Berenji who initially posed this problem and discussed an early attempt. Also to the NASA-Ames researchcenter who provided space during Hall’s sabbatical.References[1] A. Bensaid, L. Hall, J. Bezdek, L. Clarke, M. Silbiger, and et.al. Validity-guided (re)clustering for imagesegmentation. IEEE Transactions on Fuzzy Systems, 4(2):112–123, 1996.[2] H. Berenji. Machine learning in fuzzy control. In Iizuka 90, pages 231–234, 1990.[3] H. Berenji and P. Khedkar. Learning and tuning fuzzy logic controllers through reinforcements. IEEETransactions on Neural Networks, 3(5):724–740, 1992.[4] J. Bezdek and S. Pal, editors. Fuzzy Models for Pattern Recognition. IEEE Press, 1992.[5] G. Box and G. Jenkins. Time Series Analysis: Forecasting and Control. Holden Day, 1970.[6] R. Cannon, J. Dave, and J. Bezdek. Efficient implementation of the fuzzy c-means clustering algorithms.IEEE Transactions on Pattern Analysis and Machine Intelligence, 8:248–255, 1986.[7] Z. Chi and H. Yan. ID3-derived fuzzy rules and optimized defuzzification for handwritten numeral recognition.IEEE Transactions on Fuzzy Systems, 4(1):24–31, February 1996.[8] L. Hall and P. Lande. Generating fuzzy rules from data. In FUZZ-IEEE’96, pages 1757–1762, 1996.[9] L. Hall and P. Lande. Generating fuzzy rules from decision trees. In Seventh International Fuzzy SystemsAssociation World Conference 97, pages I–417–423, 1997.[10] J.-R. Jang. Anfis: Adaptive-network-based fuzzy inference systems. IEEE Transactions on Systems, Manand Cybernetics, 23(3):665–685, 1993.[11] J.-S. Jang and N. Gulley. Fuzzy Logic Toolbox for Use with Matlab. The MathWorks, Inc., 24 Prime ParkWay, Natick Mass., 1995.[12] L. Koczy and K. Hirota. Size reduction by interpolation in fuzzy rule bases. IEEE Transactions on Systems,Man and Cybernetics, 27(1):14–25, Feb. 1997.[13] P. Lande. Automated fuzzy controller learning and generation by rule extraction from classical decision trees.Master’s thesis, University of South Florida, Tampa, Fl. 33620, Dept. of CSE, 1995.[14] P. Murphy and M. Pazzani. Exploring the decision forest: An empirical investigation of occam’s razor indecision tree induction. Journal of Artificial Intelligence Research, 1:257–275, 1994.[15] S. Murthy, S. Kasif, and S. Salzberg. A system for induction of oblique decision trees. Journal of ArtificialIntelligence Research, 2:1–33, 1994.[16] O. Nelles, M. Fisher, and B. Muller. Fuzzy rule extraction by a genetic algorithm and constrained nonlinearoptimization. In Fifth IEEE International Conference on Fuzzy Systems, pages 213–219, 1996.[17] R. Osborne. Fuzzy clips 6.02a documentation. Technical Report http://ai.iit.nrc.ca/fuzzy/fuzzy.html, CanadianResearch Council, Institute for Information Technology, 1995.[18] I. Ozyurt and L. Hall. Uncertainty Analysis in Engineering, billal ayyub, ed. Fuzzy Genetic Algorithm BasedApproach to Machine Learning. Kluwer Academic, 1998.[19] J. Quinlan. Improved use of continuous attributes in c4.5. Journal of Artificial Intelligence Research, 4:77–90,1996.9
Page 1 and 2: The Generation of Fuzzy Rules from
Page 3 and 4: Wind 75PlayFigure 1. A simple decis
Page 5 and 6: experimentation showed that creatin
Page 7: is often called the Box-Jenkins [5]

The Generation of Fuzzy Rules from Decision Trees - Department of ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?