6th European Conference - Academic Conferences

Suzan Arslanturk et al.
valued dataset. Different types of syntactic functions are applied to the dataset to generate the class
labels. Since it is a fully controlled scenario varying number of relevant, irrelevant and redundant features
are placed in the dataset. However, the simulated dataset doesn’t lend itself to real longitudinal dataset of
clinical trials and the functions used in this paper does not care about any particular attribute when
deciding about the resultant, which does not fit to what we usually see in real world biomedical datasets.
In this research, on the other hand, several feature selection algorithms are applied to a simulation based
dataset with longitudinal trials in order to compare the performances of the algorithms in terms of different
noise levels and different missing value levels and combination of both. Multicollinearity is added to the
dataset to evaluate the most robust algorithms when dependencies between attributes are in question.
Same attribute selection techniques are also applied to the combination of the longitudinal datasets. A
simulation dataset is chosen since full control over the dataset is achieved.
This paper is organized as follows. In Section 2 we explain different attribute selection techniques.
Section 3 outlines how the database is created and how the rules are embedded into the dataset. In
Section 4 the programming language used to embed the rules and the data mining tool to perform the
feature selection algorithms are explained. Section 6 and Section 7 present the results.
2. Attribute selection techniques
Attribute selection techniques can be categorized into three different categories: filter, wrapper and
embedded methods. In filtering methods, the feature selection method takes place before any learning
algorithm. The undesirable attributes are filtered out before the classification step. All the training data is
used in filtering methods (Hall 2003). In embedded methods, the learning algorithm has its own feature
selection algorithm embedded in it (Molina 2002). J48 decision tree classification algorithm is a common
example of an embedded method. In wrapper mode, on the other hand, the feature selection algorithm
uses the learning algorithm as a sub-routine (John 1994).
Five different attribute selection methods are applied to the dataset. Wrapper methods which are
correlation based feature selection and Relief, a filtering method, information gain and an embedded
method J48 decision tree based feature selection are applied to the dataset and the results are compared
in terms of sensitivity and specificities.
2.1 Correlation based feature selection
Correlation based feature selection evaluates the dependencies between attributes and eliminate the
ones which are correlated to each other. The irrelevant and redundant data has to be removed as much
as possible. After the feature selection, the remaining data has to be highly correlated with the class and
uncorrelated with each other.
As equation 1 (Ghiselli, 1964) formalizes :
Merits = (1)
The feature subset S contains k different features where rfc is the feature to class correlation and rff is the
feature to feature correlation. In order to have a good feature selection algorithm the merit has to be
maximized.
Symmetrical uncertainty can be evaluated as follows where H(X) and H(Y) are marginal entropies.
Symmetrical uncertainty = 2.0 x (2)
2.2 Consistency based feature selection
The consistency of the class is evaluated by first figuring out all different combinations of the attributes.
For each different combination the consistency is calculated by differentiating the number of occurrences
of a particular attribute from the cardinality of the majority class (Hall 1998).
19