SEKE 2012 Proceedings - Knowledge Systems Institute

to d (true negatives), w hile F APR in the reference to a (true
negatives).
The presented above fitness functions are defined fo r a
binary (two-category) classi fication problem. In order to
focus on a single class, a speci al processing of confusion
matrix is performed during evaluation of models. This
processing means collapsing a multi-class confusion matrix
into a tw o-class matrix. In such a case, the multi-class
matrix is being trans formed into a num ber (equal to the
number of classes) of matrixes. Each of these matrixes is
related to two classes: a class of interest – let say a class A ,
and the other class – let say a class non-A – obtained by
fusion of all other classes. Using these two classes, all
classification rates (accuracy , sensitivity, specificity and
precision) are calculated.
As it has been explain earlier, each m odel is evaluated
by a set of objectives – a set of fitness functions where each
of them is related to a single class. E ach model is
“measured” by a set of fitness function:
or
The proposed m ethod is used to construct FNNs therefore
there is one more thing that need to be explained – how the
classification is determine based on the fuzzy output. In
order to cope w ith that, a m ethod with a Threshold Value
(Section II) is used here. If the output of model associated
with one of the classes is above the threshold then it
indicates that a given data point belongs to this class. It if
happens that two outputs are above the threshold – the
model indicates that the point should belong to both classes.
This result is seen as misclassification.
D. FNN and MOO: Pruning and Merging Processes
The Pareto-based optimization process generates a
number of models that form a Pareto surface. Such an
optimization process is repeated w ith different fitness
functions and more models are generated. A ll these models
are used in a process of knowledge – rules – extraction.
Before the rules extraction takes place, a pruning process of
models is performed. Its purpose is to reduce complexity of
the models by decreasing a number of input variables to
AND/OR neurons.
Model pruning is carried out by changing the thresholds
for OR and AND neurons. For an AND neuron, the input
variable has significant influe nce on the final output when
the weight for this variable to the neuron is close to 0. For
an OR neuron, the input has st ronger effect on the output if
the weight is closer to 1. So, for an AND neuron, the inputs
with weight values close to 1 will not contribute much to the
output. The same happens on an OR neuron if the weights
values are close to 0. If we change the threshold values to
close to1 for AND neurons, and close to 0 for OR neurons,
we could find inputs that are not very useful. By changing
threshold values, we could cut off some non-significant
inputs and simply model, possibly with little change to the
accuracy of the model. In our experiments, we change the
threshold for AND and OR neur ons separately, while
ensuring the output accuracy will not drop more than 5%.
After we trim models for each class, we extract rules,
select some of them, and combine the selected rules together
into a single m odel for classification of all classes. In the
selection process, we compare outputs of all rule sets
(models 1 ) constructed for each class . The rule set that
provides the best classificatio n rate for a single class is
considered as the part of the final model . The process is
performed for each class.
V. DATA DESCRIPTION
An experiment has been perform ed to generate software
data required for illustration of the proposed approach to
construct FNN models. In the experiment, objects of the
system EvIdent [12] have been independently analyzed by
three software architects and r anked according to their
quality attributes: complexity, maintainability and usability.
Quantitative software measures of these objects have been
compiled.
A. Software System Description
EvIdent is a user-friendly, algorithm -rich, graphical
environment for the detection, investigation, and
visualization of novelty and/or change in a set of images as
they evolve in tim e or frequency. It is w ritten in Java and
C++ and based on VIStA, an application-programming
interface (API) developed at the National Resea rch Council.
The VIStA API is written in Java and offers a generalized
data model, an extensible al gorithm framework, and a suite
of graphical user interface constructs.
B. Data Set
Only Java-based EvIdent/VIStA objects have been used
here. For each of the 366 software objects, three
programmers, named ‘A’, ‘D’ and ‘V’, w ere asked to
independently rank objects’ maintainability, complexity and
usability from 1 (very poor) to 5 (very good). At the sam e
time, a set of 64 softw are metrics was calculated for each
object. As the result, the collected data set consists of 366
data points represented by a set of 64 software metrics and
the values assigned to each point by three programmers.
For the purpose of the experiments presented in the
paper, we have combined rankings (objects) 1 and 2 into the
class1, have renamed rank (objects) 3 into the class2, and
have combined rankings (objects) 3 and 4 into the class3.
Despite this, the objects are very unevenly distributed
among the three classes. All three programmers have
identified most of the objects as belonging to the class3.
Using the “standard” approach to construct m odels – best
overall classification rate – the m odels would “concentrate”
on the class3 ignoring the class1 and class2. However, in
the case of software engineer ing applications the most
important are class1 and class2, and rules generated for
them. Objects of these classes need to recognized and better
understood.
1 In the case of an FNN the terms “rule set” and “model” are exchangeable.
The FNN is de facto a set of rules.
109