SEKE 2012 Proceedings - Knowledge Systems Institute

aggregations is captured by the interm ediate values of the
connections of the OR neuron in the output layer.
We use the OR/AND neuron as the fundamental part of
the fuzzy neural network model. However, we add
connections from OR and AND neur ons from the first layer
of OR/AND neurons to OR neurons of the second layer of
the OR/AND neurons. As the result we combi ne several
OR/AND neurons to form a uni ted model. In this model,
each neuron in the first layer of each OR/ AND neuron is
connected to all OR neurons of the second layer.
A single OR/AND neuron represents one out put of a
modeled system. The final output of the model is
determined via comparing the outputs of all OR/AND
neurons. In the case of classification – this comparison
identifies the “w inning” class. The FNN structure dealing
with 3 outputs – classes – is shown in Figure 2.
There are tw o ways to im plement the class selection
function. One is using Maximum Output, and another uses
Threshold Value. For the Maximum Output we compare the
output values of all OR neurons. The one with the maximum
value is selected, and the output of the model is generated
according to the index of this output. For the Threshold
Value, a threshold value t is defined. The output values of
each OR neuron is compared with t. If there is one and only
one output with its value greater than or equal to t, the
output of the model is the class represented by this OR
neuron. Otherwise, the output of the model is not
determined.
Output for class 1 (O1)
Output for class N (On)
Class
selection
Figure 2. General structure of FNN
Final output: class #
The most widely used algorithm to construct a FNN is
the back propagation [3]. The objective used during that
process is focused on minimization of the sum of squared
errors between predicted and original outputs. The main
advantage of this approach is its simplicity.
An interesting and important approach to construct
FNNs has emerged with the introduction of evolutionary
computing [4, 5]. Different evolutionary algorithms are
applied for optimization-based learning of FNNs [6, 7].
They allow us to focus not only on param etric but also on
structural optimization of the FNNs [8].
III. MULTI-OBJECTIVE OPTIMIZATION
A multi-objective optimization problem is an
optimization problem involving several criteria or model
design objectives. M any real-world problem s involve
simultaneous optimization of several incommensurable and
often competing requirements. While in a single-objective
optimization the optimal solution is usually clearly defined,
this does not hold for multi -objective optimization
problems. If the objectives are opposing, then the problem is
to find the best possible design or model that satisfies
multiple objectives. In this case, instead of a single solution
a set of alternative solutions is obtained.
Given a set of solutions to the problem , their partial
ordering can be determined using the principle of
dominance: a solution is clearly better than (dom inating)
another solution, if it is better or equal in all objectives, but
at least better in one objective. U sing this principle, the set
of best solutions is found by removing all solutions that are
dominated by at least one other solution. This set of
indifferent solutions is referred to as a Pareto set.
Generally speaking, solving m ulti-objective problems is
difficult. In an attem pt to solve these problems in an
acceptable timeframe, specific multi-objective evolutionary
algorithms [9, 10, 11] have been developed.
IV.
FNN CONSTRUCTION
A. Construction of Models with Skewed Data
The evolutionary-based optimization offers a great
flexibility in exploiting different objective functions. T his
allows for building FN Ns that satisfy very com plex
objective functions addressing different features of the
network, as well as dealing w ith imbalanced data sets. T he
traditional techniques for constructing networks lead to the
development of models that ar e “good” for biggest classes,
but “ignore” smallest classes.
B. FNN and MOO: Concept
In order to construct models based on data sets with
uneven distribution of classes, as well as to extract rules
identifying relationships among data attributes representing
these classes, a novel approach for model development is
proposed.
In a nutshell, the m ain idea of the approach is to replace
construction of a single FNN with development of a number
of FNNs where each of them takes care of a single class.
This process leads to finding the best m odel for each class.
These single-class models are used to extract if-then rules
describing the relationships among data attributes for each
class. These rules are then combined and pruned to create a
model that gives good classification ra te and “treats all
classes in an uniform way”.
The development of FNNs for different classes is done
simultaneously using an evolutionary -based technique –
Pareto multi-objective evolutionary strategy. This multiobjective
optimization targets all class at the sam e time.
This leads to developm ent of a num ber of different m odels.
A subset of these m odels constitutes a Pareto surface. Each
model that belongs to this subset is non -dominated – it
means that none of the models from the whole set has better
performance across all classes.
An additional advantage of application of an
evolutionary-based optimization method is flexibility in
selection of objectives that control a construct ion process of
FNNs. In such case, the proposed approach allows for
107