Fuzzy Modelling - COST Action IC0702

Fuzzy Modelling: 

Fundamentals, Design, and 

Challenges 

IFSA 2009 

Lisbon, July 20, 2009

Roadmap 

Overview 

Motivation 

Fundamental quests 

Type-2 fuzzy 

models and 

interpretability 

General architecture 

and functional modules 

General architecture 

Functional modules 

Fuzzy models and fuzzy modeling 

Graph-oriented 

fuzzy models 

Verification 

& Validation 

(V & V) 

Multimodal and 

collaborative fuzzy 

models 

Direction-based 

fuzzy models 

Design of 

Information granules 

Incremental 

fuzzy models 

Linguistic 

models

Fuzzy models – historical perspective 

Fuzzy models 

Neurofuzzy models 

Hybrid fuzzy models 

FUZZY SETS 

COMPUTATIONAL INTELLIGENCE

Fuzzy Models: some 

statistics 

query : fuzzy model and neurofuzzy 

35000 

30000 

25000 

20000 

15000 

10000 

5000 

0 

1 3 5 7 9 11 13 15 17 19 21 23 25 27 

1970 1980 

2009 

Google Scholar, April 2009

Fuzzy modeling: an overview 

Plethora of methodologies and architectures of fuzzy models 

Hybrid design strategies (fuzzy, neurofuzzy, evolutionary 

techniques) 

Predominantly numeric nature of results (fuzzy models with numeric 

decoding modules) 

Multiobjective nature of fuzzy models with several conflicting criteria 

•Accuracy 

•Interpretability 

•stability

Fuzzy modeling: 

Fundamental quests 

Dominant role of designer (user) in system modeling 

Sound design practices which help the designer assume 

active role throughout whole development process 

Models easily adjustable to current requirements imposed by 

the problem; effective realization of tradeoffs between 

accuracy and interpretability 

Successive refinements of models 

Controlled design effort (avoidance of excessively long learning, …)

Fuzzy models with 

Information granules: a retrospective 

Information granules are formed in multivariable input space. 

With each of them comes some local model associating 

inputs with the output 

curse of dimensionality 

Examples - Rule-based systems 

If cond 1 

is A 1 

and cond 2 

is A 2 

… then conclusion is B 

If cond 1 

is A 1 

and cond 2 

is A 2 

… then conclusion is f(x, a) 

If (cond 1 

, cond 2 

…cond 3 

) is R then conclusion is B 

Relational constraint

Example 

If (cond 1 

, cond 2 

…cond n 

) is R then conclusion is B 

If (cond 1 

, cond 2 

…cond n 

) is R then conclusion is f(x,a) 


aggregation

Fuzzy sets and interfaces 

Fuzzy sets (and sets) do not exist in real-world 

To interact with the world one has to construct interfaces 

(encoders and decoders) 

Encoder 

Fuzzy set-based 

processing 

Decoder 

Interfacing

Digital processing: an analogy 

D/A 

Digital 

Processing 

A/D

Functional modules of interfaces 

Encoders The objective is to translate input data into some internal format 

acceptable for processing at level of fuzzy sets 

Decoders The objective is to convert the results of processing of fuzzy sets into 

some format acceptable by the external world (typically in the form of some 

numeric quantities) 

For encoding and decoding we engage a collection of fuzzy sets – information 

granules

Encoding 

Given is a collection of fuzzy sets A 1 

, A 2 

, …, A c 

; express 

some numeric input x in R in terms of these fuzzy sets 

x [ A 1 

(x) A 2 

(x)… A c 

(x)] 

Nonlinear mapping from R to c-dimensional unit hypercube

Decoding 

(a) decoding completed on a basis of a single fuzzy set 

(b) Decoding realized on a basis of a certain finite family of fuzzy sets and 

levels of their activation.

Decoding process: a single fuzzy set 

Single fuzzy set B develop a single numeric representative

Single fuzzy set decoding: centre of gravity 

Solution to the following optimization problem 

V 

= 

∫ 

X 

B(x)[x 

− 

xˆ] 2 dx 

dV = 0 

2 

dxˆ 

∫ B(x)[x − xˆ]dx = 

X 

0

Single fuzzy set decoding: augmented strategies 

Augmented centre of gravity 

xˆ 

= 

∫ 

x∈X:B(x) 

≥β 

∫ 

x∈X:B(x) 

≥β 

B(x)xdx 

B(x)dx 

xˆ 

= 

∫ 

x∈X:B(x) 

≥β 

∫ 

B 

γ 

B 

γ 

x∈X:B(x) 

≥β 

(x)xdx 

(x)dx

Decoding: a collection of fuzzy sets 

x 

xˆ 

ENCODER 

DECODER 

Numeric 

Input 

(multidimensional) 

Granular 

representation 

Numeric 

Output 

(multidimensional) 

•One-dimensional case 

•Multivariable case- to be studied later

Decoding: one-dimensional (scalar) case 

Codeboook – a finite family of fuzzy sets {A 1 

, A 2 

, …, A c 

} 

A 1 A 2 A i A i+1 

1/2 

v i v i+1 

x

Design of Information Granules 

Development of multivariable fuzzy sets (information granules) 

R 1 

, R 2 

, …, R c 

Fuzzy clustering as a constructive vehicle of forming 

information granules

Fuzzy Clustering: Fuzzy C-Means (FCM) 

Given data x 1 

, x 2 

, …, x N 

, determine its structure by 

forming a collection of information granules – fuzzy sets 

Objective function 

Q 

= 

c 

∑ 

i= 

1 

N 

∑ 

k= 

1 

u 

m 

ik 

|| 

x 

k 

− 

v 

i 

|| 

2

FCM – representation fundamentals 

c 

∑ 

i=1 

0 < 

u ik 

=1, k =1,2,..., N 

N 

∑ uik < N, i = 

k= 

1 

1,2,..., c

FCM – optimization 

Q 

= 

c 

∑ 

i= 

1 

N 

∑ 

k= 

1 

u 

m 

ik 

|| 

x 

k 

− 

v 

i 

|| 

2 

Minimize 

subject to 

(a) prototypes 

(b) partition matrix

Optimization - details 

Partition matrix – the use of Lagrange multipliers 

V = 

c 

∑ 

m 

u ik 

d ik 

i=1 

c 

∑ 

i=1 

2 + λ( u ik 

−1) 

d ik = ||x k -v i || 2 

λ –Lagrange multiplier 

∂V 

= 0 ∂V 

∂u st 

∂λ = 0

Optimization – partition matrix (1) 

∑ 

∑ 

= 

= 

− 

+ 

= 

c 

1 

i 

ik 

2 

ik 

c 

1 

i 

m 

ik 1) 

u 

λ( 

d 

u 

V 

0 

λ 

V 

0 

u 

V 

st 

= 

∂ 

∂ 

= 

∂ 

∂ 

λ 

d 

mu 

u 

V 

2 st 

1 

m 

st 

st 

+ 

= 

∂ 

∂ 

− 

d 

m 

λ 

u 1 

m- 

2 

st 

m-1 

1 

st 

⎟ ⎠ ⎞ 

⎜ 

⎝ 

⎛ 

− 

= ∑ = 

⎟ 

⎠ 

⎞ 

⎜ 

⎝ 

⎛ 

− 

= 

− 

− c 1 

j 

1 

m 

2 

jt 

1 

m 

1 

1 

d 

m 

λ 

∑ 

= 

⎟ 

⎠ 

⎞ 

⎜ 

⎝ 

⎛ 

− 

= 

− 

− 

c 

1 

j 

1 

m 

2 

jt 

1 

m 

1 

d 

1 

m 

λ 

∑ 

= 

− 

⎟ 

⎟ 

⎠ 

⎞ 

⎜ 

⎜ 

⎝ 

⎛ 

= 

c 

1 

j 

1 

m 

1 

2 

jt 

2 

st 

st 

d 

d 

1 

u

Optimization- prototypes (2) 

Q 

= 

c 

∑ 

i= 

1 

N 

∑ 

k= 

1 

u 

m 

ik 

n 

∑ (x 

j= 

1 

kj 

− 

v 

ij 

) 

2 

Euclidean distance 

Gradient of Q with respect to v s ∑ u (x − v ) = 

N 

k= 

1 

m 

ik 

kt 

st 

0 

v 

st 

= 

N 

∑ 

u 

k= 

1 

N 

∑ 

k= 

1 

m 

ik 

u 

x 

m 

ik 

kt

Fuzzy C-Means (FCM): An overview 

procedure FCM-CLUSTERING (x) returns prototypes and partition matrix 

input : data x = {x 1, x 2, ..., x k} 

local: fuzzification parameter: m 

threshold: ε 

norm: ||.|| 

INITIALIZE-PARTITION-MATRIX 

t ← 0 

repeat 

for i=1:c do 

N 

m 

∑ u 

ik 

(t) xk 

v ← 

k = 1 

i (t) 

compute prototypes 

N 

m 

∑ uik 

(t) 

k = 1 

for i = 1:c do 

for k = 1:N do 

update partition matrix 

1 

uik 

(t + 1) = 

2/(m−1) 

c ⎛ 

⎞ 

∑ ⎜ 

|| xk 

− vi 

(t) || 

⎟ 

j= 

1 

⎜ 

⎟ 

⎝ 

|| xk 

− v j(t) 

|| 

⎠ 

update partition matrix 

t ← t + 1 

until ||U(t+1)-U(t)|| ≤ ε 

return U, V

Geometry of information granules 

n=1 

1 

1 

1 

1 

1 

1 

A( x, 

1.2) 

A( x, 

2) 

A( x, 

3.5) 

B( x, 

1.2) 

0.5 

B( x, 

2) 

0.5 

B( x, 

3.5) 

0.5 

C( x, 

1.2) 

C( x, 

2) 

C( x, 

3.5) 

0 

0 

1 2 3 

0.5 x 

3.5 

2.265× 

10 − 7 

0 

1 2 3 

0.5 x 

3.5 

2.18× 

10 − 3 

0 

1 2 3 

0.5 x 

3.5 

m =1.2 m =2.0 m =3.5

Fuzzy Clustering: choosing granularity of 

information granules 

Cluster validity measures… 

Reconstruction criterion 

Given original numeric datum x k 

, express it in terms of 

clusters (information granules) and re-construct it. 

The reconstruction error is a measure expressing 

differences between original datum and its reconstruction 

V 

N 

= ∑|| 

k= 

1 

x − 

ˆ 

2 

k 

xk 

||

Multivariable encoding and decoding: a global 

view 

Encoding 

i 0 

Decoding 

{v 1 , v 2 , …, v c } 

VQ 

use of sets – 

Vector Quantization 

(VQ) 

Prototypes 

v 1 , v 2 , …, v c 

Prototypes 

v 1 , v 2 , …, v c 

Encoding 

u 1 , u 2 , …,u c-1 

Decoding 

FVQ 

use of fuzzy sets – 

Fuzzy Vector Quantization 

(FVQ) 

Prototypes 

v 1 , v 2 , …, v c 

Prototypes 

v 1 , v 2 , …, v c

Fuzzy Vector Quantization 

The codebook formed through fuzzy clustering (FCM) producing 

A finite collection of prototypes v 1 

, v 2 

, …, v c 

. 

Given any new input x we realize its encoding and decoding 

Let us recall: 

Encoding – representation of x in terms of the prototypes 

Decoding – development of external representation of the result of 

processing realized at the level of information granules

u 

i 

( x) 

∈[0,1], 

∑ u 

i 

(x) = 1 

c 

i= 

1 

Fuzzy Vector Quantization: 

Encoding 

The optimization problem 

c 

∑ u 

i= 

1 

m 

2 

i 

|| x − vi 

|| 

Minimize w.r.t. u i 

subject to 

u 

i 

( x) 

∈[0,1], 

∑ u 

i 

(x) = 1 

c 

i= 

1 

u 

i 

( x) 

= 

⎛ 

∑⎜ 

|| 

⎝ || 

1 

x − v 

x − v 

i 

j 

|| ⎞ 

⎟ 

|| 

⎠ 

2 

m−1

u 

i 

( x) 

∈[0,1], 

∑ u 

i 

(x) = 1 

c 

i= 

1 


Decoding 

Reconstruct original mutidimensional input x 

c 

m 

2 

2 

(ˆ) x = u ˆ 

i 

|| x v 

i 

|| 

i= 

1 

Q ∑ − 

minimize 

xˆ 

= 

c 

∑ u 

i= 

1 

c 

∑ u 

i= 

1 

m 

i 

v 

m 

i 

i

u 

i 

( x) 

∈[0,1], 

∑ u 

i 

(x) = 1 

c 

i= 

1 


Decoding error

Example – computing 

Input-output relationship 

z 1 

z i 

z c 

y 

= 

c 

∑ 

i= 

1 

z 

i 

u 

i 

( x) 

u ( x) 

= 

i 

c 

∑ 

j= 

1 

⎛ 

⎜ 

|| x 

⎝ || x 

1 

− 

− 

v 

v 

i 

j 

|| ⎞ 

⎟ 

|| 

⎠ 

2 

m−1

Examples (1) 

y 

5 

m=1.2 

m=4.0 

v 1 

= -1, v 2 

= 2.5 v 3 

= 6.1; z 1 

= 6, z 2 

= -4, z 3 

= 2 

0 

m=2.0 

5 

− 5 

4 2 0 2 4 

x 

x 

5 

y 

5 

y 

5 

Change of prototypes 

in input space 

0 

0 

5 

0 2 4 6 

− 0 

x 

x 

7 

5 

0 2 4 6 

x

Fuzzy Models: A General Taxonomy 

Source of data/knowledge 

single source 

Multiple sources 

Direction –based 

architectures 

Graph–oriented 

architectures

The architectural blueprint 

of 

fuzzy models

Preamble 

As modeling is realized at higher, more abstract level, fuzzy 

models give rise to a general architecture in which we 

highlight three main functional modules, that is 

– input interface 

– processing module 

– output interface

A general architecture 

Fuzzy model 

Domain 

knowledge 

Processing 

Interface 

Interface 

Data 

Decision, control signal, 

class assignment…

Main categories of fuzzy 

models: An overview

Main categories of models: 

An overview 

Diversified landscape of fuzzy models - selected categories: 

– tabular fuzzy models 

– rule-based fuzzy models 

– fuzzy relational models including associative memories 

– fuzzy decision trees 

– fuzzy neural networks 

– fuzzy cognitive maps 

– ….

Some design considerations 

• Expressive power 

• Processing capabilities 

• Design schemes and ensuing optimization 

• Interpretability 

• Ability to deal with heterogeneous data 

• ….

Tabular fuzzy models 

• Table of relationships between the variables of the system 

granulated by some fuzzy sets. 

• Easy to build and interpret 

• Limited processing capabilities (not included as a part of the model) 

B 1 B 2 B 3 B 4 B 5 

A 1 

A 2 

C 3 

A 3 

C 1

Rule-based fuzzy models 

• Highly modular and easily expandable fuzzy models 

• Composed of a family of conditional (If – then) statements (rules) 

• Fuzzy sets occur in their conditions and conclusions 

• Standard format 

If condition 1 

is A and condition 2 

is B and … and 

condition n 

is W 

then conclusion is Z 

• Conditions ≡ rule antecedent 

• Conclusions ≡ rule consequent

Rule-based fuzzy models: 

granularity and quality of rules 

Low High 

granularity of conclusion 

general condition 

(highly applicable rule) 

and very specific 

conclusion. High quality 

rule 

high generality of the 

rule, low specificity of the 

conclusion, average 

quality of the rule 

Low High 

granularity of condition 

condition and conclusion 

highly specific; lack of 

generalization; very 

limited relevance of the 

rule 

limited generality 

(specific condition) and 

lack of specificity of 

conclusion; low quality 

rule

Fuzzy rule-based model: design 

If x is B i then y = f i (x, p i ), i=1, 2, …,c 

Determination of condition parts (information granules) of the rules: 

Fuzzy clustering (e.g., Fuzzy C-Means, FCM) commonly 

Used to build information granules B i 

Determination of conclusion parts of the rules: 

Estimation of parameters (p i ) –optimization problem; 

global minimum of the problem could be achieved

Rule-based fuzzy models 

fuzzy rule-based models with linear local models 

-if x is B i then y = a iT x 

Output as an aggregation of local models and activation 

levels 

ŷ 

c 

k = ∑ ui 

( xk 

) 

i= 

1 

a 

T 

i 

x 

k


development (1) 

2 

N 

1 

k 

k 

k ) 

ŷ 

Q ∑ (y 

= 

− 

= 

k 

T 

i 

k 

c 

1 

i 

i 

k ) 

( 

u 

ŷ 

x 

a 

x 

∑ 

= 

= 

∑ 

= 

− 

⎟ 

⎟ 

⎠ 

⎞ 

⎜ 

⎜ 

⎝ 

⎛ 

− 

− 

= 

c 

1 

j 

1 

m 

2 

j 

k 

i 

k 

ik 

|| 

|| 

|| 

|| 

1 

u 

v 

x 

v 

x 

⎥ 

⎥ 

⎥ 

⎥ 

⎦ 

⎤ 

⎢ 

⎢ 

⎢ 

⎢ 

⎣ 

⎡ 

= 

ck 

2k 

1k 

T 

c 

T 

2 

T 

1 

k ] 

... 

[ 

ŷ 

z 

z 

z 

a 

a 

a 

⎥ 

⎥ 

⎥ 

⎥ 

⎦ 

⎤ 

⎢ 

⎢ 

⎢ 

⎢ 

⎣ 

⎡ 

= 

c 

2 

1 

a 

a 

a 

a 

k 

k 

i 

ik ) 

( 

u 

x 

x 

z =


development (2) 

⎥ 

⎥ 

⎥ 

⎥ 

⎦ 

⎤ 

⎢ 

⎢ 

⎢ 

⎢ 

⎣ 

⎡ 

= 

N 

2 

1 

y 

y 

y 

y 

) 

(Z 

) 

(Z 

Q 

T 

y 

a 

y 

a 

− 

− 

= 

y 

a 

T 

1 

T 

opt 

Z 

Z) 

(Z 

− 

= 

⎥ 

⎥ 

⎥ 

⎥ 

⎦ 

⎤ 

⎢ 

⎢ 

⎢ 

⎢ 

⎣ 

⎡ 

= 

cN 

c2 

c1 

1N 

12 

11 

Z 

z 

z 

z 

z 

z 

z

Fuzzy relational structures: 

A general taxonomy 

t-conorms 

nullnorms 

infimum (min) 

uninorms 

min-uninorm composition 

t-norms 

inf-s composition 

ordinal sum 

max-min composition 

supremum (max) 

sup-min composition 

implications 

sup-t composition

Fuzzy decision trees 

• Generalization of decision trees 

A={a 1 , a 2 , a 3 } 

∈ 

B={b 1 , b 2 } C={c 1 , c 2 , c 3 , c 4 } 

≥ 

a 3 , c 1 

• Traversal of tree depending on the values of the attributes: 

only a single path traversed and a single terminal node 

reached


• Traversal of a number of paths leading to a number of 

terminal nodes (reachability levels) 

A = {A 1 , A 2 , A 3 } 

B = {B 1 , B 2 } C = {C 1 , C 2 , C 3 , C 4 } 

µ 1 µ 2 µ 3 µ 4 µ 5 µ 6 reachability


• Traversal of a number of paths leading to a number of 

terminal nodes (reachability levels) 

x 

A = {A 1 , A 2 , A 3 } 

C = {C 1 , C 2 , C 3 , C 4 } 

y 

µ = A 1 (x) t C 2 (y) reachability

Fuzzy neural networks 

• Architectures in which we combine adaptive properties of 

neural networks with interpretability (transparency) of 

fuzzy sets 

• A suite of fuzzy logic neurons: 

– aggregative neurons (and, or neurons) 

– referential neurons (dominance, equality, inclusion…) 

• Learning mechanisms could be applied to adjustment 

of connections of neurons 

• Each neuron comes with a well-defined semantics; the 

network could be easily interpreted once the training has 

been completed

Fuzzy neural networks: 

Examples of architectures 

• Use of and and or neurons (logic processor) 

and 

or

Fuzzy neural networks: 

Example of architectures 

• Use of and, or and referential (ref) neurons 

ref 

and 

or

Fuzzy cognitive maps 

• Representation of concepts and linkages between concepts 

• Directed graph: concepts are nodes; linkages are edges 

A 

+ 

+ 

- 

- 

- 

C 

D 

- 

B 

• A, B, C, and D = concepts. 

• Inhibition (-) or excitation (+) between the concepts (nodes)

Verification and validation 

of fuzzy models

Verification and validation 

of fuzzy models 

• Verification and Validation (V&V) are concerned with the 

development of the model and assessment of its usefulness 

• Verification is concerned with the analysis of the underlying 

processes of constructing the fuzzy model do we follow sound 

design principles ? 

“Are we building the product right?” 

• Validation is concerned with ensuring that the model (product) 

meets the requirements of the customer 

“Are we building the right product?”

Verification of fuzzy models 

• Sound design principles 

– iterative development process 

– assessment of accuracy 

– generalization capabilities 

– complexity of the model (Occam’s principle) 

– high level of autonomy of the model

Fuzzy models: accuracy 

• Two ways of expressing accuracy: 

– numeric level 

– internal level (information granules)


• Numeric level of expressing accuracy 

Interface 

Interface 

x k 

y k 

target k 

Processing 

Minimized 

error


• Accuracy expressed at the level of fuzzy sets 

Interface 

Interface 

x k 

Processing 

t k 

target k 

u k 

Minimized 

error

Training, validation, and 

testing data 

• To avoid potential bias in assessment of accuracy, data are split into 

– training 

– validation 

– testing subsets 

• Training - testing 

– typically 60-40% split 

– 10 fold cross-validation (90-10% split) 

– leave one out strategy

Validation of fuzzy 

models 

• Are we building the right model? 

• More difficult to quantify: 

Interpretability 

– transparency of fuzzy models 

– stability of the fuzzy model 

…. 

• Very often validation criteria are in conflict 

•Curse of dimensionality versus 

transparency 

Accuracy

Spiral scheme of model development 

* incremental design, implementation and testing 

* multidimensional space of fundamental characteristics 

Stability 

Accuracy 

Interpretability 

Knowledge 

Representation

Graph-oriented fuzzy models



single source 



architectures 


architectures

Fuzzy cognitive maps 

• Representation of concepts and linkages between concepts 

• Directed graph: concepts are represented as nodes; 

linkages (associations) are edges 

A 

+ 

+ 

- 

- 

- 

C 

D 

- 

B 

• A, B, C, and D = concepts. 

• Inhibition (-) or excitation (+) between the concepts (nodes)

Architecture and computing 

node-i 

w ij 

A i 

(k+1)= f( 

w ik 

node-k 

node-j 

Activation of node 

c 

∑ 

A i (k+1)= f( w A (k) + w ) 

j= 

1 

ij 

j 

0i

Architecture and computing: 

more computing 

Activation of node – nonlinearity with adjustable steepness 

Higher-order dynamics: A i (k+1)= F(k, k-1, k-2)…etc

Design: key issues 

Nodes – result of abstraction, a collection of 

interacting concepts, representation of knowledge 

Strength (intensity) of interaction – 

+1 , -1 cognitive maps 

[-1, 1] fuzzy cognitive maps




INFORMATION GRANULES AND INFORMATION GRANULATION 

amplitude =high & 

change of amplitude=low 

A 1 , A 2 ,…, A c 

5 

4 

3 

2 

1 

0 

-5 -3 -1 

-1 

1 3 5 

-2 

-3 

-4 

-5 

5 

4 

3 

2 

1 

0 

-1 

0 50 100 150 200 250 300 350 400 

-2 

-3 

-4 

-5




Strength (intensity) of interaction – 

+1 , -1 cognitive maps 

[-1, 1] fuzzy cognitive maps 

Learning – parametric optimization

Learning of connections 

V = 

N c 

c 

1 

∑ ∑|| 

B (k + 1) − f ( ∑ w 

(N −1)c 

k= 

1 

i 

i 

i= 1 

j= 

1 

ij 

A 

i 

(k) + 

w 

0i 

,σ 

i 

) || 

2 

Parametric optimization 

Fuzzy clustering 

5 

4 

3 

2 

1 

0 

-5 -3 -1 

-1 

1 3 5 

-2 

-3 

-4 

-5

Example (1) 

5 

4 

3 

2 

1 

0 

-1 

0 50 100 150 200 250 300 350 400 

-2 

-3 

-4 

-5 

5 

4 

3 

2 

1 

0 

-1 

0 50 100 150 200 250 300 350 400 

-2 

-3 

-4 

-5 

5 

4 

3 

2 

1 

0 

-5 -3 -1 

-1 

1 3 5 

-2 

-3 

-4 

-5

Example (2) 

(-L, -L) 

(-S, -M) 

(L, L) 

(L, S) 

(-S , S) 

5 

4 

∆x k 

3 

2 

1 

0 

-5 -3 -1 

-1 

1 3 

x k 

5 

-2 

-3 

-4 

-5

Fuzzy cognitive maps: 

extensions 

A 

C 

and 

or 

D 

B 

E

C 

B 

D3 

Fuzzy cognitive maps: 

hierarchy of concepts 

A 

D 

or 

Level of information granularity 

D1 

D2

Linguistic models: 

from data to granular architectures

Linguistic (granular) modeling: 

Main objectives 

Direct and constructive usage of information granules 

Active role of designer in the formation of focus of the linguistic 

model 

High interpretability 

Reduced learning curve – rapid prototyping with possibilities of 

further direct refinements

Clustering and revealed structure vis-àvis 

modeling agenda 

Clustering (fuzzy clustering) is a direction-free mechanism of 

development of information granules, viz. there is no distinction 

between independent (input) and dependent (output) objects to 

be clustered 

One could cluster (group) 

(a) Input data x 1 , x 2 , … x N 

(b) Concatenated data of input and output [x 1 y 1 ], [x 2 y 2 ],…, 

[x N y N ]

Clustering and revealed structure vis-àvis 

modeling agenda 

Existing drawbacks: 

(a) Structure in the input space – clusters could be heterogeneous 

in terms of output data 

(b) Dimensionality of input space is higher than the output space 

formation of proper distance function

Context-based clustering 

To align the agenda of fuzzy clustering with the principles of fuzzy 

modeling, the following features are considered: 

Active role of the designer [customization of the model] 

The structural backbone of the model is fully reflective of relationships 

between information granules in the input and output space 

Clustering : construct clusters in input space X 

Context-based Clustering : construct clusters in input space X 

given some context expressed in 

output space Y

Selected references 

W. Pedrycz, Conditional fuzzy C-Means, Pattern Recognition Letters, 17, 1996, 625- 

632. 

W. Pedrycz, Conditional fuzzy clustering in the design of radial basis function neural 

networks, IEEE Trans. on Neural Networks, 9, 1998, 601-612. 

W. Pedrycz, A Vasilakos, Linguistic models and linguistic modeling, IEEE Trans. on 

Systems, Man and Cybernetics, 29, 1999, 745-757. 

W. Pedrycz and K. Kwak, Granular models as a framework of user-centric system 

modeling, IEEE Trans. on Systems, Mans, and Cybernetics – Part A, 2006, 727-745. 

W. Pedrycz, K.C. Kwak, The development of incremental models, IEEE Trans. on 

Fuzzy Systems, 15, 3, 2007, 507-518 

W. Pedrycz, J. Valente de Oliveira, Development of fuzzy encoding and decoding 

through fuzzy clustering, IEEE Transactions on Instrumentation and Measurement, 

2008 

L. A. Zadeh, Fuzzy sets and information granularity, In: M. Gupta, R. Ragade, R. Yager 

(Eds.) Advances in Fuzzy Set Theory and Applications, Noth-Holland, Amsterdam, 

1979, 3-18.

Context-based clustering: 

Computing considerations 

structure 

structure 

context 

Data 

Data

Context-based clustering 

Context-based Clustering : construct clusters in input space X 

given some context expressed in 

output space Y 

Context – hint (piece of domain knowledge) 

provided by designer who actively impacts the 

development of the model


Context design 

Context – hint (piece of domain knowledge) 

provided by designer who actively impacts the 

development of the model. As such, context 

is imposed by the designer at the beginning 

Realization of context 

Designer focus information granule (fuzzy set) 

(a) Designer, and (b) clustering of scalar data in output space 

Context – fuzzy set (set) formed in the output space


Modeling 

Determine structure in input space 

given the output is high 


given the output is medium 


given the output is low 

Input space (data)


examples 

Find a structure of customer data [clustering] 

Find a structure of customer data considering 

customers making weekly purchases in the 

range [$1,000 $3,000] 

no context 

context 


customers making weekly purchases at the level of 

around $ 2,500 


customers making significant weekly purchases who 

are young 

context 

context 

(compound)

Context-oriented FCM 

Data (x k 

, target k 

), k=1,2,…,N 

Contexts: fuzzy sets W 1 

, W 2 

, …, W p 

w jk 

= W i 

(target k 

) membership of j-th context for k-th data 

Context-driven partition matrix 

U (W ) 

j 

c 

N 

⎧ 

= ⎨u 

ik 

∈[ 0,1] 

| ∑ u 

ik 

= w 

jk 

∀k 

and 0 < ∑ u 

⎩ 

i= 1 

k= 

1 

ik 

< 

N 

⎫ 

∀i⎬ 

⎭

Context-oriented FCM: 

Optimization flow 

Objective function 

Q 

c 

N 

= ∑∑ 

i= 1 k= 

1 

u 

m 

2 

ik 

|| x 

k 

− v 

i 

|| 

Subject to constraint U in U(W j 

) 

Iterative adjustment of partition matrix and prototypes 

u 

ik 

= 

c 

∑ 

j= 

1 

⎛ 

⎜ 

⎜ 

⎝ 

x 

x 

k 

k 

w 

jk 

− v 

− v 

i 

j 

⎞ 

⎟ 

⎟ 

⎠ 

2 

m−1 

v 

i 

N 

∑ 

k= 

1 

= 

N 

∑ 

k= 

1 

u x 

m 

ik 

u 

m 

ik 

k

From context-based FCM to 

a web of information granules 

context 

fuzzy set 

Input space

From context-based FCM to 

a web of information granules 

T 

1 

T 

2 

T 

3 

context 

fuzzy sets 

Input space

Contexts and their design on 

a basis of experimental evidence 

Selection of triangular fuzzy sets – minimization of reconstruction 

error 

W i W i+1 

m i m i+1 

y 0 

y

From neurons to 

granular neurons 

Neurons in neural networks are nonlinear mappings from R n to R 

(or [0,1]) 

Main properties 

•Connections are numeric 

•Inputs are numeric 

•Output is numeric 

Granular neurons: 

•Connections are information granules (intervals, fuzzy sets…) 

•Inputs are information granules (intervals, fuzzy sets…)

Granular neurons 

u 1 

∑ 

Y= N(u 1 

, u 2 

, ..,u c 

, A 1 

, A 2 

, .., A c 

) = (Ai ⊗ ui) 

A 1 

c 

i= 

1⊕ 

u 2 

Σ 

Y 

Algebraic operations on 

information granules 

u c 

A c


computing (1) 

Interval connections A i 

= [a i- 

, a i+ 

] 

u i 

- positive inputs 

A ⊗ i 

u i 

= [a i- 

u i 

, a i+ 

u i 

]


computing (2) 

Z i 

- fuzzy number 

Y(y) = sup {min(Z 1 

(y 1 

), Z 2 

(y 2 

), …, Z n 

(y n 

))} 

subject to 

y = y 1 

+ y 2 

+…+y n


characteristics 

u1=a u 2 

= 1-a A 1 

= [0.3 3] A 2 

= [1.4 7] 

α 

y


characteristics 

Connections represented as triangular fuzzy numbers 

A i 

= 

Y 

c 

=< ∑a 

∑ ∑ 

iui, 

miui, 

i= 

1 

c 

i= 

1 

c 

i= 

1 

b 

i 

u 

i 

> 

α 

x 

α 

x

Architecture of granular model 

Given a collection of context fuzzy sets W 1 

, W 2 

, …, W p 

(those are provided by designer; active model formation) 

Two-phase process: 

For each context, construct clusters in input space 

(context-based clustering) 

Arrange information granules into a web of associations 

of information granules formed in the input space and output 

space 

Rapid prototyping (modeling)

Architecture of granular model 

Σ 

Σ 

Σ 

Y 

x 

Σ 

Context-based 

clusters 

Contexts

Model evaluation: 

Performance analysis 

Y 

Y= 

y - 

y target y + 

Numeric quantification of performance 

Granular quantification of performance


Numeric quantification 

Y 

Y= 

y target 

y - 

y + 

N 

k= 

1 

2 

k k 

) 

∑(y − target


Granular quantification 

Y 

Y= 

y target 

y - 

y + 

N 

∑ 

k= 

1 

(1− 

Y(targetk 

))

Enhancements of granular 

models 

Introduction of bias term to granular neuron 

Optimization of contexts – focus of the granular model


models 

Introduction of bias term to granular neuron 

Σ 

Σ 

Σ 

Y 

target k 

W 1 

W 0 

x 

W 2 

+ y k 

y − 

y 

+ 

Context-based 

clusters 

Σ 

Contexts 

M 

M 

W p 

Σ Y 

=< y 

− 

, y 

, 

y 

+ 

>


models - bias 

M 

W 1 

W 2 

M 

W p 

W 0 

target k 

+ y k 

y − 

y 

+ 

Σ Y 

=< y 

− 

, y 

, 

y 

+ 

> 

w 

1 

N 

0 

= − ∑ (target 

k 

− y 

k 

) 

N k= 

1 

lower bound 

p 

∑ 

t= 

1 

z 

t 

w 

+ t− w 

0 

modal value ∑ z 

tw 

p 

t= 

1 

t 

+ w 

0 

upper bound 

p 

∑ 

t= 

1 

z w 

t 

+ t+ w 

0


models – context optimization 

T 

Conditional 

clustering 

Context 

optimization

Enhancements of granular models – 

context optimization (2) 

T 

max 

P 

1 

N 

N 

∑ Y( x 

k= 

1 

k 

)(target 

k 

) 

P- parameters of contexts 

Conditional 

clustering 

Context 

optimization 

min P 

1 

N 

N 

∑ 

k= 

1 

(b 

k 

− a 

k 

)

Incremental granular models 

Adopting a construct of a linear regression as a first-principle global model, refine 

it through granular models that capture remaining and more localized 

nonlinearities of the system 

fuzzy model = linear regression & local granular models

Incremental granular models 

(a) 

(b) 

(c)

Incremental granular models: 

the concept 

DATA 

{x k , target k } 

Linear regression 

{x k , e k } 

Residuals 

Incremental 

Granular 

model 

Granular model

Contexts in incremental granular 

models 

Context space E 

Input space R n

Architecture of the incremental 

granular model 

x 

Linear 

Regression 

z 

Σ 

Y 

INCREMENTAL 

MODEL 

Σ 

Σ 

Σ 

E 

bias 

Σ 

Context-based 

clustering 

fuzzy numbers 

(granular information 

processed)

Topology of the model 

u 11 

u 1i 

u 1c 

Σ 

ξ 1 

M 

u t1 

M 

W 1 

x 

u ti 

u tc 

Σ 

ξ t 

W t 

Σ 

E 

M 

u p1 

M 

W p 

u pi 

u pc 

Σ 

ξ p 

Context-based 

centers 

Contexts 

E 

= 

W 

ξ 

ξ 

1 

⊗ 

1 

⊕ W2 

⊗ 

2 

⊕....Wn 

⊗ξ 

n

General design strategy 

A. Design of a linear regression in the input – output space, z = L(x; b) with b 

denoting a vector of the regression hyperplane, b =[a a 0 ] T . On the basis of 

the original data set formed is a collection of input-error pairs, (x k , e k ) where 

e k = target-L(x k ,a) 

B. Construction of the collection of contexts- fuzzy sets in the space of error of 

the regression model E 1 , E 2 , …,E p . Typically triangular fuzzy sets are 

considered 

C. Context-based FCM completed in the input space and induced by the 

individual fuzzy sets of context. 

D. Summation of the activation levels of the clusters induced by the 

corresponding contexts and their overall aggregation 

E. The granular result of the incremental model is affected by eventual bias

Example (1) 

“spiky” function 

spiky(x) 

= 

⎧max(x,G(x)) 

⎨ 

⎩min(x, 

−G(x) 

+ 

2) 

if 0 ≤ x ≤ 1 

if 1 < x ≤ 2 

G(x) 

⎛ − (x − c) 

exp 

⎜ 

⎝ 2σ 

= 

2 

2 

⎟ ⎞ 

⎠

Example (2) 

RMSE = 

1 

N 

N 

∑ 

k= 

1 

(yk − targetk 

) 

2 

p=c=6 

p=c=5 

p=c=4 

p=c=3

Example (3) 

c=p=5, m=2.2

Two-dimensional function

Two-dimensional function: 

results 

Linear model 

Error and distribution of 

prototypes

Two-dimensional function: 

Contexts and results 

1 

membership grades 

0.8 

0.6 

0.4 

LN MN NZ MP LP 

0.2 

0 

-80 -60 -40 -20 0 20 40 

e

Experimental data 

Machine Learning and MIS 

Number of 

contexts and 

clusters 

RMSE 

(Training data) 

RMSE 

(Test data) 

Automobile 

MPG 

data 

Boston 

Housing 

data 

CPU 

Performance 

data 

MIS data 

Linear regression 0.194 0.295 

Incremental model p=c=6 0.142 0.285 






Incremental model p=c=6 0.896 0.773

From multitude of perceptions to global 

architectures of fuzzy models 

Multimodality: 

different perspectives: 

granularity of information, 

variables (features) involved 

Different data

Fuzzy models: 

hierarchical and distributed perspective 

granularity/hierarchy 

set of features



single source 



architectures 


architectures

Modeling with for spatio-temporal 

data 

Model 

MODEL-1 MODEL-2 MODEL-3 

time 

Data-1 Data-2 Data-3 



time 



time

Two general development strategies 

SELECTION OF A “MEANINGFUL” SUBSET OF 

INFORMATION GRANULES


(1) HIERARCHICAL DEVELOPMENT OF INFORMATION 

GRANULES (INFORMMATION GRANULES OF HIGHER 

TYPE) 


Type -2 


Type -1



GRANULES AND THE USE OF VIEWPOINTS 


Type -2 

viewpoints 


Type -1



GRANULES – A MODE OF SUCCESSIVE CONSTRUCTION

Main design phases 

Φi[1] 

Ai[1] 

vi[1] 

F1 

Determination of information granules 

Fii 

data 

Φi[ii] 

Ai[ii] 

vi[ii] 

Φi[p] 

Ai[p] 

zi 

z1 

zc 

Construction of associated local 

models 

vi[p] 

Fp 

F

A global granular view: 

choosing representative information granules 

Φ i[1] 

A i[1] 

v i[1] 

F 1 

prototypes z 1 

, z 2 

, …, z c 

data 

Φ i[ii] 

A i[ii] 

F ii 

heterogeneous space 

v i[ii] 

Recall “ standard” expression: 

z i 

z c 

Φ i[p] 

A i[p] 

v i[p] 

F p 

z 1 

F 

A 

i 

( x)[ii] 

= 

c[ii] 

∑ 

j= 

1 

⎛ 

⎜ 

|| x − v 

⎝ || x − v 

1 

i 

[ii] || ⎞ 

⎟ 

[ii] || 

⎠ 

j 

2/(m −1) 

ii

Information granules and 

their representatives 

Represent v k 

[ii] with the use of z 1 

, z 2 

, …, z c 

u i 

(v k 

[ii]) = 

c 

∑ 

j=1 

1 

⎛ || v k 

[ii]− z i 

|| Fii ∩F 

⎜ 

⎝ || v k 

[ii]− z j 

|| Fii ∩F 

⎞ 

⎟ 

⎠ 

2/(m−1) 

z 1 

z 2 

v 1 [ii] 

z c 

F 

F ii

Representation of fuzzy sets: 

two performance measures 

Entropy measure 

Reconstruction criterion (error)

Expressing performance through 

entropy measure 

p 

∑ 

ii= 

1 

c 

∑ 

i= 

1 

c[ii] 

∑ 

k= 

1 

H(u 

i 

( v 

k 

[ii]))

Reconstruction error 

Q = 

p 

∑ 

ii= 

1 

c[ii] 

∑ 

k= 

1 

2 

|| v ˆ( v 

k[ii]) 

− v 

k[ii] || 

F 

ii 

where 

vˆ 

( v 

k 

[ii]) 

c 

∑ 

m 

m 

= u ( v [ii]) z vˆ 

( v [ii]) = u ( v [ii]) z / u ( v [ii]) 

i= 

1 

m 

i 

k 

i 

k 

c 

∑ 

i= 

1 

i 

k 

i 

c 

∑ 

i= 

1 

i 

k 

Requirement of “coverage” condition 

c 

U 

k= 

1 

F 

p 

U 

= F 

i k 

i= 

1 

i

Optimization problem 

Form a collection of prototypes Z = {z 1 

, z 2 

, …, z c 

} such that 

entropy (or reconstruction error) 

is minimized while satisfying coverage criterion 

c 

U 

k= 

1 

F 

p 

U 

= F 

i k 

i= 

1 

i 

Min Z 

Q subject to 

c 

U 

k= 

1 

F 

p 

U 

= F 

i k 

i= 

1 

i 

Optimization of fuzzification coefficient (m) 

Min Z 

Q subject to m>1 and 

c 

U 

k= 

1 

F 

p 

U 

= F 

i k 

i= 

1 

i

Design of local models 

Φi[1] 

Ai[1] 

vi[1] 

F1 

data 

Φi[ii] 

Ai[ii] 

vi[ii] 

Fii 

zi 

zc 

Φi[p] 

Ai[p] 

vi[p] 

Fp 

z1 

F

Knowledge sharing 

knowledge 

data 

Structure 

Model (e.g., fuzzy rule-based) 

Predictor 

Decision-making strategy 

….. 

Signature of knowledge 

phenomenon, process, system… 

Structure – clusters [prototypes] 

Model: condition and conclusion parts 

- if x is A i 

then y is f i 

(x, a i 

)

Knowledge sharing and 

collaboration 

knowledge 

data-2 

Structure 


Predictor 


….. 


knowledge 

data-1 

Structure 


Predictor 


….. 


knowledge 

Structure 


Predictor 


….. 

data-P 


phenomenon, process, system…

Knowledge sharing and 

collaboration 

knowledge 

data-2 

Structure 


Predictor 


….. 


knowledge 

data-1 

Structure 


Predictor 


….. 


knowledge 

Structure 


Predictor 


….. 

data-P 

Signature of knowledge

Collaborative 

structure development (1) 

Information 

granules 

data-1 

data-2 

data-P 


Collaborative 

structure development (2) 

Information 

granules of 

higher type 

Information 

granules 

data-1 

data-2 

data-P 


Collaborative clustering 

Information 

Information 

granules Information 

granules 

granules 

data-1 data-1 data-1 

data-2 data-2 data-2 

phenomenon, phenomenon, process, process, system… 

phenomenon, process, system… system… 

data-P data-P data-P 

Discover a structure in a collaborative fashion by communicating 

findings produced at the level of local data sites. 

Exchange of findings in the form of information granules 

constructed for each data site 

Further usage of such findings in refining and directing (navigating) 

search in local data

Collaborative structure determination: 

Information granules of higher order 

Information 

Information 

granules of 

granules higher type of 

higher type 

Prototypes 

(higher order) 

Clustering 

Information 

Information 

granules 

granules 

data-1 

data-1 

data-2 

data-2 



data-P 

data-P 

prototypes 

D[1] D[2] D[P]

Determining correspondence between 

clusters 

Prototypes 

(higher order) 

z j 

Clustering 

Select prototypes in D[1], D[2], …, D[p] associated with z j 

with the highest degree of membership

Determining correspondence between 

clusters 

z j 

D[ii] 

v i 

[ii] 

λ 

ij 

[ii] 

= 

c[ii] 

∑ 

k= 

1 

⎛ || v 

⎜ 

⎝ 

|| v 

i 

k 

1 

[ii] − z 

[ii] − z 

j 

j 

|| ⎞ 

⎟ 

|| 

⎠ 

2 

Prototype i 0 

associated with prototype z j 

λ 

[ii] 

= 

max 

i0 j 

i= 

1,2,...,c[ii] 

λ 

ij

Family of associated prototypes 

Prototype i 1 

in D[1] associated with prototype z j 

Prototype i 2 

Prototype i p 

in D[2] associated with prototype z j 

… 

in D[p] associated with prototype z j 

v 

i 

1 

[1], 

v 

i 

2 

[2],...., 

v 

i 

p 

[P] 

λ 

i 

1 

, 

λ 

i 

2 

,...., 

λ 

i 

p

From numeric prototypes to 

granular prototypes 

v 

i 

1 

[1], 

v 

i 

2 

[2],...., 

v 

i 

p 

[P] 

λ 

i 

1 

, 

λ 

i 

2 

,...., 

λ 

i 

p 

individual coordinate of the associated prototypes: 

a 1 

a 2 

…. a p 

µ 1 

µ 2 

…. µ p 

R 

[0,1] 

Information granule

The principle of justifiable granularity: 

Interval representation 

a 1 

a 2 

…. a p 

1 

µ 1 

µ 2 

…. µ p 

0 

b 

a 0 

d 

if a i 

∈ [b,d] then elevate to membership grades to 1 

required change : 1- µ i


Interval representation 

a 1 

a 2 

…. a p 

1 

µ 1 

µ 2 

…. µ p 

0 

b 

a 0 

d 

if a i 

∉ [b,d] then reduce membership grades to 0 

required change: µ i


optimization criterion 

1 

0 

z 1 

z 2 

∑ 

a i ∈[b,d] 

∑ 

a i ∉[b,d] 

Min b,d ∈R:b≤d 

{ (1− µ i 

) + µ i 

}

Hyperbox prototypes 

i ≠ 

j: 

H 

i 

∩ H 

j 

= 

∅ 

(the number of 

H i 

H j 

level) 

clusters at the aggregation

Interval-valued fuzzy sets 

and granular prototypes 

x 

H i 

H j

Interval-valued fuzzy sets 

and granular prototypes 

v i 

x 

|| 

x − v 

i 

|| 

min 

| 

x 

− 

v 

i 

|| 

max 

Bounds of distances computed coordinate-wise

Interval-valued fuzzy sets: 

membership function 

∑ 

∑ 

= 

− 

− 

= 

− 

+ 

⎟ 

⎟ 

⎠ 

⎞ 

⎜ 

⎜ 

⎝ 

⎛ 

− 

− 

= 

⎟ 

⎟ 

⎠ 

⎞ 

⎜ 

⎜ 

⎝ 

⎛ 

− 

− 

= 

c 

1 

j 

1 

2 

min 

j 

max 

i 

i 

c 

1 

j 

1 

2 

max 

j 

min 

i 

i 

|| 

|| 

|| 

|| 

1 

) 

( 

u 

|| 

|| 

|| 

|| 

1 

) 

( 

u 

m 

m 

v 

x 

v 

x 

x 

v 

x 

v 

x 

x 

Upper bound 

Lower bound


Structure refinement 

Information 

Information 

granules of 

granules of 

higher type 

higher type 

Information 

Information 

granules 

granules 

data-1 

data-1 

data-2 

data-2 

data-P 

data-P 

Feedback 

and structure 

refinement 




Structure refinement 

Information 

Information 

granules of 

granules higher of type 

higher type 

Information 

Information 

granules 

granules 

data-1 

data-1 

data-2 

data-2 



data-P 

data-P 

Iterate 

Clustering at the local level 

Sharing findings and clustering at the higher (global) level 

Assessment of quality of clusters in light of the global structure 

γ i (U)[ii] 

Refinement of clustering 

Q[ii] = 

c[ii] 

2 

∑ ∑ γi (U)[ii]|| xk 

− vi[ii] 

|| 

i = 1 x ∈X[ii] 

k 

Until termination criterion satisfied

Towards enhanced interpretability of 

fuzzy models 

Fuzzy model


results of decision support models 

User 

Qualitative membership degrees 

A(x)=high 

R(A, B)=medium 

Numeric membership degrees 

A(x)=0.87 

Fuzzy 

model 

R(A, B)=0.51

Interpretability of fuzzy sets through 

type-2 fuzzy sets 

H M L 

Linguistic descriptors of membership 

Numeric membership degrees 

linguistic 

quantification 

x

Interpretability of fuzzy sets through 

type-2 fuzzy sets 

H M L 

A 

L L L M H H M M M L L 

linguistic 

quantification 

x 

x 

B 1 

, B 2 

, …, B r 

defined in [0,1] – treated as qualitative evaluators 

The least uncertain representation of A in terms of {B 1 

, B 2 

, …, B r 

} 

V 

= 

∫∑ 

X 

r 

i= 

1 

H(B i (A(x)))dx 

Min


fuzzy models 

User 

Interpretability layer 

Fuzzy model 

Information 

granules 

Data/ 

environment

Fuzzy Modelling - COST Action IC0702

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?