Mathematics in Independent Component Analysis

Chapter 4. Neurocomputing 64:223-234, 2005 99
p
Mutual information of recovered sources
1
0.8
0.6
0.4
0.2
1
0.2 0.4 0.6 0.8 1
1
2
0.6
0.4
3
5
4
0.4 0.6
0.4 0.6
0.2 0.4 0.6 0.8 1
q
0
0.8
0.6
0.4
0.6
0.2
0.4
p × g−1 q ) ◦ (f0.5 × g0.5) ◦ A � �
S
5
4
3
2
1
�
− log
0
MI � A −1 ◦ (f −1
2
1
0
0
1
1
gq
2 3
0.5
fp
p = 0.1
p = 1.0
q = 0.1
q = 1.0
0
0 1 2 3
Fig. 2. Simulation of the separability result using two families of nonlinearities with
fp(x) = 1
10p log
� √
x+ x2 +4e−20p 2e−10p �
and gq = y y
4 | 4 |3q−0.5 . The left plot displays a color
plot together with overlayed contours of a separation measure depending on the
parameters p, q used for recovery. The separation quality is measured using the
negative logarithm of the mutual information of the recovered sources. The region
around the separation point p = q = 0.5 is also displayed in more detail.
The components of the postnonlinearity will be taken from two families of
functions described by
fp(x) = 1
10p log
� √
x + x2 + 4e−20p �
2e −10p
and gq(y) = y
4
� �
�y
�3q−0.5
� �
� �
4
with p, q varying between 0 and 1. The first component of the nonlinearity,
fp models a sensors which saturates with varying strength and the second
component gq describes a polynomial activation of the sensor with varying
degree, see figure 2, right hand side.
In the simulation, an independent uniformly distributed random vector (3000
samples in [−1, 1] 2 ) is mixed postnonlinearly by the matrix A = ( 2.6 1.4
0.7 3.3 ) and
the diagonal nonlinearity
⎛
⎜
f ⎝ x
⎞ ⎛
1
⎟ ⎜ 5
⎠ = ⎝
y
log� e5 2
x + 1
2
√ �⎞
e10x2 + 4
1 y |y| 16
⎟
⎠ =
⎛
⎜
⎝ f0.5(x)
⎞
⎟
⎠
g0.5(y)
To recover the sources the family (fp, gq) −1 of diagonal nonlinearities is used
together with the inverse of A.
10