Mathematics in Independent Component Analysis

Chapter 5. IEICE TF E87-A(9):2355-2363, 2004 107
4
where y ′ := Wx and y := Vx.
In applications, V is usually chosen to be the projection
along the first principal components in order to
reduce noise [14]. In this case it is easy to see that
indeed VA is invertible as needed in the theorem.
3. Quadratic ICA
The model of quadratic ICA is introduced and separability
and algorithms are studied in this context.
3.1 Model
Let x be an m-dimensional random vector. Consider
the following quadratic or bilinear unmixing model
y := g(x, x) (6)
for a fixed bilinear mapping g : R m × R m → R n .
The components of the bilinear mapping are quadratic
forms, which can be parameterized by symmetric matrices.
So the above is equivalent to
yi := x ⊤ G (i) x (7)
for symmetric matrices G (i) and i = 1, . . . , n. If G (i)
kl
are the coefficients of G (i) , this means
yi =
m�
m�
k=1 l=1
G (i)
kl xkxl
for i = 1, . . . , n.
A special case of this model can be constructed as
follows: Decompose the symmetric coefficient matrices
into
G (i) = E (i)⊤ Λ (i) E (i) ,
where E (i) is orthogonal and Λ (i) diagonal. In order
to explicitly invert the above model (after restriction
to a subset for invertibility) we now assume that these
coordinate changes E (i) are all the same i.e.
for i = 1, . . . , n. Then
E = E (i)
yi = (Ex) ⊤ Λ (i) (Ex) =
m�
k=1
Λ (i)
kk (Ex)2 k
(8)
where Λ (i)
kk are the coefficients on the diagonal of Λ(i) .
Setting
⎛
⎜
Λ := ⎝
11 . . . Λ (1) ⎞
nn
. .
. ..
. ⎟
. ⎠
Λ (1)
Λ (n)
11 . . . Λ (n)
nn
yields a two-layered unmixing model
y = Λ ◦ (.) 2 ◦ E ◦ x, (9)
IEICE TRANS. FUNDAMENTALS, VOL.E87–A, NO.9 SEPTEMBER 2004
x1
x2
xm
.
E1m
E2m
Enm
(.) 2
(.) 2
.
(.) 2
Λ1n
Λ2n
E Λ
Λnn
Fig. 1 Simplified quadratic unmixing model y = Λ◦(.) 2 ◦E◦x.
s1
s2
sn
� Λ −1 �
.
1n
�
−1 Λ �
2n
� Λ −1 �
nn
√
√
� E −1 �
.
√
1n
� E −1 �
2n
� E −1 �
Λ −1 E −1
Fig. 2 Simplified square root mixing model x = E −1 ◦ √ ◦
Λ −1 ◦ s.
where (.) 2 is to be read as the componentwise square of
each element. This can be interpreted as a two-layered
feed-forward neural network, see figure 1.
The advantage of the restricted model from equation
8 is that now the model can easily be explicitly
inverted. We assume that Λ is invertible and that
Ex only takes values in one quadrant — otherwise the
model cannot be inverted. Without loss of generality
let this be the first quadrant that is assume
(Ex)i ≥ 0
for i = 1 . . . , m. Then model 9 is invertible and the corresponding
inverse model (mixing model) can be easily
expressed as
mn
.
.
y1
y2
yn
x1
x2
xm
x = E −1 ◦ √ ◦ Λ −1 ◦ s (10)
with E −1 = E ⊤ . Here we write s as the domain of the
model in order to distinguish between the recoveries y
given by the unmixing model. Ideally, the two are the
same. The inverse model is shown in figure 2.
3.2 Separability
Constructing a new random vector by arranging the