Mathematics in Independent Component Analysis

S U M M A R Y T his �le contains all �gures of the manuscript
1.2.
in the same size as in the manuscript itself.
Uniqueness keyissues words: in independent component analysis 11
x 1
x 2
x m
.
E 1m
E 2m
E n m
(.) 2
(.) 2
.
(.) 2
Λ 1n
Λ 2n
E Λ
Λ n n
F ig. 1 Simpli�ed quadratic unmixing model y = Λ (.) 2 E x .
Figure 1.4: Simplified quadratic unmixing model y = Λ ◦ (.) 2 ◦ E ◦ x. If Ex only takes values
in one quadrant, then the model is invertible and its inverse model again follows a multilayer
perceptron structure x = E ⊤ ◦ √ ◦ Λ −1 ◦ y.
Several studies have employed quadratic forms as a generative process of data. Abed-Meraim
et al. (1996) suggested analyzing mixtures by second-order polynomials using a linearization for
the mixtures, which however in general destroys the assumption of independence. Leshem (1999)
proposed a whitening scheme based on quadratic forms in order to enhance linear separation
of time-signals in algorithms such as SOBI (Belouchrani et al., 1997). Similar quadratic mixing
models are also considered in Georgiev (2001) and Hosseini and Deville (2003). These are
studies in which the mixing model is assumed to be quadratic in contrast to the quadratic unmixing
model (1.4). For demixing into independent components by quadratic forms, Bartsch and
Obermayer (2003) suggested applying linear ICA to second-order terms of data, and Hashimoto
(2003) proposed an algorithm based on minimization of Kullback-Leibler divergence. However,
no identifiability was studied; instead they focused on the application to natural images.
In (Theis and Nakamura, 2004), we defined the above quadratic unmixing process and derived
a generative model. We then reduced the quadratic model to an overdetermined linear model,
where more observations than sources are given, by embedding y into R m(m+1)/2 ; this can be
done by taking the monomials xixj as new variables. Using some linear algebra, we then derived
the following identifiability theorem for overdetermined ICA:
Theorem 1.2.5 (Uniqueness of overdetermined ICA). Let x = As with independent n-dimensional
random vector s and full-rank m × n-matrix A with m ≥ n, and let the n × m matrix W
be chosen such thatMWx anuscript is independent. received Furthermore October 11, assume 2002. that s has at most one Gaussian
component and thatMthe anuscript variancesrevised of s exist. October Then there 11, 2002. exist a permutation matrix P and an
invertible scaling matrix F inalL manuscript with W = LP(A received J anuary 15, 2003.
†
T he authors are with the L ab. for A dvanced B rain Signal
P rocessing, B rain Science I nstitute, R I K E N, W ako-shi,
Saitama 351-0198 J apan.
On leave from the I nstitute of B iophysics, U niversity
of R egensburg, 93051 R egensburg, G ermany.
a) E -mail: fabian@theis.name
b) E -mail: wakakoh@brain.riken.jp
⊤A) −1A⊤ + C and CA = 0.
.
y 1
y 2
y n