Mathematics in Independent Component Analysis

Chapter 3. Signal Processing 84(5):951-956, 2004 85
954 F.J. Theis / Signal Processing 84 (2004) 951 – 956
Theorem 4.1 (Separability of complex linear BSS).
Let A ∈ Gl(n; C) and S a complex independent random
vector. Assume one of the following:
i. S has at most one Gaussian component and the
(complex) covariance of S exists.
ii. S has no Gaussian component.
If AS is again independent 1 then A is equivalent
to the identity.
Here, the complex covariance of S is de ned by
Cov(S)=E((S − E(S))(S − E(S)) ∗ );
where the asterix denotes the transposed and complexly
conjugated vector.
Comon has shown this for the real case [5]; for
the complex case a complex version of the Darmois–
Skitovitch theorem is needed as provided in Section 3.
Theorem 4.1 indeed proves separability of the complex
linear BSS model, because if X = AS and W is
a demixing matrix such that WX is independent, then
WA ∼ I, soW −1 ∼ A as desired. And it also calculates
the indeterminacies of complex ICA, because if
W and V are ICAs of X, then both VX and WV −1 VX
are independent, so WV −1 ∼ I and hence W ∼ V.
Proof. Denote X := AS.
First assume case ii: S has no Gaussian component
at all. Then A =(aij) is equal to the identity, because
if not there exist i1 �= i2 and j with ai1jai2j �= 0.
Applying Corollary 3.4 to Xi1 and Xi2 then shows
that Sj is Gaussian, which is a contradiction to
Assumption ii.
Now assume that the covariance exists and that S
has at most one Gaussian component. First we will
show using complex decorrelation that we can assume
A to be unitary. Without loss of generality assume
that all random vectors are centered. By assumption
Cov(X) is diagonal, so let D1 be diagonal invertible
with Cov(X) =D2 1 . Note that D1 is real. Similarly
let D2 be diagonal invertible with Cov(S)=D2 2 . Set
Y := D −1
1 X and T := D−1
2 S that is normalize X and
S to covariance I. Then
Y = D −1
1
X = D−1AS
= D−1
1
1 AD2T
1 Indeed, we only need that AS are pairwise mutually independent.
and T, D −1
1 AD2 and Y satisfy the assumption and
D −1
1 AD2 is unitary because
I = Cov(Y)
= E(YY ∗ )
= E(D −1
1 AD2TT ∗ D2A ∗ D −1
1 )
=(D −1 −1
1 AD2)(D1 AD2) ∗ :
If we assume A I, then using the fact that A is
unitary there exist indices i1 �= i2 and j1 �= j2 with
ai∗j∗ �= 0. By assumption
Xi1 = ai1j1 Sj1 + ai1j2 Sj2 + ···
Xi2 = ai2j1
Sj1 + ai2j2
Sj2 + ···
are independent, and in both Xi1 and Xi2 the variables
Sj1 and Sj2 appear non-trivially, so by the complex Skitovitch–Darmois
Theorem 3.4 Sj1 and Sj2 are Gaussian,
which is a contradiction to the fact that at most
one source is Gaussian.
5. Indeterminacies of multidimensional ICA
In this section, we want to analyze indeterminacies
of so-called multidimensional independent component
analysis. The idea of this generalization of
ICA is that we do not require full independence of the
transform Y but only mutual independence of certain
tuples Yi1;:::;Yi2. If the size of all tuples is restricted to
one, this reduces to original ICA. In general, of course
the tuples could have di erent sizes, but for the sake
of simplicity we assume that they all have the same
length (which then necessarily has to divide the total
dimension).
Multidimensional ICA has rst been introduced by
Cardoso [3] using geometrical motivations. Hyvarinen
and Hoyer then presented a special case of multidimensional
ICA which they called independent
subspace analysis [9]; there the dependence within
a k-tuple is explicitly modelled enabling the authors
to propose better algorithms without having to
resort to the problematic multidimensional density
estimation. A di erent extension of ICA is given by
topographic ICA [10], where dependencies between
all components are assumed. A special case of multidimensional
ICA is complex ICA as presented in