Mathematics in Independent Component Analysis

Chapter 3. Signal Processing 84(5):951-956, 2004 83
952 F.J. Theis / Signal Processing 84 (2004) 951 – 956
Section 5 nally deals with separability of multidimensional
ICA (group ICA).
2. Notation
Let K ∈{R; C} be either the real or the complex
numbers. For m; n ∈ N let Mat(m × n; K)
be the K-vectorspace of real, respectively, complex
m × n matrices, Gl(n; K) := {W ∈ Mat(n ×
n; K) | det(W) �= 0} be the general linear group of
K n . I ∈ Gl(n; K) denotes the unit matrix. For ∈ C
we write Re( ) for its real and Im( ) for its imaginary
part.
An invertible matrix L ∈ Gl(n; K) is said to be a
scaling matrix, if it is diagonal. We say two matrices
B, C ∈ Mat(m × n; K) are (K−)equivalent, B ∼ C, if
C can be written as C = BPL with a scaling matrix
L ∈ Gl(n; K) and an invertible matrix with unit vectors
in each row (permutation matrix) P ∈ Gl(n; K). Note
that PL = L ′ P for some scaling matrix L ′ ∈ Gl(n; K),
so the order of the permutation and the scaling matrix
does not play a role for equivalence. Furthermore,
if B ∈ Gl(n; K) with B ∼ I, then also B −1 ∼ I,
and more general if BC ∼ A, then C ∼ B −1 A.So
two matrices are equivalent if and only if they di er
by right-multiplication by a matrix with exactly one
non-zero entry in each row and each column. If K=R,
the two matrices are the same except for permutation,
sign and scaling, if K = C, they are the same except
for permutation, sign, scaling and phase-shift.
3. A multivariate version of the Skitovitch–
Darmois theorem
The original Skitovitch–Darmois theorem shows a
non-trivial connection between Gaussian distributions
and stochastic independence. More precisely, it states
that if two linear combinations of non-Gaussian independent
random variables are again independent, then
each original random variable can appear only in one
of the two linear combinations. It has been proved independently
by Darmois [6] and Skitovitch [14]; in a
more accessible form, the proof can be found in [12].
Separability of linear BSS as shown by Comon [5]
is a corollary of this theorem, although recently separability
has also been shown without it [17].
Theorem 3.1 (Skitovitch–Darmois theorem). Let
L1 = �n i=1 iXi and L2 = �n i=1 iXi with X1;:::;Xn
independent real random variables and j, j ∈ R for
j =1;:::;n. If L1 and L2 are independent, then all Xj
with j j �= 0are Gaussian.
The converse is true if we assume that � n
j=1 j j =
0: If all Xj with j j �= 0 are Gaussian and
� n
j=1 j j = 0, then L1 and L2 are independent. This
follows because then L1 and L2 are uncorrelated, and
with all common variables being normal then also
independent.
Theorem 3.2 (Multivariate S–D theorem). Let L1 �
=
n
i=1 AiXi and L2 = �n i=1 BiXi with mutually independent
k-dimensional random vectors Xj and invertible
matrices Aj; Bj ∈ Gl(k; R) for j =1;:::;n. If
L1 and L2 are mutually independent, then all Xj are
Gaussian.
Here Gaussian (or jointly Gaussian) means that
each component of the random vector is a Gaussian.
Obviously, those Gaussians can have non-trivial correlations.
This extension of Theorem 3.1 to random vectors
has rst been noted by Skitovitch [15] and shown by
Ghurye and Olkin [8]. Zinger gave a di erent proof
for it in his Ph.D. thesis [18].
We need the following corollary:
Corollary 3.3. Let L1 = � n
i=1 AiX i and L2 =
� n
i=1 BiX i with mutually independent k-dimensional
random vectors X j and matrices Aj; Bj either zero or
in Gl(k; R) for j =1;:::;n. If L1 and L2 are mutually
independent, then all X j with AjBj �= 0are Gaussian.
Proof. We want to throw out all X j with AjBj =0.
Then Theorem 3.2 can be applied. Let j be given with
AjBj=0. Without loss of generality assume that Bj=0.
If also Aj = 0, then we can simply leave out X j since
it does not appear in both L1 and L2. Assume Aj �=
0. By assumption X j and X 1 ;:::;X j−1 ; X j+1 ;:::;X n
are mutually independent, then so are X j and L2 because
Bj = 0. Hence both −AjX j , L2 and L1, L2
are mutually independent, so also the two linear combinations
L1 − AjX j and L2 of the n − 1 variables
X 1 ;:::;X j−1 ; X j+1 ;:::;X n are mutually independent.
After successive application of this recursion we can