Mathematics in Independent Component Analysis

1.3. Dependent component analysis 23
x
s
L
P
A
(a) ICA
x
s
L
P
A
(b) ISA with fixed groupsize
x
s
L
P
A
(c) general ISA
Figure 1.9: Linear factorization models for a random vector x = As and the resulting indeterminacies,
where L denotes a one- or higher dimensional invertible matrix (scaling) and P
a permutation, to be applied only along the horizontal line as indicated in the figures. The
small horizontal gaps denote statistical independence. One of the key differences between the
models is that general ISA may always be applied to any random vector x, whereas ICA and
its generalization, fixed-size ISA, yield unique results only if x follows the corresponding model.
Of course, some restriction is necessary, otherwise no decomposition would be enforced at all.
The key idea in Theis (2007), see chapter 8, is to allow only irreducible components defined as
random vectors without lower-dimensional independent components.
The advantage of this formulation now is that it can clearly be applied to any random vector,
although of course a trivial decomposition might be the result in the case of an irreducible random
vector. Obvious indeterminacies of an ISA of x are scalings i.e. invertible transformations within
each si and permutation of si of the same dimension. These are already all indeterminacies as
shown by the following theorem:
Theorem 1.3.3 (Existence and Uniqueness of ISA). Given a random vector X with existing
covariance, an ISA of X exists and is unique except for permutation of components of the same
dimension and invertible transformations within each independent component and within the
Gaussian part.
Here, no Gaussians had to be excluded from S as in the previous uniqueness theorems,
because the dimension reduction result from section 1.5.2 has been used. For details we refer to
Theis (2007) and Gutch and Theis (2007). The connection of the various factorization models
and the corresponding uniqueness results are illustrated in figure 1.9.
Again, we turned this uniqueness result into a separation algorithm, this time by considering
the JADE-source condition based on fourth-order cumulants. The key idea was to translate
irreducibility into maximal block-diagonality of the source condition matrices Ci(s). Algorithmically,
JBD was performed using JD first using theorem 1.3.2, followed by permutation and
block-size identification (Theis, 2007, algorithm 1). So far, we did not implement a sophisticated
clustering step but only a straight-forward thresholding method for block-size determination.
First results when using more elaborate clustering techniques are promising.