Mathematics in Independent Component Analysis

16 Chapter 1. Statistical machine learning of biomedical data
In contrast to this sometimes called hard-whitening technique, soft-whitening tries to avoid
a bias towards second-order statistics and uses a non-orthogonal joint diagonalization algorithm
(Pham, 2001, Yeredor, 2002, Ziehe et al., 2003b) by jointly diagonalizing the source conditions
Ci(x) together with the mixture covariance matrix E(xx ⊤ ). Then possible estimation errors in
the second-order part do not influence the total error disproportionately high.
Depending on the source conditions, various algorithms have been proposed in the literature.
Table 1.1 gives an overview over the algorithms together with the references, the source model,
the condition matrices and the optimization algorithm. For more details and references, we refer
to Theis and Inouye (2006).
Multidimensional autodecorrelation
In Theis et al. (2004e), see chapter 6, we considered BSS algorithms based on time-decorrelation
and the resulting source condition. Corresponding JD-based algorithms include AMUSE (Tong
et al., 1991) and extensions such as SOBI (Belouchrani et al., 1997) and TDSEP (Ziehe and
Mueller, 1998). They rely on the fact that the data sets have non-trivial autocorrelations. We
extended them to data sets having more than one direction in the parametrization such as
images. For this, we replaced one-dimensional autocovariances by multidimensional autocovariances
defined by
�
Cτ1,...,τM (s) := E s(z1 + τ1, . . . , zM + τM)s(z1, . . . , zM) ⊤�
(1.7)
where the s is centered and the expectation is taken over (z1, . . . , zM). Cτ1,...,τM (s) can be
estimated given equidistant samples by replacing random variables by sample values and expectations
by sums as usual.
A typical example for non-trivial multidimensional autocovariances is a source data set, in
which each component si represents an image of size h × w. Then the data is of dimension
M = 2 and samples of s are given at indices z1 = 1, . . . , h, z2 = 1, . . . , w. Classically, s(z1, z2)
is transformed to s(t) by fixing a mapping from the two-dimensional parameter set to the onedimensional
time parametrization of s(t), for example by concatenating columns or rows in the
case of a finite number of samples (vectorization). If the time structure of s(t) is not used, as in
all classical ICA algorithms in which i.i.d. samples are assumed, this choice does not influence
the result. However, in time-structure based algorithms such as AMUSE and SOBI results can
vary greatly depending on the choice of this mapping.
The advantage of using multidimensional autocovariances lies in the fact that now the multidimensional
structure of the data set can be used more explicitly. For example, if row concatenation
is used to construct s(t) from the images, horizontal lines in the image will only
give trivial contributions to the autocovariance. Figure 1.6 shows the one- and two-dimensional
autocovariance of the Lena image for varying τ respectively (τ1, τ2) after normalization of the
image to variance 1. Clearly, the two-dimensional autocovariance does not decay as quickly with
increasing radius as the one-dimensional covariance. Only at multiples of the image height, the
one-dimensional autocovariance is significantly high i.e. captures image structure.
For details as well as extended simulations and examples, we refer to Theis et al. (2004e)
and related work by Schießl et al. (2000), Schöner et al. (2000).