Mathematics in Independent Component Analysis

Chapter 6. LNCS 3195:726-733, 2004 115
2 Fabian J. Theis, Anke Meyer-Baese, and Elmar W. Lang
where the source signals s(t) have diagonal autocovariances
Rs(τ) := E � (s(t + τ) − E(s(t)))(s(t) − E(s(t)) ⊤�
for all τ, and the additive noise n(t) is modelled by a stationary, temporally
and spatially white zero-mean process with variance σ2 . x(t) is observed, and
the goal is to recover A and s(t). Having found A, s(t) can be estimated by
A−1x(t), which is optimal in the maximum-likelihood sense (if the density of
n(t) is maximal at 0, which is the case for usual noise models such as Gaussian or
Laplacian noise). So the BSS task reduces to the estimation of the mixing matrix
A. Extensions of the above model include for example the complex case [4] or the
allowance of different dimensions for s(t) and x(t), where the case of larger mixing
dimension can be easily reduced to the presented complete case by dimension
reduction resulting in a lower noise level [6].
By centering the processes, we can assume that x(t) and hence s(t) have zero
mean. The autocovariances then have the following structure
Rx(τ) = E � x(t + τ)x(t) ⊤� =
� ARs(0)A ⊤ + σ 2 I τ = 0
ARs(τ)A ⊤ τ �= 0
Clearly, A (and hence s(t)) can be determined by equation 1 only up to permutation
and scaling of columns. Since we assume existing variances of x(t)
and hence s(t), the scaling indeterminacy can be eliminated by the convention
Rs(0) = I. In order to guarantee identifiability of A except for permutation
from the above model, we have to additionally assume that there exists a delay
τ such that Rs(τ) has pairwise different eigenvalues (for a generalization see [4],
theorem 2). Then using the spectral theorem it is easy to see from equation 2
that A is determined uniquely by x(t) except for permutation.
2 AMUSE and SOBI
Equation 2 also gives an indication of how to perform BSS i.e. how to recover A
from x(t). The usual first step consists of whitening the no-noise term ˜x(t) :=
As(t) of the observed mixtures x(t) using an invertible matrix V such that V˜x(t)
has unit covariance. V can simply be estimated from x(t) by diagonalization of
the symmetric matrix R˜x(0) = Rx(0) − σ2I, provided that the noise variance
σ2 is known. If more signals than sources are observed, dimension reduction can
be performed in this step, and the noise level can be reduced [6].
In the following without loss of generality, we will therefore assume that
˜x(t) = As(t) has unit covariance for each t. By assumption, s(t) also has
unit covariance, hence I = E � As(t)s(t) ⊤A⊤� = ARs(0)A⊤ = AA⊤ so A is
orthogonal. Now define the symmetrized autocovariance of x(t) as ¯ � Rx(τ) :=
1 Rx(τ) + (Rx(τ)) ⊤� . Equation 2 shows that also the symmetrized autocovari-
2
ance x(t) factors, and we get
¯Rx(τ) = A ¯ Rs(τ)A ⊤
(2)
(3)