Mathematics in Independent Component Analysis

116 Chapter 6. LNCS 3195:726-733, 2004
SOBI based on multi-dimensional autocovariances 3
for τ �= 0. By assumption ¯ Rs(τ) is diagonal, so equation 3 is an eigenvalue decomposition
of the symmetric matrix ¯ Rx(τ). If we furthermore assume that ¯ Rx(τ)
or equivalently ¯ Rs(τ) has n different eigenvalues, then the above decomposition
i.e. A is uniquely determined by ¯ Rx(τ) except for orthogonal transformation
in each eigenspace and permutation; since the eigenspaces are one-dimensional
this means A is uniquely determined by equation 3 except for permutation. In
addition to this separability result, A can be recovered algorithmically by simply
calculating the eigenvalue decomposition of ¯ Rx(τ) (AMUSE, [3]).
In practice, if the eigenvalue decomposition is problematic, a different choice
of τ often resolves this problem. Nontheless, there are sources in which some
components have equal autocovariances. Also, due to the fact that the autocovariance
matrices are only estimated by a finite amount of samples, and due to
possible colored noise, the autocovariance at τ could be badly estimated. A more
general BSS algorithm called SOBI (second-order blind identification) based on
time decorrelation was therefore proposed by Belouchrani et al. [4]. In addition
to only diagonalizing a single autocovariance matrix, it takes a whole set of autocovariance
matrices of x(t) with varying time lags τ and jointly diagonalizes
the whole set. It has been shown that increasing the size of this set improves
SOBI performance in noisy settings [1].
Algorithms for performing joint diagonalization of a set of symmetric commuting
matrices include gradient descent on the sum of the off-diagonal terms,
iterative construction of A by Givens rotation in two coordinates [7] (used in the
simulations in section 4), an iterative two-step recovery of A [8] or more recently
a linear least-squares algorithm for diagonalization [9], where the latter two algorithms
can also search for non-orthogonal matrices A. Joint diagonalization
has been used in BSS using cumulant matrices [10] or time autocovariances [4,5].
3 Multidimensional SOBI
The goal of this work is to improve SOBI performance for random processes
with a higher dimensional parametrization i.e. for data sets where the random
processes s and x do not depend on a single variable t, but on multiple variables
(z1, . . . , zM ). A typical example is a source data set, in which each component
si represents an image of size h × w. Then M = 2 and samples of s are given at
z1 = 1, . . . , h, z2 = 1, . . . , w. Classically, s(z1, z2) is transformed to s(t) by fixing
a mapping from the two-dimensional parameter set to the one-dimensional time
parametrization of s(t), for example by concatenating columns or rows in the
case of a finite number of samples. If the time structure of s(t) is not used, as
in all classical ICA algorithms in which i.i.d. samples are assumed, this choice
does not influence the result. However, in time-structure based algorithms such
as AMUSE and SOBI results can vary greatly depending on the choice of this
mapping, see figure 2.
Without loss of generality we again assume centered random vectors. Then
define the multidimensional covariance to be
Rs(τ1, . . . , τM ) := E � s(z1 + τ1, . . . , zM + τM)s(z1, . . . , zM ) ⊤�