Mathematics in Independent Component Analysis

178 Chapter 12. LNCS 3195:718-725, 2004
Postnonlinear overcomplete blind source separation using sparse sources 3
functions with domain R and write f = f1 × . . . × fm. The goal of overcomplete
postnonlinear k-SCA is to determine the mixing functions f and A and the
sources s given only x.
Without loss of generality consider only the complete and the overcomplete
case (i.e. m ≤ n). In the following we will assume that the sources are sparse of
level k := m − 1 and that the components fi of f are continuously differentiable
with f ′ i (t) �= 0. This is equivalent to saying that the fi are continuously differentiable
with continuously differentiable inverse functions (diffeomorphisms).
2.2 Identifiability
Definition 2. Let A be an m × n matrix. Then A is said to be mixing if A has
at least two nonzero entries in each row. And A = (aij)i=1...m,j=1...n is said to
be absolutely degenerate if there are two columns k �= l such that a 2 ik = λa2 il for
all i and fixed λ �= 0 i.e. the normalized columns differ only by the sign of the
entries.
Postnonlinear overcomplete SCA is a generalization of linear overcomplete
SCA, so the indeterminacies of postnonlinear SCA contain at least the indeterminacies
of linear overcomplete SCA: A can only be reconstructed up to scaling
and permutation. Also, if L is an invertible scaling matrix, then
f(As) = (f ◦ L) � (L −1 A)s � ,
so f and A can interchange scaling factors in each component.
Two further indeterminacies occur if A is either not mixing or absolutely
degenerate. In the first case, this means that fi cannot be identified if the ith
row of A contains only one non-zero element. In the case of an absolutely
degenerate mixing matrix, � sparseness � alone cannot detect the nonlinearity as the
1 1
counterexample A = and arbitrary f1 ≡ f2 shows.
1 −1
If s is an n-dimensional random vector, its image (or the support of its
density) is denoted as im s := {s(t)}.
Theorem 3 (Identifiability). Let s be an n-dimensional k-sparse random vector
(k < m), and x an m-dimensional random vector constructed from s as in
equation 2. Furthermore assume that
(i) s is fully k-sparse in the sense that im s equals the union of all k-dimensional
coordinate spaces (in which it is contained by the sparsity assumption),
(ii) A is mixing and not absolutely degenerate,
(iii) every m × m-submatrix of A is invertible.
If x = ˆf( ˆs) is another representation of x as in equation 2 with ˆs satisfying
the same conditions as s, then there exists an invertible scaling L with f = ˆf ◦ L,
and invertible scaling and permutation matrices L ′ , P ′ with A = LÂL′ P ′ .