Mathematics in Independent Component Analysis

Chapter 12. LNCS 3195:718-725, 2004 177
2 Fabian J. Theis and Shun-ichi Amari
If an n-dimensional vector is (n − 1)-sparse, that is it includes at least one
zero component, it is simply said to be sparse. The goal of Sparse Component
Analysis of level k (k-SCA) is to decompose a given m-dimensional random
vector x into
x = As (1)
with a real m × n-matrix A and an n-dimensional k-sparse random vector s. s
is called the source vector, x the mixtures and A the mixing matrix. We speak
of complete, overcomplete or undercomplete k-SCA if m = n, m < n or m > n
respectively. In the following without loss of generality we will assume m ≤ n
because the undercomplete case can be easily reduced to the complete case by
projection of x.
Theorem 1 (Matrix identifiability). Consider the k-SCA problem from equation
1 for k := m − 1 and assume that every m × m-submatrix of A is invertible.
Furthermore let s be sufficiently rich represented in the sense that for any index
set of n − m + 1 elements I ⊂ {1, ..., n} there exist at least m samples of s such
that each of them has zero elements in places with indexes in I and each m − 1
of them are linearly independent. Then A is uniquely determined by x except for
left-multiplication with permutation and scaling matrices.
Theorem 2 (Source identifiablity). Let H be the set of all x ∈ R m such
that the linear system As = x has an (m − 1)-sparse solution s. If A fulfills
the condition from theorem 1, then there exists a subset H0 ⊂ H with measure
zero with respect to H, such that for every x ∈ H \ H0 this system has no other
solution with this property.
The above two theorems show that in the case of overcomplete BSS using
(m−1)-SCA, both the mixing matrix and the sources can uniquely be recovered
from x except for the omnipresent permutation and scaling indeterminacy. We
refer to [8] for proofs of these theorems and algorithms based upon them. We
also want to note that the present source recovery algorithm is quite different
from the usual sparse source recovery using l1-norm minimization [7] and linear
programming. In the case of sources with sparsity as above, the latter will not
be able to detect the sources.
2 Postnonlinear overcomplete SCA
2.1 Model
Consider n-dimensional k-sparse sources s with k < m. The postnonlinear mixing
model [9] is defined to be
x = f(As) (2)
with a diagonal invertible function f with f(0) = 0 and a real m × n-matrix A.
Here a function f is said to be diagonal if each component fi only depends on
xi. In abuse of notation we will in this case interpret the components fi of f as