Mathematics in Independent Component Analysis

1.4. Sparseness 27
Theorem 1.4.3 (SCA conditions). Assume that m ≤ n ≤ T and the matrix X ∈ R m×T satisfies
the following conditions:
� �
n
(i) the columns of X lie in the union H of m−1 different hyperplanes, each column lies in
only one such hyperplane, each hyperplane contains at least m columns of X such that each
m − 1 of them are linearly independent.
(ii) for each i ∈ {1, ..., n} there exist p =
� �
n−1
m−2
different hyperplanes {Hi,j} p
j=1
that their intersection Li = � p
k=1 Hi,j is one dimensional subspace.
(iii) any m different Li span the whole R m .
in H such
Then the matrix X is uniquely representable (up to permutation and scaling) as SCA, satisfying
the conditions of theorem 1.4.1.
Algorithms for SCA
In Georgiev et al. (2004, 2005c), we also proposed an algorithm based on random sampling for
reconstructing the mixing matrix and the sources, however it could not easily be applied in noisy
settings and high dimensions due to the involved combinatorial searches. Therefore, we derived
a novel, robust algorithm for SCA in Theis et al. (2007a), see chapter 11. The key idea was that
if the sources are of sufficiently high sparsity, the mixtures are clustered along hyperplanes in the
mixture space. Based on this condition, the mixing matrix could be reconstructed; furthermore,
this property turned out to be robust against noise and outliers.
The proposed algorithm employed a generalization of the Hough transform in order to detect
the hyperplanes in the mixture space, see figure 1.12. This leads to an algorithmically robust
matrix and source identification. The Hough-based hyperplane estimation does not depend on
the source dimension n, only on the mixture dimension m. With respect to applications, this
implies that n can be quite large and hyperplanes will still be found if the grid resolution used
in the Hough transform is sufficiently high. Increase of the grid resolution (in polynomial time)
results in increased accuracy also for higher source dimensions n.
For applications of the proposed SCA algorithms in signal processing and biomedical data
analysis, we refer to section 1.6.3 and Georgiev et al. (2006, 2005a,b), Theis et al. (2007a). More
elaborate source reconstruction methods, after knowing the mixing matrix A were discussed in
Theis et al. (2004a).
Postnonlinear generalization
In Theis and Amari (2004), see chapter 12, we considered the generalization of SCA to postnonlinear
mixtures, see section 1.2.2. As before the data x(t) = f(As(t)) is assumed to be
linearly mixed followed by a componentwise nonlinearity, see equation (1.3). However, now the
(m × n)-matrix A is allowed to be ‘wide’ i.e. the more complicated overcomplete situation is
treated and m < n. By using sparseness of s(t), we were still able to recover the system: