Mathematics in Independent Component Analysis

272 Chapter 20. Signal Processing 86(3):603-623, 2006
sparseness is measured by combining the Euclidean norm �.�2 and the 1-norm
�x�1 := �
i |xi| as follows:
√
n − �x�1/�x�2
sparseness(x) := √ (3)
n − 1
if x ∈ R n \ {0}. So sparseness(x) = 1 (maximal) if x contains n −1 zeros, and
it reaches zero if the absolute value of all coefficients of x coincide.
The devised algorithm is based on the iterative application of a gradient decent
step and a projection step, thus restricting the search to the subspace of sparse
solutions. We perform the factorization using the publicly available Matlab
library nmfpack 1 , which is used in [14].
So NMF decomposes X into nonnegative A and nonnegative S. The assumption
that A has nonnegative coefficients is very well fulfilled by s-EMG recordings;
however as seen before the sources also have negative entries. In order
to be able to apply the algorithms, we therefore preprocess the data using the
function
⎧
⎪⎨ 0, x < 0
κ(x) =
(4)
⎪⎩ x, x ≥ 0
to cut off non-zero values; this yields the new random vector (sample matrix)
X+ := (κ(X1), . . .,κ(Xn)) ⊤ . For comparison, we also construct a new sample
set by simply leaving out samples that have at least one negative value. Here
we model this by the random vector X∗.
2.2.3 Sparse component analysis
Sparse component analysis (SCA) [3,4] requires strong sparseness in the sources
only — this is then sufficient to decompose the observations. In order to define
the SCA model, a vector x ∈ R n is said to be k-sparse if x has at most
k non-zero entries. This k-sparseness implies k0-sparseness for k0 ≥ k. If an
n-dimensional vector is k-sparse for k = n − 1, it is simply said to be sparse.
The goal of sparse component analysis of level k (k-SCA) is to decompose X
into X = AS as above such that each sample (i.e. column) of S is k-sparse.
In the following we will assume k = n − 1.
Note that, in contrast to the ICA model, the above model is not translation
invariant. However, it is easy to see that if instead of A we allow an affine
linear transformation, the translation constant can be determined from X
only as long as the sources are non-deterministic. In other words, instead of
assuming k-sparseness of the sources we could also assume that at any time
1 http://www.cs.helsinki.fi/u/phoyer/
11