Mathematics in Independent Component Analysis

278 Chapter 20. Signal Processing 86(3):603-623, 2006
JADE NMF NMF∗ sNMF sNMF∗ SCA sources
mean kurtosis 10.80 8.48 8.80 8.85 9.64 7.14 11.4
sparseness 0.286 0.268 0.295 0.270 0.302 0.201 0.252
σ(A✷) 2.91 2.02 2.11 1.99 2.36 0.96 3.24
Table 2
Sparseness measures of the recovered sources using the various methods. In the first
row, the mean kurtosis is calculated (the higher, the more ‘spiky’ the signal). The
second row gives the sparseness index from equation (3) (the higher, the sparser
the signal) and the third row the cost function (5) employed in SCA (the lower, the
better the SCA criterion is fulfilled). Optimal values are printed in bold face.
AJADE ANMF ANMF∗ AsNMF AsNMF∗ ASCA
E1(A + ✷A) 1.39 3.27 2.96 3.41 2.56 3.86
JADE clearly outperforms the other methods — this will be explained in the
next paragraph. However we have to add that the signal generation additionally
involves a slightly nonlinear filter, so we can only estimate the real mixing
matrix A from the sources and the mixtures, which yielded a non-negligible
error. Hence this result indicates that JADE could best approximate the linearized
system.
One key question in this work is how the different models induce sparseness.
Clearly, sparseness is a rather ambiguous term, so we calculate three
indices (kurtosis, ‘sparseness’ from equation (3) and σ from (5)) for the real
as well as the recovered sources using the proposed methods, see Tab. 2. As
expected, ICA gives the highest kurtosis among all methods, whereas sparse
NMF yields highest values of the sparseness criterion. And SCA has the lowest
i.e. best value regarding k-sparseness measured by σ(A). We further see that
both the kurtosis and the sparseness criterion seem to be somewhat related
with regards to the data set as high values in both indices are achieved by
both JADE and sNMF. The k-sparseness criterion, which fixes only the zero-
(semi)norm i.e. requires a fixed amount of zeros in the data without additional
requirements about the other values does not induce as high sparseness when
measured using kurtosis/sparseness and vice versa. Finally by looking at the
sparseness indices of the real sources, we can now understand why JADE outperformed
the other methods in this toy data setting — indeed kurtosis of the
sources is high and the mutual information low, see Fig. 6(b). But in terms of
sparseness and especially σ, the sources are not as sparse as expected. Hence
the (sparse) NMF and mainly the SCA algorithm could not perform as well
as JADE when compared to the original sources as noted above. However, we
will see that in the case of real s-EMG signals, this distinction will break; furthermore,
sparse NMF turns out to be more robust against noise than JADE,
as is shown in the following.
17