Mathematics in Independent Component Analysis

212 Chapter 15. Neurocomputing, 69:1485-1501, 2006
0.5 1 1.5 2 2.5 3 3.5 4
10
10
5
0
-5
-10
Original Signal Noisy Signal MDL based local ICA Threshold based local ICA
0.5 1 1.5 2 2.5 3 3.5 4
×10 3
5
0
-5
-10
10
5
0
-5
-10
0.5 1 1.5 2 2.5 3 3.5 4
-10
-15
-15
0.5 1 1.5 2 2.5 3 3.5 4
×10 3
10
5
0
-5
0.5 1 1.5 2 2.5 3 3.5 4
10
10
5
0
-5
-10
0.5 1 1.5 2 2.5 3 3.5 4
5
0
-5
-10
0.5 1 1.5 2 2.5 3 3.5 4
8
8
6
4
2
0
-2
-4
-6
-8
-8
0.5 1 1.5 2 2.5 3 3.5 4
Fig. 1. Comparison between MDL and threshold denoising of an artificial signal
with known SNR = 0. The feature space dimension was M = 40 and the number
of clusters was K = 35. The (MDL achieved an SNR = 8.9 dB and the Threshold
criterion an SNR = 10.5 dB)
MDL criterion favors some over-modelling of the signal subspace, i.e. it tends
to underestimate the number of noise components in the registered signals.
In [17] the conditions, such as the noise not being completely white, which
lead to a strong over-modelling are identified. Over-modelling also happens
frequently, if the eigenvalues of the covariance matrix related with noise components,
are not sufficiently close together and are not separated from the
signal components by a gap. In those cases a clustering criterion for the eigenvalues
seems to yield better results, but it is not as generic as the MDL
criterion.
4.1.2 Comparisons between LICA and LPCA
Consider the artificial signal shown in figure 1 with varying additive Gaussian
white noise. We apply the LICA denoising algorithm using either an MDL criterion
or a threshold criterion for parameter selection. The results are depicted
in figure 2.
The first and second diagram of figure 2 compare the performance, here the
enhancement of the SNR and the mean square error, of LPCA and LICA
depending on the input SNR. Note that a source SNR of 0 describes a case
where signal and noise have the same strength, while negative values indicate
situations where the signal is buried int the noise. The third graph shows the
difference in kurtosis of the original signal and the source signal in dependence
on the input SNR. All three diagrams were generated with the same data set,
i.e. the same signal and, for a given input SNR, the same additive noise.
These results suggest that a LICA approach is more effective when the signal
is infested with a large amount of noise, whereas a LPCA seems better suited
for signals with high SNRs. This might be due to the nature of our selection of
subspaces based on kurtosis or variance of the autocorrelation as the comparison
of higher statistical moments of the restored data, like kurtosis, indicate
that noise reduction can be enhanced if we are using a LICA approach.
15
×10 3
×10 3
6
4
2
0
-2
-4
-6