Mathematics in Independent Component Analysis

PSfrag replacements
PSfrag replacements
f quadratic ICA for n = 2
PSfrag replacements
f quadratic ICA for n = 2
f quadratic ICA for n = 3
Chapter 5. IEICE TF E87-A(9):2355-2363, 2004 111
8
Mean SNR (dB)
Mean SNR (dB)
Mean SNR (dB)
45
40
35
30
25
20
15
10
Performance of quadratic ICA for n = 2
Bilinear ICA
Linear ICA
5
0 2000 4000 6000 8000 10000 12000
Number of samples
Performance of quadratic ICA for n = 3
35
Bilinear ICA
Linear ICA
30
25
20
15
10
5
0 2000 4000 6000 8000 10000 12000
Number of samples
Performance of quadratic ICA for n = 4
30
Bilinear ICA
Linear ICA
25
20
15
10
5
0 2000 4000 6000
Number of samples
8000 10000 12000
Fig. 8 Quadratic ICA performance versus sample set size in
dimensions 2 (top), 3 (middle) and 4 (bottom). Mean was taken
over 1000 runs with Λ −1 and E −1 having uniformly distributed
coefficients in [0, 1] n respectively [−1, 1] n . E −1 was orthogonalized
(an orthogonal basis was constructed out of the columns
of E −1 ) because in experiments the algorithm in higher dimension
turned out to be quite sensitive to high condition number of
E −1 . So E −1 = E ⊤ in accordance with the model from equation
9. The sources were uniformly distributed in [0, 1] n .
IEICE TRANS. FUNDAMENTALS, VOL.E87–A, NO.9 SEPTEMBER 2004
condition number of Λ −1 was very high. By leaving
out these cases, the performance of quadratic ICA noticeably
rises in comparison to linear ICA.
4.3 Quadratic ICA — image data
Finally we deal with a real-world example, that is, analysis
of natural images. We applied quadratic ICA to a
set of small patches collected from database of natural
images made by van Hateren [18]. We show two
dominant linear filters and corresponding eigenvalues
of each obtained quadratic form in figure 9. Most obtained
quadratic forms have one or two dominant linear
filters and these linear filters are selective for local
bar stimuli. The other eigenvalues all are substantially
smaller and lie centered around zero in a roughly Gaussian
distribution, see eigenvalue distribution in figure
9. If a quadratic form has two dominant filters, position,
orientation and spatial frequency selectiveness of
these two filters are very similar. Two corresponding
eigenvalues have different signs. In summary, values of
all quadratic forms correspond to squared simple cell
outputs. This result is qualitatively similar to the result
obtained in [2]. Simple-cell-like features are more
prominent in our results. Also, in section 4 of [9], the
obtained quadratic forms correspond to squared simple
cell outputs.
5. Conclusion
This paper treats nonlinear ICA by reducing it to the
case of linear ICA. After formally stating separability
results of overdetermined ICA, these results are applied
to quadratic and polynomial ICA. The presented algorithm
consists of linearization and application of linear
ICA. In order to apply PCA projection also in high dimensions,
a block calculation of the covariance matrix
is suggested. The algorithms are then used for artificial
data sets, where they show good performance for a
sufficiently high number of samples. In the application
to natural image data, characteristics of the obtained
quadratic forms are qualitatively the same as the results
of Bartsch and Obermayer [2] and they correspond to
squared simple cell outputs. In future work, we want
to further investigate a possible enhancement of the
separability result as well as a more detailed extension
to ICA with polynomial and maybe analytic mappings.
Furthermore, bilinear ICA with additional noise (for example
multiplicative noise) could be considered. Also,
issues such as model and parameter selection will have
to be treated, if higher order models are allowed.
Acknowledgments
We thank the anonymous reviewers for their valuable
suggestions, which improved the original manuscript.
FT was partially supported by the DFG in the grant