Mathematics in Independent Component Analysis

50 Chapter 1. Statistical machine learning of biomedical data
1.7 Outlook
We considered the (mostly linear) factorization problem
x(t) = As(t) + n(t). (1.18)
Often, the noise n(t) was not explicitly modeled but included in s(t). We assumed that x(t)
is known as well as some additional information about the system itself. Depending on the
assumptions, different problems and algorithmic solutions can be derived to solve such inverse
problems:
• statistically independent s(t): we proved that in this case ICA could solve (1.18) uniquely,
see section 1.2; this holds even in some nonlinear generalizations.
• approximate spatiotemporal independence in A and s(t): we provided a very robust,
simple algorithm for the spatiotemporal separation based on joint diagonalization, see
section 1.3.2.
• statistical independence between groups of sources s(t): again, uniqueness except for transformation
within the blocks can be proven, and the constructive proof results in a simple
update algorithm as in the linear case, see section 1.3.3.
• sparseness of the sources s(t): in the context of SCA, we relaxed the assumptions of single
source sparseness to multi-dimensional sparseness constraints, for which we were still able
to prove uniqueness and to derive an algorithm, see section 1.4.1.
• non-negativity of A and s(t): a sparse extension of the plain NMF model was analyzed
in terms of existence an uniqueness, and a generalization for lp-sparse sources has been
proposed in order to better approximate combinatorial sparseness in the l0-sense, see
section 1.4.2.
• Gaussian or noisy components in s(t): we proposed various denoising and dimension reduction
schemes, and proved uniqueness of a non-Gaussian signal subspace in section 1.5.
Finally, in section 1.6, we applied some of the above methods to biomedical data sets recorded
by functional MRI, surface EMG and optical microscopy.
Other work
In this summary, data analysis was discussed from the viewpoint of data factorization models
such as (1.18). Before concluding, some other works of the author in related areas should be
mentioned:
Primarily only discussed in mathematics, optimization on Lie groups has become an important
topic in the field of machine learning and neural networks, since many cost functions are
defined on parameter spaces that more naturally obey a non-Euclidean geometry. Consider for
example Amari’s natural gradient (Amari, 1998): simply by taking into account the geometry