Mathematics in Independent Component Analysis

40 Chapter 1. Statistical machine learning of biomedical data
Other metrics may be defined on Gn,p, and they result in different Riemannian geometries
on the manifold. Optimization on non-Euclidean geometries is non-trivial, and has been studied
for a long time, see for example Edelman et al. (1999) and references therein. For instance in
the context of ICA, Amari’s seminal paper (Amari, 1998) on taking into account the geometry
of the search space Gl(n) yielded a considerable increase in performance and accuracy. Learning
in these matrix manifolds has been reviewed in Theis (2005b) and extended in Squartini and
Theis (2006).
In order to apply batch k-means to (Gn,p, d), we only had to solve the cluster assignment
equation (1.17). It turned out that for this, no elaborate optimization was necessary, instead
a closed form solution that only needs eigenvalue decomposition could be found. We state this
with the following theorem, proved in Gruber and Theis (2006):
Theorem 1.5.3 (Grassmann centroids). The centroid [C] ∈ Gn,p of a set of points [V1], . . . , [Vl] ∈
Gn,p according to (1.17) is spanned by p independent eigenvectors corresponding to the smallest
eigenvalues of the generalized cluster correlation l −1 � l
i=1 ViV ⊤ i .
Application to nonnegative matrix factorization
Detecting clusters in multiple samples drawn from a Grassmannian is a problem arising in
various applications. In Gruber and Theis (2006), we applied this to NMF in order to illustrate
the feasibility of the proposed algorithm.
Consider the matrix factorization problem (1.12) with the additional non-negativity constraints
S, A ≥ 0. If we assume that S spans the whole first quadrant, then X = AS is a conic
hull with cone edges spanned by the columns of A. After projection onto the standard simplex,
the conic hull reduces to the convex hull, and the projected, known mixture data set X lies
within a convex polytope of the order given by the number of rows of S. Hence we face the
problem of identifying n edges of a sampled polytope in R m−1 .
In two dimensions (after reduction of m = 3), this implies the task of finding the k edges
of a polytope where only samples in the inside are known. We used the Quickhull algorithm
(Barber et al., 1993) to construct the convex hull thus identifying the possible edges of the
polytope. However due to finite samples the identified polytope has far too many edges. Therefore,
we applied affine Grassmann n-means clustering—with samples weighted according to their
volume—to these edges in order to identify the n bounding edges, see example in figure 1.20.
Biomedical applications of other matrix factorization methods are discussed in the next
chapter. We only shortly want to mention Meyer-Bäse et al. (2005), where we applied NMF
and related unsupervised clustering techniques for the self-organized segmentation of biomedical
image time-series data describing groups of pixels exhibiting similar properties of local signal
dynamics.