Mathematics in Independent Component Analysis

164 Chapter 11. EURASIP JASP, 2007
Fabian J. Theis et al. 3
a1
a3
a2
(a) Three hyperplanes span{ai,
aj} for 1 ≤ i < j ≤ 3 in the
3 × 3 case
a1
a3
a2
(b) Hyperplanes from (a) visualized
by intersection with the
sphere
a1
a3
a4
a2
(c) Six hyperplanes span{ai,
aj} for 1 ≤ i < j ≤ 4 in the
3 × 4 case
Figure 1: Visualization of the hyperplanes in the mixture space {x(t)} ⊂ R 3 . Due to the source sparsity, the mixtures are generated by only
two matrix columns ai, aj, and are hence contained in a union of hyperplanes. Identification of the hyperplanes gives mixing matrix and
sources.
Data: samples x(1), . . . , x(T)
Result: estimated mixing matrix �A
Hyperplane identification.
(1) Cluster the samples x(t) in � �
n
m−1 groups such that the span
of the elements of each group produces one distinct
hyperplane Hi.
Matrix identification.
(2) Cluster the normal vectors to these hyperplanes in the
smallest number of groups Gj, j = 1, . . . , n (which gives the
number of sources n) such that the normal vectors to the
hyperplanes in each group Gj lie in a new hyperplane �Hj.
(3) Calculate the normal vectors �aj to each hyperplane
�Hj, j = 1, . . . , n.
(4) The matrix �A with columns �aj is an estimate of the mixing
matrix (up to permutation and scaling of the columns).
Algorithm 1: SCA matrix identification algorithm.
illustrated in Figure 1: by assuming sufficiently high sparsity
of the sources, the mixture space clusters along a union of
hyperplanes, which uniquely determine both mixing matrix
and sources.
The matrix and source identification algorithm from [9]
are recalled in Algorithms 1 and 2. We will present a modification
of the matrix identification part—the same source
identification algorithm (Algorithm 2) will be used in the experiments.
The “difficult” part of the matrix identification
algorithm lies in the hyperplane detection; in Algorithm 1, a
random sampling and clustering technique is used. Another
more efficient algorithm for finding the hyperplanes containing
the data has been developed by Bradley and Mangasarian
[21], essentially by extending k-means batch clustering.
Their so-called k-plane clustering algorithm in the special case
of hyperplanes containing 0 is shown in Algorithm 3. The
Data: samples x(1), . . . , x(T) and estimated mixing matrix �A
Result: estimated sources �s(1), . . . ,�s(T)
(1) Identify the set of hyperplanes H produced by taking the
linear hull of every subsets of the columns of �A with m − 1
elements
for t ← 1, . . . , T do
(2) Identify the hyperplane H ∈ H containing x(t), or, in
the presence of noise, identify the one to which the
distance from x(t) is minimal and project x(t) onto H
to �x.
(3) If H is produced by the linear hull of column vectors
�ai1, . . . , �aim−1, find coefficients λi(j) such that
�x = � m−1
j=1 λi(j)�ai(j).
(4) Construct the solution �s(t): it contains λi(j) at index i(j)
for j = 1, . . . , m − 1, the other components are zero.
end
Algorithm 2: SCA source identification algorithm.
finite termination of the algorithm is proven in [21, Theorem
3.7]. We will later compare the proposed Hough algorithm
with the k-hyperplane algorithm. The k-hyperplane algorithm
has also been extended to a more general, orthogonal
k-subspace clustering method [22, 23] thus allowing a search
not only for hyperplanes but also for lower-dimensional subspaces.
3. HOUGH TRANSFORM
The Hough transform is a classical method for locating
shapes in images, widely used in the field of image processing;
see [10, 24]. It is robust to noise and occlusions and is
used for extracting lines, circles, or other shapes from images.
In addition to these nonlinear extensions, it can also be
made more robust to noise using antialiasing techniques.