Mathematics in Independent Component Analysis

1.3. Dependent component analysis 21
ISA with known group structure via joint block diagonalization
In order to solve ISA with fixed block-size k or at least known block structure, we will use a
generalization of joint diagonalization, which searches for block-structures instead of diagonality.
We are not interested in the order of the blocks, so the block-structure is uniquely specified by
fixing a partition n = m1 + . . . + mr of n and set m := (m1, . . . , mr) ∈ N r . An n × n matrix is
said to be m-block diagonal if it is of the form
⎛
M1
⎜
⎝ .
· · ·
. ..
0
.
⎞
⎟
⎠
0 · · · Mr
with arbitrary mi × mi matrices Mi.
As generalization of JD in the case of known the block structure, the joint m-block diagonalization
problem is defined as the minimization of
f m ( Â) :=
K�
�Â⊤Ci − diagm ( Â⊤CiÂ)�2F i=1
(1.10)
with respect to the orthogonal matrix Â, where diagm (M) produces a m-block diagonal matrix
by setting all other elements of M to zero. Indeterminacies of any m-JBD are m-scaling
i.e. multiplication by an m-block diagonal matrix from the right, and m-permutation defined
by a permutation matrix that only swaps blocks of the same size.
A few algorithms to actually perform JBD have been proposed, see Abed-Meraim and Belouchrani
(2004), Févotte and Theis (2007a). In the following we will simply perform joint
diagonalization and then permute the columns of A to achieve block-diagonality—in experiments
this turns out to be an efficient solution to JBD, although other more sophisticated pivot
selection strategies for JBD are of interest (Févotte and Theis, 2007b). The fact that JD induces
JBD has been conjectured by Abed-Meraim and Belouchrani (2004), and we were able to give
a partial answer with the following theorem:
Theorem 1.3.2 (JBD via JD). Any block-optimal JBD of the Ci’s (i.e. a zero of f m ) is a local
minimum of the JD-cost-function f from equation (1.5).
Clearly not any JBD minimizes f, only those such that in each block of size mk, f( Â) when
restricted to the block is maximal over A ∈ O(mk), which we denote as block-optimal. The
proof is given in Theis (2007), see chapter 8.
In the case of k-ISA, where m = (k, . . . , k), we used this result to propose an explicit
algorithm (Theis, 2005a, see chapter 7). Consider the BSS model from equation (1.1). As usual
by preprocessing we may assume whitened observations x, so A is orthogonal. For the density
ps of the sources, we therefore get ps(s0) = px(As0). Its Hessian transforms like a 2-tensor,
which locally at s0 (see section 1.2.1) guarantees
Hln ps (s0) = Hln px◦A(s0) = AHln px (As0)A ⊤ . (1.11)