Mathematics in Independent Component Analysis

Chapter 16. Proc. ICA 2006, pages 917-925 231
An intuitive notion of how to choose the reduced dimension n is to require that WGX is
maximally Gaussian, and hence WNX non-Gaussian.
The dimension reduction problem itself can of course also be formulated within a
generative model, which leads to the following linear mixing model
X=ANSN+ AGSG
such that SN and SG are independent, and SG Gaussian. Then (AN, AG) −1 = (W ⊤ N , W⊤ G )⊤ .
This model includes the general noisy ICA model X=ANSN+ G, where G is Gaussian
and SN is also assumed to be mutually independent; the dimension reduction then means
projection onto the signal subspace, which might be deteriorated by the noise G along
the subspace — the components of G orthogonal to the subspace will be removed.
However (1) is more general in the sense that it does not assume mutual independence
of SN, only independence of SN and SG.
The paper is organized as follows: In the next section, we first discuss obvious
indeterminacies of NGSA and possible regularizations. We then present our main result,
theorem 1, and give an explicit proof in a special case. The general proof is divided
up into a series of lemmas, the proofs of which are omitted due to lack of space. In
section 2, some simulations are performed to validate the uniqueness result. A practical
algorithm for performing NGSA essentially using the idea of separated characteristic
functions from the proof is presented in the co-paper [6].
1 Uniqueness of NGSA-based dimension reduction
This contribution aims at providing conditions such that the decomposition (1) is unique.
More precisely, we will show under which conditions the non-Gaussian as well as the
Gaussian subspace is unique.
1.1 Indeterminacies
Clearly, the matrices AN and AG in the decomposition (1) cannot be unique — multiplication
from the right using any invertible matrix leaves the model invariant: X=
ANSN+ AGSG= (ANBN)(B−1 −1
N SN)+(AGBG)(B G SG) with BN∈ Gl(n), BG∈ Gl(d− n),
because B−1 N SN and B−1 G SG are again independent, and B−1 G SG Gaussian.
An additional indeterminacy comes into play due to the fact that we do not want to
fix the reduced dimension in advance. Given a realization of the model (1) with d