Mathematics in Independent Component Analysis

More documents

Recommendations

Info

82 Chapter 3. Signal Processing 84(5):951-956, 2004 Abstract Signal Processing 84 (2004) 951 – 956 www.elsevier.com/locate/sigpro Fast communication Uniqueness of complex and multidimensional independent component analysis F.J. Theis ∗ Institute of Biophysics, University of Regensburg, Universitaetsstr. 31, D93040 Regensburg, Germany Received 25 September 2003 A complex version of the Darmois–Skitovitch theorem is proved using a multivariate extension of the latter by Ghurye and Olkin. This makes it possible to calculate the indeterminacies of independent component analysis (ICA) with complex variables and coe cients. Furthermore, the multivariate Darmois–Skitovitch theorem is used to show uniqueness of multidimensional ICA, where only groups of sources are mutually independent. ? 2004 Elsevier B.V. All rights reserved. PACS: 84.40.Ua; 89.70.+c; 07.05.Kf Keywords: Complex ICA; Multidimensional ICA; Separability 1. Introduction The task of independent component analysis (ICA) is to transform a given random vector into a statistically independent one. ICA can be applied to blind source separation (BSS), where it is furthermore assumed that the given vector has been mixed using a xed set of independent sources. Good textbook-level introductions to ICA are given in [4,11]. BSS is said to be separable if the mixing structure can be blindly recovered except for obvious indeterminacies. In [5], Comon shows separability of linear real BSS using the Skitovitch–Darmois theorem. He notes that his proof for the real case can also be extended to the complex setting. However, a complex version of ∗ Tel.: +49-941-9432924; fax: +49-941-9432479. E-mail addresses: fabian.theis@mathematik.uni-regensburg.de, fabian@theis.name (F.J. Theis). 0165-1684/$ - see front matter ? 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.sigpro.2004.01.008 the Skitovitch–Darmois theorem is needed, which, to the knowledge of the author, has not been shown in the literature, yet. In this work we will provide such a theorem, which is then used to prove separability of complex BSS. Separability and uniqueness of BSS is already included in the de nition of what is commonly called a ‘contrast’ [5]. Hence it has been widely studied, but in the setting of complex BSS to the knowledge of the author separability has only been shown under the additional assumption of non-zero cumulants of the sources [5,13]. The paper is organized as follows: In the next section, basic terms and notations are introduced. Section 3 states the well-known Skitovitch–Darmois theorem and a multivariate extension thereof; furthermore, a complex version of it is derived. The following Section 4 then introduces the complex linear blind source separation model and shows its separability.
Chapter 3. Signal Processing 84(5):951-956, 2004 83 952 F.J. Theis / Signal Processing 84 (2004) 951 – 956 Section 5 nally deals with separability of multidimensional ICA (group ICA). 2. Notation Let K ∈{R; C} be either the real or the complex numbers. For m; n ∈ N let Mat(m × n; K) be the K-vectorspace of real, respectively, complex m × n matrices, Gl(n; K) := {W ∈ Mat(n × n; K) | det(W) �= 0} be the general linear group of K n . I ∈ Gl(n; K) denotes the unit matrix. For ∈ C we write Re( ) for its real and Im( ) for its imaginary part. An invertible matrix L ∈ Gl(n; K) is said to be a scaling matrix, if it is diagonal. We say two matrices B, C ∈ Mat(m × n; K) are (K−)equivalent, B ∼ C, if C can be written as C = BPL with a scaling matrix L ∈ Gl(n; K) and an invertible matrix with unit vectors in each row (permutation matrix) P ∈ Gl(n; K). Note that PL = L ′ P for some scaling matrix L ′ ∈ Gl(n; K), so the order of the permutation and the scaling matrix does not play a role for equivalence. Furthermore, if B ∈ Gl(n; K) with B ∼ I, then also B −1 ∼ I, and more general if BC ∼ A, then C ∼ B −1 A.So two matrices are equivalent if and only if they di er by right-multiplication by a matrix with exactly one non-zero entry in each row and each column. If K=R, the two matrices are the same except for permutation, sign and scaling, if K = C, they are the same except for permutation, sign, scaling and phase-shift. 3. A multivariate version of the Skitovitch– Darmois theorem The original Skitovitch–Darmois theorem shows a non-trivial connection between Gaussian distributions and stochastic independence. More precisely, it states that if two linear combinations of non-Gaussian independent random variables are again independent, then each original random variable can appear only in one of the two linear combinations. It has been proved independently by Darmois [6] and Skitovitch [14]; in a more accessible form, the proof can be found in [12]. Separability of linear BSS as shown by Comon [5] is a corollary of this theorem, although recently separability has also been shown without it [17]. Theorem 3.1 (Skitovitch–Darmois theorem). Let L1 = �n i=1 iXi and L2 = �n i=1 iXi with X1;:::;Xn independent real random variables and j, j ∈ R for j =1;:::;n. If L1 and L2 are independent, then all Xj with j j �= 0are Gaussian. The converse is true if we assume that � n j=1 j j = 0: If all Xj with j j �= 0 are Gaussian and � n j=1 j j = 0, then L1 and L2 are independent. This follows because then L1 and L2 are uncorrelated, and with all common variables being normal then also independent. Theorem 3.2 (Multivariate S–D theorem). Let L1 � = n i=1 AiXi and L2 = �n i=1 BiXi with mutually independent k-dimensional random vectors Xj and invertible matrices Aj; Bj ∈ Gl(k; R) for j =1;:::;n. If L1 and L2 are mutually independent, then all Xj are Gaussian. Here Gaussian (or jointly Gaussian) means that each component of the random vector is a Gaussian. Obviously, those Gaussians can have non-trivial correlations. This extension of Theorem 3.1 to random vectors has rst been noted by Skitovitch [15] and shown by Ghurye and Olkin [8]. Zinger gave a di erent proof for it in his Ph.D. thesis [18]. We need the following corollary: Corollary 3.3. Let L1 = � n i=1 AiX i and L2 = � n i=1 BiX i with mutually independent k-dimensional random vectors X j and matrices Aj; Bj either zero or in Gl(k; R) for j =1;:::;n. If L1 and L2 are mutually independent, then all X j with AjBj �= 0are Gaussian. Proof. We want to throw out all X j with AjBj =0. Then Theorem 3.2 can be applied. Let j be given with AjBj=0. Without loss of generality assume that Bj=0. If also Aj = 0, then we can simply leave out X j since it does not appear in both L1 and L2. Assume Aj �= 0. By assumption X j and X 1 ;:::;X j−1 ; X j+1 ;:::;X n are mutually independent, then so are X j and L2 because Bj = 0. Hence both −AjX j , L2 and L1, L2 are mutually independent, so also the two linear combinations L1 − AjX j and L2 of the n − 1 variables X 1 ;:::;X j−1 ; X j+1 ;:::;X n are mutually independent. After successive application of this recursion we can
Page 1 and 2:
Statistical machine learning of bio
Page 3:
to Jakob
Page 6 and 7:
vi Preface
Page 8 and 9:
viii CONTENTS 3 Signal Processing 8
Page 11 and 12:
Chapter 1 Statistical machine learn
Page 13 and 14:
1.1. Introduction 5 auditory cortex
Page 15 and 16:
1.2. Uniqueness issues in independe
Page 17 and 18:
1.2. Uniqueness issues in independe
Page 19 and 20:
S U M M A R Y T his �le contains
Page 21 and 22:
1.3. Dependent component analysis 1
Page 23 and 24:
Table 1.1: BSS algorithms based on
Page 25 and 26:
Page 27 and 28:
Page 29 and 30:
Page 31 and 32:
Page 33 and 34:
1.4. Sparseness 25 1.4 Sparseness O
Page 35 and 36:
1.4. Sparseness 27 Theorem 1.4.3 (S
Page 37 and 38:
1.4. Sparseness 29 R 3 R 3 A BSRA R
Page 39 and 40: 1.4. Sparseness 31 Sparse projectio
Page 41 and 42: 1.5. Machine learning for data prep
Page 51 and 52: 1.6. Applications to biomedical dat
Page 59 and 60: 1.7. Outlook 51 noise data signal i
Page 61: Part II Papers 53
Page 64 and 65: 56 Chapter 2. Neural Computation 16
Page 92 and 93: 84 Chapter 3. Signal Processing 84(
Page 98 and 99: 90 Chapter 4. Neurocomputing 64:223
Page 112 and 113: 104 Chapter 5. IEICE TF E87-A(9):23
Page 122 and 123: 114 Chapter 6. LNCS 3195:726-733, 2
Page 132 and 133: 124 Chapter 7. Proc. ISCAS 2005, pa
Page 138 and 139: 130 Chapter 8. Proc. NIPS 2006 Towa
Page 140 and 141:
132 Chapter 8. Proc. NIPS 2006 1.3
Page 142 and 143:
134 Chapter 8. Proc. NIPS 2006 stru
Page 144 and 145:
136 Chapter 8. Proc. NIPS 2006 5.5
Page 146 and 147:
138 Chapter 8. Proc. NIPS 2006
Page 148 and 149:
140 Chapter 9. Neurocomputing (in p
Page 150 and 151:
Page 152 and 153:
Page 154 and 155:
Page 156 and 157:
Page 158 and 159:
Page 160 and 161:
Page 162 and 163:
Page 164 and 165:
156 Chapter 10. IEEE TNN 16(4):992-
Page 166 and 167:
Page 168 and 169:
Page 170 and 171:
162 Chapter 11. EURASIP JASP, 2007
Page 172 and 173:
Page 174 and 175:
Page 176 and 177:
Page 178 and 179:
Page 180 and 181:
Page 182 and 183:
Page 184 and 185:
176 Chapter 12. LNCS 3195:718-725,
Page 186 and 187:
178 Chapter 12. LNCS 3195:718-725,
Page 188 and 189:
180 Chapter 12. LNCS 3195:718-725,
Page 190 and 191:
182 Chapter 12. LNCS 3195:718-725,
Page 192 and 193:
184 Chapter 12. LNCS 3195:718-725,
Page 194 and 195:
186 Chapter 13. Proc. EUSIPCO 2005
Page 196 and 197:
Page 198 and 199:
Page 200 and 201:
192 Chapter 14. Proc. ICASSP 2006 S
Page 202 and 203:
194 Chapter 14. Proc. ICASSP 2006 A
Page 204 and 205:
196 Chapter 14. Proc. ICASSP 2006
Page 206 and 207:
198 Chapter 15. Neurocomputing, 69:
Page 208 and 209:
Page 210 and 211:
Page 212 and 213:
Page 214 and 215:
Page 216 and 217:
Page 218 and 219:
Page 220 and 221:
Page 222 and 223:
Page 224 and 225:
Page 226 and 227:
Page 228 and 229:
Page 230 and 231:
Page 232 and 233:
Page 234 and 235:
Page 236 and 237:
Page 238 and 239:
230 Chapter 16. Proc. ICA 2006, pag
Page 240 and 241:
Page 242 and 243:
Page 244 and 245:
Page 246 and 247:
Page 248 and 249:
240 Chapter 17. IEEE SPL 13(2):96-9
Page 250 and 251:
Page 252 and 253:
Page 254 and 255:
Page 256 and 257:
Page 258 and 259:
Page 260 and 261:
252 Chapter 19. LNCS 3195:977-984,
Page 262 and 263:
254 Chapter 19. LNCS 3195:977-984,
Page 264 and 265:
256 Chapter 19. LNCS 3195:977-984,
Page 266 and 267:
258 Chapter 19. LNCS 3195:977-984,
Page 268 and 269:
260 Chapter 19. LNCS 3195:977-984,
Page 270 and 271:
262 Chapter 20. Signal Processing 8
Page 272 and 273:
Page 274 and 275:
Page 276 and 277:
Page 278 and 279:
Page 280 and 281:
Page 282 and 283:
Page 284 and 285:
Page 286 and 287:
Page 288 and 289:
Page 290 and 291:
Page 292 and 293:
Page 294 and 295:
Page 296 and 297:
Page 298 and 299:
Page 300 and 301:
292 Chapter 21. Proc. BIOMED 2005,
Page 302 and 303:
Page 304 and 305:
Page 306 and 307:
298 BIBLIOGRAPHY Barber, C., Dobkin
Page 308 and 309:
300 BIBLIOGRAPHY Georgiev, P. (2001
Page 310 and 311:
302 BIBLIOGRAPHY Keck, I., Theis, F
Page 312 and 313:
304 BIBLIOGRAPHY Schießl, I., Sch
Page 314 and 315:
306 BIBLIOGRAPHY Theis, F. and Inou
show all

Mathematics in Independent Component Analysis

Create successful ePaper yourself

Delete template?

Save as template?