Separation theorem for independent subspace analysis and its consequences

Pattern Recognition

Volumn 45, Issue 4, 2012, Pages 1782-1791

  Szabó, Zoltán ^a   Póczos, Barnabás ^b   Lrincz, András ^a  

^a EÖTVÖS LORÁND UNIVERSITY   (Hungary)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Complex valued models; Controlled models; Independent subspace analysis; Linear systems; Nonparametric source dynamics; Partially observed systems; Post nonlinear systems; Separation principles

Indexed keywords

CONTROLLED MODELS; INDEPENDENT SUBSPACE ANALYSIS; NON-PARAMETRIC; OBSERVED SYSTEMS; POST-NONLINEAR; SEPARATION PRINCIPLES;

COMPUTER VISION; DYNAMICS; INDEPENDENT COMPONENT ANALYSIS; LINEAR SYSTEMS; SIGNAL PROCESSING;

SEPARATION;

EID: 83655191961     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2011.09.007     Document Type: Article

References (100)

1. C. Jutten, and J. Hérault Blind separation of sources: an adaptive algorithm based on neuromimetic architecture Signal Processing 24 1991 1 10 (Pubitemid 21679270)
2. J. Cardoso, and A. Souloumiac Blind beamforming for non-Gaussian signals IEE Proceedings F 140 6 1993 362 370
3. P. Comon Independent component analysis, a new concept? Signal Processing 36 3 1994 287 314
4. M.S. Bartlett, J.R. Movellan, and T.J. Sejnowski Face recognition by independent component analysis IEEE Transactions on Neural Networks 13 6 2002 1450 1464 (Pubitemid 35428531)
5. P.C. Yuen, and J.H. Lai Face representation using independent component analysis Pattern Recognition 35 6 2002 1247 1257 (Pubitemid 34262333)
6. H. Neemuchwalaa, A. Hero, and P. Carson Image matching using alpha-entropy measures and entropic graphs Signal Processing 85 2 2005 277 296 (Pubitemid 41536591)
7. M.J. McKeown, S. Makeig, G.G. Brown, T.-P. Jung, S.S. Kindermann, A.J. Bell, and T.J. Sejnowski Analysis of fMRI data by blind separation into independent spatial components Human Brain Mapping 6 1998 160 188 (Pubitemid 28247339)
8. A.J. Bell, and T.J. Sejnowski The 'independent components' of natural scenes are edge filters Vision Research 37 1997 3327 3338 (Pubitemid 27493806)
9. R. Jenssen, and T. Eltoft Independent component analysis for texture segmentation Pattern Recognition 36 13 2003 2301 2315 (Pubitemid 36947214)
10. A. Hyvärinen, and E. Oja Independent component analysis: algorithms and applications Neural Networks 13 4-5 2000 411 430 (Pubitemid 30447427)
11. A. Cichocki, and S. Amari Adaptive Blind Signal and Image Processing 2002 Wiley
12. A. Hyvärinen, J. Karhunen, and E. Oja Independent Component Analysis 2001 Wiley
13. S. Choi, A. Cichocki, H. Park, and S. Lee Blind source separation and independent component analysis Neural Information Processing: Letters and Reviews 6 2005 1 57
14. J. Cardoso, Multidimensional independent component analysis, in: ICASSP 1998, pp. 19411944.
15. A. Hyvärinen, and P.O. Hoyer Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces Neural Computation 12 2000 1705 1720
16. H. Kim, S. Choi, and S. Bang Membership scoring via independent feature subspace analysis for grouping co-expressed genes IJCNN 2003 1690 1695
17. A. Sharma, and K.K. Paliwal Subspace independent component analysis using vector kurtosis Pattern Recognition 39 2006 2227 2232 (Pubitemid 44175449)
18. F.J. Theis Blind signal separation into groups of dependent signals using joint block diagonalization ISCAS 2005 5878 5881
19. L.D. Lathauwer, B.D. Moor, and J. Vandewalle Fetal electrocardiogram extraction by blind source subspace separation IEEE Transactions on Biomedical Engineering 47 5 2000 567 572 (Pubitemid 30316973)
20. H. Stögbauer, A. Kraskov, S.A. Astakhov, and P. Grassberger Least dependent component analysis based on mutual information Physical Review E: Statistical, Nonlinear, and Soft Matter Physics 70 6 2004 066123
21. S. Ma, X.-L. Li, N. Correa, T. Adali, and V. Calhoun Independent subspace analysis with prior information for fMRI data ICASSP 2010 1922 1925
22. F. Kohl, G. Wübbeler, D. Kolossa, C. Elster, M. Bär, and R. Orglmeister Non-independent BSS: a model for evoked MEG signals with controllable dependencies ICA 2009 443 450
23. B.A. Olshausen, and D.J. Field Emergence of simple-cell receptive field properties by learning a sparse code for natural images Nature 381 1996 607 609 (Pubitemid 26177476)
24. Y. Nishimori, S. Akaho, and M.D. Plumbley Riemannian optimization method on the flag manifold for independent subspace analysis ICA 2006 295 302 (Pubitemid 43998428)
25. H. Choi, and S. Choi Relative gradient learning for independent subspace analysis IJCNN 2006 3919 3924 (Pubitemid 351369853)
26. C.S. Santos, J.E. Kögler, and E.D.M. Hernandez Using independent subspace analysis for selecting filters used in texture processing ICIP 2005 465 468 (Pubitemid 44538679)
27. Q. Le, W. Zou, S. Yeung, and A. Ng Stacked convolutional independent subspace analysis for action recognition CVPR 2011 3361 3368
28. A. Hyvärinen, P.O. Hoyer, and M. Inki Topographic independent component analysis Neural Computation 13 7 2001 1527 1558 (Pubitemid 33595031)
29. S.Z. Li, X. Lv, and H. Zhang View-subspace analysis of multi-view face patterns RATFG-RTS 2001 125 132
30. M.A. Casey, and A. Westner Separation of mixed audio sources by independent subspace analysis ICMC 2000 154 161
31. Z. Fan, J. Zhou, and Y. Wu Motion segmentation based on independent subspace analysis ACCV 2004
32. J.K. Kim, and S. Choi Tree-dependent components of gene expression data for clustering ICANN 2006 837 846 (Pubitemid 44561941)
33. Z. Szabó, B. Póczos, and A. Lrincz Undercomplete blind subspace deconvolution Journal of Machine Learning Research 8 2007 1063 1095 (Pubitemid 46804653)
34. A. Hyvärinen, Independent component analysis for time-dependent stochastic processes, in: ICANN 1998, pp. 541546.
35. M.S. Pedersen, J. Larsen, U. Kjems, and L.C. Parra A survey of convolutive blind source separation methods Springer Handbook of Speech Processing 2007 Springer
36. C. Jutten, and J. Karhunen Advances in blind source separation (BSS) and independent component analysis (ICA) for nonlinear systems International Journal of Neural Systems 14 5 2004 267 292
37. J. Anemüller, T.J. Sejnowski, and S. Makeig Complex independent component analysis of frequency-domain electroencephalographic data Neural Networks 16 2003 1311 1323 (Pubitemid 37386370)
38. V. Calhoun, T. Adali, G. Pearlson, P. van Zijl, and J. Pekar Independent component analysis of fMRI data in the complex domain Magnetic Resonance in Medicine 48 2002 180 192 (Pubitemid 34721601)
39. J. Anemüller, J. Duann, T.J. Sejnowski, and S. Makeig Spatio-temporal dynamics in fMRI recordings revealed with complex independent component analysis Neurocomputing 69 13-15 2006 1502 1512 (Pubitemid 43906067)
40. K. Chan, T.-W. Lee, and T.J. Sejnowski Variational Bayesian learning of ICA with missing data Neural Computation 15 8 2003 1991 2011 (Pubitemid 37035555)
41. A.T. Cemgil, C. Févotte, and S.J. Godsill Variational and stochastic inference for Bayesian source separation Digital Signal Processing 17 2007 891 913 (Pubitemid 47176909)
42. J. Anemüller Second-order separation of multidimensional sources with constrained mixing system ICA 2006 16 23 (Pubitemid 43998394)
43. T.M. Cover, and J.A. Thomas Elements of Information Theory 1991 Wiley New York, USA
44. L. Ma, and L. Zhang Bayesian estimation of overcomplete independent feature subspaces for natural images ICA 2007 746 753
45. A. Hyvärinen, and M. Inki Estimating overcomplete independent component bases for image windows Journal of Mathematical Imaging and Vision 17 2 2002 139 152 (Pubitemid 35427052)
46. H. Hotelling Analysis of a complex of statistical variables into principal components Journal of Educational Psychology 24 1933 417 441
47. B. Póczos, and A. Lrincz Independent subspace analysis using k-nearest neighborhood distances ICANN 2005 163 168
48. E.G. Learned-Miller, and J.W. Fisher III ICA using spacings estimates of entropy Journal of Machine Learning Research 4 2003 1271 1295
49. F.J. Theis Uniqueness of complex and multidimensional independent component analysis Signal Processing 84 5 2004 951 956
50. F.J. Theis Towards a general independent subspace analysis NIPS 2007 1361 1368
51. S. Amari, A. Cichocki, H.H. Yang, A new learning algorithm for blind signal separation, in: NIPS, 1996, pp. 757763.
52. Z. Szabó, and B. Póczos Nonparametric independent process analysis EUSIPCO 2011 1718 1722
53. K.-T. Fang, S. Kotz, and K.W. Ng Symmetric Multivariate and Related Distributions 1990 Chapman and Hall
54. p-norm spherical distribution Journal of Statistical Planning and Inference 60 2 1997 241 260
55. S. Takano The inequalities of Fisher information and entropy power for dependent variables Proceedings of the Seventh Japan-Russia Symposium on Probability Theory and Mathematical Statistics 1995
56. K. Abed-Meraim, and A. Belouchrani Algorithms for joint block diagonalization EUSIPCO 2004 209 212
57. F.J. Theis Multidimensional independent component analysis using characteristic functions EUSIPCO 2005
58. K. Nordhausen, and H. Oja Independent subspace analysis using three scatter matrices Austrian Journal of Statistics 40 1-2 2011 93 101
59. B. Póczos, B. Takács, and A. Lrincz Independent subspace analysis on innovations ECML 2005 698 706
60. R.H. Lambert, Multichannel Blind Deconvolution: FIR Matrix Algebra and Separation of Multipath Mixtures, Ph.D. Thesis, University of Southern California, 1996.
61. V. Petrov Central limit theorem for m-dependent variables Proceedings of the All-Union Conference on Probability Theory and Mathematical Statistics 1958 38 44
62. A. Neumaier, and T. Schneider Estimation of parameters and eigenmodes of multivariate autoregressive models ACM Transactions on Mathematical Software 27 1 2001 27 57 (Pubitemid 33609114)
63. T. Schneider, and A. Neumaier Algorithm 808: ARfit - a Matlab package for the estimation of parameters and eigenmodes of multivariate autoregressive models ACM Transactions on Mathematical Software 27 1 2001 58 65 (Pubitemid 33609115)
64. S. Choi Differential learning algorithms for decorrelation and independent component analysis Neural Networks 19 10 2006 1558 1567 (Pubitemid 44793169)
65. R. Rajagopal, and L.C. Potter Multivariate MIMO FIR inverses IEEE Transactions on Image Processing 12 2003 458 465
66. Z. Szabó, B. Póczos, and A. Lrincz Undercomplete blind subspace deconvolution via linear prediction ECML 2007 740 747
67. B. Póczos, Z. Szabó, M. Kiszlinger, and A. Lrincz Independent process analysis without a priori dimensional information ICA 2007 252 259
68. Z. Szabó Complete blind subspace deconvolution ICA 2009 138 145
69. S. Choi, and A. Cichocki Blind signal deconvolution by spatio-temporal decorrelation and demixing Neural Networks for Signal Processing 7 1997 426 435
70. A. Mansour, C. Jutten, P. Loubaton, Subspace method for blind separation of sources in convolutive mixture, in: EUSIPCO 1996, pp. 20812084.
71. S. Icart, R. Gautier, Blind separation of convolutive mixtures using second and fourth order moments, in: ICASSP 1996, pp. 30183021.
72. A. Gorokhov, P. Loubaton, Multiple-input multiple-output ARMA systems: second order blind identification for signal extractions, in: SSAP 1996, pp. 348351.
73. Z. Szabó, B. Póczos, G. Szirtes, and A. Lrincz Post nonlinear independent subspace analysis ICANN 2007 677 686
74. A. Taleb, and C. Jutten Source separation in post-nonlinear mixtures IEEE Transactions on Signal Processing 10 47 1999 2807 2820 (Pubitemid 30514923)
75. A. Ziehe, M. Kawanabe, S. Harmeling, and K.-R. Müller Blind separation of post-nonlinear mixtures using linearizing transformations and temporal decorrelation Journal of Machine Learning Research 4 7-8 2003 1319 1338
76. J. Solé-Casals, C. Jutten, and D. Pham Fast approximation of nonlinearities for improving inversion algorithms of PNL mixtures and Wiener systems Signal Processing 85 2005 1780 1786 (Pubitemid 40940839)
77. J. Eriksson Complex random vectors and ICA models: identifiability, uniqueness and separability IEEE Transactions on Information Theory 52 3 2006 1017 1029 (Pubitemid 46444889)
78. Z. Szabó, and A. Lrincz Real and complex independent subspace analysis by generalized variance ICARN 2006 85 88
79. Y. Nishimori, S. Akaho, and M.D. Plumbley Natural conjugate gradient on complex flag manifolds for complex independent subspace analysis ICANN 2008 165 174
80. P. Krishnaiah, and J. Lin Complex elliptically symmetric distributions Communications in Statistics 15 12 1986 3693 3718
81. Z. Szabó, and A. Lrincz Towards independent subspace analysis in controlled dynamical systems ICARN 2008 9 12
82. Z. Szabó Autoregressive independent process analysis with missing observations ESANN 2010 159 164
83. H.-M. Krolzig Markov Switching Vector Autoregressions. Modelling, Statistical Inference and Application to Business Cycle Analysis 1997 Springer
84. B. Póczos, and A. Lrincz Identification of recurrent neural networks by Bayesian interrogation techniques Journal of Machine Learning Research 10 2009 515 554
85. J.T. Lomba Estimation of dynamic econometric models with errors in variables Lecture Notes in Economics and Mathematical Systems vol. 339 1990 Springer
86. A. García-Hiernaux, J. Casals, and M. Jerez Fast estimation methods for time series models in state-space form Journal of Statistical Computation and Simulation 79 2 2009 121 134
87. K.R. Kadiyala, and S. Karlsson Numerical methods for estimation and inference in Bayesian VAR-models Journal of Applied Economics 12 1997 99 132
88. D. Bosq Nonparametric statistics for stochastic processes: estimation and prediction Lecture Notes in Statistics 1998 Springer
89. N. Hilgert, B. Portier, Strong Uniform Consistency and Asymptotic Normality of a Kernel Based Error Density Estimator in Functional Autoregressive Models, Technical Report, 2009 〈 http://arxiv.org/abs/0905.2327v1 〉.
90. Z. Szabó, B. Póczos, and A. Lrincz Cross-entropy optimization for independent process analysis ICA 2006 909 916 (Pubitemid 43998504)
91. R.Y. Rubinstein, and D.P. Kroese The Cross-Entropy Method 2004 Springer
92. G.H. Hardy, and S.I. Ramanujan Asymptotic formulae in combinatory analysis Proceedings of the London Mathematical Society 17 1 1918 75 115
93. J.V. Uspensky Asymptotic formulae for numerical functions which occur in the theory of partitions Bulletin of the Russian Academy of Sciences 14 6 1920 199 218
94. B. Póczos, and A. Lrincz Non-combinatorial estimation of independent autoregressive sources Neurocomputing Letters 69 2006 2416 2419 (Pubitemid 44340616)
95. Z. Szabó, B. Póczos, and A. Lrincz Auto-regressive independent process analysis without combinatorial efforts Pattern Analysis & Applications 13 2010 1 13
96. Z. Szabó, and A. Lrincz Independent subspace analysis can cope with the curse of dimensionality Acta Cybernetica (Symp. Intell. Syst. 2006) 18 2007 213 221
97. P. Gruber, H.W. Gutch, and F.J. Theis Hierarchical extraction of independent subspaces of unknown dimensions ICA 2009 259 266
98. F.R. Bach, and M.I. Jordan Beyond independent components: trees and clusters Journal of Machine Learning Research 4 2003 1205 1233
99. D. Yan, L. Huang, and M.I. Jordan Fast approximate spectral clustering KDD 2009 907 916
100. B. Póczos, and A. Lrincz Independent subspace analysis using geodesic spanning trees ICML 2005 673 680 (Pubitemid 43183392)