Independent component analysis: Recent advances

Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences

Volumn 371, Issue 1984, 2013, Pages

  Hyvärinen, Aapo ^a  

^a UNIVERSITY OF HELSINKI   (Finland)

Author keywords

Blind source separation; Causal analysis; Independent component analysis; Non Gaussianity

Indexed keywords

BLIND SOURCE SEPARATION; MATHEMATICAL TRANSFORMATIONS;

BASIC MODELS; BASIC THEORY; CAUSAL ANALYSIS; CAUSAL RELATIONS; CLASSICAL METHODS; INDEPENDENT COMPONENTS; LINEAR TRANSFORM; MULTIPLE DATA SETS; MULTIVARIATE STATISTICAL METHOD; NON-GAUSSIAN; NONGAUSSIANITY; NONNORMAL; PROBABILISTIC METHODS; RANDOM VECTORS; THREE WAY DATA; UNDERLYING COMPONENTS;

INDEPENDENT COMPONENT ANALYSIS;

EID: 84874119249     PISSN: 1364503X     EISSN: None     Source Type: Journal    
DOI: 10.1098/rsta.2011.0534     Document Type: Review

References (93)

1. Jutten C, Hérault J. 1991 Blind separation of sources, part I: an adaptive algorithm based on neuromimetic architecture. Signal Process. 24, 1-10. (doi:10.1016/0165-1684(91)90079-X)
2. Hyvärinen A, Oja E. 2000 Independent component analysis: algorithms and applications. Neural Netw. 13, 411-430. (doi:10.1016/S0893-6080(00)00026-5)
3. Hyvärinen A, Karhunen J, Oja E. 2001 Independent component analysis. London, UK: Wiley Interscience.
4. Comon P, Jutten C. 2010 Handbook of blind source separation. New York, NY: Academic Press.
5. Comon P. 1994 Independent component analysis: a new concept? Signal Process. 36, 287-314.(doi:10.1016/0165-1684(94)90029-9)
6. Eriksson J, Koivunen V. 2004 Identifiability, separability, uniqueness of linear ICA models. IEEE Signal Process. Lett. 11, 601-604. (doi:10.1109/LSP. 2004.830118)
7. Pham D-T, Garrat P. 1997 Blind separation of mixture of independent sources through a quasi-maximum likelihood approach. IEEE Trans. Signal Process. 45, 1712-1725. (doi:10.1109/ 78.599941)
8. Bell AJ, Sejnowski TJ. 1995 An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7, 1129-1159. (doi:10.1162/neco.1995.7.6.1129)
9. Nadal J-P, Parga N. 1994 Non-linear neurons in the low noise limit: a factorial codemaximizes information transfer. Network 5, 565-581. (doi:10.1088/0954-898X/5/4/008)
10. Cardoso J-F. 1997 Infomax and maximum likelihood for source separation. IEEE Lett. Signal Process. 4, 112-114. (doi:10.1109/97.566704)
11. Hyvärinen A. 1999 Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 10, 626-634. (doi:10.1109/72.761722)
12. Amari S-I, Cichocki A, Yang H. 1996 A new learning algorithm for blind source separation. In Advances in neural information processing systems, vol. 8 (eds DS Touretzky, MC Mozer, ME Hasselmo), pp. 757-763. Cambridge, MA: MIT Press.
13. Hyvärinen A. 2010 Pairwise measures of causal direction in linear non-gaussian acyclic models. In Proc. Asian Conf. Machine Learning, Tokyo, Japan, vol. 13, pp. 1-16.
14. Hyvärinen A, Smith SM. Submitted. Pairwise likelihood ratios for estimation of non-Gaussian structural equation models.
15. Dodge Y, Rousson V. 2001 On asymmetric properties of the correlation coefficient in the regression setting. Am. Stat. 55, 51-54. (doi:10.1198/ 000313001300339932)
16. Shimizu S, Hoyer PO, Hyvärinen A, Kerminen A. 2006 A linear non-Gaussian acyclic model for causal discovery. J. Mach. Learn. Res. 7, 2003-2030.
17. Lacerda G, Spirtes P, Ramsey J, Hoyer PO. 2008 Discovering cyclic causal models by independent components analysis. In Proc. 24th Conf. Uncertainty in Artificial Intelligence (UAI2008), Helsinki, Finland.
18. Shimizu S, Inazumi T, Sogawa Y, Hyvärinen A, Kawahara Y, Washio T, Hoyer PO, Bollen K. 2011 DirectLiNGAM: a direct method for learning a linear non-Gaussian structural equation model. J. Mach. Learn. Res. 12, 1225-1248.
19. Hyvärinen A, Zhang K, Shimizu S, Hoyer PO. 2010 Estimation of a structural vector autoregression model using non-Gaussianity. J. Mach. Learn. Res. 11, 1709-1731.
20. Ramsey JD, Hanson SJ, Glymour C. 2011 Multi-subject search correctly identifies causal connections and most causal directions in the DCM models of the Smith et al. simulation study. NeuroImage 58, 838-848. (doi:10.1016/j. neuroimage.2011.06.068)
21. Shimizu S. 2012 Joint estimation of linear non-Gaussian acyclic models. Neurocomputing 81, 104-107. (doi:10.1016/j.neucom.2011.11.005)
22. Hoyer PO, Janzing D, Mooij J, Peters J, Schölkopf B. 2009 Nonlinear causal discovery with additive noisemodels. In Advances in neural information processing systems, vol. 21, pp. 689-696. Cambridge, MA: MIT Press.
23. Zhang K, Hyvärinen A. 2009 On the identifiability of the post-nonlinear causalmodel. In Proc. 25th Conf. on Uncertainty in Artificial Intelligence (UAI2009), Montréal, Canada, pp. 647-655.
24. Himberg J, Hyvärinen A, Esposito F. 2004 Validating the independent components of neuroimaging time-series via clustering and visualization. NeuroImage 22, 1214-1222. (doi:10.1016/j.neuroimage.2004.03.027)
25. Meinecke F, Ziehe A, Kawanabe M, Müller K-R. 2002 A resampling approach to estimate the stability of one-dimensional or multidimensional independent components. IEEE Trans. Biomed. Eng. 49, 1514-1525. (doi:10.1109/TBME.2002.805480)
26. Hyvärinen A. 2011 Testing the ICA mixing matrix based on inter-subject or inter-session consistency. NeuroImage 58, 122-136. (doi:10.1016/j.neuroimage.2011.05.086)
27. Esposito F, Scarabino T, Hyvärinen A, Himberg J, Formisano E, Comani S, Tedeschi G, Goebel R, Seifritz E, Salle FD. 2005 Independent component analysis of fMRI group studies by selforganizing clustering. NeuroImage 25, 193-205. (doi:10.1016/j.neuroimage.2004.10.042)
28. Hyvärinen A, Ramkumar P. Submitted. Testing independent components by inter-subject or inter-session consistency.
29. Calhoun VD, Liu J, Adali T. 2009 A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, ERP data. NeuroImage 45, S163-S172. (doi:10.1016/j.neuroimage. 2008.10.057)
30. Varoquaux G, Gramfort A, Pedregosa F, Michel V, Thirion B. 2011 Multi-subject dictionary learning to segment an atlas of brain spontaneous activity. In Information processing in medical imaging, pp. 562-573. Lecture Notes in Computer Science. Germany: Springer.
31. Hyvärinen A, Smith SM. 2012 Computationally efficient group ICA for large groups. In Human Brain Mapping Meeting, Beijing, China, 10-14 June 2012.
32. Harshman RA. 1970 Foundations of the PARAFAC procedure: models and conditions for an explanatory multimodal factor analysis. UCLA Working Papers Phonetics 16, 1-84.
33. Beckmann CF, Smith SM. 2005 Tensorial extensions of independent component analysis for group fMRI data analysis. NeuroImage 25, 294-311. (doi:10.1016/j.neuroimage.2004.10.043)
34. Belouchrani A, Meraim KA, Cardoso J-F, Moulines E. 1997 A blind source separation technique based on second order statistics. IEEE Trans. Signal Process. 45, 434-444. (doi:10.1109/78.554307)
35. Pham D-T, Cardoso J-F. 2001 Blind separation of instantaneous mixtures of non stationary sources. IEEE Trans. Signal Process. 49, 1837-1848. (doi:10.1109/78.942614)
36. Yeredor A. 2010 Second-order methods based on color. In Handbook of blind source separation (eds P Comon, C Jutten), pp. 227-279. New York, NY: Academic Press.
37. Cardoso J-F, Le Jeune M, Delabrouille J, BetouleM, Patanchon G. 2008 Component separation with flexible models: application to multichannel astrophysical observations. IEEE J. Sel. Top. Signal Process. 2, 735-746. (doi:10.1109/JSTSP.2008.2005346)
38. Hyvärinen A, Hurri J, Hoyer PO. 2009 Natural image statistics. Berlin, Germany: Springer.
39. Hyvärinen A, Hoyer PO. 2000 Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces. Neural Comput. 12, 1705-1720. (doi:10.1162/089976600300015312)
40. Hyvärinen A, Hoyer PO, Inki M. 2001 Topographic independent component analysis. Neural Comput. 13, 1527-1558. (doi:10.1162/ 089976601750264992)
41. Hyvärinen A, Köster U. 2006 FastISA: a fast fixed-point algorithm for independent subspace analysis. In Proc. Eur. Symp. Artificial Neural Networks, Bruges, Belgium.
42. Gruber P, Gutch HW, Theis FJ. 2009 Hierarchical extraction of independent subspaces of unknown dimensions. In Proc. Int. Conf. on Independent Component Analysis and Blind Signal Separation (ICA2009), Paraty, Brazil, pp. 259-266.
43. Karklin Y, Lewicki MS. 2005 A hierarchical Bayesian model for learning nonlinear statistical regularities in natural signals. Neural Comput. 17, 397-423. (doi:10.1162/0899766053011474)
44. Karklin Y, Lewicki MS. 2009 Emergence of complex cell properties by learning to generalize in natural scenes. Nature 457, 83-86. (doi:10.1038/nature07481)
45. Ranzato M, Krizhevsky A, Hinton GE. 2010 Factored 3-way restricted Boltzmann machines for modeling natural images. In Proc. 13th Int. Conf. on Artificial Intelligence and Statistics (AISTATS2010), Sardinia, Italy, 13-15 May 2010.
46. Osindero S,WellingM, Hinton GE. 2006 Topographic productmodels applied to natural scene statistics. Neural Comput. 18, 381-414. (doi:10.1162/ 089976606775093936)
47. Köster U, Hyvärinen A. 2010 A two-layer model of natural stimuli estimated with score matching. Neural Comput. 22, 2308-2333. (doi:10.1162/NECO-a-00010)
48. Hinton GE. 2002 Training products of experts by minimizing contrastive divergence. Neural Comput. 14, 1771-1800. (doi:10.1162/089976602760128018)
49. Hyvärinen A. 2005 Estimation of non-normalized statistical models using score matching. J. Mach. Learn. Res. 6, 695-709.
50. Gutmann MU, Hyvärinen A. 2012 Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. J. Mach. Learn. Res. 13, 307-361.
51. Hyvärinen A, Hurri J. 2004 Blind separation of sources that have spatiotemporal variance dependencies. Signal Process. 84, 247-254. (doi:10.1016/j.sigpro.2003.10.010)
52. Kawanabe M, Müller K-R. 2005 Estimating functions for blind separation when sources have variance dependencies. J. Mach. Learn. Res. 6, 453-482.
53. Hyvärinen A. 1998 Independent component analysis for time-dependent stochastic processes. In Proc. Int. Conf. on Artificial Neural Networks (ICANN'98), Skövde, Sweden, pp. 135-140.
54. Gómez-Herrero G, Atienza M, Egiazarian K, Cantero J. 2008 Measuring directional coupling between EEG sources. NeuroImage 43, 497-508. (doi:10.1016/j.neuroimage.2008.07.032)
55. Zhang K, Hyvärinen A. 2011 A general linear non-Gaussian state-space model: identifiability, identification, applications. In Proc. Asian Conf. on Machine Learning, Tokyo, Japan, pp. 113-128.
56. Haufe S, Tomioka R, Nolte G, Müller K-R, Kawanabe M. 2010 Modeling sparse connectivity between underlying brain sources for EEG/MEG. IEEE Trans. Biomed. Eng. 57, 1954-1963. (doi:10.1109TBME.2010.2046325)
57. (doi:10.1109/TBME.2010.2046325) 57. Zhang K, Chan L. 2006 An adaptive method for subband decomposition ICA. Neural Comput. 18, 191-223. (doi:10.1162/089976606774841620)
58. Lahat D, Cardoso J-F, Messer H. 2012 Joint block diagonalization algorithms for optimal separation of multidimensional components. In Latent variable analysis and signal separation, vol. 7191, pp. 155-162. Berlin, Germany: Springer.
59. Sasaki H, Gutmann MU, Shouno H, Hyvärinen A. 2012 Topographic analysis of correlated components. In Proc. Asian Conf. on Machine Learning, Singapore.
60. Bach FR, Jordan MI. 2003 Beyond independent components: trees and clusters. J. Mach. Learn. Res. 4, 1205-1233.
61. Zoran D,Weiss Y. 2010 The 'tree-dependent components' of natural images are edge filters. In Advances in neural information processing systems, vol. 22. Cambridge, MA: MIT Press.
62. Hinton GE. 2007 Learning multiple layers of representation. Trends Cogn. Sci. 11, 428-434. (doi:10.1016j.tics.2007.09.004)
63. Hinton GE, Salakhutdinov RR. 2006 Reducing the dimensionality of data with neural networks. Science 313, 504-507. (doi:10.1126/science.1127647)
64. Lee H, Grosse R, Ranganath R, Ng AY. 2011 Unsupervised learning of hierarchical representations with convolutional deep belief networks. Commun. ACM 54, 95-103. (doi:10.11452001269.2001295)
65. Tong L, Liu R-W, Soon VC, Huang Y-F. 1991 Indeterminacy and identifiability of blind identification. IEEE Trans. Circuits Syst. 38, 499-509. (doi:10.1109/31.76486)
66. Hyvärinen A. 2001 Complexity pursuit: separating interesting components from time-series. Neural Comput. 13, 883-898. (doi:10.1162/ 089976601300014394)
67. Hyvärinen A. 2005 A unifying model for blind separation of independent sources. Signal Process. 85, 1419-1427. (doi:10.1016/j.sigpro.2005. 02.003)
68. Pham D-T. 2002 Exploiting source non-stationary and coloration in blind source separation. In Proc. Int. Conf. on Digital Signal Processing (dSP2002), pp. 151-154. IEEE.
69. Beale EML, Mallows CL. 1959 Scale mixing of symmetric distributions with zero means. Ann. Math. Stat. 30, 1145-1151. (doi:10.1214/aoms/1177706099)
70. Zibulevsky M, Pearlmutter BA. 2001 Blind source separation by sparse decomposition in a signal dictionary. Neural Comput. 13, 863-882. (doi:10.1162/089976601300014385)
71. Pham D-T, Cardoso J-F. 2003 Source adaptive blind source separation: Gaussian models and sparsity. In SPIE Conf., Wavelets: Applications in Signal and Image Processing. SPIE.
72. Kisilev P, Zibulevsky M, Zeevi YY. 2003 A multiscale framework for blind separation of linearly mixed signals. J. Mach. Learn. Res. 4, 1339-1363.
73. Gribonval R, Zibulevsky M. 2010 Sparse component analysis. In Handbook of blind source separation (eds P Comon, C Jutten). New York, NY: Academic Press.
74. Hyvärinen A, Ramkumar P, Parkkonen L, Hari R. 2010 Independent component analysis of short-time Fourier transforms for spontaneous EEG/MEG analysis. NeuroImage 49, 257-271.(doi:10.1016/j.neuroimage.2009.08.028)
75. Eriksson J, Koivunen V. 2006 Complex random vectors and ICA models: identifiability, uniqueness, separability. IEEE Trans. Inf. Theory 52, 1017-1029. (doi:10.1109/TIT.2005.864440)
76. Bingham E, Hyvärinen A. 2000 A fast fixed-point algorithm for independent component analysis of complex valued signals. Int. J. Neural Syst. 10, 1-8. (doi:10.1142/ SO129065700000028)
77. Hyvärinen A. 1999 The fixed-point algorithm and maximum likelihood estimation for independent component analysis. Neural Process. Lett. 10, 1-5. (doi:10.1023/A:1018647011077)
78. Chen A, Bickel PJ. 2006 Efficient independent component analysis. Ann. Stat. 34, 2824-2855.(doi:10.1214009053606000000939)
79. Vlassis N, Motomura Y. 2001 Efficient source adaptivity in independent component analysis. IEEE Trans. Neural Netw. 12, 559-566. (doi:10.1109/72. 925558)
80. Hastie T, Tibshirani R. 2003 Independent components analysis through product density estimation. In Advances in neural information processing 15 (Proc. NIPS.*2002). Cambridge. MA: MIT Press.
81. Learned-Miller EG, Fisher JW. 2003 ICA using spacings estimates of entropy. J. Mach. Learn. Res. 4, 1271-1295.
82. Bach FR, JordanMI. 2002 Kernel independent component analysis. J. Mach. Learn. Res. 3, 1-48.
83. Gretton A, Fukumizu K, Teo C-H, Song L, Schölkopf B, Smola A. 2008 A kernel statistical test of independence. In Advances in neural information processing systems, vol. 20. Cambridge, MA: MIT Press.
84. Stögbauer H, Kraskov A, Astakhov SA, Grassberger P. 2004 Least-dependent-component analysis based on mutual information. Phys. Rev. E 70, 066123. (doi:10.1103/PhysRevE.70.
85. Paatero P, Tapper U. 1994 Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5, 111-126. (doi:10.1002/ env.3170050203)
86. Paatero P. 1997 Least squares formulation of robust non-negative factor analysis. Chemometrics Intell. Lab. Syst. 37, 23-35. (doi:10.1016/S0169-7439(96) 00044-5)
87. Lee DD, Seung HS. 1999 Learning the parts of objects by non-negative matrix factorization. Nature 401, 788-791. (doi:10.1038/44565)
88. Hoyer PO, Hyvärinen A. 2002 A multi-layer sparse coding network learns contour coding from natural images. Vision Res. 42, 1593-1605. (doi:10.1016/S0042-6989(02)00017-2)
89. Cichocki A, Zdunek R, Phan A-H, Amari S-I. 2009 Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis. London, UK:Wiley.
90. Donoho DL, Stodden V. 2004 When does non-negative matrix factorization give a correct decomposition into parts? In Advances in neural information processing 16 (Proc. NIPS*2003). Cambridge, MA: MIT Press.
91. Hoyer PO. 2004 Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5, 1457-1469.
92. Plumbley MD 2003 Algorithms for non-negative independent component analysis. IEEE Trans. Neural Netw. 14, 534-543. (doi:10.1109/TNN.2003.810616)
93. Plumbley MD, Cicocki A, Bro R. 2010 Non-negative mixtures. In Handbook of blind source separation (eds P Comon, C Jutten). New York, NY: Academic Press.