Unsupervised feature extraction by time-contrastive learning and Nonlinear ICA

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 3772-3780

  Hyvärinen, Aapo ^a,b   Morioka, Hiroshi ^a  

^a UNIVERSITY OF HELSINKI   (Finland)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

FEATURE EXTRACTION; LINEAR TRANSFORMATIONS; MATHEMATICAL TRANSFORMATIONS;

IDENTIFIABILITY; NON-LINEAR ICA; NON-STATIONARITIES; NONLINEAR INDEPENDENT COMPONENT ANALYSIS; NONSTATIONARY; OPTIMAL DISCRIMINATIONS; TIME SEGMENTS; UNSUPERVISED FEATURE LEARNING;

INDEPENDENT COMPONENT ANALYSIS;

EID: 85019195046     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. L. B. Almeida. MISEP - linear and nonlinear ICA based on mutual information. J. of Machine Learning Research, 4: 1297-1318, 2003.
2. M. J. Brookes et al. Investigating the electrophysiological basis of resting state networks using magnetoen-cephalography. Proc. Natl. Acad. Sci., 108(40): 16783-16788, 2011.
3. P. Comon. Independent component analysis - a new concept? Signal Processing, 36: 287-314, 1994.
4. F. de Pasquale et al. A cortical core for dynamic integration of functional networks in the resting human brain. Neuron, 74(4): 753-764, 2012.
5. L. Dinh, D. Krueger, and Y. Bengio. NICE: Non-linear independent components estimation. arXiv: 1410.8516 [cs.LG], 2015.
6. P. Földiák. Learning invariance from transformation sequences. Neural Computation, 3: 194-200, 1991.
7. X. Glorot and Y. Bengio. Understanding the difficulty of training deep feedforward neural networks. In AISTATS'10, 2010.
8. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, pages 2672-2680, 2014.
9. R. Goroshin, J. Bruna, J. Tompson, D. Eigen, and Y. LeCun. Unsupervised feature learning from temporal data. arXiv: 1504.02518, 2015.
10. M. U. Gutmann, R. Dutta, S. Kaski, and J. Corander. Likelihood-free inference via classification. arXiv: 1407.4981 [stat.CO], 2014.
11. M. U. Gutmann and A. Hyvärinen. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. J. of Machine Learning Research, 13: 307-361, 2012.
12. S. Harmeling, A. Ziehe, M. Kawanabe, and K.-R. Müller. Kernel-based nonlinear blind source separation. Neural Comput., 15(5): 1089-1124, 2003.
13. G. E. Hinton. Learning multiple layers of representation. Trends Cogn. Sci., 11: 428-434, 2007.
14. G. E. Hinton and R. S. Zemel. Autoencoders, minimum description length, and helmholtz free energy. Adv. Neural Inf. Process. Syst., 1994.
15. A. Hyvärinen. Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw., 10(3): 626-634, 1999.
16. A. Hyvärinen. Blind source separation by nonstationarity of variance: A cumulant-based approach. IEEE Transactions on Neural Networks, 12(6): 1471-1474, 2001.
17. A. Hyvärinen, J. Hurri, and P. O. Hoyer. Natural Image Statistics. Springer-Verlag, 2009.
18. A. Hyvärinen and P. Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural Netw., 12(3): 429-439, 1999.
19. S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. CoRR, abs/1502.03167, 2015.
20. C. Jutten, M. Babaie-Zadeh, and J. Karhunen. Nonlinear mixtures. Handbook of Blind Source Separation, Independent Component Analysis and Applications, pages 549-592, 2010.
21. D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv: 1312.6114 [stat.ML], 2014.
22. K. Matsuoka, M. Ohya, and M. Kawamoto. A neural net for blind separation of nonstationary signals. Neural Netw., 8(3): 411-419, 1995.
23. H. Mobahi, R. Collobert, and J. Weston. Deep learning from temporal coherence in video. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 737-744, 2009.
24. D.-T. Pham and J.-F. Cardoso. Blind separation of instantaneous mixtures of non stationary sources. IEEE Trans. Signal Processing, 49(9): 1837-1848, 2001.
25. P. Ramkumar, L. Parkkonen, R. Hari, and A. Hyvärinen. Characterization of neuromagnetic brain rhythms over time scales of minutes using spatial independent component analysis. Hum. Brain Mapp., 33(7): 1648-1662, 2012.
26. H. Sprekeler, T. Zito, and L. Wiskott. An extension of slow feature analysis for nonlinear blind source separation. J. of Machine Learning Research, 15(1): 921-947, 2014.
27. J. T. Springenberg and M. Riedmiller. Learning temporal coherent features through life-time sparsity. In Neural Information Processing, pages 347-356. Springer, 2012.
28. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15(1): 1929-1958, 2014.
29. Y. Tan, J. Wang, and J.M. Zurada. Nonlinear blind source separation using a radial basis function network. IEEE Transactions on Neural Networks, 12(1): 124-134, 2001.
30. H. Valpola. From neural PCA to deep unsupervised learning. In Advances in Independent Component Analysis and Learning Machines, pages 143-171. Academic Press, 2015.
31. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res., 11: 3371-3408, 2010.
32. L. Wiskott and T. J. Sejnowski. Slow feature analysis: Unsupervised learning of invariances. Neural Comput., 14(4): 715-770, 2002.