Flexible and efficient implementations of Bayesian independent component analysis

Neurocomputing

Volumn 71, Issue 1-3, 2007, Pages 221-233

  Winther, Ole ^a   Petersen, Kaare Brandt ^a  

^a TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

Author keywords

Empirical Bayes; Independent component analysis; Mean field methods; Variational methods

Indexed keywords

ALGORITHMS; BAYESIAN NETWORKS; COVARIANCE MATRIX; OPTIMIZATION; PARAMETERIZATION; VARIATIONAL TECHNIQUES;

EMPIRICAL BAYES; EXPECTATION MAXIMIZATION ALGORITHM; MEAN FIELD METHODS;

INDEPENDENT COMPONENT ANALYSIS;

ALGORITHM; ARTICLE; BAYES THEOREM; COMPUTER PROGRAM; CONTROLLED STUDY; COVARIANCE; INDEPENDENT COMPONENT ANALYSIS; PERFORMANCE; PRIORITY JOURNAL; PROCESS OPTIMIZATION;

EID: 35648952111     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2007.01.007     Document Type: Article

References (27)

1. Attias H. A variational Bayesian framework for graphical models. In: Leen T., Dietterich T.G., and Tresp V. (Eds). Advances in Neural Information Processing Systems vol. 12 (2000), MIT Press, Cambridge 209-215
2. Bell A.J., and Sejnowski T.J. An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7 6 (1995) 1129-1159
3. O. Bermond, J.F. Cardoso, Approximate likelihood for noisy mixtures, in: Proceedings of the ICA Conference, 1999.
4. Girolami M. A variational method for learning sparse and overcomplete representations. Neural Comput. 13 (2001) 2517-2532
5. T. Heskes, M. Opper, W. Wiegerinck, O. Winther, O. Zoeter, Approximate inference techniques with expectation constraints, J. Stat. Mech. Theory Exp. (2005) P11015.
6. Hojen-Sorensen P., Winther O., and Hansen L.K. Mean-field approaches to independent component analysis. Neural Comput. 14 (2002) 889-918
7. Hyvärinen A. Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Networks 10 (1999) 626-634
8. Jordan M.I., Ghahramani Z., Jaakkola T.S., and Saul L.K. An introduction to variational methods for graphical models. Mach. Learn. 37 (1999) 183-233
9. Lee D.D., and Seung H.S. Algorithms for non-negative matrix factorization. In: Leen T.K., Dietterich T.G., and Tresp V. (Eds). Advances in Neural Information Processing Systems (NIPS) vol. 13 (2001) 556-562
10. MacKay D.J.C. Information Theory, Inference, and Learning Algorithms (2003), Cambridge University Press, Cambridge
11. Minka T. Expectation propagation for approximate Bayesian inference, Doctoral Dissertation (2001), MIT Media Lab
12. Miskin J.W., and MacKay D.J.C. Ensemble learning for blind source separation. In: Roberts S., and Everson R. (Eds). Independent Component Analysis: Principles and Practice (2001), Cambridge University Press, Cambridge
13. R.M. Neal, Geoffrey Hinton, A new view of the EM algorithm that justifies incremental and other variants, Technical Report, Department of Computer Science, University of Toronto, 1993.
14. H.B. Nielsen, UCMINF-an algorithm for unconstrained nonlinear optimization, Technical Report IMM-Rep-2000-19, Technical University of Denmark, 2000.
15. Olsson R.K., Petersen K.B., and Lehn-Schioler T. State-space models-from the EM algorithm to a gradient approach. Neural Comput. 19 4 (2007)
16. Opper M., and Winther O. Gaussian processes for classification: mean field algorithms. Neural Comput. 12 (2000) 2655-2684
17. Opper M., and Winther O. Adaptive and self-averaging Thouless-Anderson-Palmer mean field theory for probabilistic modeling. Phys. Rev. E 64 (2001) 056131
18. Opper M., and Winther O. Expectation consistent approximate inference. J. Mach. Learn. Res. 6 (2005) 2177-2204
19. K.B. Petersen, Mean field ICA, Ph.D. Thesis, Technical University of Denmark, 2005.
20. K.B. Petersen, O. Winther, Explaining slow convergence of EM in low noise linear mixtures, Technical Report 2005-2, Informatics and Mathematical Modelling, Technical University of Denmark, 2005.
21. K.B. Petersen, O. Winther, The EM algorithm in independent component analysis, in: International Conference on Acoustics, Speech, and Signal Processing, 2005.
22. Petersen K.B., Winther O., and Hansen L.K. On the slow convergence of EM and VBEM in low-noise linear models. Neural Comput. 17 9 (2005)
23. R. Salakhutdinov, S. Roweis, Adaptive overrelaxed bound optimization methods, in: Proceedings of International Conference on Machine Learning, ICML, International Conference on Machine Learning, ICML, 2003.
24. H. Valpola, Bayesian ensemble learning for nonlinear factor analysis, Ph.D. Thesis, Helsinki University of Technology, Espoo, 2000.
25. Vanduffel W., Fize D., Mandeville J.B., Nelissen K., Van Hecke P., Rosen B.R., Tootell R.B., and Orban G.A. Visual motion processing investigated using contrast agent-enhanced fMRI in awake behaving monkeys. Neuron 32 (2001) 565-577
26. O. Winther, K.B. Petersen, Bayesian independent component analysis: variational methods and non-negative decompositions, Digital Signal Process, 2007, doi:10.1016.j.dsp.2007.01.003.
27. Yedidia J.S., Freeman W.T., and Weiss Y. Generalized belief propagation. In: Leen T.K., Dietterich T.G., and Tresp V. (Eds). Advances in Neural Information Processing Systems (NIPS) vol. 13 (2000) 689-695