Variational and stochastic inference for Bayesian source separation

Digital Signal Processing: A Review Journal

Volumn 17, Issue 5, 2007, Pages 891-913

  Cemgil, A Taylan ^a   Févotte, Cédric ^b   Godsill, Simon J ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b TELECOM PARISTECH   (France)

Author keywords

Gibbs sampler; Markov chain Monte Carlo; Source separation; Variational Bayes

Indexed keywords

COMPUTATIONAL METHODS; COMPUTER SIMULATION; GAUSSIAN NOISE (ELECTRONIC); MARKOV PROCESSES; MESSAGE PASSING; MONTE CARLO METHODS; STOCHASTIC MODELS;

BAYESIAN SOURCE SEPARATION; GIBBS SAMPLER; STATE SPACES; STOCHASTIC INFERENCE;

BLIND SOURCE SEPARATION;

EID: 34547454757     PISSN: 10512004     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.dsp.2007.03.008     Document Type: Article

References (49)

1. A. Mohammad-Djafari, A Bayesian estimation method for detection, localisation and estimation of superposed sources in remote sensing, in: SPIE'97, San Diego, July 1997
2. A. Mohammad-Djafari, A Bayesian approach to source separation, in: Proc. 19th International Workshop on Bayesian Inference and Maximum Entropy Methods (MaxEnt99), Boise, USA, August 1999
3. K.H. Knuth, Bayesian source separation and localization, in: SPIE'98: Bayesian Inference for Inverse Problems, San Diego, July 1998, pp. 147-158
4. K.H. Knuth, A Bayesian approach to source separation, in: Proc. 1st International Workshop on Independent Component Analysis and Signal Separation, Aussois, France, January 1999, pp. 283-288
5. K.H. Knuth, H.G. Vaughan, Convergent Bayesian formulations of blind source separation and electromagnetic source estimation, in: Maximum Entropy and Bayesian Methods (MaxEnt), Munich, 1998, pp. 217-226
6. Rowe D.B. A Bayesian approach to blind source separation. J. Interdisciplin. Math. 5 1 (2002) 49-76
7. Rowe D.B. Multivariate Bayesian Statistics: Models for Source Separation and Signal Unmixing (2003), Chapman & Hall, New York
8. McKay D.J.C. Information Theory, Inference and Learning Algorithms (2003), Cambridge Univ. Press, Cambridge
9. Miskin J., and Mackay D. Ensemble learning for blind source separation. In: Roberts S.J., and Everson R.M. (Eds). Independent Component Analysis (2001), Cambridge Univ. Press, Cambridge 209-233
10. Hyvärinen A., Karhunen J., and Oja E. Independent Component Analysis (2001), Wiley, New York
11. Cichocki A., and Amari S.I. Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications (2003), Wiley, New York
12. C. Févotte, S.J. Godsill, P.J. Wolfe, Bayesian approach for blind separation of underdetermined mixtures of sparse sources, in: Proc. 5th International Conference on Independent Component Analysis and Blind Source Separation (ICA 2004), Granada, Spain, 2004, pp. 398-405
13. C. Févotte, S.J. Godsill, A Bayesian approach for blind separation of sparse sources, IEEE Trans. Speech and Audio Processing, in press, available at http://persos.mist-technologies.com/~cfevotte/
14. Zibulevsky M., Pearlmutter B.A., Bofill P., and Kisilev P. Blind source separation by sparse decomposition. In: Roberts S.J., and Everson R.M. (Eds). Independent Component Analysis: Principles and Practice (2001), Cambridge Univ. Press, Cambridge
15. A. Jourjine, S. Rickard, O. Yilmaz, Blind separation of disjoint orthogonal signals: Demixing n sources from 2 mixtures, in: Proc. ICASSP-5, Istanbul, Turkey, June 2000, pp. 2985-2988
16. Ghahramani Z., and Beal M. Propagation algorithms for variational Bayesian learning. Neural Inform. Process. Syst. 13 (2000)
17. M. Wainwright, M.I. Jordan, Graphical models, exponential families, and variational inference, Technical Report 649, Department of Statistics, UC Berkeley, September 2003
18. Bishop C.M. Pattern Recognition and Machine Learning (2006), Springer-Verlag, New York
19. Attias H. Independent factor analysis. Neural Comput. 11 4 (1999) 803-851
20. H. Lappalainen, Ensemble learning for independent component analysis, in: Proceedings of Int. Workshop on Independent Component Analysis and Signal Separation (ICA'99), Aussois, France, 1999, pp. 7-12
21. H. Valpola, Nonlinear independent component analysis using ensemble learning: Theory, in: Proceedings of the Second International Workshop on Independent Component Analysis and Blind Signal Separation, ICA 2000, Helsinki, Finland, 2000, pp. 251-256
22. Girolami M. A variational method for learning sparse and overcomplete representations. Neural Comput. 13 11 (2001) 2517-2532
23. Hojen-Sorensen P., Winther O., and Hansen L.K. Mean-field approaches to independent component analysis. Neural Comput. 14 (2002) 889-918
24. Chan K., Lee T.W., and Sejnowski T.J. Variational Bayesian learning of ica with missing data. Neural Comput. 15 (2003) 1991-2011
25. O. Winther, K.B. Petersen, Flexible and efficient implementations of Bayesian independent component analysis, Neurocomputing, 2006, submitted for publication
26. Kschischang F.R., Frey B.J., and Loeliger H.-A. Factor graphs and the sum-product algorithm. IEEE Trans. Inform. Theory 47 2 (2001) 498-519
27. A.T. Cemgil, C. Fevotte, S.J. Godsill, Blind separation of sparse sources using variational EM, in: 13th European Signal Processing Conference, Antalya, Turkey, 2005. EURASIP. URL http://www-sigproc.eng.cam.ac.uk/~cf269/eusipco05/sound_files.html
28. S. Moussaoui, D. Brie, A. Mohammad-Djafari, C. Carteret, Separation of non-negative mixture of non-negative sources using a Bayesian approach and MCMC sampling, IEEE Trans. Signal Process., in press
29. In: Gilks W.R., Richardson S., and Spiegelhalter D.J. (Eds). Markov Chain Monte Carlo in Practice (1996), CRC Press, London
30. Metropolis N., Rosenbluth A., Rosenbluth M., Teller A., and Teller E. Equations of state calculations by fast computing machines. J. Chem. Phys. 21 (1953) 1087-1091
31. Hastings W.K. Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57 (1970) 97-109
32. Roberts G.O., and Rosenthal J.S. Markov chain Monte Carlo: Some practical implications of theoretical results. Can. J. Statist. 26 (1998) 5-31
33. W. Wiegerinck, Variational approximations between mean field theory and the junction tree algorithm, in: UAI (16th conference), 2000, pp. 626-633
34. Cover T.M., and Thomas J.A. Elements of Information Theory (1991), Wiley, New York
35. Neal R.M., and Hinton G.E. A view of the EM algorithm that justifies incremental, sparse, and other variants. Learning in Graphical Models (1999), MIT Press, Cambridge, MA. 0-262-60032-3 355-368
36. Winn J., and Bishop C. Variational message passing. J. Machine Learn. Res. 6 (2005) 661-694
37. Ilin A., and Valpola H. On the effect of the form of the posterior approximation in variational learning of ica models. Neural Process. Lett. 22 2 (2005)
38. D. Barber, W. Wiegerinck, Tractable variational structures for approximating graphical models, in: M. Kearns, S. Solla, D. Cohn (Eds.), Advances in Neural Information Processing Systems (NIPS), 1999, pp. 183-189
39. Mallat S. A Wavelet Tour of Signal Processing (1998), Academic Press, San Diego
40. Vincent E., Gribonval R., and Févotte C. Performance measurement in blind audio source separation. IEEE Trans. Speech Audio Process. 14 4 (2006) 1462-1469
41. L.K. Hansen, K.B. Petersen, Monaural ICA of white noise mixtures is hard, in: Proceedings of ICA 2003, pp. 815-820
42. Petersen K.B., Winther O., and Hansen L.K. On the slow convergence of EM and VBEM in low noise linear mixtures. Neural Comput. 17 (2005) 1-6
43. Neal R.M. Suppressing random walks in Markov chain Monte Carlo using ordered overrelaxation. In: Jordan M.I. (Ed). Learning in Graphical Models (1998), Kluwer Academic, Dordrecht 205-225
44. Doucet A., Godsill S., and Andrieu C. On sequential Monte Carlo sampling methods for Bayesian filtering. Statist. Comput. 10 3 (2000) 197-208
45. E. Sudderth, A. Ihler, W. Freeman, A. Willsky, Nonparametric belief propagation, in: Proceedings of IEEE Computer Vision and Pattern Recognition Conference (CVPR), 2003
46. M. Briers, A. Doucet, S.S. Singh, K. Weekes, Particle filters for graphical models, in: Proceedings of Nonlinear Statistical Signal Processing Workshop. IEEE, 2006
47. T. Minka, Divergence measures and message passing. Technical Report MSR-TR-2005-173, Microsoft Research, Cambridge, 2005
48. Opper M., and Winther O. Expectation consistent approximate inference. J. Machine Learn. Res. (2005) 2177-2204
49. M. Davies, N. Mitianoudis, A simple mixture model for sparse overcomplete ICA, IEE Proceedings on Vision, Image and Signal Processing, February 2004