The variational approximation for Bayesian inference: Life after the EM algorithm

IEEE Signal Processing Magazine

Volumn 25, Issue 6, 2008, Pages 131-146

  Tzikas, Dimitris G ^a   Likas, Aristidis C ^b   Galatsanos, Nickolaos P ^b  

^a UNIVERSITY OF IOANNINA   (Greece)

^b UNIVERSITY OF PATRAS   (Greece)

Author keywords

Approximation theory; Bayes methods; Bayes theorem; Bayesian inference; Bayesian methods; Belief networks; Computational modeling; Estimation; History; Inference algorithms; Inference mechanisms; Laplace method; Random variables; Signal processing; Signal processing algorithms; Variational approximation; Variational techniques

Indexed keywords

APPROXIMATION ALGORITHMS; APPROXIMATION THEORY; ASYMPTOTIC ANALYSIS; BAYESIAN NETWORKS; COMPUTATION THEORY; ESTIMATION; HISTORY; KNOWLEDGE BASED SYSTEMS; LAPLACE TRANSFORMS; RANDOM VARIABLES; SIGNAL PROCESSING; VARIATIONAL TECHNIQUES;

BAYES METHOD; BAYES THEOREM; BAYESIAN INFERENCE; BAYESIAN METHODS; COMPUTATIONAL MODEL; INFERENCE ALGORITHM; INFERENCE MECHANISM; LAPLACE METHODS; SIGNAL PROCESSING ALGORITHMS; VARIATIONAL APPROXIMATION;

INFERENCE ENGINES;

EID: 85032751295     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2008.929620     Document Type: Article

References (31)

1. S.M. Stigler, "Thomas Bayes's inference," J. Roy. Statis. Soc. A vol. 145, pp. 250-258, 1982.
2. P.S. Laplace, "Mémoire sur la probabilité des causes par les événements," Mémoires de mathématique et de physique presentés á l'Académie royale des sciences par divers savants & lus dans ses assemblées, vol. 6, pp. 621-656, 1774.
3. S.M. Stigler, "Laplace's 1774 memoir on inverse probability," Statist. Sci., vol. 1, no. 3, pp. 359-363, 1986.
4. A. Dempster, N. Laird, and D. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Roy. Statis. Soc. A, vol. 39, no. 1, pp. 1-38, 1977.
5. W. Jones, L. Byars, and M. Casey, "Positron emission tomographic images and expectation maximization: A VLSI architecture for multiple iterations per second," IEEE Trans. Nuclear Sci., vol. 35, no. 1, pp. 620-624, 1988.
6. Z. Liang and H. Hart, "Bayesian reconstruction in emission computerized tomography," IEEE Trans. Nuclear Sci., vol. 35, no. 1, pp. 788-792, 1988.
7. S. Newcomb, "A generalized theory of the combination of observations so as to obtain the best result," Amer. J. Math., vol. 8, pp. 343-366, 1886.
8. G. McLachlan and T. Krishnan, The EM Algorithm and Extensions. New York: Wiley, 1997.
9. T.K. Moon, "The EM algorithm in signal processing," IEEE Signal Process. Mag., vol. 13, no. 6, pp. 47-60, 1996.
10. V. Smídl and A. Quinn, The Variational Bayes Method in Signal Processing. New York: Springer-Verlag, 2005.
11. C. Robert and G. Cassela, Monte Carlo Statistical Methods. New York: Springer-Verlag, 1999.
12. C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan, "An introduction to MCMC for machine learning," Mach. Learn., vol. 50, no. 1, pp. 5-43, 2003.
13. C. Bishop, Pattern Recognition and Machine Learning. New Yoek: Springer- Verlag, 2006.
14. C. Borgelt and R. Kruse, Graphical Models Methods for Data Analysis and Mining. Hoboken, NJ: Wiley, 2002.
15. R. Neapolitan, Learning Bayesian Networks. Englewood Cliffs, NJ: Prentice- Hall, 2004.
16. R.M. Neal and G.E. Hinton, "A view of the EM algorithm that justifies incremental, sparse and other variants," in Learning in Graphical Models, M.I. Jordan, Ed. Cambridge, MA: MIT Press, 1998, pp. 355-368.
17. S.M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory. Englewood Cliffs, NJ: Prentice-Hall, 1993.
18. R. Weinstock, Calculus of Variations. New York: Dover, 1974.
19. T.S. Jaakkola, "Variational methods for inference and learning in graphical models," Ph.D. dissertation, Dept. Elect. Eng. Comp. Sci., MIT, 1997.
20. M. Jordan, Z. Gharamani, T. Jaakkola, and L. Saul, "An introduction to variational methods for graphical models," in Learning in Graphical Models, M. Jordan, Ed., Cambridge, MA: MIT Press, 1998, pp. 105-162.
21. G. Parisi, Statistical Field Theory. Reading, MA: Addison-Wesley, 1988.
22. C. Bishop and M. Tipping, "Variational Relevance Vector Machines," in Proc. 16th Conf. Uncertainty in Artificial Intelligence, 2000, pp. 46-53.
23. G. Chantas, N. Galatsanos, and A. Likas, "Bayesian restoration using a new nonstationary edge preserving image prior," IEEE Trans. Image Processing, vol. 15, no. 10, pp. 2987-2997, Oct. 2006.
24. G. Chantas, N. Galatsanos, and N. Woods, "Super resolution based on fast registration and maximum a posteriori reconstruction," IEEE Trans. Image Processing, vol. 16, no. 7, pp. 1821-1830, July 2007.
25. D. Tzikas, A. Likas, and N. Galatsanos, "Variational Bayesian Blind Image Deconvolution with Student-T Priors," in Proc. IEEE Int. Conf. Image Processing, (ICIP), San Antonio, TX, Sept. 2007, vol. 1, pp. 109-112.
26. A. Mairgiotis, N. Galatsanos, and Y. Yang, "New additive watermark detectors based on a hierarchical spatially adaptive image model," IEEE Trans. Information Forensics and Security, vol. 3, no. 1, pp. 29-37, Mar. 2008.
27. H. Attias, "A Variational Bayesian Framework for Graphical Models," Proc. NIPS 12, MIT Press, 2000, pp. 209-216.
28. A. Corduneanu and C. Bishop, "Variational bayesian model selection for mixture distributions," in Proc. AI and Statistics Conf., Jan. 2001, pp. 27-34.
29. C. Constantinopoulos and A. Likas, "Unsupervised learning of gaussian mixtures based on variational component splitting," IEEE Trans. Neural Networks, vol. 18, no. 3, pp. 745-755, Mar. 2007.
30. K. Blekas, A. Likas, N. Galatsanos, and I.E. Lagaris, "A spatially-constrained mixture model for image segmentation," IEEE Trans. Neural Networks, vol. 16, no. 2, pp. 494-498, 2005.
31. C. Nikou, N. Galatsanos, and A. Likas, "A class-adaptive spatially variant mixture model for image segmentation," IEEE Trans. Image Processing, vol. 16, no. 4, pp. 1121-1130, Apr. 2007.