Variational bayesian methods for multimedia problems

IEEE Transactions on Multimedia

Volumn 16, Issue 4, 2014, Pages 1000-1017

  Chen, Zhaofu ^a   Babacan, S Derin ^b   Molina, Rafael ^c   Katsaggelos, Aggelos K ^a  

^a Northwestern University   (United States)

^b GOOGLE INC   (United States)

^c UNIVERSITY OF GRANADA   (Spain)

Author keywords

Bayes methods; graphical models; inverse problems; multimedia signal processing; variational Bayes

Indexed keywords

BAYESIAN NETWORKS; GRAPHIC METHODS; INFERENCE ENGINES; INVERSE PROBLEMS; MULTIMEDIA SIGNAL PROCESSING; PROBABILITY DISTRIBUTIONS; STOCHASTIC SYSTEMS;

BAYES METHOD; EXPECTATION-MAXIMIZATION ALGORITHMS; GRAPHICAL MODEL; KULLBACK-LEIBLER DIVERGENCE; LOOPY BELIEF PROPAGATION; PROBABILISTIC GRAPHICAL MODELS; VARIATIONAL BAYES; VARIATIONAL BAYESIAN METHODS;

APPROXIMATION ALGORITHMS;

EID: 84901054636     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2014.2307692     Document Type: Article

References (66)

1. D. Barber, Bayesian Reasoning and Machine Learning. Cambridge,U.K.: Cambridge Univ. Press, 2012.
2. K. P.Murphy,Machine Learning: A Probabilistic Perspective. Cambridge,MA, USA: MIT Press, 2012.
3. S. J. D. Prince, Computer Vision: Models, Learning, and Inference.Cambridge, U.K.: Cambridge Univ. Press, 2012.
4. P. Chen and D. Suter, "Recovering the missing components in a largenoisy low-rank matrix: Application to SFM," IEEE Trans. PatternAnal. Mach. Intell., vol. 26, no. 8, pp. 1051-1063, 2004.
5. H. Ji, C. Liu, Z. Shen, and Y. Xu, "Robust video denoising using lowrank matrix completion," in Proc. IEEE Conf. Computer Vision andPattern Recognition, 2010, pp. 1791-1798.
6. J. P. Haldar and Z. Liang, "Spatiotemporal imaging with partially separablefunctions: A matrix recovery approach," in Proc. IEEE Int. Symp.Biomedical Imaging: From Nano to Macro, 2010, pp. 716-719.
7. N. Srebro, "Learning with matrix factorization," Ph.D. dissertation,Massachusetts Inst. Technol., Cambridge, MA, USA, 2004.
8. Z. Liu and L. Vandenberghe, "Interior-point method for nuclear normapproximation with application to system identification," SIAM J. MatrixAnal. Applicat., vol. 31, no. 3, pp. 1235-1256, 2009.
9. S. Oh, A. Montanari, and L. Karbasi, "Sensor network localizationfrom local connectivity: Performance analysis for the MDS-MAP algorithm,"in Proc. IEEE Information Theory Workshop, 2010, pp. 1-5.
10. J. Wang, Y. Shi, W. Ding, and B. Yin, "A low-rank matrix completionbased intra prediction for H.264/AVC," in Proc. IEEE 13th Int. WorkshopMultimedia Signal Processing, 2011, pp. 1-6.
11. N. Barzigar, A. Roozgard, S. Cheng, and P. Verma, "An efficient videodenoising method using decomposition approach for low-rank matrixcompletion," in Conf. Record 46th Asilomar Conf. Signals, Systemsand Computers, 2012, pp. 1684-1687.
12. M. D. Dao, D. T. Nguyen, Y. Cao, and T. D. Tran, "Video concealmentvia matrix completion at high missing rates," in Proc. Asilomar Conf.Signals, Systems and Computers, 2010, pp. 758-762.
13. D. Babacan, M. Luessi, R. Molina, and A. K. Katsaggelos, "SparseBayesian methods for low-rankmatrix estimation," IEEE Trans. SignalProcess., vol. 60, no. 8, pp. 3964-3977, Aug. 2012.
14. A. Wagner, J. Wright, A. Ganesh, Z. Zhou, H. Mobahi, and Y. Ma,"Toward a practical face recognition system: Robust alignment and illuminationby sparse representation," IEEE Trans. Pattern Anal.Mach.Intell., vol. 34, no. 2, pp. 372-386, 2012.
15. T. Goodall, S. Gibson, and M. C. Smith, "Parallelizing principalcomponent analysis for robust facial recognition using CUDA," Proc.Symp. App. Accelerators in High Perf. Comput., pp. 121-124, 2012.
16. C. H. Papadimitriou, P. Raghavan, H. Tamaki, and S. Vempala, "Latentsemantic indexing: A probabilistic analysis," Elsevier J. Comput. Syst.Sci., vol. 61, no. 2, pp. 217-235, 2000.
17. Y. Peng, A. Ganesh, J. Wright, W. Xu, and Y. Ma, "RASL: Robustalignment by sparse and low-rank decomposition for linearly correlatedimages," in Proc. IEEE Conf. Comput. Vis. and Pattern Recognit.,2010, pp. 763-770.
18. P. Huang, S. D. Chen, P. Smaragdis, and M. Hasegawa-Johnson, "Singing-voice separation from monaural recordings using robustprincipal component analysis," in Proc. IEEE Int. Conf. Acoustics,Speech and Signal Processing, 2012, pp. 57-60.
19. W. Tan, G. Cheung, and Y. Ma, "Face recovery in conference videostreaming using robust principal component analysis," in Proc. 18thIEEE Int. Conf. Image Processing, 2011, pp. 3225-3228.
20. X. Wang, W. Wan, and G. Liu, "Multi-task low-rank and sparse matrixrecovery for human motion segmentation," in Proc. 19th IEEE Int.Conf. Image Processing, 2012, pp. 897-900.
21. Z. Chen, R. Molina, and A. K. Katsaggelos, "A variational approachfor sparse component estimation and low-rank matrix recovery," J.Commun., vol. 8, no. 9, pp. 600-611, 2013.
22. M. Beal, "Variational algorithms for approximate Bayesian inference,"Ph.D. dissertation, Univ. College London, London, U.K., 2003.
23. J. Miskin, "Ensemble learning for independent component analysis,"Ph.D. dissertation, Astrophysics Group, Univ. Cambridge, Cambridge,U.K., 2000.
24. C. Bishop, Pattern Recognition and Machine Learning. NewYork,NY, USA: Springer, 2006.
25. S. T. Jaakkola and I. M. Jordan, "Bayesian parameter estimation viavariational methods," Statist. Comput., vol. 10, no. 1, pp. 25-37, 2000.
26. D. G. Tzikas, C. L. Likas, and N. P. Galatsanos, "The variational approximationfor Bayesian inference," IEEE Signal Process. Mag., vol.25, no. 6, pp. 131-146, 2008.
27. C. W. Fox and S. J. Roberts, "A tutorial on variational Bayesian inference,"Artif. Intell. Rev., vol. 38, no. 2, pp. 85-95, 2012.
28. S. Dupont and J. Luettin, "Audio-visual speech modeling for continuousspeech recognition," IEEE Trans. Multimedia, vol. 2, no. 3, pp.141-151, 2000.
29. A. V. Nefian, L. Liang, X. Pi, X. Liu, and K. P. Murphy, "DynamicBayesian networks for audio-visual speech recognition," EURASIP J.Adv. Signal Process., vol. 2002, no. 11, pp. 1274-1288, 2002.
30. Z. Liu, F. Jiang, G. Tian, S. Wang, F. Sato, S. Meltzer, and M. Tan,"Sparse logistic regression with penalty for biomarker identification,"Statist. App. Genet. Molec. Biol., vol. 6, no. 1, 2007.
31. A. Kabán and R. Durrant, "Learning with vs-norm regularisationwith exponentially many irrelevant features," Mach. Learn. andKnowl. Discov. Databases, Lecture Notes in Comput. Sci., vol. 5211,pp. 580-596, 2008.
32. D. Koller and N. Friedman, Probabilistic Graphical Models-Principlesand Techniques. Cambridge, MA, USA: MIT Press, 2009.
33. M. I. Jordan, Z. Ghahramani, T. S. Jaakola, and L. K. Saul, "An introductionto variational methods for graphical models," in Learningin Graphical Models. Cambridge, MA, USA: MIT Press, 1998, pp.105-162.
34. S. S. Geman and D. Geman, "Stochastic relaxation, Gibbs distributionsand the Bayesian restoration of images," IEEE Trans. Pattern Anal.Mach. Intell., vol. 6, pp. 721-741, 1984. (Pubitemid 15453722)
35. J. Grimmer, "An introduction to Bayesian inference via variational approximations,"Polit. Anal., vol. 19, no. 1, pp. 32-47, 2010.
36. S. D. Babacan, R.Molina,M. N. Do, and A. K. Katsaggelos, "Bayesianblind deconvolution with general sparse image priors," in Proc. Eur.Conf. Computer Vision, 2012, pp. 341-355.
37. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine,"J. Mach. Learn. Res., vol. 1, pp. 211-244, 2001. (Pubitemid 33687203)
38. T. Buchgraber, "Variational sparse Bayesian learning: Centralized anddistributed processing," Ph.D. dissertation, Graz Univ. Technol., Graz,Austria, 2013.
39. A. D. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihoodfrom incomplete data via the E-M algorithm," J. Roy. Statist. Soc., SeriesB, vol. 39, pp. 1-37, 1977.
40. G. Parisi, Statistical Field Theory. Reading, MA, USA: Addison-Wesley, 1988.
41. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge,U.K.: Cambridge Univ. Press, 2004.
42. R. T. Rockafellar, Convex analysis. Princeton, NJ, USA: PrincetonUniv. Press, 1970.
43. J. A. Palmer, K. Kreutz-Delgado, D. P. Wipf, and B. D. Rao, "VariationalEM algorithms for non-Gaussian latent variable models," inAdvances in Neural Information Processing Systems 18, Y. Weiss,B. Schölkopf, and J. Platt, Eds. Cambridge, MA, USA: MIT Press,2006, pp. 1059-1066.
44. D. J. C. MacKay, Information Theory, Inference & Learning Algorithms.New York, NY, USA: Cambridge Univ. Press, 2007.
45. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks ofPlausible Inference. San Francisco, CA, USA: Morgan Kaufmann,1988.
46. T. P. Minka, "A family of algorithms for approximate Bayesian inference,"Ph.D. dissertation, Massachusetts Inst. Technol., Cambridge,MA, USA, 2001.
47. T. P. Minka, "Expectation propagation for approximate Bayesian inference,"in Proc. Conf. Uncertainty in Artificial Intelligence, 2001, pp.362-369.
48. C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for MachineLearning. Cambridge, MA, USA: MIT Press, 2006.
49. S. D. Babacan, R. Molina, and A. K. Katsaggelos, " VariationalBayesian blind deconvolution using a total variation prior," IEEETrans. Image Process., vol. 18, no. 1, pp. 12-26, Jan. 2009.
50. A. M. McIvor, "Background subtraction techniques," in Proc. Imageand Vision Computing New Zealand 2000, Reveal Limited, Auckland,New Zealand.
51. M. Piccardi, "Background subtraction techniques: A review," inProc. IEEE Int. Conf. Syst., Man and Cybernetics, 2004, vol. 4, pp.3099-3104. (Pubitemid 40408308)
52. A. Patcha and J. Park, "An overview of anomaly detection techniques:Existing solutions and latest technological trends," Elsevier Comput.Netw., vol. 51, no. 12, pp. 3448-3470, 2007. (Pubitemid 46921030)
53. V. Chandola, A. Banerjee, and V. Kumar, "Anomaly detection: Asurvey," ACM Comput. Surveys, vol. 41, no. 3, pp. 151-1558,2009.
54. C. Wang, D. Blei, and F. Li, "Simultaneous image classification andannotation," in Proc. IEEE Comput. Vision & Pattern Recognit., 2009,pp. 1903-1910.
55. D.M. Blei, A. Y. Ng, andM. I. Jordan, "Latent Dirichlet allocation,"J. Mach. Learn. Res., vol. 3, pp. 993-1022, 2003.
56. D. M. Blei and J. D. McAuliffe, "Supervised topic models," inAdvances in Neural Information Processing Systems 20, J. Platt, D.Koller, Y. Singer, and S. Roweis, Eds. Cambridge, MA, USA: MITPress, 2008, pp. 121-128.
57. J. Zeng, W. K. Cheung, and J. Liu, "Learning topic models by beliefpropagation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 5,pp. 1121-1134, 2013.
58. X.Wang, X. Ma, and W. E. L. Grimson, "Unsupervised activity perceptionin crowded and complicated scenes using hierarchical Bayesianmodels," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 3, pp.539-555, 2009.
59. B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, "LabelMe:A database and web-based tool for image annotation," Int. J.Comput. Vision, vol. 77, no. 1-3, pp. 157-173, 2008.
60. S. Watanabe, Y. Minami, A. Nakamura, and N. Ueda, " VariationalBayesian estimation and clustering for speech recognition," IEEETrans. Speech Audio Process., vol. 12, no. 4, pp. 365-381, 2004.
61. Y. Tam and T. Schultz, "Dynamic language model adaptation usingvariational Bayes inference," in Proc. INTERSPEECH, 2005, pp. 5-8. (Pubitemid 43907987)
62. W. Penny, S. Kiebel, and K. Friston, "Variational Bayesian inferencefor fMRI time series," NeuroImage, vol. 19, no. 3, pp. 727-741, 2003. (Pubitemid 36918216)
63. X. Li and Y. Zheng, "Patch-based video processing: A variationalBayesian approach," IEEE Trans. Circuits Syst. Video Technol., vol.19, no. 1, pp. 27-40, 2009.
64. I. Pruteanu-Malinici and L. Carin, "Infinite hidden Markov models forunusual-event detection in video," IEEE Trans. Image Process., vol.17, no. 5, pp. 811-822, May 2008. (Pubitemid 351597171)
65. Y. Bazi and F.Melgani, "Gaussian process approach to remote sensingimage classification," IEEE Trans. Geosci. Remote Sens., vol. 48, no.1, pp. 196-197, 2010.
66. Y. Bazi and F. Melgani, "Classification of hyperspectral remotesensing images using Gaussian processes," in Proc. IEEE Int. Geoscienceand Remote Sensing Symp., 2008, pp. 1013-1016.