Sparse bayesian methods for low-rank matrix estimation

IEEE Transactions on Signal Processing

Volumn 60, Issue 8, 2012, Pages 3964-3977

  Babacan, S Derin ^a   Luessi, Martin ^b   Molina, Rafael ^c   Katsaggelos, Aggelos K ^d  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b MASSACHUSETTS GENERAL HOSPITAL   (United States)

^c UNIVERSITY OF GRANADA   (Spain)

^d Northwestern University   (United States)

Author keywords

Bayesian methods; low rankness; matrix completion; outlier detection; robust principal component analysis; sparse Bayesian learning; sparsity; variational Bayesian inference

Indexed keywords

BAYESIAN METHODS; LOW-RANKNESS; MATRIX COMPLETION; OUTLIER DETECTION; ROBUST PRINCIPAL COMPONENT ANALYSIS; SPARSE BAYESIAN LEARNING; SPARSITY; VARIATIONAL BAYESIAN;

BAYESIAN NETWORKS; INFERENCE ENGINES; PRINCIPAL COMPONENT ANALYSIS;

ESTIMATION;

EID: 84863899797     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2012.2197748     Document Type: Article

References (50)

1. E. J. Candès and B. Recht, "Exact matrix completion via convex optimization," Found. Comput. Math., vol. 9, pp. 717-772, 2008.
2. E. J. Candès and T. Tao, "The power of convex relaxation: Near-optimal matrix completion," IEEE Trans. Inf. Theory, vol. 56, no. 5, pp. 2053-2080, 2009.
3. E. J. Candès,X. Li, Y. Ma, and J. Wright, "Robust principal component analysis?," CoRR, vol. abs/0912. 3599, 2009.
4. Z. Zhou, X. Li, J. Wright, E. Candès, and Y. Ma, "Stable principal component pursuit," in Proc. IEEE Int. Symp. Inf. Theory (ISIT), 2010.
5. C. R. Johnson, "Matrix completion problems: A survey," in Proc. Symp. Appl. Math., 1990, vol. 40, pp. 171-198.
6. Z. Liu and L. Vandenberghe, "Interior-point method for nuclear norm approximation with application to system identification," SIAM J. Matrix Anal. Appl., vol. 31, no. 3, pp. 1235-1256, 2009.
7. S. Oh, A. Karbasi, and A. Montanari, "Sensor network localization from local connectivity: Performance analysis for the MDS-MAP algorithm," in Proc. IEEE Inf. Theory Workshop (ITW 2010), 2010.
8. N. Srebro, "Learning with matrix factorizations," Ph. D., Mass. Insti. Technol., Cambridge, MA, 2004.
9. P. Chen and D. Suter, "Recovering the missing components in a large noisy low-rank matrix: Application to SFM," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, pp. 1051-1063, 2004.
10. H. Ji, C. Liu, Z. Shen, and Y. Xu, "Robust video denoising using low rank matrix completion," in Proc. IEEE Conf. Comput. Vision Pattern Recogn. (CVPR 2010), 2010.
11. Z.-P. Liang, "Spatiotemporal imaging with partially separable functions," in Proc. NFSI-ICFBI, 2007, pp. 181-182.
12. P. J. Huber and E. M. Ronchetti, Robust Statistics. New York:Wiley, 2009.
13. F. D. la Torre and M. J. Black, "A framework for robust subspace learning," Int. J. Comput. Vision, vol. 54, pp. 117-142, 2003.
14. Q. Ke and T. Kanade, "Robust l1 norm factorization in the presence of outliers and missing data by alternative convex programming," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., 2005.
15. M. A. Fischler and R. C. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM, vol. 24, pp. 381-385, 1981.
16. J. Gao, "Robust l1 principal component analysis and its Bayesian variational inference," Neural Computat., vol. 20, pp. 555-578, 2008.
17. J. Luttinen, A. Ilin, and J. Karhunen, "Bayesian robust PCA for incomplete data," in Proc. Int. Conf. Independ. Compon. Anal. Signal Separat., 2009.
18. A. Wagner, J. Wright, A. Ganesh, Z. Zhou, H. Mobahi, and Y. Ma, "Towards a practical face recognition system: Robust alignment and illumination by sparse representation," IEEE Trans. Pattern Anal. Mach. Intell. (PAMI), Jul. 2010, submitted for publication.
19. C. Papadimitriou, P. Rghavan, H. Tamaki, and S. Vempala, "Latent semantic indexing, a probabilistic analysis," J. Comput. Syst. Sci., vol. 61, pp. 217-235, 2000.
20. Y. Peng, A. Ganesh, J. Wright, W. Xu, and Y. Ma, "RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images," IEEE Trans. Pattern Anal. Mach. Intell. (PAMI), Jul. 2010, submitted for publication.
21. B. Recht, M. Fazel, and P. A. Parrilo, "Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization," SIAM Rev., vol. 52, no. 3, pp. 471-501, 2010.
22. R. H. Keshavan, S. Oh, and A. Montanari, "Matrix completion from a few entries," IEEE Trans. Inf. Theory, vol. 56, no. 6, pp. 2980-2998, 2010.
23. E. J. Candès and Y. Plan, "Matrix completion with noise," Proc. IEEE, vol. 98, no. 6, pp. 925-936, 2009.
24. V. Chandrasekharan, S. Sanghavi, P. Parillo, and A. S. Wilsky, "Ranksparsity incoherence for matrix decomposition," SIAM J. Optim., vol. 21, no. 2, Jun. 2011.
25. J.-F. Cai, E. J. Candès, and Z. Shen, "A singular value thresholding algorithm for matrix completion," SIAM J. Optimiz., vol. 20, no. 4, pp. 1956-1982, 2008.
26. P. Jain, R. Meka, and I. S. Dhillon, "Guaranteed rank minimization via singular value projection," Neural Inf. Process. Syst., pp. 937-945, 2010.
27. S. Ma, D. Goldfarb, and L. Chen, "Fixed point and Bregman iterative methods for matrix rank minimization," Math. Program., vol. 128, no. 1-2, pp. 321-353, Jun. 2011.
28. K. Lee and Y. Bresler, "ADMiRA: Atomic decomposition for minimum rank approximation," IEEE Trans. Inf. Theory, vol. 56, no. 9, pp. 4402-4416, Sep. 2010.
29. J. Paisley and L. Carin, "A nonparametric Bayesian model for kernel matrix completion," in Proc. ICASSP, Dallas, TX, Apr. 2010.
30. N. Ding, Y. A. Qi, R. Xiang, I. Molloy, and N. Li, "Nonparametric Bayesian matrix factorization by Power-EP," in Proc. AISTATS, 2010, vol. 9, pp. 169-176.
31. Y. J. Lim and Y. W. Teh, "Variational Bayesian approach to movie rating prediction," in Proc. KDD Cup and Workshop, 2007.
32. M. Zhou, C. Wang,M. Chen, J. Paisley,D. Dunson, and L. Carin, "Nonparametric Bayesian matrix completion," in Proc. IEEE Sens. Array Multichannel Signal Process. Workshop (SAM), Oct. 2010.
33. F. Wood and T. L. Griffiths, "Particle filtering for nonparametric Bayesian matrix factorization," in Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press, 2006, pp. 1513-1520.
34. R. Salakhutdinov and A. Mnih, "Bayesian probabilistic matrix factorization using Markov chain Monte Carlo," in Proc. 25th Int. Conf. Mach. Learn., New York, 2008, pp. 880-887.
35. Z. Lin, A. Ganesh, J. Wright, L. Wu, M. Chen, and Y. Ma, "Fast convex optimization algorithms for exact recovery of a corrupted low-rankmatrix," SIAM J. Optimiz., 2009, submitted for publication.
36. Z. Lin, M. Chen, L. Wu, and Y. Ma, "The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices," Univ. Illinois at Urbana-Champaign, Champaign, IL, Tech. Rep., 2010.
37. X. Ding, L. He, and L. Carin, "Bayesian robust principal component analysis," IEEE Trans. Image Process., vol. 20, no. 12, pp. 3419-3430, 2011.
38. M. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., pp. 211-244, 2001. (Pubitemid 33687203)
39. T. Raiko, A. Ilin, and J. Karhunen, J. Kok, J. Koronacki, R. Mantaras, S. Matwin, D. Mladenic, and A. Skowron, Eds., "Principal component analysis for large scale problems with lots of missing values," in Proc. 18th Eur. Conf. Mach. Learn. (ECML 2007), Berlin/Heidelberg, 2007, vol. 4701, pp. 691-698.
40. V. Y. F. Tan and C. Févotte, "Automatic relevance determination in nonnegative matrix factorization," in Proc. Workshop on Signal Process. Adapt. Sparse Struct. Represent. (SPARS'09), 2009.
41. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization," in Proc. NIPS 13, 2001, pp. 556-562.
42. J. Haldar and D. Hernando, "Rank-constrained solutions to linear matrix equations using power factorization," IEEE Signal Process. Lett., vol. 16, no. 7, pp. 584-587, 2009.
43. R. M. Neal, Bayesian Learning for Neural Networks. NewYork: Springer, 1996.
44. M. Beal, "Variational algorithms for approximate Bayesian inference," Ph. D. dissertation, The Gatsby Computational Neurosci. Unit, University College, London, U. K., 2003.
45. C. M. Bishop, Pattern Recognition and Machine Learning. New York: Springer-Verlag, 2006.
46. B. Lakshminarayanan, G. Bouchard, and C. Archambeau, "Robust Bayesian matrix factorization," in Proc. Int. Conf. on Artif. Intell. Statist. (AISTATS), 2011.
47. M. E. Tipping, "Sparse kernel principal component analysis," in Proc. NIPS, 2000, pp. 633-639.
48. C. M. Bishop, "Bayesian PCA," in Proc. NIPS, 1999, pp. 382-388.
49. C. M. Bishop, "Variational principal components," in Proc. 9th Int. Conf. Artif. Neural Netw. ICANN'99, 1999, pp. 509-514.
50. K. Mardia, J. Kent, and J. Bibby, Multivariate Analysis. NewYork: Academic, 1979.