Variational bayesian matrix factorization for bounded support data

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 37, Issue 4, 2015, Pages 876-889

  Ma, Zhanyu ^a   Teschendorff, Andrew E ^b,c   Leijon, Arne ^d   Qiao, Yuanyuan ^a   Zhang, Honggang ^a   Guo, Jun ^a  

^a BEIJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

^b MAX PLANCK INSTITUTE   (Germany)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

^d ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

Author keywords

Bayesian estimation; bioinformatics; bounded support data; collaborative filtering; extended factorized approximation; Nonnegative matrix factorization; relative convexity; variational inference

Indexed keywords

BAYESIAN NETWORKS; BIOINFORMATICS; COLLABORATIVE FILTERING; FACTORIZATION;

BAYESIAN ESTIMATIONS; BOUNDED SUPPORT; EXTENDED FACTORIZED APPROXIMATION; NONNEGATIVE MATRIX FACTORIZATION; RELATIVE CONVEXITY; VARIATIONAL INFERENCE;

MATRIX ALGEBRA;

EID: 84924746875     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2014.2353639     Document Type: Article

References (59)

1. P. Paatero and U. Tapper, "Positive matrix factorization: A nonnegative factor model with optimal utilization of error estimates of data values, " Environmetrics, vol. 5, pp. 111-126, 1994.
2. D. D. Lee and H. S. Seung, "Learning the parts of objects with nonnegative matrix factorization, " Nature, vol. 401, pp. 788-791, 1999.
3. S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, "Indexing by latent semantic analysis, " J. Amer. Soc. Inf. Sci., vol. 41, no. 6, pp. 391-407, 1990.
4. D. Guillamet and J. Vitri-a, "Non-negative matrix factorization for face recognition, " in Topics in Artificial Intelligence, ser. Lecture Notes in Computer Science, vol. 2504. New York, NY, USA: Springer, 2002, pp. 336-344.
5. C. Fevotte, N. Bertin, and J.-L. Durrieu, "Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis, " Neural Comput., vol. 21, no. 3, pp. 793-830, 2009.
6. M. Hoffman, D. Blei, and P. Cook, "Bayesian nonparametric matrix factorization for recorded music, " in Proc. Int. Conf. Mach. Learn., 2010, pp. 439-446.
7. C. Fevotte and J. Idier, "Algorithms for nonnegative matrix factorization with the beta-divergence, " Neural Comput., vol. 23, no. 9, pp. 2421-2456, Sep. 2011.
8. S. Kirbiz and P. Smaragdis, "An adaptive time-frequency resolution approach for non-negative matrix factorization based single channel sound source separation, " in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., 2011, pp. 253-256.
9. K. W. Wilson, B. Raj, P. Smaragdis, and A. Divakaran, "Speech denoising using nonnegative matrix factorization with priors, " in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., 2008, pp. 4029-4032.
10. K. W. Wilson, B. Raj, and P. Smaragdis, "Regularized nonnegative matrix factorization with temporal dependencies for speech denoising, " in Proc. Interspeech Int. Conf. Spoken Lang. Process., 2008, pp. 411-414.
11. R. Salakhutdinov and A. Mnih, "Bayesian probabilistic matrix factorization using Markov chain Monte Carlo, " in Proc. Int. Conf. Mach. Learn., 2008, pp. 880-887.
12. R. Salakhutdinov and A. Mnih, "Probabilistic matrix factorization, " in Proc. Adv. Neural Inf. Process. Syst., 2008, pp. 1257-1264.
13. T. Raiko, A. Ilin, and J. Karhunen, "Principal component analysis for sparse high-dimensional data, " in Neural Information Processing, M. Ishikawa, K. Doya, H. Miyamoto, and T. Yamakawa, Eds, Berlin, Germany: Springer-Verlag, 2008, pp. 566-575.
14. U. Paquet, B. Thomson, and O. Winther, " A hierarchical model for ordinal matrix factorization, " Statist. Comput., vol. 22, no. 4, pp. 945-957, 2012.
15. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization, " in Proc. Adv. Neural Inf. Process. Syst. 13, 2001, pp. 556-562.
16. P. O. Hoyer, "Non-negative matrix factorization with sparseness constraints, " J. Mach. Learn. Res., vol. 5, pp. 1457-1469, Dec. 2004.
17. D. Guillamet, M. Bressan, and J. Vitri-a, "A weighted non-negative matrix factorization for local representations, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., vol. 1, 2001, pp. 942-947.
18. P. A. Højen-Sørensen, O. Winther, and L. K. Hansen, "Mean-field approaches to independent component analysis, " Neural Comput., vol. 14, no. 4, pp. 889-918, 2002.
19. M. N. Schmidt and H. Laurberg, "Nonnegative matrix factorization with Gaussian process priors, " Comput. Intell. Neurosci., vol. 2008, p. 8, 2008.
20. M. N. Schmidt, O. Winther, and L. K. Hansen, "Bayesian nonnegative matrix factorization, " in Independent Component Analysis and Signal Separation, ser. Lecture Notes in Computer Science. New York, NY, USA: Springer, 2009.
21. J. Besag, "On the statistical analysis of dirty pictures, " J. Royal Statist. Soc., vol. 48, no. 3, pp. 259-302, 1986.
22. A. T. Cemgil, "Bayesian inference in non-negative matrix factorisation models, " Comput. Intell. Neurosci., vol. 2009, p. 4, July 2009.
23. D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent Dirichlet allocation, " J. Mach. Learn. Res., vol. 3, pp. 993-1022, Jan. 2003.
24. Y. Ji, C. Wu, P. Liu, J. Wang, and K. R. Coombes, "Application of beta-mixture models in bioinformatics, " Bioinformat. Appl. Note, vol. 21, pp. 2118-2122, 2005.
25. Z. Ma and A. Leijon, "PDF-optimized LSF vector quantization based on beta mixture models, " in Proc. 11th Annu. Conf. Int. Speech Commun. Assoc., 2010, pp. 2374-2377.
26. Z. Ma and A. Leijon, "Bayesian estimation of beta mixture models with variational inference, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 11, pp. 2160-2173, Nov. 2011.
27. Z. Ma, "Non-Gaussian statistical models and their applications, " Ph.D. dissertation, KTH-Royal Institute of Technology, Stockholm, Sweden, 2011.
28. A. E. Teschendorff, F. Marabita, M. Lechner, T. Bartlett, J. Tegner, D. Gomez-Cabrero, and S. Beck, "A beta-mixture quantile normalisation method for correcting probe design bias in illumina infinium 450k DNA methylation data, " Bioinformatics, vol. 29, no. 2, pp. 189-196, Nov. 2012.
29. Z. Ma and A. E. Teschendorff, "A variational Bayes beta mixture model for feature selection in DNA methylation studies, " J. Bioinformat. Comput. Biol., vol. 11, no. 4, p. 1350005, 2013.
30. J. A. Palmer, "Relative convexity, " ECE Dept., Univ. California, San Diego, San Diego, CA, USA, Tech. Rep., 2003.
31. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
32. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul, "An introduction to variational methods for graphical models, " Mach. Learn., vol. 37, no. 2, pp. 183-233, 1999.
33. C. M. Bishop, Pattern Recognition Machine Learning. New York, NY, USA: Springer, 2006.
34. T. S. Jaakkola and M. I. Jordan, "Bayesian parameter estimation via variational methods, " Statist. Comput., vol. 10, pp. 25-37, 2000.
35. T. S. Jaakkola, "Tutorial on variational approximation methods, " in Proc. Adv. Mean Field Methods, 2001, pp. 129-159.
36. D. M. Blei and J. D. Lafferty, "Correlated topic models, " in Proc. Adv. Neural Inf. Process. Syst., 2006.
37. D. M. Blei and J. D. Lafferty, "A correlated topic model of Science, " Ann. Appl. Statist., vol. 1, pp. 17-35, 2007.
38. M. Braun and J. McAuliffe, "Variational inference for large-scale models of discrete choice, " J. Amer. Statist. Assoc., vol. 105, pp. 324-335, 2010.
39. N. Bouguila, D. Ziou, and E. Monga, "Practical Bayesian estimation of a finite beta mixture through Gibbs sampling and its applications, " Statist. Comput., vol. 16, pp. 215-225, 2006.
40. Z. Ma, P. K. Rana, J. Taghia, M. Flierl, and A. Leijon, "Bayesian estimation of Dirichlet mixture model with varitional inference, " Pattern Recognit., vol. 47, no. 9, pp. 3143-3157, Sep. 2014.
41. M. Brookes, The Matrix Reference Manual, 2005, http://www.psi. toronto.edu/matrix/intro.html
42. K. D. Schmidt, "On the covariance of monotone functions of a random variable, " Lehrstuhl fur Versicherungsmathematik, Technische Universitat Dresden, Tech. Rep., 2003.
43. M. Egozcue, L. F. Garcia, and W. K. Wong, "On some covariance inequalities for monotonic and non-monotonic functions, " J. Inequalities Pure Appl. Math., vol. 10, pp. 1-16, 2009.
44. F. A. Quintana, J. S. Liu, and G. E. Pino, "Monte Carlo em with importance reweighting and its applications in random effects models, " Comput. Statist. Data Anal., vol. 29, no. 4, pp. 429-444, Feb. 1999.
45. G. Moffa and J. Kuipers, "Sequential Monte Carlo em for multivariate probit models, " Comput. Statist. Data Anal., vol. 72, no. 0, pp. 252-272, 2014.
46. [Online]. Available: http://cs.nyu.edu/-roweis/data.html
47. X. Su and T. M. Khoshgoftaar, "A survey of collaborative filtering techniques, " Adv. Artif. Intell., vol. 2009, p. 4, Jan. 2009.
48. J. S. Breese, D. Heckerman, and C. Kadie, "Empirical analysis of predictive algorithms for collaborative filtering, " in Proc. 14th Conf. Uncertainty Artif. Intell., 1998, pp. 43-52.
49. J. L. Herlocker, J. Konstan, L. G. Terveen, and J. T. Riedl, "Evaluating collaborative filtering recommender systems, " ACM Trans. Inf. Syst., vol. 22, no. 1, pp. 5-53, Jan. 2004.
50. [Online]. Available: http://www.netflixprize.com/
51. A. P. Feinberg, R. Ohlsson, and S. Henikoff, "The epigenetic progenitor origin of human cancer, " Nat. Rev. Genetics, vol. 7, pp. 21-33, Jan. 2006.
52. P. A. Jones and S. B. Baylin, "The epigenomics of cancer, " Cell, vol. 128, no. 4, pp. 683-692, Feb. 2007.
53. J. Zhuang, M. Widschwendter, and A. E. Teschendorff, "A comparison of feature selection and classification methods in DNA methylation studies using the Illumina Infinium platform, " BMC Bioinformatics, vol. 13, p. 59, 2012.
54. E. A. Houseman, B. C. Christensen, R. F. Yeh, C. J. Marsit, M. R. Karagas, M. Wrensch, H. H. Nelson, J. Wiemels, S. Zheng, J. K. Wiencke, and K. T. Kelsey, "Model-based clustering of DNA methylation array data: A recursive-partitioning algorithm for high-dimensional data arising as a mixture of beta distributions, " Bioinformatics, vol. 9, p. 365, 2008.
55. M. Bibikova, Z. Lin, L. Zhou, E. Chudin, E. W. Garcia, B. Wu, D. Doucet, N. J. Thomas, Y. Wang, E. Vollmer, T. Goldmann, C. Seifart, W. Jiang, D. L. Barker, M. S. Chee, J. Floros, and J. B. Fan, "High-throughput DNA methylation profiling using universal bead arrays, " Genome Res., vol. 16, no. 3, pp. 383-393, Mar. 2006.
56. M. Widschwendter, H. Fiegl, D. Egle, E. Mueller-Holzner, G. Spizzo, C. Marth, D. J. Weisenberger, M. Campan, J. Young, I. Jacobs, and P. W. Laird, "Epigenetic stem cell signature in cancer, " Nat. Genetics, vol. 39, no. 2, pp. 157-158, Feb 2007.
57. C. Curtis, S. P. Shah, S. F. Chin, G. Turashvili, O. M. Rueda, M. J. Dunning, D. Speed, A. G. Lynch, S. Samarajiwa, Y. Yuan, S. Grf, G. Ha, G. Haffari, A. Bashashati, R. Russell, S. McKinney, A. Langerod, A. Green, E. Provenzano, G. W. G, S. Pinder, P. Watson, F. Markowetz, L. Murphy, I. Ellis, A. Purushotham, A. L. Borresen-Dale, J. D. Brenton, S. Tavare, C. Caldas, and S. Aparicio, "The genomic and transcriptomic architecture of 2, 000 breast tumours reveals novel subgroups, " Nature, vol. 486, no. 7403, pp. 346-352, Apr. 2012.
58. V. Plerou, P. Gopikrishnan, B. Rosenow, L. A. Amaral, T. Guhr, and H. E. Stanley, "Random matrix approach to cross correlations in financial data, " Phys. Rev. E: Statist., Nonlinear, and Soft Matter Phys., vol. 65, p. 066126, Jun. 2002.
59. A. E. Teschendorff, J. Zhuang, and M. Widschwendter, "Independent surrogate variable analysis to deconvolve confounding factors in large-scale microarray profiling studies, " Bioinformatics, vol. 27, no. 11, pp. 1496-1505, Jun. 2011.