Maximum marginal likelihood estimation for nonnegative dictionary learning in the gamma-poisson model

IEEE Transactions on Signal Processing

Volumn 60, Issue 10, 2012, Pages 5163-5175

  Dikmen, Onur ^a   Févotte, Cédric ^b  

^a AALTO UNIVERSITY   (Finland)

^b INSTITUT TELECOM   (France)

Author keywords

Automatic relevance determination; model order selection; Monte Carlo EM; nonnegative matrix factorization; sparse coding; variational EM

Indexed keywords

AUTOMATIC RELEVANCE DETERMINATION; MODEL-ORDER SELECTION; MONTE CARLO; NONNEGATIVE MATRIX FACTORIZATION; SPARSE CODING; VARIATIONAL EM;

ALGORITHMS; ESTIMATION; EXPANSION; FACE RECOGNITION; OPTIMIZATION;

MAXIMUM LIKELIHOOD ESTIMATION;

EID: 84866511641     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2012.2207117     Document Type: Article

References (29)

1. D. D. Lee and H. S. Seung, "Learning the parts of objects by non negative matrix factorization," Nature, vol. 401, pp. 788-791, 1999.
2. W. H. Richardson, "Bayesian-based iterative method of image restoration," J. Opt. Soc. Amer., vol. 62, pp. 55-59, 1972.
3. L. B. Lucy, "An iterative technique for the rectification of observed distributions," Astron. J., vol. 79, pp. 745-754, 1974.
4. M. S. Lewicki and T. J. Sejnowski, "Learning overcomplete representations," Neural Comput., vol. 12, pp. 337-365, 2000.
5. J. F. Canny, "GaP: A factor model for discrete data," in Proc. 27th ACM Int. Conf. Res. Develop. Inform. Retrieval (SIGIR), Sheffield, U.K., 2004, pp. 122-129.
6. R. Nallapati, W. W. Cohen, S. Ditmore, J. Lafferty, and K. Ung, "Multiscale topic tomography," in Proc. 13th ACM SIGKDD Int. Conf. Knowledge Discovery Data Mining (KDD'07), San Jose, CA, 2007.
7. M. Titsias, "The infinite gamma-poisson feature model," presented at the Adv. Neural Inform. Process. Syst. (NIPS'07), Vancouver, BC, Canada, 2007.
8. W. L. Buntine and A. Jakulin, "Discrete component analysis," Lecture Notes in Comput. Sci., vol. 3940, pp. 1-33, 2006.
9. A. T. Cemgil, "Bayesian inference for nonnegative matrix factorisation models," Comput. Intell. Neurosci., p. 17, 2009, Article ID 785152, doi: 10.1155/2009/785152.
10. D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent Dirichlet allocation," J. Mach. Learn.Res., vol. 3, pp. 993-1022, 2003.
11. T. Hofmann, "Probabilistic latent semantic indexing," presented at the 22nd Int. Conf. Res. Develop. Inform. Retrieval (SIGIR), Berkeley, CA, 1999.
12. S. Chib, "Marginal likelihood from the Gibbs output," J. Amer. Statist. Assoc., vol. 90, no. 432, pp. 1313-1321, 1995.
13. T. Bertin-Mahieux, D. P. Ellis, B. Whitman, and P. Lamere, "The million song dataset," presented at the 12th Int. Soc. Music Inform. Retrieval Conf. (ISMIR'11), Miami, FL, 2011.
14. O. Dikmen and C. Févotte, "Maximum marginal likelihood estimation for nonnegative dictionary learning," in Proc. Int. Conf. Acoust. Speech Signal Process.(ICASSP'11), Prague, Czech Republic, 2011.
15. C. Févotte and A. T. Cemgil, "Nonnegative matrix factorisations as probabilistic inference in composite models," in Proc. 17th Eur. Signal Process. Conf. (EUSIPCO), Glasgow, Scotland, Aug. 2009, pp. 1913-1917.
16. C. Févotte and J. Idier, "Algorithms for nonnegative matrix factorization with the beta-divergence," Neural Comput., vol. 23, no. 9, Sep. 2011.
17. M. N. Schmidt, O. Winther, and L. K. Hansen, "Bayesian non-negative matrix factorization," in Proc. 8th Int. Conf. Independent Component Anal. Signal Separation (ICA'09), Paraty, Brazil, Mar. 2009.
18. J. Eggert and E. Körner, "Sparse coding and NMF," in Proc. IEEE Int. Joint Conf. Neural Netw., Budapest, Hungary, 2004, pp. 2529-2533.
19. A. Lefèvre, F. Bach, and C. Févotte, "Itakura-Saito nonnegative matrix factorization with group sparsity," in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Prague, Czech Republic, May 2011.
20. A. P. Dempster, N. M. Laird, and D. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Roy. Statist. Soc., vol. 1, no. 39, pp. 1-38, 1977.
21. M. J. Beal and Z. Ghahramani, "The variational Bayesian EM algorithm for incomplete data: With application to scoring graphical model structures," in Bayesian Statistics 7, J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West, Eds. London, U.K.: Oxford Univ. Press, 2003.
22. G. C. G. Wei and M. A. Tanner, "A Monte Carlo implementation of the EM algorithm and the poor man's data augmentation algorithms," J. Amer. Statist. Assoc., vol. 85, no. 411, pp. 699-704, 1990.
23. S. Geman and D. Geman, "Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 6, no. 6, pp. 721-741, Jun. 1984.
24. E. Lehmann, Theory of Point Estimation. New York: Wiley, 1983.
25. D. Donoho and V. Stodden, "When does non-negative matrix factorization give a correct decomposition into parts?," in Advances in Neural Information Processing Systems 16, S. Thrun, L. Saul, and B. Schölkopf, Eds. Cambridge, MA: MIT Press, 2004.
26. B. Heisele, T. Poggio, and M. Pontil, "Face detection in still gray images," Center for Biological and Computational Learning, MIT, Cambridge, MA, A.I. Memo 1687, 2000.
27. V. Y. F. Tan and C. Févotte, "Automatic relevance determination in nonnegative matrix factorization," in Proc. Workshop Signal Process. Adaptative Sparse Structured Representations (SPARS), St-Malo, France, 2009.
28. S. D. Babacan, M. Luessi, R. Molina, and A. K. Katsaggelos, "Low-rank matrix completion by variational sparse bayesian learning," in Proc. Int. Conf. Acoust., Speech. Signal Process. (ICASSP'11), Prague, Czech Republic, 2011, pp. 2188-2111.
29. C. Févotte, N. Bertin, and J.-L. Durrieu, "Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis," Neural Comput., vol. 21, no. 3, pp. 793-830, Mar. 2009.