Fast bregman divergence NMF using taylor expansion and coordinate descent

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2012, Pages 307-315

  Li, Liangda ^a   Lebanon, Guy ^a   Park, Haesun ^a  

^a Georgia Institute of Technology   (United States)

Author keywords

bregman divergences; euclidean distance; non negative matrix factorization; taylor series expansion

Indexed keywords

ALGEBRAIC MANIPULATIONS; BREGMAN DIVERGENCES; COORDINATE DESCENT; EUCLIDEAN DISTANCE; FROBENIUS NORM; LATENT STRUCTURES; MINIMIZATION PROBLEMS; NON-NEGATIVITY; NONNEGATIVE MATRIX FACTORIZATION; REAL WORLD DATA; TAYLOR EXPANSIONS; TAYLOR SERIES EXPANSIONS;

DATA MINING; HUMAN COMPUTER INTERACTION; SIGNAL RECONSTRUCTION; SINGULAR VALUE DECOMPOSITION; TAYLOR SERIES;

ALGORITHMS;

EID: 84866033934     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2339530.2339582     Document Type: Conference Paper

References (34)

1. http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html.
2. A. Banerjee. Optimal bregman prediction and jensena̧ŕs equality. In In Proc. International Symposium on Information Theory (ISIT), page 2004, 2004.
3. A. Banerjee, S. Merugu, I. S. Dhillon, and J. Ghosh. Clustering with bregman divergences. J. Mach. Learn. Res., 6:1705-1749, December 2005.
4. P. Breheny and J. Huang. Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection. Annals of Applied Statistics, 5(1):232-253, 2011.
5. A. Cichocki and A.-H. Phan. Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Transactions on Fundamentals of Electronics, 92:708-721, 2009.
6. A. Cichocki and R. Zdunek. Nmflab for signal and image processing. In tech. rep, Laboratory for Advanced Brain Signal Processing, Saitama, Japan, 2006. BSI, RIKEN.
7. A. Cichocki, R. Zdunek, and S. A. A.-H. Phan. Nonnegative matrix and tensor factorizations: Applications to exploratory multi-way data analysis and blind source separation. New York, USA, 2009. Wiley.
8. I. S. Dhillon and S. Sra. Generalized nonnegative matrix approximations with bregman divergences. In Neural Information Proc. Systems, pages 283-290, 2005.
9. C. Ding, T. Li, and W. Peng. On the equivalence between non-negative matrix factorization and probabilistic latent semantic indexing. Comput. Stat. Data Anal., 52:3913-3927, April 2008.
10. J. Fan and R. Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456).
11. C. Fevotte, N. Bertin, and J.-L. Durrieu. Nonnegative matrix factorization with the itakura-saito divergence: With application to music analysis. Neural Comput., 21:793-830, March 2009.
12. M. Figueiredo, R. Nowak, and S. J. Wright. Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE J. of Selected Topics in Signal Proc, 1:586-598, 2007.
13. J. Friedman, T. Hastie, and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1), 1 2010.
14. N. Gillis and F. Glineur. Accelerated multiplicative updates and hierarchical als algorithms for nonnegative matrix factorization. Neural Comput., 24(4):1085-1105, 4 2012.
15. M. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 49(6), 1952.
16. T. Hofmann. Probabilistic latent semantic indexing. In SIGIR '99, pages 50-57, New York, NY, USA, 1999. ACM.
17. C.-J. Hsieh and I. S. Dhillon. Fast coordinate descent methods with variable selection for non-negative matrix factorization. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '11, pages 1064-1072, New York, NY, USA, 2011. ACM.
18. C.-J. Hsieh, M. A. Sustik, I. S. Dhillon, and P. Ravikumar. Sparse inverse covariance matrix estimation using quadratic approximation. In Advances in Neural Information Processing Systems 24, pages 2330-2338, 2011.
19. H. Kim and H. Park. Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics, 23:1495-1502, June 2007.
20. H. Kim and H. Park. Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J. Matrix Anal. Appl., 30:713-730, July 2008.
21. J. Kim, Y. He, and H. Park. Algorithms for nonnegative matrix and tensor factorizations: A unified view based on block coordinate descent framework. Under review.
22. J. Kim and H. Park. Toward faster nonnegative matrix factorization: A new algorithm and comparisons. IEEE International Conference on Data Mining, 0:353-362, 2008.
23. J. Kim and H. Park. Fast nonnegative matrix factorization: An active-set-like method and comparisons. In SIAM Journal on Scientific Computing, 2011.
24. G. Lebanon. Axiomatic geometry of conditional models. Information Theory, IEEE Transactions, 51:1283-1294, April 2005.
25. D. D. Lee and H. S. Seung. Algorithms for non-negative matrix factorization. In NIPS, pages 556-562. MIT Press, 2000.
26. Y. Li and S. Osher. Coordinate descent optimization for l1 minimization with application to compressed sensing; a greedy algorithm. Inverse Probl. Imaging, 3(3).
27. C.-J. Lin. Projected gradient methods for non-negative matrix factorization. Neural Computation, 19:2756-2779, October 2007.
28. C. Y. Lin and E. Hovy. Automatic evaluation of summaries using n-gram co-occurrence statistics. In NAACL, pages 71-78, Morristown, NJ, USA, 2003. Association for Computational Linguistics.
29. R. Mazumder, J. Friedman, and T. Hastie. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 106(495).
30. S. D. Pietra, V. D. Pietra, and J. Lafferty. Duality and auxiliary functions for bregman distances. Technical report, School of Computer Science, Carnegie Mellon University, 2002.
31. A. P. Singh and G. J. Gordon. A unified view of matrix factorization models. In Proceedings of the European conference on Machine Learning and Knowledge Discovery in Databases - Part II, ECML PKDD '08, pages 358-373, Berlin, Heidelberg, 2008. Springer-Verlag.
32. S. Wang and D. Schuurmans. Learning continuous latent variable models with bregman divergences. In In Proc. IEEE International Conference on Algorithmic Learning Theory, page 2004, 2003.
33. T. Wu and K. Lange. Coordinate descent algorithms for lasso penalized regression. The Annals of Applied Statistics, 2(1):224-244, 2008.
34. S. Yun and K.-C. Toh. A coordinate gradient descent method for l1-regularized convex minimization. Computational Optimization and Applications, 48(2).