Fast coordinate descent methods with variable selection for non-negative matrix factorization

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2011, Pages 1064-1072

  Hsieh, Cho Jui ^a   Dhillon, Inderjit S ^a  

^a University of Texas at Austin   (United States)

Author keywords

Algorithms; Experimentation; Performance

Indexed keywords

CONSTRAINED OPTIMIZATION; CONVEX OPTIMIZATION; DATA MINING; IMAGE PROCESSING; LEAST SQUARES APPROXIMATIONS; MATRIX ALGEBRA; NEWTON-RAPHSON METHOD;

COORDINATE DESCENT METHODS; DIMENSION REDUCTION METHOD; EFFECTIVE DIMENSIONS; EXPERIMENTATION; KL-DIVERGENCE; NON NEGATIVES; NONNEGATIVE MATRIX FACTORIZATION; PERFORMANCE; TEXT-MINING; VARIABLES SELECTIONS;

NON-NEGATIVE MATRIX FACTORIZATION;

EID: 80052651461     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2020408.2020577     Document Type: Conference Paper

References (23)

1. M. Berry, M. Browne, A. Langville, P. Pauca, and R. Plemmon. Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics and Data Analysis, 2007. Submitted.
2. D. Bersekas and J. Tsitsiklis. Parallel and distributed computation. Prentice-Hall, 1989.
3. A. Cichocki and A.-H. Phan. Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Transaction on Fundamentals, E92-A(3):708-721, 2009.
4. E. Gaussier and C. Goutte. Relation between PLSA and NMF and implications. 28th Annual International ACM SIGIR Conference, 2005.
5. E. F. Gonzales and Y. Zhang. Accelerating the Lee-Seung algorithm for non-negative matrix factorization. Technical report, Department of Computational and Applied Mathematics, Rice University, 2005.
6. L. Grippo and M. Sciandrone. On the convergence of the block nonlinear Gauss-Seidel method under convex constraints. Operations Research Letters, 26:127-136, 2000. (Pubitemid 30564332)
7. P. O. Hoyer. Non-negative sparse coding. In Proceedings of IEEE Workshop on Neural Networks for Signal Processing, pages 557-565, 2002.
8. P. O. Hoyer. Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research, 5:1457-1469, 2004.
9. C.-J. Hsieh and I. S. Dhillon. Fast coordinate descent methods with variable selection for non-negative matrix factorization. Department of Computer Science TR-11-06, University of Texas at Austin, 2011.
10. D. Kim, S. Sra, and I. S. Dhillon. Fast Newton-type methods for the least squares nonnegative matrix appoximation problem. Proceedings of the Sixth SIAM International Conference on Data Mining, pages 343-354, 2007.
11. J. Kim and H. Park. Non-negative matrix factorization based on alternating non-negativity constrained least squares and active set method. SIAM Journal on Matrix Analysis and Applications, 30(2):713-730, 2008.
12. J. Kim and H. Park. Toward faster nonnegative matrix factorization: A new algorithm and comparisons. Proceedings of the IEEE International Conference on Data Mining, pages 353-362, 2008.
13. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, November 1998. MNIST database available at http://yann.lecun.com/exdb/mnist/.
14. D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401:788-791, 1999.
15. D. D. Lee and H. S. Seung. Algorithms for non-negative matrix factorization. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, pages 556-562. MIT Press, 2001.
16. D. D. Lewis, Y. Yang, T. G. Rose, and F. Li. RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361-397, 2004.
17. Y. Li and S. Osher. Coordinate descent optimization for l1 minimization with application to compressed sensing; a greedy algorithm. Inverse Probl. Imaging, 3(3):487-503, 2009.
18. C.-J. Lin. Projected gradient methods for non-negative matrix factorization. Neural Computation, 19:2756-2779, 2007.
19. C. Liu, H. chih Yang, J. Fan, L.-W. He, and Y.-M. Wang. Distributed Non-negative Matrix Factorization for Web-Scale Dyadic Data Analysis on MapReduce. 2010.
20. P. Paatero and U. Tapper. Positive matrix factorization: A non-negative factor model with optimal utilization of error. Environmetrics, 5:111-126, 1994.
21. S. Perkins, K. Lacker, and J. Theiler. Grafting: Fast, incremental feature selection by gradient descent in function space. Journal of Machine Learning Research, 3:1333-1356, 2003.
22. J. Piper, P. Pauca, R. Plemmons, and M. Giffin. Object characterization from spectral data using nonnegative factorization and information theory. In Proceedings of AMOS Technical Conference, 2004.
23. R. Zdunek and A. Cichocki. Non-negative matrix factorization with quasi-newton optimization. Eighth International Conference on Artificial Intelligence and Soft Computing, ICAISC, pages 870-879, 2006. (Pubitemid 44089876)