Fast nonnegative matrix factorization: An active-set-like method and comparisons

SIAM Journal on Scientific Computing

Volumn 33, Issue 6, 2011, Pages 3261-3281

  Kim, Jingu ^a   Park, Haesun ^a  

^a Georgia Institute of Technology   (United States)

Author keywords

Active set method; Block principal pivoting method; Dimension reduction; Lower rank approximation; Nonnegative matrix factorization; Nonnegativity constrained least squares

Indexed keywords

DATA MINING; MATRIX ALGEBRA; MATRIX FACTORIZATION; PROBLEM SOLVING;

ACTIVE-SET METHODS; BLOCK PRINCIPAL PIVOTING METHOD; CONSTRAINED LEAST SQUARES; DIMENSION REDUCTION; LOW RANK APPROXIMATIONS; NON-NEGATIVITY; NONNEGATIVE LEAST SQUARES; NONNEGATIVE MATRIX FACTORIZATION; NONNEGATIVITY-CONSTRAINED LEAST SQUARE; PIVOTING METHODS;

LEAST SQUARES APPROXIMATIONS;

EID: 84863012243     PISSN: 10648275     EISSN: None     Source Type: Journal    
DOI: 10.1137/110821172     Document Type: Article

References (35)

1. M. Berry, M. Browne, A. Langville, V. Pauca, and R. Plemmons, Algorithms and applications for approximate nonnegative matrix factorization, Comput. Statist. Data Anal., 52 (2007), pp. 155-173. (Pubitemid 47331703)
2. R. Bro and S. De Jong, A fast non-negativity-constrained least squares algorithm, J. Chemometrics, 11 (1997), pp. 393-401. (Pubitemid 127478570)
3. J. Brunet, P. Tamayo, T. Golub, and J. Mesirov, Metagenes and molecular pattern discovery using matrix factorization, Proc. Natl. Acad. Sci., 101 (2004), pp. 4164-4169. (Pubitemid 38405900)
4. J. Cantarella and M. Piatek, tsnnls: A Solver for Large Sparse Least Squares Problem with Non-negative Variables, ArXiv Computer Science e-prints, 2004.
5. A. Cichocki and A.-H. Phan, Fast local algorithms for large scale nonnegative matrix and tensor factorizations, IEICE Trans. Fundamentals Electron. Commun. Comput. Sci., E92-A (2009), pp. 708-721.
6. A. Cichocki, R. Zdunek, and S.-I. Amari, Hierarchical ALS algorithms for nonnegative matrix and 3d tensor factorization, in Independent Component Analysis and Signal Separation, Lecture Notes in Comput. Sci. 4666, Springer, New York, 2007, pp. 169-176.
7. A. Cichocki, R. Zdunek, A. H. Phan, and S.-I. Amari, Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation, Wiley, Hoboken, NJ, 2009.
8. K. Devarajan, Nonnegative matrix factorization: An analytical and interpretive tool in computational biology, PLoS Comput. Biol., 4 (2008), e1000029.
9. I. Dhillon and S. Sra, Generalized nonnegative matrix approximations with Bregman divergences, in Advances in Neural Information Processing Systems 18, Y. Weiss, B. Schölkopf, and J. Platt, eds., MIT Press, Cambridge, MA, 2006, pp. 283-290.
10. N. Gillis and F. Glineur, Nonnegative Factorization and the Maximum Edge Biclique Problem, CORE Discussion Paper 2008/64, Universite Catholique de Louvain, 2008.
11. E. F. Gonzalez and Y. Zhang, Accelerating the Lee-Seung Algorithm for Non-negative Matrix Factorization, Technical report, Department of Computational and Applied Mathematics, Rice University, 2005.
12. L. Grippo and M. Sciandrone, On the convergence of the block nonlinear Gauss-Seidel method under convex constraints, Oper. Res. Lett., 26 (2000), pp. 127-136. (Pubitemid 30564332)
13. N.-D. Ho, Nonnegative Matrix Factorization Algorithms and Applications, Ph.D. Thesis, Universite Catholique de Louvain, 2008.
14. J. J. Júdice and F. M. Pires, A block principal pivoting algorithm for large-scale strictly monotone linear complementarity problems, Comput. Oper. Res., 21 (1994), pp. 587-596.
15. D. Kim, S. Sra, and I. S. Dhillon, Fast Newton-type methods for the least squares nonnegative matrix approximation problem, in Proceedings of the Seventh SIAM International Conference on Data Mining, SIAM, Philadelphia, 2007, pp. 343-354.
16. H. Kim and H. Park, Sparse non-negative matrix factorizations via alternating non-negativityconstrained least squares for microarray data analysis, Bioinform., 23 (2007), pp. 1495-1502. (Pubitemid 47244474)
17. H. Kim and H. Park, Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 713-730.
18. J. Kim and H. Park, Sparse Nonnegative Matrix Factorization for Clustering, Technical report GT-CSE-08-01, 2008, Georgia Institute of Technology.
19. J. Kim and H. Park, Toward faster nonnegative matrix factorization: A new algorithm and comparisons, in Proceedings of the 2008 Eighth IEEE International Conference on Data Mining (ICDM), 2008, pp. 353-362.
20. J. Kim and H. Park, Fast active-set-type algorithms for l1-regularized linear regression, in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS), J. Mach. Learning Res. W&CP, 9 (2010), pp. 397-404.
21. C. L. Lawson and R. J. Hanson, Solving Least Squares Problems, Prentice Hall, 1974.
22. D. D. Lee and H. S. Seung, Learning the parts of objects by non-negative matrix factorization, Nature, 401 (1999), pp. 788-791.
23. D. D. Lee and H. S. Seung, Algorithms for non-negative matrix factorization, in Advances in Neural Information Processing Systems 13, MIT Press, Cambridge, MA, 2001, pp. 556-562.
24. C.-J. Lin, Projected gradient methods for nonnegative matrix factorization, Neural Comput., 19 (2007), pp. 2756-2779.
25. L. Li, G. Lebanon, and H. Park, Fast Algorithm for Non-negative Matrix Factorization with Bregman Divergences via their Relationship to Euclidean Distance, manuscript, 2011.
26. S. Z. Li, X. Hou, H. Zhang, and Q. Cheng, Learning spatially localized, parts-based representation, in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 2001, pp. 207-212.
27. P. Paatero and U. Tapper, Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values, Environmetrics, 5 (1994), pp. 111-126.
28. P. Paatero, Least squares formulation of robust non-negative factor analysis, Chemometrics Intell. Lab. Syst., 37 (1997), pp. 23-35. (Pubitemid 27242783)
29. V. P. Pauca, J. Piper, and R. J. Plemmons, Nonnegative matrix factorization for spectral data analysis, Linear Algebra Appl., 416 (2006), pp. 29-47. (Pubitemid 43737212)
30. V. P. Pauca, F. Shahnaz, M. W. Berry, and R. J. Plemmons, Text mining using nonnegative matrix factorizations, in Proceedings of the Fourth SIAM International Conference on Data Mining, SIAM, Philadelphia, 2004, pp. 452-456.
31. L. F. Portugal, J. J. Judice, and L. N. Vicente, A comparison of block pivoting and interior-point algorithms for linear least squares problems with nonnegative variables, Math. Comp., 63 (1994), pp. 625-643.
32. R. Tibshirani, Regression shrinkage and selection via the lasso, J. Roy. Statist. Soc. Ser. B, 58 (1996), pp. 267-288.
33. M. H. Van Benthem and M. R. Keenan, Fast algorithm for the solution of large-scale nonnegativity-constrained least squares problems, J. Chemometrics, 18 (2004), pp. 441-450. (Pubitemid 41327531)
34. S. A. Vavasis, On the complexity of nonnegative matrix factorization, SIAM J. Optim., 20 (2009), pp. 1364-1377.
35. W. Xu, X. Liu, and Y. Gong, Document clustering based on non-negative matrix factorization, in SIGIR '03: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM Press, New York, 2003, pp. 267-273.