Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method

SIAM Journal on Matrix Analysis and Applications

Volumn 30, Issue 2, 2008, Pages 713-730

  Kim, Hyunsoo ^a   Park, Haesun ^a  

^a Georgia Institute of Technology   (United States)

Author keywords

Active set method; Karush Kuhn Tucker (KKT) conditions; Lower rank approximation; Nonnegative Matrix factorization; Nonnegativity constrained least squares; Two block coordinate descent method

Indexed keywords

ACTIVE SET METHOD; KARUSH-KUHN-TUCKER (KKT) CONDITIONS; LOWER RANK APPROXIMATION; NONNEGATIVE MATRIX FACTORIZATION; NONNEGATIVITY CONSTRAINED LEAST SQUARES; TWO-BLOCK COORDINATE DESCENT METHOD;

ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; CURVE FITTING; ECHO SUPPRESSION; FACTORIZATION; LIGHT MEASUREMENT; MATRIX ALGEBRA; QUADRATIC PROGRAMMING; SET THEORY;

LEAST SQUARES APPROXIMATIONS;

EID: 67349093319     PISSN: 08954798     EISSN: 10957162     Source Type: Journal    
DOI: 10.1137/07069239X     Document Type: Article

References (37)

1. M. W. BERRY, M. BROWNE, A. N. LANGVILLE, V. P. PAUCA, and R. J. PLEMMONS, Algorithms and applications for approximate nonnegative matrix factorization, Comput. Statist. Data Anal., 52 (2007), pp. 155-173.
2. D. P. BERTSEKAS, Nonlinear Programming, 2nd ed., Athena Scientific, Belmont, MA, 1999.
3. R. BRO AND S. DE JONG, A fast non-negativity-constrained least squares algorithm, J. Chemometrics, 11 (1997), pp. 393-401.
4. J. P. BRUNET, P. TAMAYO, T. R. GOLUB, and J. P. MESIROV, Metagenes and molecular pattern discovery using matrix factorization, Proc. Natl. Acad. Sci. USA, 101 (2004), pp. 4164-4169.
5. M. CHU, F. DIELE, R. PLEMMONS, and S. RAGNI, Optimality, Computation and Interpretation of Nonnegative Matrix Factorization, preprint, 2004.
6. C. DING, T. LI, W. PENG, and H. PARK, Orthogonal nonnegative matrix tri-factorizations for clustering, in Proceedings of the International Conference on Knowledge Discovery and Data Mining (KDD 2006), ACM Press, New York, 2006, pp. 126-135.
7. D. DUECK, Q. D. MORRIS, AND B. J. FREY, Multi-way clustering of microarray data using probabilistic sparse matrix factorization, Bioinform., 21 (2005), pp. il44-il51.
8. Y. GAO and G. CHURCH, Improving molecular cancer class discovery through sparse nonnegative matrix factorization, Bioinform., 21 (2005), pp. 3970-3975.
9. G. H. GOLUB and C. F. VAN LOAN, Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, MD, 1996.
10. T. R. GOLUB, D. K. SLONIM, P. TAMAYO, C. HUARD, M. GAASENBEEK, J. P. MESIROV, H. COLLER, M. L. LOH, J. R. DOWNING, M. A. CALIGIURI, C. D. BLOOMFIELD, and E. S. LANDER, Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring, Science, 286 (1999), pp. 531-537.
11. E. F. GONZALES and Y. ZHANG, Accelerating the Lee-Seung Algorithm for Non-negative Matrix Factorization, Tech. report, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 2005.
12. L. GRIPPO AND M. SCIANDRONE, On the convergence of the block nonlinear Gauss-Seidel method under convex constraints, Oper. Res. Lett., 26 (2000), pp. 127-136.
13. P. O. HOYER, Non-negative sparse coding, in Proceedings of the IEEE Workshop on Neural Networks for Signal Processing, 2002, pp. 557-565.
14. P. O. HOYER, Non-negative matrix factorization with sparseness constraints, J. Mach. Learn. Res., 5 (2004), pp. 1457-1469.
15. D. KIM, S. SRA, and I. S. DHILLON, Fast Newton-type methods for the least squares nonnegative matrix approximation problem, in Proceedings of the Seventh SIAM International Conference on Data Mining, SIAM, Philadelphia, 2007, pp. 343-354.
16. H. KIM and H. PARK, Sparse non-negative matrix factorizations via alternating non-negativityconstrained least squares, in Proceedings of the IASTED International Conference on Computational and Systems Biology (CASB2006), D.-Z. Du, ed., 2006, pp. 95-100.
17. H. KIM and H. PARK, Cancer class discovery using non-negative matrix factorization based on alternating non-negativity- constrained least squares, in Proceedings of the Third International Symposium on Bioinformatics Research and Applications, ISBRA 2007, Lecture Notes in Comput. Sci. 4463, Springer-Verlag, New York, 2007, pp. 477-487.
18. H. KIM and H. PARK, Sparse non-negative matrix factorizations via alternating non-negativityconstrained least squares for microarray data analysis, Bioinform., 23 (2007), pp. 1495-1502.
19. P. M. KIM and B. TIDOR, Subsystem identification through dimensionality reduction of largescale gene expression data, Genome Res., 13 (2003), pp. 1706-1718.
20. C. L. LAWSON and R. J. HANSON, Solving Least Squares Problems, Prentice-Hall, Englewood Cliffs, NJ, 1974.
21. D. D. LEE and H. S. SEUNG, Learning the parts of objects by non-negative matrix factorization, Nature, 401 (1999), pp. 788-791.
22. D. D. LEE and H. S. SEUNG, Algorithms for non-negative matrix factorization, in Advances in Neural Information Processing 13, MIT Press, Cambridge, MA, 2001, pp. 556-562.
23. S. Z. LI, X. HOU, H. ZHANG, and Q. CHENG, Learning spatially localized parts-based representations, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2001, pp. 207-212.
24. C. J. LIN, Projected Gradient Methods for Non-negative Matrix Factorization, Tech. Report Information and Support Service ISSTECH-95-013, Department of Computer Science, National Taiwan University, 2005.
25. W. LIU and J. YI, Existing and New Algorithms for Nonnegative Matrix Factorization, Tech. report, University of Texas, Austin, 2003.
26. MATLAB User's Guide, The MathWorks, Natick, MA, 1992.
27. P. PAATERO and U. TAPPER, Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values, Environmetrics, 5 (1994), pp. 111-126.
28. H. PARK and H. KIM, One-sided non-negative matrix factorization and non-negative centroid dimension reduction for text classification, in Proceedings of the Workshop on Text Mining at the Sixth SIAM International Conference on Data Mining, M. Castellanos and M. W. Berry, eds., SIAM, Philadelphia, 2006.
29. A. PASCUAL-MONTANO, J. M. CARAZO, K. KOCHI, D. LEHMANN, and R. D. PASCUAL-MARQUI, Nonsmooth nonnegative matrix factorization (nsNMF), IEEE Trans. Pattern Anal. Machine Intell., 28 (2006), pp. 403-415.
30. V. P. PAUCA, J. PIPER, and R. J. PLEMMONS, Nonnegative matrix factorization for spectral data analysis, Linear Algebra Appl., 416 (2006), pp. 29-47.
31. V. P. PAUCA, F. SHAHNAZ, M. W. BERRY, and R. J. PLEMMONS, Text mining using nonnegative matrix factorizations, in Proceedings of the Fourth SIAM International Conference on Data Mining, SIAM, Philadelphia, 2004, pp. 452-456.
32. J. PIPER, V. P. PAUCA, R. J. PLEMMONS, AND M. GIFFIN, Object characterization from spectral data using nonnegative factorization and information theory, in Proceedings of the Amos Technical Conference, 2004.
33. S. L. POMEROY, P. TAMAYO, M. GAASENBEEK. L. M. STURLA, M. ANGELO. M. E. MCLAUGHLIN, J. Y. KIM, L. C. GOUMNEROVA, P. M. BLACK, C. LAU, J. C. ALLEN. D. ZAGZAG. J. M. OLSON, T. CURRAN, C. WETMORE, J. A. BIEGEL. T. POGGIO. S. MUKHERJEE. R. RIFKIN. A. CALIFANO, G. STOLOVITZKY. D. N. LOUIS, J. P. MESIROV. E. S. LANDER, and T. R. GOLUB, Prediction of central nervous system embryonal tumour outcome based on gene expression, Nature, 415 (2002), pp. 436-442.
34. S. SRA and I. S. DHILLON, Nonnegative Matrix Approximation: Algorithms and Applications, Tech. Report 06-27, University of Texas, Austin, 2006.
35. R. TIBSHIRANI, Regression shrinkage and selection via LASSO, J. Roy. Statist. Soc. B, 58 (1996), pp. 267-288.
36. M. H. VAN BENTHEM and M. R. KEENAN, Fast algorithm for the solution of large-scale nonnegativity-constrained least squares problems, J. Chemometrics, 18 (2004), pp. 441-450.
37. R. ZDUNEK AND A. CICHOCKI, Non-negative matrix factorization with quasi-Newton optimization, in Proceedings of the Eighth International Conference on Artificial Intelligence and Soft Computing (ICAISC), 2006, pp. 870-879.