A fast algorithm for nonnegative matrix factorization and its convergence

IEEE Transactions on Neural Networks and Learning Systems

Volumn 25, Issue 10, 2014, Pages 1855-1863

  Li, Li Xin ^a   Wu, Lin ^b   Zhang, Hui Sheng ^a   Wu, Fang Xiang ^b  

^a NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

^b UNIVERSITY OF SASKATCHEWAN   (Canada)

Author keywords

Auxiliary function; convergence; multiplicative updates; nonnegative matrix factorization (NMF); optimization; stationary point

Indexed keywords

AUXILIARY FUNCTIONS; CONVERGENCE; MULTIPLICATIVE UPDATES; NONNEGATIVE MATRIX FACTORIZATION; STATIONARY POINTS;

OPTIMIZATION;

EID: 84907819416     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2296627     Document Type: Article

References (37)

1. H. Hotelling, "Analysis of a complex of statistical variables into principal components," J. Educ. Psychol., vol. 24, no. 6, pp. 417-441, Sep. 1933.
2. M. Bishop, Pattern Recognition and Machine Learning. New York, NY, USA: Springer-Verlag, 2007.
3. A. Hyvarinen, "Fast and robust fixed-point algorithms for independent component analysis," IEEE Trans. Neural Netw., vol. 10, no. 3, pp. 626-634, May 1999.
4. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York, NY, USA: Wiley, 2001
5. J. C. Liao, R. Boscolo, Y.-L. Yang, L. M. Tran, C. Sabatti, and V. P. Roychowdhury, "Network component analysis: Reconstruction of regulatory signals in biological systems," in Proc. Nat. Acad. Sci. United States Amer., vol. 100. Dec. 2003, pp. 15522-15527.
6. R. Boscolo, C. Sabatti, J. C. Liao, and V. P. Roychowdhury, "A generalized framework for network component analysis," IEEE/ACM Trans. Comput. Biol. Bioinf., vol. 2, no. 4, pp. 289-301, Oct./Dec. 2005.
7. D. Lee and H. S. Seung, "Learning the parts of objects by nonnegative matrix factorization," Nature, vol. 401, no. 6755, pp. 788-791, Oct. 1999.
8. P. Paatero and U. Tapper, "Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values," Environmetrics, vol. 5, no. 2, pp. 111-126, Jun. 1994.
9. P. Paatero, "Least squares formulation of robust non-negative factor analysis," Chemometrics Intell. Lab. Syst., vol. 37, no. 1, pp. 23-35, May 1997.
10. P. Paatero, "The multilinear engine-A table-driven least squares program for solving multilinear problems, including the n-way parallel factor analysis model," J. Comput. Graph. Statist., vol. 8, no. 4, pp. 1-35, 1999.
11. L. P. Tian, L. Liu, and F.-X. Wu, "Matrix decomposition methods in bioinformatics," Current Bioinf., vol. 8, no. 2, pp. 259-266, Apr. 2013.
12. H. Kim and H. Park, "Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis," Bioinformatics, vol. 23, no. 12, pp. 1495-1502, 2007.
13. K. Devarajan, "Nonnegative matrix factorization: An analytical and interpretive tool in computational biology," PLoS Comput. Biol., vol. 4, no. 7, p. e1000029, Jul. 2008.
14. D. Greene, G. Cagney, N. Krogan, and P. Cunningham, "Ensemble non-negative matrix factorization methods for clustering protein-protein interactions," Bioinformatics, vol. 24, no. 15, pp. 1722-1728, 2008.
15. Y. Li and A. Ngom, "Nonnegative least-squares methods for the classification of high dimensional biological data," IEEE/ACM Trans. Comput. Biol. Bioinf., vol. 10, no. 2, pp. 447-456, Mar./Apr. 2013.
16. Y. Li and A. Ngom, "Classification approach based on non-negative least squares," Neurocomputing, vol. 118, pp. 41-57, Oct. 2013.
17. Q. Qi, Y. Zhao, M. Li, and R. Simon, "Non-negative matrix factorization of gene expression profiles: A plug-in for BRB-ArrayTools," Bioinformatics, vol. 25, no. 4, pp. 545-547, 2009.
18. F. Shahnaz, M. W. Berry, V. Pauca, and R. J. Plemmons, "Document clustering using nonnegative matrix factorization," Inf. Process. Manag., vol. 42, no. 2, pp. 373-86, Mar. 2006.
19. W. Xu, X. Liu, and Y. Gong, "Document clustering based on nonnegative matrix factorization," in Proc. 26th Annu. Int. ACM SIGIR Conf. Res. Develop. Inf. Retr., 2003, pp. 267-73.
20. M. Berry, M. Browne, A. Langville, V. Pauca, and R. Plemmons, "Algorithms and applications for approximate nonnegative matrix factorization," Comput. Statist. Data Anal., vol. 52, no. 1, pp. 155-173, 2007.
21. P. O. Hoyer, "Non-negative matrix factorization with sparseness constraints," J. Mach. Learn. Res., vol. 5, pp. 1457-1469, Jan. 1, 2014.
22. A. Cichocki, S. Amari, R. Zdunek, R. Kompass, G. Hori, and Z. He, "Extended SMART algorithms for non-negative matrix factorization," in Artificial Intelligence and Soft Computing (Lecture Notes in Computer Science), vol. 4029. New York, NY, USA: Springer-Verlag, 2006, pp. 548-562.
23. G. Zhou, Z. Yang, S. Xie, and J. M. Yang, "Online blind source separation using incremental nonnegative matrix factorization with volume constraint," Trans. Neural Netw., vol. 22, no. 4, pp. 550-560, Apr. 2011.
24. R. Gaujoux and C. Seoighe, "A flexible R package for nonnegative matrix factorization," BMC Bioinf., vol. 11, p. 367, Jul. 2010.
25. Y. Li and A. Ngom, "The non-negative matrix factorization toolbox for biological data mining," BMC Sour. Code Biol. Med., vol. 8, no. 1, pp. 1-10, 2013.
26. Y. X. Wang and Y.-J. Zhang, "Nonnegative matrix factorization: A comprehensive review," IEEE Trans. Knowl. Data Eng., vol. 25, no. 6, pp. 1336-1353, Jun. 2013.
27. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization," in NIPS. Cambridge, MA, USA: MIT Press, 2000.
28. E. Gonzalez and Y. Zhang, "Accelerating the Lee-Seung algorithm for nonnegative matrix factorization," Dept. Comput. Appl. Math., Rice Univ., Houston, TX, USA, Tech. Rep. TR-05-02, 2005.
29. C.-J. Lin, "On the convergence of multiplicative update algorithms for nonnegative matrix factorization," IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1589-1597, Nov. 2007.
30. H. Kim and H. Park, "Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method," SIAM J. Matrix Anal Appl., vol. 30, no. 2, pp. 713-730, May 2008.
31. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2009.
32. C.-J. Lin, "Projected gradient methods for nonnegative matrix factorization," Neural Comput., vol. 19, no. 10, pp. 2756-2779, Oct. 2007.
33. M. Chu, F. Diele, R. Plemmons, and S. Ragni. (2004). Optimality, computation, and interpretations of nonnegative matrix factorizations [Online]. Available: http://www.wfu.edu/~plemmons
34. P. Pauca, J. Piper, and R. Plemmons, "Nonnegative matrix factorization for spectral data analysis," Linear Algebra Appl., vol. 416, no. 1, pp. 29-47, Jul. 2006.
35. C. Ding, X. He, and H. Simon, "On the equivalence of nonnegative matrix factorization and spectral clustering," in Proc. 5th SIAM Int. Conf. Data Mining, Newport Beach, CA, USA, 2005, pp. 606-615.
36. Z. Yang and E. Oja, "Linear and nonlinear projective nonnegative matrix factorization," IEEE Trans. Neural Netw., vol. 21, no. 5, pp. 734-749, May 2010.
37. P. M. Fitzpatrick, Advanced Calculus, 2nd ed. Providence, RI, USA: AMS, 2006.