Fast nonnegative matrix/tensor factorization based on low-rank approximation

IEEE Transactions on Signal Processing

Volumn 60, Issue 6, 2012, Pages 2928-2940

  Zhou, Guoxu ^a,b   Cichocki, Andrzej ^a,c   Xie, Shengli ^d  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^b SOUTH CHINA UNIVERSITY OF TECHNOLOGY   (China)

^c SYSTEMS RESEARCH INSTITUTE   (Poland)

^d GUANGDONG UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

Low rank approximation; Nonnegative matrix factorization (NMF); Nonnegative Tucker decomposition (NTD); Principle component analysis (PCA)

Indexed keywords

CONVERGENCE SPEED; EUCLIDEAN DISTANCE; LARGE-SCALE PROBLEM; LATENT FACTOR; LOW RANK; LOW RANK APPROXIMATIONS; MATRIX FACTORIZATIONS; MEMORY REQUIREMENTS; NON-NEGATIVE MATRIX; NON-NEGATIVE MATRIX FACTORIZATION ALGORITHMS; NONNEGATIVE MATRIX FACTORIZATION; NONNEGATIVE TUCKER DECOMPOSITIONS; NONNEGATIVITY CONSTRAINTS; PRINCIPLE COMPONENT ANALYSIS; SYNTHETIC AND REAL DATA; TENSOR FIELDS;

ALGORITHMS; APPROXIMATION THEORY; COMPUTATIONAL COMPLEXITY; FACTORIZATION; PRINCIPAL COMPONENT ANALYSIS;

MATRIX ALGEBRA;

EID: 84861153505     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2012.2190410     Document Type: Review

References (45)

1. A. Cichocki, R. Zdunek, A.-H. Phan, and S. Amari, Nonnegative Matrix and Tensor Factorizations: Applications to ExploratoryMulti-Way Data Analysis and Blind Source Separation. Chichester, U.K.:Wiley, 2009.
2. J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 2000.
3. P.-G. Martinsson, V. Rokhlin, and M. Tygert, "A randomized algorithm for the decomposition of matrices," Appl. Comput. Harmon. Anal., vol. 30, no. 1, pp. 47-68, 2011.
4. G. Tomasi, "Practical and computational aspects in chemometric data analysis," Ph.D. dissertation, Food ScienceDept., RoyalVeterinary and Agricultural Univ., Copenhagen, Denmark, 2006.
5. D. D. Lee and H. S. Seung, "Learning the parts of objects by non-negative matrix factorization," Nature, vol. 401, no. 6755, pp. 788-791, Oct. 21, 1999.
6. M. Berry, M. Browne, A. Langville, V. Pauca, and R. Plemmons, "Algorithms and applications for approximate nonnegative matrix factorization," Comput. Stat. Data Anal., vol. 52, no. 1, pp. 155-173, Sep. 2007.
7. P. Paatero and U. Tapper, "Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values," Environmetrics, vol. 5, no. 2, pp. 111-126, 1994.
8. D. Cai, X. He, J. Han, and T. Huang, "Graph regularized nonnegative matrix factorization for data representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 8, pp. 1548-1560, Aug. 2011.
9. N. Guan, D. Tao, Z. Luo, and B. Yuan, "Manifold regularized discriminative nonnegative matrix factorization with fast gradient descent," IEEE Trans. Image Process., vol. 20, no. 7, pp. 2030-2048, July 2011.
10. Y. Chen, L. Wang, and M. Dong, "Non-negative matrix factorization for semisupervised heterogeneous data coclustering," IEEE Trans. Knowl. Data Eng., vol. 22, no. 10, pp. 1459-1474, Oct. 2010.
11. C. Ding, T. Li, and M. Jordan, "Convex and semi-nonnegative matrix factorizations," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 1, pp. 45-55, Jan. 2010.
12. J.Kimand H. Park, "Toward faster nonnegativematrix factorization: A new algorithm and comparisons," in IEEE 8th Int. Conf. Data Mining (ICDM), Dec. 2008, pp. 353-362.
13. H. Kim and H. Park, "Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method," SIAM J. Matrix Anal. Appl., vol. 30, no. 2, pp. 713-730, 2008.
14. F. Wang and P. Li, "Efficient nonnegative matrix factorization with random projections," in 10th SIAM Int. Conf. Data Minding (SDM), 2010, pp. 281-292.
15. C. Boutsidis and P. Drineas, "Random projections for the nonnegative least-squares problem," Dec. 2008 [Online]. Available: http://arxiv.org/abs/0812.4547, ArXiv e-prints
16. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization," in Advances in Neural Information Processing Systems 13, T. K. Leen, T. G. Dietterich, and V. Tresp, Eds. Cambridge, MA: MIT Press, 2000, pp. 556-562.
17. R. Bro, "Multi-way analysis in the food industry: Models, algorithms& applications," Food Technol., p. 309, 1998 [Online]. Available: http://curis.ku.dk/ws/files/13035961/Rasmus-Bro.pdf
18. Z. He, A. Cichocki, S. Xie, and K. Choi, "Detecting the number of clusters in-way probabilistic clustering," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 11, pp. 2006-2021, 2010.
19. C.-J. Lin, "On the convergence of multiplicative update algorithms for nonnegative matrix factorization," IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1589-1596, Nov. 2007.
20. D. G. Luenberger, Linear and Nonlinear Programming, 2nd ed. Norwell, MA: Kluwer, 2003.
21. P. O. Hoyer, "Non-negative matrix factorization with sparseness constraints," J. Mach. Learn. Res., vol. 5, pp. 1457-1469, Nov. 2004.
22. G. Zhou, S. Xie, Z. Yang, J.-M. Yang, and Z. He, "Minimum- volumeconstrained nonnegative matrix factorization: Enhanced ability of learning parts," IEEE Trans. Neural Netw., vol. 22, no. 10, pp. 1626-1637, Oct. 2011.
23. L. De Lathauwer, B. De Moor, and J. Vandewalle, "A multilinear singular value decomposition," SIAM J. Matrix Anal. Appl., vol. 21, pp. 1253-1278, 2000.
24. L. De Lathauwer, B. De Moor, and J. Vandewalle, "On the best rank-1 and rank-(r1,r2,rn) approximation of higher-order tensors," SIAM J. Matrix Anal. Appl., vol. 21, no. 4, pp. 1324-1342, 2000.
25. T. G. Kolda and B. W. Bader, "Tensor decompositions and applications," SIAM Rev., vol. 51, no. 3, pp. 455-500, 2009.
26. A. Cichocki, "Generalized component analysis and blind source separation methods for analyzing mulitchannel brain signals," in Statistical and Process Models for Cognitive Neuroscience and Aging. Mahwah, NJ: Lawrence Erlbaum Assoc., 2007, pp. 201-272.
27. J. Carroll and J.-J. Chang, "Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart-Young decomposition," Psychometrika, vol. 35, no. 3, pp. 283-319, September 1970.
28. R. A. Harshman, "Foundations of the PARAFAC procedure: Models and conditions for an 'explanatory' multi-modal factor analysis," UCLA Working Papers in Phonet., vol. 16, no. 1, pp. 1-84, 1970.
29. L. Tucker, "Some mathematical notes on three-mode factor analysis," Psychometrika, vol. 31, no. 3, pp. 279-311, Sep. 1966.
30. M. Mørup, L. K. Hansen, and S. M. Arnfred, "Algorithms for sparse nonnegative Tucker decompositions," Neural Comput., vol. 20, no. 8, pp. 2112-2131, Aug. 2008.
31. A. H. Phan and A. Cichocki, "Extended HALS algorithm for nonnegative Tucker decomposition and its applications for multiway analysis and classification," Neurocomput., vol. 74, no. 11, pp. 1956-1969, 2011.
32. M. W. Mahoney, "Randomized algorithms for matrices and data," 4, 2011 [Online]. Available: http://arxiv.org/abs/1104.5557, ArXiv e-prints
33. P. Drineas, R. Kannan, and M. W. Mahoney, "Fast Monte Carlo algorithms for matrices II: Computing a low-rank approximation to a matrix," SIAM J. Comput., vol. 36, no. 1, pp. 158-183, July 2006.
34. N. Halko, P.-G. Martinsson, and J. A. Tropp, "Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions," Dec. 2010 [Online]. Available: http://arxiv.org/abs/0909. 4061
35. A. Cichocki and H. A. Phan, "Fast local algorithms for large scale nonnegative matrix and tensor factorizations," IEICE Trans. Fund. Electron., Commun., Comput. Sci., vol. E92-A, no. 3, pp. 708-721, 2009.
36. C. F. Caiafa and A. Cichocki, "Generalizing the column-row matrix decomposition to multi-way arrays," Linear Algebra Appl., vol. 433, no. 3, pp. 557-573, 2010.
37. N. Vannieuwenhoven, R. Vandebril, and K. Meerbergen, "On the truncated multilinear singular value decomposition," Numerical Analysis and Applied Mathematics Section, Comput. Sci. Dept., K.U.Leuven, Leuven, Belgium, 2011 [Online]. Available: http://www.cs.kuleuven.be/publicaties/rapporten/tw/ TW589.abs.html
38. C.-J. Lin, "Projected gradient methods for nonnegative matrix factorization," Neural Comput., vol. 19, no. 10, pp. 2756-2779, Oct. 2007.
39. E. J. Candes, X. Li, Y. Ma, and J. Wright, "Robust principal component analysis?" Dec. 2009 [Online]. Available: http://arxiv.org/abs/ 0912.3599, ArXiv e-prints
40. A. Cichocki and R. Zdunek, "NMFLAB for signal processing," 2006 [Online]. Available: http://www.bsp.brain.riken.jp/ICALAB/nmflab. html
41. R. Zdunek, H. A. Phan, and A. Cichocki, "Damped Newton iterations for nonnegative matrix factorization," Austral. J. Intell. Inf. Process. Syst., vol. 12, no. 1, pp. 16-22, 2010.
42. ORL Database [Online]. Available: http://www.zjucadcg.cn/dengcai/Data/ FaceData.html
43. D. D. Lewis, Y. Yang, T. G. Rose, and F. Li, "RCV1: A new benchmark collection for text categorization research," J. Mach. Learn. Res., vol. 5, pp. 361-397, Dec. 2004.
44. W.-Y. Chen, Y. Song, H. Bai, C.-J. Lin, and E. Y. Chang, "Parallel spectral clustering in distributed systems," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 3, pp. 568-586, 2011.
45. S. A. Nene, S. K. Nayar, and H. Murase, Columbia Object Image Library (COIL-100), 1996 [Online]. Available: http://www.cs.columbia. edu/CAVE/software/softlib/coil-100.php