Nonnegative matrix and tensor factorizations: An algorithmic perspective

IEEE Signal Processing Magazine

Volumn 31, Issue 3, 2014, Pages 54-65

  Zhou, Guoxu ^a   Cichocki, Andrzej ^a   Zhao, Qibin ^a   Xie, Shengli ^b  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^b SOUTH CHINA UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

BLIND SOURCE SEPARATION; FEATURE EXTRACTION; MATRIX ALGEBRA; TENSORS;

LOW-RANK MATRICES; MODEL REDUCTION; MULTIWAY ARRAYS; NON-NEGATIVE MATRIX AND TENSOR FACTORIZATIONS; TENSOR FACTORIZATION;

FACTORIZATION;

EID: 85032751462     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2014.2298891     Document Type: Article

References (48)

1. A. Cichocki, R. Zdunek, A. H. Phan, and S.-I. Amari, Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-Way Data Analysis and Blind Source Separation. Chichester, U.K.: Wiley, 2009.
2. D. D. Lee and H. S. Seung, "Learning the parts of objects by non-negative matrix factorization," Nature, vol. 401, no. 6755, pp. 788-791, Oct. 1999.
3. P. Paatero and U. Tapper, "Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values," Environmetrics, vol. 5, no. 2, pp. 111-126, 1994
4. G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, "Tensor displays: Compressive light field synthesis using multilayer displays with directional backlighting," ACM Trans. Graph. (Proc. SIGGRAPH), vol. 31, no. 4, pp. 1-11, 2012.
5. A. Kumar, V. Sindhwani, and P. Kambadur, "Fast conical hull algorithms for near-separable non-negative matrix factorization," in JMLR W&CP, ser. 1, S. Dasgupta and D. McAllester, Eds. Atlanta, GA: Microtome, 2013, vol. 28, pp. 231-239.
6. C. Thurau, K. Kersting, and C. Bauckhage, "Convex non-negative matrix factorization in the wild," in Proc. 2009 9th IEEE Int. Conf. Data Mining, ser. ICDM'09. Washington, D.C.: IEEE Comput. Soc., 2009, pp. 523-532.
7. D. Donoho and V. Stodden, "When does non-negative matrix factorization give a correct decomposition into parts?" in Advances in Neural Information Processing 16 (Proc. NIPS*2003), vol. 16. Cambridge, MA: MIT Press, 2004.
8. H. Laurberg, M. G. Christensen, M. D. Plumbley, L. K. Hansen, and S. H. Jensen. (2008). Theorems on positive data: On the uniqueness of NMF. Computat. Intell. Neurosci. [Online] http://www.hindawi.com/journals/cin/2008/764206/.
9. N. Gillis, Sparse and unique nonnegative matrix factorization through data preprocessing. J. Mach. Learn. Res., vol. 13, no. 11, 3349-3386, Nov. 2012.
10. K. Huang, N. Sidiropoulos, and A. Swami, "Non-negative matrix factorization revisited: Uniqueness and algorithm for symmetric decomposition," IEEE Trans. Signal Processing, vol. 62, no. 1, pp. 211-224, 2014.
11. G. Zhou, S. Xie, Z. Yang, J.-M. Yang, and Z. He, "Minimum-volume- constrained nonnegative matrix factorization: Enhanced ability of learning parts," IEEE Trans. Neural Networks, vol. 22, no. 10, pp. 1626-1637, Oct. 2011.
12. J. Kim, Y. He, and H. Park, "Algorithms for nonnegative matrix and tensor factorizations: A unified view based on block coordinate descent framework," J. Global Optim., vol. 58, no. 2, pp. 1-35, 2013.
13. G. Zhou, A. Cichocki, and S. Xie, "Fast nonnegative matrix/tensor factorization based on low-rank approximation," IEEE Trans. Signal Processing, vol. 60, no. 6, pp. 2928-2940, June 2012.
14. C.-J. Lin, "On the convergence of multiplicative update algorithms for nonnegative matrix factorization," IEEE Trans. Neural Networks, vol. 18, no. 6, pp. 1589-1596, Nov. 2007. (Pubitemid 350148414)
15. J. Kim and H. Park, "Fast nonnegative matrix factorization: An active-set-like method and comparisons," SIAM J. Sci. Comput., vol. 33, no. 6, pp. 3261-3281, 2011.
16. N. Guan, D. Tao, Z. Luo, and B. Yuan, "NeNMF: An optimal gradient method for nonnegative matrix factorization," IEEE Trans. Signal Processing, vol. 60, no. 6, pp. 2882-2898, 2012.
17. D. G. Luenberger, Linear and Nonlinear Programming, 2nd ed. Norwell, MA: Kluwer, 2003.
18. Y. Zhang, R. Li, and C.-L. Tsai, "Regularization parameter selections via generalized information criterion," J. Amer. Stat. Assoc., vol. 105, no. 489, pp. 312-323, 2010.
19. E. J. Candès, X. Li, Y. Ma, and J. Wright, "Robust principal component analysis?," J. ACM (JACM), vol. 58, no. 3, pp. 11:1-11:37, June 2011.
20. M. D. Craig, "Minimum-volume transforms for remotely sensed data," IEEE Trans. Geosci. Remote Sens., vol. 32, no. 3, pp. 542-552, 1994.
21. J. M. P. Nascimento and J. M. Bioucas-Dias, "Vertex component analysis: A fast algorithm to unmix hyperspectral data," IEEE Trans. Geosci. Remote Sens., vol. 43, no. 4, pp. 898-910, Apr. 2005. (Pubitemid 40476033)
22. V. Bittorf, B. Recht, C. Ré, and J. A. Tropp, "Factoring nonnegative matrices with linear programs," in Proc. Advances in Neural Information Processing Systems (NIPS), vol. 25, pp. 1223-1231, 2012.
23. N. Gillis and S. A. Vavasis, "Fast and robust recursive algorithms for separable nonnegative matrix factorization," IEEE Trans. Pattern Anal. Mach. Intell., to be published.
24. C. Ding, T. Li, and M. Jordan, "Convex and semi-nonnegative matrix factorizations," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 1, pp. 45-55, Jan. 2010.
25. Y.-O. Li, T. Adali, W. Wang, and V. Calhoun, "Joint blind source separation by multiset canonical correlation analysis," IEEE Trans. Signal Processing, vol. 57, no. 10, pp. 3918-3929, Oct. 2009.
26. E. F. Lock, K. A. Hoadley, J. S. Marron, and A. B. Nobel, "Joint and individual variation explained (JIVE) for integrated analysis of multiple data types," Ann. Appl. Stat., vol. 7, no. 1, pp. 523-542, 2013.
27. V. Calhoun, J. Liu, and T. Adali, "A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data," NeuroImage, vol. 45, no. 1, pp. 163-172, 2009.
28. S. Ma, R. Phlypo, V. D. Calhoun, and T. Adali, "Capturing group variability using IVA: a simulation study and graph-theoretical analysis," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), Vancouver, BC, May 2013, pp. 3128-3132.
29. G. Zhou, A. Cichocki, and S. Xie. (2013). Common and individual features analysis: Beyond canonical correlation analysis. [Online]. Available: http://arxiv.org/abs/1212.3913
30. C. Ding, T. Li, W. Peng, and H. Park, "Orthogonal nonnegative matrix tri-factorizations for clustering," in Proc. 12th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, ser. KDD'06. New York: ACM, 2006, pp. 126-135. (Pubitemid 44535510)
31. T. G. Kolda and B. W. Bader, "Tensor decompositions and applications," SIAM Rev., vol. 51, no. 3, pp. 455-500, 2009.
32. A. Cichocki, D. Mandic, A.-H. Phan, C. Caiafa, G. Zhou, Q. Zhao, and L. De Lathauwer, "Tensor decompositions for signal processing application," IEEE Signal Processing Mag., to be published.
33. Q. Zhao, G. Zhou, T. Adali, L. Zhang, and A. Cichocki, "Kernelization of tensor- based models for multiway data analysis," IEEE Signal Processing Mag., vol. 30, no. 4, pp. 137-148, 2013.
34. R. Bro and C. A. Andersson, "Improving the speed of multiway algorithms: Part II: Compression," Chemomet. Intell. Lab. Syst., vol. 42, no. 1-2, pp. 105-113, 1998. (Pubitemid 28440365)
35. N. D. Sidiropoulos and R. Bro, "On the uniqueness of multilinear decomposition of n-way arrays," J. Chemomet., vol. 14, no. 3, pp. 229-239, 2000.
36. M. Mørup, L. K. Hansen, and S. M. Arnfred, "Algorithms for sparse nonnegative Tucker decompositions," Neural Computat., vol. 20, no. 8, pp. 2112-2131, Aug. 2008.
37. L. De Lathauwer, B. De Moor, and J. Vandewalle, "A multilinear singular value decomposition," SIAM J. Matrix Anal. Applicat., vol. 21, no. 4, pp. 1253-1278, Mar.-May 2000. (Pubitemid 33217068)
38. L.-H. Lim and P. Comon, "Nonnegative approximations of nonnegative tensors," J. Chemomet., vol. 23, no. 7-8, pp. 432-441, 2009.
39. J.-P. Royer, N. Thirion-Moreau, and P. Comon, "Computing the polyadic decomposition of nonnegative third order tensors," Signal Process., vol. 91, no. 9, pp. 2159-2171, 2011.
40. J. Kim and H. Park, "Fast nonnegative tensor factorization with an active-set- like method," in High-Performance Scientific Computing, M. W. Berry, K. A. Gallivan, E. Gallopoulos, A. Grama, B. Philippe, Y. Saad, and F. Saied, Eds. London: Springer, 2012, pp. 311-326.
41. H. Phan, P. Tichavsky, and A. Cichocki, "Fast alternating LS algorithms for high order CANDECOMP/PARAFAC tensor factorizations," IEEE Trans. Signal Processing, vol. 61, no. 19, pp. 4834-4846, 2013.
42. G. Zhou, A. Cichocki, and S. Xie, "Accelerated canonical polyadic decomposition by using mode reduction," IEEE Trans. Neural Networks Learn. Syst., vol. 24, no. 12, pp. 2051-2062, Dec. 2013.
43. G. Zhou and A. Cichocki, "Canonical polyadic decomposition based on a single mode blind source separation," IEEE Signal Processing Lett., vol. 19, no. 8, pp. 523-526, Aug. 2012.
44. C.-J. Hsieh and I. S. Dhillon, "Fast coordinate descent methods with variable selection for non-negative matrix factorization," in Proc. 17th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, ser. KDD'11. New York: ACM, 2011, pp. 1064-1072.
45. D. Cai, X. He, J. Han, and T. Huang, "Graph regularized nonnegative matrix factorization for data representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 8, pp. 1548-1560, Aug. 2011.
46. L. Van Der Maaten and C. Detection, "Visualizing data using t-SNE," J. Mach. Learn. Res., vol. 9, pp. 2579-2605, Nov. 2008.
47. F. Nie, C. Ding, D. Luo, and H. Huang, "Improved minmax cut graph clustering with nonnegative relaxation," in Machine Learning and Knowledge Discovery in Databases (Series Lecture Notes in Computer Science). Berlin/Heidelberg: Springer, 2010, vol. 6322, pp. 451-466.
48. C. Liu, H.-c. Yang, J. Fan, L.-W. He, and Y.-M. Wang, "Distributed nonnegative matrix factorization for web-scale dyadic data analysis on MapReduce," in Proc. 19th Int. Conf. World Wide Web, New York: ACM, 2010, pp. 681-690.