Efficient Nonnegative Tucker Decompositions: Algorithms and Uniqueness

IEEE Transactions on Image Processing

Volumn 24, Issue 12, 2015, Pages 4990-5003

  Zhou, Guoxu ^a,b   Cichocki, Andrzej ^b,c   Zhao, Qibin ^b   Xie, Shengli ^a  

^a GUANGDONG UNIVERSITY OF TECHNOLOGY   (China)

^b RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^c SYSTEMS RESEARCH INSTITUTE   (Poland)

Author keywords

dimensionality reduction; Nonnegative alternating least squares; Tucker decompositions

Indexed keywords

ALGORITHMS; APPROXIMATION ALGORITHMS; COST FUNCTIONS; DIGITAL STORAGE; TENSORS;

ALTERNATING LEAST SQUARES; COMPUTATION TIME; CURSE OF DIMENSIONALITY; DIMENSIONALITY REDUCTION; NONNEGATIVE TUCKER DECOMPOSITIONS; STORAGE COMPLEXITY; TUCKER DECOMPOSITIONS; UNIQUENESS PROPERTIES;

COMPUTATIONAL EFFICIENCY;

EID: 84959556564     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2015.2478396     Document Type: Article

References (45)

1. P. Comon and C. Jutten, Eds., Handbook of Blind Source Separation: Independent Component Analysis and Applications, 1st ed. New York, NY, USA: Academic, Mar. 2010.
2. S. Yan, D. Xu, Q. Yang, L. Zhang, X. Tang, H.-J. Zhang, "Multilinear discriminant analysis for face recognition," IEEE Trans. Image Process., vol. 16, no. 1, pp. 212-220, Jan. 2007.
3. D. Tao, X. Li, X. Wu, S. J. Maybank, "General tensor discriminant analysis and Gabor features for gait recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 10, pp. 1700-1715, Oct. 2007.
4. A. Cichocki, R. Zdunek, A. H. Phan, S.-I. Amari, Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-Way Data Analysis and Blind Source Separation. Chichester, U.K.: Wiley, 2009.
5. A. Rajwade, A. Rangarajan, A. Banerjee, "Image denoising using the higher order singular value decomposition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 4, pp. 849-862, Apr. 2013.
6. D. D. Lee and H. S. Seung, "Learning the parts of objects by nonnegative matrix factorization," Nature, vol. 401, no. 6755, pp. 788-791, Oct. 1999.
7. B. Li, G. Zhou, A. Cichocki, "Two efficient algorithms for approximately orthogonal nonnegative matrix factorization," IEEE Signal Process. Lett., vol. 22, no. 7, pp. 843-846, Jul. 2015.
8. Z. Li, J. Liu, Y. Yang, X. Zhou, H. Lu, "Clustering-guided sparse structural learning for unsupervised feature selection," IEEE Trans. Knowl. Data Eng., vol. 26, no. 9, pp. 2138-2150, Sep. 2014.
9. Z. Li, J. Liu, J. Tang, H. Lu, "Robust structured subspace learning for data representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 10, pp. 2085-2098, Oct. 2015.
10. P. O. Hoyer, "Non-negative sparse coding," in Proc. 12th IEEE Workshop Neural Netw. Signal Process., Sep. 2002, pp. 557-565, doi: 10.1109/JPROC.2015.2474704.
11. F. Y. Wang, C.-Y. Chi, T.-H. Chan, Y. Wang, "Nonnegative leastcorrelated component analysis for separation of dependent sources by volume maximization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 5, pp. 875-888, May 2010.
12. Y.-D. Kim and S. Choi, "Nonnegative Tucker decomposition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2007, pp. 1-8.
13. M. Mørup, L. K. Hansen, S. M. Arnfred, "Algorithms for sparse nonnegative Tucker decompositions," Neural Comput., vol. 20, no. 8, pp. 2112-2131, Aug. 2008.
14. G. Zhou, A. Cichocki, Q. Zhao, S. Xie, "Nonnegative matrix and tensor factorizations: An algorithmic perspective," IEEE Signal Process. Mag., vol. 31, no. 3, pp. 54-65, May 2014.
15. T. G. Kolda and B. W. Bader, "Tensor decompositions and applications," SIAM Rev., vol. 51, no. 3, pp. 455-500, 2009.
16. E. F. Lock, A. B. Nobel, J. S. Marron, "Comment," J. Amer. Statist. Assoc., vol. 106, no. 495, pp. 798-802, 2011.
17. G. Zhou, Q. Zhao, Y. Zhang, T. Adal, S. Xie, A. Cichocki, "Linked component analysis from matrices to high order tensors: Applications to biomedical data," Proc. IEEE, to be published, doi: 10.1109/JPROC.2015.2474704.
18. Y. Zhang, G. Zhou, Q. Zhao, J. Jin, X. Wang, A. Cichocki, "Spatialtemporal discriminant analysis for ERP-based brain-computer interface," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 21, no. 2, pp. 233-243, Mar. 2013.
19. L.-H. Lim and P. Comon, "Nonnegative approximations of nonnegative tensors," J. Chemometrics, vol. 23, nos. 7-8, pp. 432-441, 2009.
20. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization," in Advances in Neural Information Processing Systems 13, T. K. Leen, T. G. Dietterich, V. Tresp, Eds. Cambridge, MA, USA: MIT Press, 2000, pp. 556-562.
21. H. Kim and H. Park, "Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method," SIAM J. Matrix Anal. Appl., vol. 30, no. 2, pp. 713-730, 2008.
22. J. Kim and H. Park, "Fast nonnegative matrix factorization: An activeset-like method and comparisons," SIAM J. Sci. Comput., vol. 33, no. 6, pp. 3261-3281, 2011.
23. N. Guan, D. Tao, Z. Luo, B. Yuan, "NeNMF: An optimal gradient method for nonnegative matrix factorization," IEEE Trans. Signal Process., vol. 60, no. 6, pp. 2882-2898, Jun. 2012.
24. H. A. Phan and A. Cichocki, "Extended HALS algorithm for nonnegative Tucker decomposition and its applications for multiway analysis and classification," Neurocomputing, vol. 74, no. 11, pp. 1956-1969, May 2011.
25. T. G. Kolda and J. Sun, "Scalable tensor decompositions for multiaspect data mining," in Proc. IEEE 8th Int. Conf. Data Mining (ICDM), Dec. 2008, pp. 363-372.
26. L. De Lathauwer, B. De Moor, J. Vandewalle, "A multilinear singular value decomposition," SIAM J. Matrix Anal. Appl., vol. 21, no. 4, pp. 1253-1278, 2000.
27. N. D. Sidiropoulos, E. E. Papalexakis, C. Faloutsos, "A parallel algorithm for big tensor decomposition using randomly compressed cubes (PARACOMP)," in Proc. IEEE 39th Int. Conf. Acoust., Speech, Signal Process. (ICASSP), May 2014, pp. 1-5.
28. C. F. Caiafa and A. Cichocki, "Generalizing the column-row matrix decomposition to multi-way arrays," Linear Algebra Appl., vol. 433, no. 3, pp. 557-573, 2010.
29. C. E. Tsourakakis, "MACH: Fast randomized tensor decompositions," in Proc. SIAM Int. Conf. Data Mining, 2010, pp. 689-700.
30. G. Zhou, A. Cichocki, S. Xie. (2014). "Decomposition of big tensors with low multilinear rank." [Online]. Available: http://arxiv.org/abs/1412.1885
31. D. Goldfarb and Z. Qin, "Robust low-rank tensor recovery: Models and algorithms," SIAM J. Matrix Anal. Appl., vol. 35, no. 1, pp. 225-253, 2014.
32. Q. Zhao, L. Zhang, A. Cichocki. (May 2015). "Bayesian sparse Tucker models for dimension reduction and tensor completion." [Online]. Available: http://arxiv.org/abs/1505.02343
33. G. Zhou, A. Cichocki, S. Xie, "Fast nonnegative matrix/tensor factorization based on low-rank approximation," IEEE Trans. Signal Process., vol. 60, no. 6, pp. 2928-2940, Jun. 2012.
34. N. Gillis and F. Glineur, "Accelerated multiplicative updates and hierarchical ALS algorithms for nonnegative matrix factorization," Neural Comput., vol. 24, no. 4, pp. 1085-1105, 2012.
35. C. Ding, T. Li, M. I. Jordan, "Convex and semi-nonnegative matrix factorizations," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 1, pp. 45-55, Jan. 2010.
36. C. F. Van Loan and N. Pitsianis, "Approximation with Kronecker products," in Linear Algebra for Large Scale and Real-Time Applications. Norwell, MA, USA: Kluwer, 1993, pp. 293-314.
37. G. Zhou, A. Cichocki, S. Xie, "Accelerated canonical polyadic decomposition using mode reduction," IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 12, pp. 2051-2062, Dec. 2013.
38. N. Gillis, "Sparse and unique nonnegative matrix factorization through data preprocessing," J. Mach. Learn. Res., vol. 13, no. 1, pp. 3349-3386, Nov. 2012.
39. H. Laurberg, "Uniqueness of non-negative matrix factorization," in Proc. IEEE/SP 14th Workshop Statist. Signal Process., Aug. 2007, pp. 44-48.
40. K. Huang, N. D. Sidiropoulos, A. Swami, "Non-negative matrix factorization revisited: Uniqueness and algorithm for symmetric decomposition," IEEE Trans. Signal Process., vol. 62, no. 1, pp. 211-224, Jun. 2014.
41. G. Zhou, S. Xie, Z. Yang, J.-M. Yang, Z. He, "Minimum-volumeconstrained nonnegative matrix factorization: Enhanced ability of learning parts," IEEE Trans. Neural Netw., vol. 22, no. 10, pp. 1626-1637, Oct. 2011.
42. V. Bittorf, B. Recht, C. Ré, J. A. Tropp, "Factoring nonnegative matrices with linear programs," in Advances in Neural Information Processing Systems 25. Red Hook, NY, USA: Curran Associates, 2012, pp. 1223-1231.
43. N. Gillis and S. A. Vavasis, "Fast and robust recursive algorithms for separable nonnegative matrix factorization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 4, pp. 698-714, Apr. 2014.
44. N. D. Sidiropoulos and R. Bro, "On the uniqueness of multilinear decomposition of N-way arrays," J. Chemometrics, vol. 14, no. 3, pp. 229-239, 2000.
45. N. Guan, D. Tao, Z. Luo, J. Shawe-Taylor. (Jul. 2012). "Mah-NMF: Manhattan non-negative matrix factorization." [Online]. Available: http://arxiv.org/abs/1207.3438