Seeking an appropriate alternative least squares algorithm for nonnegative tensor factorizations: A novel recursive solution for nonnegative quadratic programming and NTF

Neural Computing and Applications

Volumn 21, Issue 4, 2012, Pages 623-637

  Phan, Anh Huy ^a   Cichocki, Andrzej ^a  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

ALS; Canonical polyadic decomposition (CP); Clustering; EEG analysis; NMF; Nonnegative quadratic programming; Nonnegative tensor factorization; Object classification; PARAFAC; Parallel computing

Indexed keywords

ALS; CANONICAL POLYADIC DECOMPOSITION (CP); CLUSTERING; EEG ANALYSIS; NMF; NONNEGATIVE TENSOR FACTORIZATIONS; OBJECT CLASSIFICATION; PARAFAC;

ALGORITHMS; BENCHMARKING; PARALLEL ARCHITECTURES; PARALLEL PROCESSING SYSTEMS; TENSORS;

FACTORIZATION;

EID: 84861194912     PISSN: 09410643     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00521-011-0652-0     Document Type: Article

References (36)

1. Abadir KM, Magnus JR (2005) Matrix algebra (econometric exercises). Cambridge University Press, URL http://www. amazon. com/exec/obidos/redirect?tag=citeulike07-20&path=ASIN/0521822890.
2. Albright R, Cox J, Duling D, Langville AN, Meyer CD (2006) Algorithms, initializations, and convergence for the nonnegative matrix factorization. Tech. rep., NCSU Technical Report Math 81706, URL http://www. wfu. edu/~plemmons/papers. htm.
3. Cichocki A, Phan AH (2009) Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE 92-A (3): 708-721 (invited paper).
4. Cichocki A, Zdunek R (2006) Multilayer nonnegative matrix factorization. Electron Lett 42(16): 947-948.
5. Cichocki A, Zdunek R, Choi S, Plemmons R, Amari S (2007) Nonnegative tensor factorization using Alpha and Beta divergencies. In: Proceedings of IEEE international conference on acoustics, speech, and signal processing (ICASSP07), vol III, Honolulu, pp 1393-1396.
6. Cichocki A, Washizawa Y, Rutkowski T, Bakardjian H, Phan AH, Choi S, Lee H, Zhao Q, Zhang L, Li Y (2008) Noninvasive BCIs: multiway signal-processing array decompositions. Computer 41(10): 34-42. doi: 10. 1109/MC. 2008. 431. http://www. ieeecomputersociety. org/.
7. Cichocki A, Zdunek R, Phan AH, Amari S (2009) Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation. Wiley, Chichester.
8. Gillis N, Glineur F (2008) Nonnegative factorization and the maximum edge biclique problem. CORE Discussion Papers 2008064, Universit catholique de Louvain, Center for Operations Research and Econometrics (CORE), URL http://econpapers. repec. org/RePEc: cor: louvco: 2008064.
9. Hancewicz T, Wang JH (2005) Discriminant image resolution: a novel multivariate image analysis method utilizing a spatial classification constraint in addition to bilinear nonnegativity. Chemometr Intell Lab Syst 77: 18-31.
10. Harshman R (1970) Foundations of the PARAFAC procedure: models and conditions for an explanatory multimodal factor analysis. UCLA Working Papers Phonetics 16: 1-84.
11. Higham DJ, Higham NJ (2005) MATLAB guide, 2nd edn. SIAM.
12. Higham NJ (2002) Accuracy and stability of numerical algorithms, 2nd edn. SIAM, Philadelphia, xxx+680 pp. ISBN: 0-89871-521-0.
13. Horn RA, Johnson CR (1990) Matrix analysis. Cambridge University Press, Cambridge.
14. Kim YD, Choi S (2007) Nonnegative Tucker decomposition. In: Proceedings of conference on computer vision and pattern recognition (CVPR-2007), Minneapolis, Minnesota.
15. Kivinen J, Warmuth M (1997) Exponentiated gradient versus gradient descent for linear predictors. Inform Comput 132.
16. Kolda T, Bader B (2009) Tensor decompositions and applications. SIAM Rev 51(3): 455-500.
17. Langville AN, Meyer CD, Albright R (2006) Initializations for the nonnegative matrix factorization. In: Proceedings of the twelfth ACM SIGKDD international conference on knowledge discovery and data mining, Philadelphia.
18. Lee D, Seung H (1999) Learning of the parts of objects by non-negative matrix factorization. Nature 401: 788-791.
19. Lee H, Kim YD, Cichocki A, Choi S (2007) Nonnegative tensor factorization for continuous EEG classification. Int J Neural Syst 17(4): 1-13.
20. Lee H, Yoo J, Choi S (2010) Semi-supervised nonnegative matrix factorization. IEEE Signal Process Lett 18.
21. Lin CJ (2007) Projected gradient methods for non-negative matrix factorization. Neural Comput 19(10): 2756-2779.
22. Mørup M, Hansen L, Arnfred S (2006) ERPWAVELAB a toolbox for multi-channel analysis of time-frequency transformed event related potentials. J Neurosci Methods 161(2): 361-368.
23. Morup M, Hansen L, Parnas J, Arnfred S (2006) Decomposing the time-frequency representation of EEG using non-negative matrix and multi-way factorization. Tech. rep., URL http://www2. imm. dtu. dk/pubdb/p. php?4144.
24. Nion D, De Lathauwer L (2008) An enhanced line search scheme for complex-valued tensor decompositions. Application in DS-CDMA. Signal Process 88(3): 749-755.
25. NVIDIA (2008) NVIDIA CUDA Programming Guide 2. 0. NVIDIA, Santa Clara.
26. Paatero P (1997) Least-squares formulation of robust nonnegative factor analysis. Chemometr Intell Lab Syst 37: 23-35.
27. Paatero P (1997) A weighted non-negative least squares algorithm for three-way PARAFAC factor analysis. Chemometr Intell Lab Syst 38(2): 223-242.
28. Phan AH, Cichocki A (2008) Multi-way nonnegative tensor factorization using fast hierarchical alternating least squares algorithm (HALS). In: Proceedings of The 2008 international symposium on nonlinear theory and its applications, Budapest.
29. Phan AH, Cichocki A (2011) PARAFAC algorithms for large-scale problems. Adaptive incremental learning in neural networks; learning algorithm and mathematic modelling selected papers from the international conference on neural information processing - ICONIP 2009. Neurocomputing 74(11): 1970-1984. doi: 10. 1016/j. neucom. 2010. 06. 030. http://www. sciencedirect. com/science/article/pii/S0925231211000415.
30. Rajih M, Comon P, Harshman RA (2008) Enhanced line search: a novel method to accelerate PARAFAC. SIAM J Matrix Anal Appl 30(3): 1128-1147.
31. Sha F, Saul LK, Lee DD (2002) Multiplicative updates for nonnegative quadratic programming in support vector machines. In: Advances in neural information processing systems 15. MIT Press, pp 1041-1048.
32. Sha F, Lin Y, Saul LK, Lee DD (2007) Multiplicative updates for nonnegative quadratic programming. Neural Comput 19.
33. Shashua A, Hazan T (2005) Non-negative tensor factorization with applications to statistics and computer vision. In: Proceedings of the 22-th international conference on machine learning, Bonn.
34. Wang J, Hopke P, Hancewicz T, Zhang SL (2003) Application of modified alternating least squares regression to spectroscopic image analysis. Analytica Chimica Acta 476: 93-109.
35. Zdunek R, Cichocki A (2007) Nonnegative matrix factorization with constrained second-order optimization. Signal Process 87: 1904-1916.
36. Zdunek R, Cichocki A (2008) Nonnegative matrix factorization with quadratic programming. Neurocomputing 71(10-12): 2309-2320.