Extended HALS algorithm for nonnegative Tucker decomposition and its applications for multiway analysis and classification

Neurocomputing

Volumn 74, Issue 11, 2011, Pages 1956-1969

  Phan, Anh Huy ^a   Cichocki, Andrzej ^a,b  

^a RIKEN   (Japan)

^b SYSTEMS RESEARCH INSTITUTE   (Poland)

Author keywords

Alternating least squares (ALS); DEDICOM models; Hierarchical alternating least squares (HALS); Large scale dataset; Multiway classification; Social networks analysis; Tensor decompositions; Tucker decomposition; Tucker 2; Tucker 3

Indexed keywords

ALTERNATING LEAST SQUARES; DATA SETS; DEDICOM MODELS; MULTI-WAY CLASSIFICATION; SOCIAL NETWORKS ANALYSIS; TENSOR DECOMPOSITIONS; TUCKER DECOMPOSITION; TUCKER-2; TUCKER-3;

CLUSTERING ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; COST BENEFIT ANALYSIS; COST FUNCTIONS; INVERSE PROBLEMS; MATRIX ALGEBRA; SPECTRUM ANALYSIS; TENSORS; TEXT PROCESSING;

DATA MINING;

ARTICLE; CLASSIFICATION; DATA ANALYSIS; E-MAIL; ELECTROENCEPHALOGRAM; HIERARCHICAL ALTERNATING LEAST SQUARES; LEARNING ALGORITHM; MATHEMATICAL ANALYSIS; NOISE REDUCTION; NONNEGATIVE TUCKER DECOMPOSITION; PRIORITY JOURNAL; VALIDITY;

EID: 79956048185     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2010.06.031     Document Type: Article

References (57)

1. A.N. Langville, C.D. Meyer, R. Albright, Initializations for the nonnegative matrix factorization, in: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia, USA, 2006.
2. Hancewicz T., Wang J.-H. Discriminant image resolution: a novel multivariate image analysis method utilizing a spatial classification constraint in addition to bilinear nonnegativity. Chemometrics and Intelligent Laboratory Systems 2005, 77:18-31.
3. R. Albright, J. Cox, D. Duling, A.N. Langville, C.D. Meyer, Algorithms, initializations, and convergence for the nonnegative matrix factorization, Technical Report, NCSU Technical Report Math 81706, 2006, URL http://www.wfu.edu/~lemmons/papers.htm.
4. Wang J., Hopke P., Hancewicz T., Zhang S.-L. Application of modified alternating least squares regression to spectroscopic image analysis. Analytica Chimica Acta 2003, 476:93-109.
5. Cichocki A., Zdunek R., Phan A.-H., Amari S. Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation 2009, Wiley, Chichester.
6. T. Kolda, B. Bader, Tensor decompositions and applications, SIAM Review 51 (3), September 2009, 455-500.
7. Lin C.-Y., Cao N., Liu S., Papadimitriou S., Sun J., Yan X. Smallblue: social network analysis for expertise search and collective intelligence. ICDE 2009, 1483-1486.
8. Acar E., Yener B. Unsupervised multiway data analysis: a literature survey. IEEE Transactions on Knowledge and Data Engineering 2008, 21:6-20.
9. Sun J., Papadimitriou S., Lin C.-Y., Cao N., Liu S., Qian W. Multivis: content-based social network exploration through multi-way visual analysis. SDM 2009, 1063-1074.
10. E. Acar, D.M. Dunlavy, T.G. Kolda. Link prediction on evolving data using matrix and tensor factorizations, in: Proceedings of the 2009 IEEE International Conference on Data Mining Workshops ICDMW'09, IEEE Computer Society Press, December 2009 pp. 262-269.
11. D.M. Dunlavy, T.G. Kolda, W.P. Kegelmeyer, Multilinear algebra for analyzing data with multiple linkages, Technical Report Number SAND2006-2079, Sandia National Laboratories, Albuquerque, NM and Livermore, CA, April 2006, http://www.prod.sandia.gov/cgi-bin/techlib/access-control.pl/2006/062079.pdf.
12. B. Bader, R. Harshman, T. Kolda, Temporal analysis of social networks using three-way DEDICOM, Technical Report SAND2006-2161, Sandia National Laboratories, Albuquerque, NM and Livermore, CA, April 2006.
13. J. Sun, D. Tao, S. Papadimitriou, P.S. Yu, C. Faloutsos, Incremental tensor analysis: theory and applications, ACM Transactions on Knowledge Discovery from Data 2 (3).
14. T. Kolda, B. Bader, The TOPHITS model for higher-order web link analysis, in: Proceedings of the SIAM Data Mining Conference Workshop on Link Analysis, Counterterrorism and Security, 2006.
15. Bader B., Harshman R., Kolda T. Temporal analysis of semantic graphs using ASALSAN. ICDM 2007: Proceedings of the 7th IEEE International Conference on Data Mining 2007, 33-42. doi: 10.1109/ICDM.2007.54.
16. Tucker L.R. Implications of factor analysis of three-way matrices for measurement of change. Problems in Measuring Change 1963, 122-137. University of Wisconsin Press. C.W. Harris (Ed.).
17. De Lathauwer L., Vandewalle J. Dimensionality reduction in higher-order signal processing and rank-(R1,R2,...,Rn) reduction in multilinear algebra. Linear Algebra Applications 2004, 391:31-55.
18. Muti D., Bourennane S. Multidimensional filtering based on a tensor approach. Signal Processing 2005, 85(12):2338-2353. doi: http://dx.doi.org/10.1016/j.sigpro.2004.11.029.
19. M. Vasilescu, D. Terzopoulos, Multilinear analysis of image ensembles: Tensorfaces, In: Proceedings of the European Conference on Computer Vision (ECCV), vol. 2350, Copenhagen, Denmark, 2002, pp. 447-460.
20. Mørup M., Hansen L., Arnfred S. Algorithms for sparse nonnegative Tucker decompositions. Neural Computation 2008, 20:2112-2131. URL 〈http://www2.imm.dtu.dk/pubdb/〉.
21. R.A. Harshman, Models for analysis of asymmetrical relationships among n objects or stimuli, Paper presented at the First Joint Meeting of the Psychometric Society and the Society for Mathematical Psychology, Hamilton, 1978.
22. R.A. Harshman, M.E. Lundy, Three-way DEDICOM: analyzing multiple matrices of asymmetric relationships, in: The Annual Meeting of the North American Psychometric Society, 1992, Columbus, Ohio.
23. Lundy M., Harshman R., Kruskal J. A two-stage procedure incorporating good features of both trilinear and quadrilinear models 1989, 123-130.
24. B. Bader, R. Harshman, T. Kolda, Pattern analysis of directed graphs using DEDICOM: an application to Enron Email, Technical Report SAND2006-7744, Sandia National Laboratories, Albuquerque, NM and Livermore, CA, December 2006.
25. De Lathauwer L., de Moor B., Vandewalle J. A multilinear singular value decomposition. SIAM Journal of Matrix Analysis and Applications 2001, 21:1253-1278.
26. De Lathauwer L., Moor B.D., Vandewalle J. On the best rank-1 and rank-(R1,R2, ...,RN) approximation of higher-order tensors. SIAM Journal of Matrix Analysis and Applications 2000, 21(4):1324-1342. doi: http://dx.doi.org/10.1137/S0895479898346995.
27. De Lathauwer L. Decompositions of a higher-order tensor in block terms-part I: lemmas for partitioned matrices. SIAM Journal on Matrix Analysis and Applications (SIMAX) special Issue on Tensor Decompositions and Applications 2008, 30(3):1022-1032. URL 〈http://publi-etis.ensea.fr/2008/De08e〉.
28. De Lathauwer L., Nion D. Decompositions of a higher-order tensor in block terms-part III: alternating least squares algorithms. SIAM Journal on Matrix Analysis and Applications (SIMAX) special Issue Tensor Decompositions and Applications 2008, 30(3):1067-1083. URL 〈http://publi-etis.ensea.fr/2008/DN08〉.
29. Y.-D. Kim, S. Choi, Nonnegative Tucker decomposition, in: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR-2007), Minneapolis, Minnesota, 2007.
30. Phan A.-H., Cichocki A. Fast and efficient algorithms for nonnegative Tucker decomposition. Proceedings of the 5th International Symposium on Neural Networks, Lecture Notes in Computer Science, vol. 5264 2008, 772-782. Springer, Beijing, China.
31. I. Dhillon, S. Sra, Generalized nonnegative matrix approximations with Bregman divergences, in: Neural Information Processing Systems, Vancouver, Canada, 2005, pp. 283-290.
32. A. Cichocki, R. Zdunek, S. Choi, R. Plemmons, S. Amari, Nonnegative tensor factorization using alpha and beta divergencies, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP07), vol. III, Honolulu, Hawaii, USA, 2007, pp. 1393-1396.
33. Andersson C., Bro R. The N-way toolbox for MATLAB. Chemometrics and Intelligent Laboratory Systems 2000, 52(1):1-4.
34. Tucker L. Some mathematical notes on three-mode factor analysis. Psychometrika 1966, 31:279-311.
35. A. Cichocki, A.-H. Phan, C. Caiafa, Flexible HALS algorithms for sparse non-negative matrix/tensor factorization, in: Proceedings of 18th IEEE Workshops on Machine Learning for Signal Processing, Cancun, Mexico, 2008.
36. A. Cichocki, A.-H. Phan, Fast local algorithms for large scale nonnegative matrix and tensor factorizations, IEICE 92-A (3) (2009) 708-721 (invited paper).
37. A. Cichocki, R. Zdunek, S. Amari, Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization, in: Lecture Notes in Computer Science (LNCS), vol. 4666, 2007, pp. 169-176.
38. Cichocki A., Phan A.H., Zdunek R., Zhang L.-Q. Flexible component analysis for sparse. Lecture Notes in Computer Science (LNCS) 2008, vol. 4984:811-820. Springer.
39. Daube-Witherspoon M., Muehllehner G. An iterative image space reconstruction algorthm suitable for volume ECT. IEEE Transactions on Medical Imaging 1986, 5:61-66.
40. De Pierro A. On the relation between the ISRA and the EM algorithm for positron emission tomography. IEEE Transactions on Medial Imaging 1993, 12(2):328-333.
41. Lantéri H., Soummmer R., Aime C. Comparison between ISRA and RLA algorithms: use of a Wiener filter based stopping criterion. Astronomy and Astrophysics Supplemantary Series 1999, 140:235-246.
42. Eggermont P., LaRiccia V. Maximum smoothed likelihood density estimation for inverse problems. Annals of Statistics 1995, 23(1):199-220.
43. Lee D., Seung H. Learning of the parts of objects by non-negative matrix factorization. Nature 1999, 401:788-791.
44. Nikolova M. Minimizers of cost-functions involving nonsmooth data-fidelity terms application to the processing of outliers. SIAM Journal on Numerical Analysis 2002, 40(3):965-994. doi: http://dx.doi.org/10.1137/S0036142901389165.
45. Acar R., Vogel C. Analysis of bounded variation penalty methods for ill-posed problems. IEEE Transactions on Image Processing 1994, 10:1217-1229.
46. T. Chan, H. Zhou, R. Chan, A continuation method for total variation denoising problems, 1995.
47. Goreinov S., Tyrtyshnikov E., Zamarashkin N. A theory of pseudo-skeleton approximations. Linear Algebra and Applications 1997, 261:1-21.
48. Caiafa C.F., Cichocki A. Generalizing the column-row matrix decomposition to multi-way arrays. Linear Algebra and its Applications 2010, 433:557-573.
49. A. Cichocki, R. Zdunek, NTFLAB for signal processing, Technical Report, Laboratory for Advanced Brain Signal Processing, BSI, RIKEN, Saitama, Japan, 2006, URL http://www.bsp.brain.riken.jp.
50. B. Bader, T. Kolda, MATLAB Tensor Toolbox Version 2.2, January 2007, URL http://csmr.ca.sandia.gov/tgkolda/TensorToolbox/.
51. Cichocki A., Washizawa Y., Rutkowski T., Bakardjian H., Phan A.-H., Choi S., Lee H., Zhao Q., Zhang L., Li Y. Noninvasive BCIs: multiway signal-processing array decompositions. Computer 2008, 41(10):34-42. doi: http://doi.ieeecomputersociety.org/10.1109/MC.2008.431.
52. A.-H. Phan, A. Cichocki, Analysis of interactions among hidden components for Tucker model, in: APSIPA Annual Summit and Conference, 2009, pp. 154-158.
53. H. Bakardjian, A. Cichocki, Extraction and classification of common independent components in single-trial crossmodal cortical responses, in: Proceedings of the 5th Annual Meeting of the International Multisensory Research Forum, Barcelona, Spain, 2004, pp. 26-27.
54. S.A. Nene, S.K. Nayar, H. Murase, Columbia object image library (COIL-20), Technical Report CUCS-005-96, Columbia University, February 1996.
55. S. Canu, Y. Grandvalet, V. Guigue, A. Rakotomamonjy, SVM and kernel methods Matlab toolbox, Perception Systems et Information, INSA de Rouen, Rouen, France, 2005.
56. van der Maaten L., Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research 2008, 9:2579-2605.
57. Priebe C.E., Conroy J.M., Marchette D.J., Park Y. Scan statistics on Enron graphs. Computational & Mathematical Organization Theory 2005, 11(3):229-247. doi: http://dx.doi.org/10.1007/s10588-005-5378-z.