Fast alternating LS algorithms for high order CANDECOMP/PARAFAC tensor factorizations

IEEE Transactions on Signal Processing

Volumn 61, Issue 19, 2013, Pages 4834-4846

  Phan, Anh Huy ^a   Tichavský, Petr ^b   Cichocki, Andrzej ^a,c  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^b INSTITUTE OF INFORMATION THEORY AND AUTOMATION   (Czech Republic)

^c SYSTEMS RESEARCH INSTITUTE   (Poland)

Author keywords

ALS; CANDECOMP PARAFAC (CP); Canonical decomposition; Gradient; Tensor factorization

Indexed keywords

ALS; CANDECOMP/PARAFAC; CANONICAL DECOMPOSITION; HIGHER-ORDER TENSOR; KHATRI-RAO PRODUCTS; MEMORY REQUIREMENTS; SEPARATED REPRESENTATION; TENSOR FACTORIZATION;

ALGORITHMS; FACTORIZATION; GRADIENT METHODS;

TENSORS;

EID: 84883782927     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2013.2269903     Document Type: Article

References (52)

1. R. A. Harshman, "Foundations of the PARAFAC procedure: Models and conditions for an explanatory multimodal factor analysis," UCLA Working Papers in Phonet. , vol. 16, pp. 1-84, 1970
2. J. D. Carroll and J. J. Chang, "Analysis of individual differences in multidimensional scaling via an -way generalization of Eckart-Young decomposition," Psychometrika, vol. 35, no. 3, pp. 283-319, 1970
3. C. M. Andersson and R. Bro, "Practical aspects of PARAFAC modelling of fluorescence excitation-emission data," J. Chemometr., vol. 17, pp. 200-215, 2003
4. L. De Lathauwer and J. Castaing, "Tensor-based techniques for the blind separation of DS-CDMA signals," Signal Process., vol. 87, no. 2, pp. 322-336, Feb. 2007 (Pubitemid 44602255)
5. N. D. Sidiropoulos and R. Bro, "PARAFAC techniques for signal separation," in Signal Processing Advances in Communications, P. Stoica, G. Giannakis, Y. Hua, and L. Tong, Eds. Upper Saddle River, NJ, USA: Prentice-Hall, 2000, vol. 2, ch. 4
6. A. S. Field and D. Graupe, "Topographic component (Parallel Factor) analysis of multichannel evoked potentials: Practical issues in trilinear spatiotemporal decomposition," Brain Topogr., vol. 3, no. 4, pp. 407-423, 1991
7. M. Morup, L. K. Hansen, C. S. Herrmann, J. Parnas, and S. M. Arnfred, "Parallel factor analysis as an exploratory tool for wavelet transformed event-related EEG," NeuroImage, vol. 29, no. 3, pp. 938-947, 2006 (Pubitemid 43121218)
8. W. Deburchgraeve, P. J. Cherian, M. de Vos, R. M. C. Swarte, J. H. Blok, G.H. Visser, P.Govaert, and S. van Huffel, "Neonatal seizure localization using PARAFAC decomposition," Clin. Neurophysiol., vol. 120, no. 10, pp. 1787-1796, Oct. 2009
9. P. Constantine, A. Doostan, Q.Wang, and G. Iaccarino, Center for Turbulent Research, "A surrogate accelerated Bayesian inverse analysis of the HyShot II supersonic combustion data," in Proc. Summer Program, 2010, AIAA-2011-2037
10. W. Hackbusch and B. N. Khoromskij, "Tensor-product approximation to operators and functions in high dimensions," J. Complex., vol. 23, no. 4-6, pp. 697-714, 2007 (Pubitemid 350122963)
11. H. Becker, P. Comon, L. Albera, M. Haardt, and I. Merlet, "Multi-way space-time-wave-vector analysis for EEG source separation," Signal Process., vol. 92, no. 4, pp. 1021-1031, 2012
12. B. W. Bader, M. W. Berry, and M. Browne, "Discussion tracking in Enron email using PARAFAC," in Survey of Text Mining II, M. W. Berry and M. Castellanos, Eds. London, U.K.: Springer, 2008, pp. 147-163
13. D. M. Dunlavy, T. G. Kolda, and E. Acar, "Temporal link prediction using matrix and tensor factorizations," ACM Trans. Knowl. Discov. Data, vol. 5, no. 2, Feb. 2011, pp. Article 10, 27 pages
14. A. Shashua and T. Hazan, ICML, "Non-negative tensor factorization with applications to statistics and computer vision," in Proc. 22th Int. Conf. Mach. Learn. (ICML), Bonn, Germany, 2005, pp. 792-799
15. D. González, A. Ammar, F. Chinesta, and E. Cueto, "Recent advances on the use of separated representations," Int. J. Numer. Methods Eng., vol. 81, no. 5, pp. 637-659, 2010
16. R. A. Harshman, "PARAFAC2: Mathematical and technical notes," UCLA Working Papers in Phonet., vol. 22, pp. 30-44, 1972
17. J. B. Kruskal, "Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics," Linear Algebra and Its Appl., vol. 18, no. 2, pp. 95-138, 1977
18. N. D. Sidiropoulos and R. Bro, "On the uniqueness of multilinear decomposition of N-way arrays," J. Chemometr., vol. 14, no. 3, pp. 229-239, 2000
19. C. A. Andersson and R. Bro, "The N-way toolbox for MATLAB," Chemometr. Intell. Lab. Syst., vol. 52, no. 1, pp. 1-4, 2000 (Pubitemid 30621552)
20. M. Rajih, P. Comon, and R. A. Harshman, "Enhanced line search: A novel method to accelerate PARAFAC," SIAM J. Matrix Anal. Appl., vol. 30, no. 3, pp. 1128-1147, 2008
21. G. Tomasi, "Practical and computational aspects in chemometric data analysis," Ph.D. thesis, Food Science, Quality and Technol., The Royal Veterinary and Agricultural Univ., Frederiksberg, Denmark, 2006
22. Y. Chen, D. Han, and L. Qi, "New ALS methods with extrapolating search directions and optimal step size for complex-valued tensor decompositions," IEEE Trans. Signal Process., vol. 59, no. 12, pp. 5888-5898, 2011
23. H. A. L. Kiers, "A three-step algorithm for CANDECOMP/PARAFAC analysis of large data sets with multicollinearity," J. Chemometr., vol. 12, no. 3, pp. 155-171, 1998 (Pubitemid 128478959)
24. E. Acar, D. M. Dunlavy, and T. G. Kolda, "A scalable optimization approach for fitting canonical tensor decompositions," J. Chemometr., vol. 25, no. 2, pp. 67-86, Feb. 2011
25. J.-P. Royer, P. Comon, and N. Thirion-Moreau, "Nonnegative 3-way tensor factorization via conjugate gradient with globally optimal stepsize," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2011, pp. 4040-4043
26. P. Paatero, "A weighted non-negative least squares algorithm for three-way PARAFAC factor analysis," Chemometr. Intell. Lab. Syst., vol. 38, no. 2, pp. 223-242, 1997 (Pubitemid 27512497)
27. P. Tichavský and Z. Koldovský, "Simultaneous search for all modes in multilinear models," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2010, pp. 4114-4117
28. A.-H. Phan, P. Tichavský, and A. Cichocki, "Low complexity damped Gauss-Newton algorithms for CANDECOMP/PARAFAC," SIAM J. Matrix Anal. Appl., vol. 34, no. 1, pp. 126-147, 2013
29. A.-H. Phan, P. Tichavský, and A. Cichocki, "Fast damped Gauss-Newton algorithm for sparse and nonnegative tensor factorization," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2011, pp. 1988-1991
30. L. De Lathauwer, "A link between the canonical decomposition inmultilinear algebra and simultaneousmatrix diagonalization," SIAMJ. Matrix Anal. Appl., vol. 28, pp. 642-666, 2006 (Pubitemid 47109355)
31. F. Roemer and M. Haardt, "A closed-form solution for multilinear PARAFAC decompositions," in Proc. 5th IEEE Sens. Array Multichannel Signal Process. Workshop (SAM), Jul. 2008, pp. 487-491
32. L. De Lathauwer and J. Castaing, "Blind identification of underdetermined mixtures by simultaneous matrix diagonalization," IEEE Trans. Signal Process., vol. 56, no. 3, pp. 1096-1105, 2008 (Pubitemid 351386270)
33. B. W. Bader and T. G. Kolda, "Efficient MATLAB computations with sparse and factored tensors," SIAM J. Scientif. Comput., vol. 30, pp. 205-231, 2007
34. B. W. Bader and T. G. Kolda, MATLAB Tensor Toolbox Version 2.4. Albuquerque, NM, USA: Sandia National Labs, Jan. 2010 [Online]. Available: http://csmr.ca.sandia.gov/~tgkolda/TensorToolbox/
35. B. W. Bader and T. G. Kolda, "Algorithm 862: MATLAB tensor classes for fast algorithm prototyping," ACM Trans. Math. Software, vol. 32, no. 4, pp. 635-653, 2006 (Pubitemid 44927149)
36. G. Tomasi, "Recent developments in fast algorithms for fitting the PARAFAC model," in Proc. TRICAP, Crete, Greece, 2006 [Online]. Available: http://www.telecom.tuc.gr/nikos/TRICAP2006main/TomasiTRICAP2006. ppt
37. Eigenvector Research, Inc., PLS Toolbox [Online]. Available: http:// www.eigenvector.com/software/-pls-toolbox.htm 2012
38. A. Cichocki, R. Zdunek, A.-H. Phan, and S. Amari, Nonnegative Matrix and Tensor Factorizations: Applications to ExploratoryMulti-Way Data Analysis and Blind Source Separation. Chichester, U.K.:Wiley, 2009
39. T. G. Kolda and B. W. Bader, "Tensor decompositions and applications," SIAM Rev., vol. 51, no. 3, pp. 455-500, Sep. 2009
40. S. Ragnarsson and C. F. Van Loan, "Block tensor unfoldings," SIAM J. Matrix Anal. Appl., vol. 33, no. 1, pp. 149-169, 2012
41. F. L. Hitchcock, "The expression of a tensor or a polyadic as a sum of products," J. Math. Phys., vol. 6, pp. 164-189, 1927
42. A. Smilde, R. Bro, and P. Geladi, Multi-Way Analysis: Applications in the Chemical Sciences. New York, NY, USA: Wiley, 2004
43. P. Tichavský, A.-H. Phan, and Z. Koldovský, "Cramér-Rao-induced bounds for CANDECOMP/PARAFAC tensor decomposition," IEEE Trans. Signal Process., vol. 61, no. 8, pp. 1986-1997, 2013
44. P. Tichavský, A.-H. Phan, and A. Cichocki, "A further improvement of a fast damped GAUSS-NEWTON algorithm for CANDECOMPPARAFAC tensor decomposition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2013, pp. 5964-5968
45. M. Morup, L. K. Hansen, and S. M. Arnfred, "ERPWAVELAB a toolbox for multi-channel analysis of time-frequency transformed event related potentials," J. Neurosci. Methods, vol. 161, no. 2, pp. 361-368, 2006
46. A. Karfoul, L. Albera, and L. De Lathauwer, "Iterative methods for the canonical decomposition of multi-way arrays: Application to blind underdetermined mixture identification," Signal Process., vol. 91, no. 8, pp. 1789-1802, 2011
47. G. Tomasi and R. Bro, "2.22-Multilinear models: Iterative methods," in Comprehensive Chemometrics: Chemical and Biochemical Data Analysis, S. Brown, R. Tauler, and B. Walczak, Eds. Oxford, U.K.: Elsevier, 2009, pp. 411-451
48. A. Cichocki and A.-H. Phan, "Fast local algorithms for large scale nonnegativematrix and tensor factorizations," IEICE Trans., vol. 92-A, no. 3, pp. 708-721, 2009
49. J. Chen and Y. Saad, "On the tensor SVD and the optimal low rank orthogonal approximation of tensors," SIAM J.Matrix Anal. Appl., vol. 30, no. 4, pp. 1709-1734, Jan. 2009
50. C. Tallon-Baudry, O. Bertrand, C. Delpuech, and J. Pernier, "Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human," J. Neurosci., vol. 16, no. 13, pp. 4240-4249, 1996 (Pubitemid 26192419)
51. A.-H. Phan and A. Cichocki, "Tensor decompositions for feature extraction and classification of high dimensional datasets," IEICE Nonlin. Theory and Its Appl., vol. 1, pp. 37-68, 2010, (invited paper)
52. A.-H. Phan, NFEA: Tensor Toolbox for Feature Extraction and Applications [Online]. Available: http://www.bsp.brain.riken.jp/~phan/nfea. html 2011