A scalable optimization approach for fitting canonical tensor decompositions

Journal of Chemometrics

Volumn 25, Issue 2, 2011, Pages 67-86

  Acar, Evrim ^a   Dunlavy, Daniel M ^b   Kolda, Tamara G ^c  

^a SCIENTIFIC AND TECHNOLOGICAL RESEARCH COUNCIL OF TURKEY   (Turkey)

^b SANDIA NATIONAL LABORATORIES   (United States)

^c SANDIA NATIONAL LABORATORIES   (United States)

Author keywords

CANDECOMP; Optimization; PARAFAC; Tensor decomposition; Tensor factorization

Indexed keywords

ITERATIVE METHODS; LEAST SQUARES APPROXIMATIONS; SIGNAL PROCESSING;

ALTERNATING LEAST SQUARES; CANDECOMP; CANDECOMP/PARAFAC; GRADIENT-BASED OPTIMIZATION METHOD; NONLINEAR LEAST SQUARES METHODS; OPTIMISATIONS; OPTIMIZATION APPROACH; PARAFAC; TENSOR DECOMPOSITION; TENSOR FACTORIZATION;

TENSORS;

EID: 79951898361     PISSN: 08869383     EISSN: 1099128X     Source Type: Journal    
DOI: 10.1002/cem.1335     Document Type: Article

References (58)

1. Carroll JD, Chang JJ. Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart-Young" decomposition. Psychometrika 1970; 35: 283-319.
2. Harshman RA. Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-modal factor analysis. UCLA working papers in phonetics 1970; 16: 1-84. Available at .
3. Acar E, Yener B. Unsupervised multiway data analysis: A literature survey. IEEE Transactions on Knowledge and Data Engineering 2009; 21(1): 6-20.
4. Kolda TG, Bader BW. Tensor decompositions and applications. SIAM Review 2009; 51(3): 455-500.
5. Kruskal JB. Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear Algebra and its Applications 1977; 18(2): 95-138.
6. Andersen CM, Bro R. Practical aspects of PARAFAC modeling of fluorescence excitation-emission data. Journal of Chemometrics 2003; 17(4): 200-215.
7. Bro R, Kiers HAL. A new efficient method for determining the number of components in PARAFAC models. Journal of Chemometrics 2003; 17(5): 274-286.
8. Acar E, Bingol CA, Bingol H, Bro R, Yener B. Multiway analysis of epilepsy tensors. Bioinformatics 2007; 23(13): i10-i18.
9. Mitchell BC, Burdick DS. Slowly converging PARAFAC sequences: Swamps and two-factor degeneracies. Journal of Chemometrics 1994; 8(2): 155-168.
10. Paatero P. Construction and analysis of degenerate PARAFAC models. Journal of Chemometrics 2000; 14(3): 285-299.
11. Rayens WS, Mitchell BC. Two-factor degeneracies and a stabilization of PARAFAC. Chemometrics and Intelligent Laboratory Systems 1997; 38(2): 173.
12. Stegeman A, De Lathauwer L. A method to avoid diverging components in the Candecomp/Parafac model for generic I×J×2 arrays. SIAM Journal on Matrix Analysis and Applications 2009; 30(4): 1614-1638.
13. Faber NKM, Bro R, Hopke PK. Recent developments in CANDECOMP/PARAFAC algorithms: A critical review. Chemometrics and Intelligent Laboratory Systems 2003; 65(1): 119-137.
14. Faber K. Short communication: On solving generalized eigenvalue problems using matlab. Journal of Chemometrics 1997; 11(1): 87-91. Cited by Reference [13].
15. Faber NM, Buydens LMC, Kateman G. Generalized rank annihilation method. I: Derivation of eigenvalue problems. Journal of Chemometrics 1994; 8(2): 147-154. Cited by Reference [13].
16. Faber NM, Buydens LMC, Kateman G. Generalized rank annihilation method. III: Practical implementation. Journal of Chemometrics 1994; 8(4): 273-285. Cited by Reference [13].
17. Gerritsen M, Tanis H, Vandeginste B, Kateman G. Generalized rank annihilation factor analysis, iterative target transformation factor analysis, and residual bilinearization for the quantitative analysis of data from liquid chromatography with photodiode array detection. Analytical Chemistry 1992; 64(18): 2029-2035. Cited by Reference [13].
18. Li S, Hamilton J, Gemperline P. Generalized rank annihilation method using similarity transformations. Analytical Chemistry 1992; 64(6): 599-607. Cited by Reference [13].
19. Lorber A. Features of quantifying chemical composition from two-dimensional data array by the rank annihilation factor analysis method. Analytical Chemistry 1985; 57(12): 2395-2397. Cited by Reference [13].
20. Sanchez E, Kowalski B. Generalized rank annihilation factor analysis. Analytical Chemistry 1986; 58(2): 496-499. Cited by Reference [13].
21. Wu HL, Shibukawa M, Oguma K. An alternating trilinear decomposition algorithm with application to calibration of HPLC-DAD for simultaneous determination of overlapped chlorinated aromatic hydrocarbons. Journal of Chemometrics 1998; 12(1): 1-26.
22. Chen ZP, Wu HL, Jiang JH, Li Y, Yu RQ. A novel trilinear decomposition algorithm for second-order linear calibration. Chemometrics and Intelligent Laboratory Systems 2000; 51(1): 75-86.
23. Chen ZP, Wu HL, Yu RQ. On the self-weighted alternating trilinear decomposition algorithm-the property of being insensitive to excess factors used in calculation. Journal of Chemometrics 2001; 15(5): 439-453.
24. Chen ZP, Li Y, Yu RQ. Pseudo alternating least squares algorithm for trilinear decomposition. Journal of Chemometrics 2001; 15(3): 149-167. Cited by Reference [13].
25. Jiang JH, Wu HL, Li Y, Yu RQ. Alternating coupled vectors resolution (acover) method for trilinear analysis of three-way data. Journal of Chemometrics 1999; 13: 557-578. Cited by Reference [13].
26. Jiang J, Wu H, Li Y, Yu R. Three-way data resolution by alternating slice-wise diagonalization (ASD) method. Journal of Chemometrics 2000; 14(1): 15-36.
27. Li Y, Jiang J, Wu H, Chen Z, Yu R. Alternating coupled matrices resolution method for three-way arrays analysis. Chemometrics and Intelligent Laboratory Systems 2000; 52(1): 33-43. Cited by Reference [13].
28. Tomasi G, Bro R. A comparison of algorithms for fitting the PARAFAC model. Computational Statistics & Data Analysis 2006; 50(7): 1700-1734.
29. Paatero P. A weighted non-negative least squares algorithm for three-way "PARAFAC" factor analysis. Chemometrics and Intelligent Laboratory Systems 1997; 38(2): 223-242.
30. Tomasi G, Bro R. PARAFAC and missing values. Chemometrics and Intelligent Laboratory Systems 2005; 75(2): 163-180.
31. Tomasi G. Practical and computational aspects in chemometric data analysis. Ph.D. thesis, Department of Food Science, The Royal Veterinary and Agricultural University, Frederiksberg, Denmark 2006. Available at .
32. De Lathauwer L, De Moor B, Vandewalle J. Computation of the canonical decomposition by means of a simultaneous generalized Schur decomposition. SIAM Journal on Matrix Analysis and Applications 2004; 26(2): 295-327.
33. De Lathauwer L. A link between the canonical decomposition in multilinear algebra and simultaneous matrix diagonalization. SIAM Journal on Matrix Analysis and Applications 2006; 28(3): 642-666.
34. Vorobyov SA, Rong Y, Sidiropoulos ND, Gershman AB. Robust iterative fitting of multilinear models. IEEE Transactions on Signal Processing 2005; 53(8-1): 2678-2689.
35. Kiers HAL. Towards a standardized notation and terminology in multiway analysis. Journal of Chemometrics 2000; 14(3): 105-122.
36. Smilde A, Bro R, Geladi P. Multi-Way Analysis: Applications in the Chemical Sciences. Wiley: West Sussex, England, 2004.
37. Golub GH, Van Loan CF., Matrix Computations. Johns Hopkins Univ. Press 1996.
38. Khatri CG, Rao CR. Solutions to some functional equations and their applications to characterization of probability distributions. Sankhya: The Indian Journal of Statistics, Series A 1968; 30(2): 167-180.
39. Tomasi G, Bro R. Multilinear models: Iterative methods. In Comprehensive Chemometrics, vol. 2, Brown S, Tauler R, Walczak R (eds). Elsevier: Oxford, 2009; pp. 411-451.
40. Kolda TG. Multilinear operators for higher-order decompositions. Tech. Rep. SAND2006-2081, Sandia National Laboratories, Albuquerque, New Mexico and Livermore, California. 2006.
41. Acar E, Kolda TG, Dunlavy DM. An optimization approach for fitting canonical tensor decompositions. Tech. Rep. SAND2009-0857, Sandia National Laboratories, Albuquerque, NM and Livermore, CA 2009.
42. Paatero P. The multilinear engine: A table-driven, least squares program for solving multilinear problems, including the n-way parallel factor analysis model. Journal of Computational and Graphical Statistics 1999; 8(4): 854-888.
43. Wang JH, Hopke PK. Equation-oriented system: an efficient programming approach to solve multilinear and polynomial equations by the conjugate gradient algorithm. Chemometrics and Intelligent Laboratory Systems 2001; 55(1-2): 13-22.
44. Shashua A, Hazan T. Non-negative tensor factorization with applications to statistics and computer vision. In ICML 2005: Proceedings of the 22nd International Conference on Machine Learning 2005; pp. 792-799.
45. Navasca C, De Lathauwer L, Kinderman S. Swamp reducing technique for tensor decompositions. In EUSIPCO'08: Proceedings of the 16th European Signal Processing Conference 2008.
46. Bader BW, Kolda TG. Tensor Toolbox for MATLAB, version 2.2 last accessed November, 2008. .
47. Bader BW, Kolda TG. Efficient MATLAB computations with sparse and factored tensors. SIAM Journal on Scientific Computing 2007; 30(1): 205-231.
48. Bro R. PARAFAC. Tutorial and applications. Chemometrics and Intelligent Laboratory Systems 1997; 38(2): 149-171.
49. Rajih M, Comon P, Harshman RA. Enhanced line search: A novel method to accelerate parafac. SIAM Journal on Matrix Analysis and Applications 2008; 30(3): 1128-1147.
50. Tomasi G. INDAFAC and PARAFAC3W last accessed March, 2009. .
51. Madsen K, Nielson HB, Tingleff O. Methods for non-linear least squares problems, 2nd edition. Informatics and Mathematical Modelling. Technical University of Denmark: 2004.
52. Dunlavy DM, Kolda TG, Acar E. Poblano v1.0: A Matlab toolbox for gradient-based optimization. Tech. Rep. SAND2010-1422, Sandia National Laboratories, Albuquerque, New Mexico and Livermore, California 2010.
53. Moré JJ, Thuente DJ. Line search algorithms with guaranteed sufficient decrease. ACM Transactions on Mathematical Software 1994; 20(3): 286-307.
54. Nocedal J, Wright SJ. Numerical Optimization. Springer: 1999.
55. Hager WW, Zhang H. A survey of nonlinear conjugate gradient methods. Pacific Journal of Optimiation 2006; 2(1): 35-58.
56. Bro R. Amino acids fluorescence data last accessed December, 2008. .
57. Hopke PK, Paatero P, Jia H, Ross RT, Harshman RA. Three-way (parafac) factor analysis: examination and comparison of alternative computational methods as applied to ill-conditioned data. Chemometrics and Intelligent Laboratory Systems 1998; 43: 25-42.
58. Nash SG, Sofer A. Linear and Nonlinear Programming. McGraw-Hill: New York, NY, 1996.