Tensor decompositions, alternating least squares and other tales

Journal of Chemometrics

Volumn 23, Issue 7-8, 2009, Pages 393-405

  Comon, P ^a   Luciani, X ^a   de Almeida, A L F ^a  

^a UNIVERSITY OF NICE SOPHIA ANTIPOLIS   (France)

Author keywords

Canonical decomposition; Computational complexity; Degeneracy; PARAFAC; Tensor rank; Three way array

Indexed keywords

COMPUTATIONAL COMPLEXITY;

ALTERNATING LEAST SQUARES; CANONICAL DECOMPOSITION; DEGENERACY; NUMERICAL ALGORITHMS; NUMERICAL COMPLEXITY; PARAFAC; SIMPLE++; TENSOR DECOMPOSITION; TENSOR RANKS; THREE-WAY ARRAYS;

TENSORS;

EID: 70349656252     PISSN: 08869383     EISSN: 1099128X     Source Type: Journal    
DOI: 10.1002/cem.1236     Document Type: Article

References (70)

1. Golub GH, Pereyra V. Differentiation of pseudo-inverses, separable nonlinear least square problems and other tales. In Generalized Inverses and Applications, Nashed MZ (ed.). Academic Press: New York, 1976; 303-324.
2. Sidiropoulos ND, Giannakis GB, Bro R. Blind PARAFAC receivers for DS- CDMA systems. Trans. Sig. Proc. 2000; 48(3): 810-823.
3. DeLathauwer L, Castaing J. Tensor-based techniques for the blind separation of DS-CDMA signals. Signal Processing 2007; 87(2): 322-336.
4. de Almeida ALF, Favier G, Mota JCM. PARAFAC-based unified tensor modeling for wireless communication systems with application to blind multiuser equalization. Signal Processing 2007; 87(2): 337-351.
5. Comon P. Tensor decompositions. In Mathematics in Signal Processing V, McWhirter JG, Proudler IK (eds). Clarendon Press: Oxford, UK, 2002; 1-24.
6. Kruskal JB. Three-way arrays: Rank and uniqueness of trilinear decompositions. Linear Algebra Appl. 1977; 18: 95-138.
7. Strassen V. Rank and optimal computation of generic tensors. Linear Algebra Appl. 1983; 52: 645-685.
8. Bürgisser P, Clausen M, Shokrollahi MA. Algebraic Complexity Theory, vol. 315. Springer: Berlin, 1997.
9. Smilde A, Bro R, Geladi P. Multi-Way Analysis. Wiley: Chichester, 2004.
10. Bro R. Parafac, tutorial and applications. Chemom. Intel. Lab. Syst. 1997; 38: 149-171.
11. Comon P, Rajih M. Blind identification of under-determined mixtures based on the characteristic function. Signal Processing 2006; 86(9): 2271-2281.
12. Kruskal JB, Harshman RA, Lundy ME. How 3-MFA data can cause degenerate Parafac solutions, among other relationships. In Multiway Data Analysis, Coppi R, Bolasco S (eds). Elsevier Science: North-Holland, 1989; 115-121.
13. Rajih M, Comon P, Harshman R. Enhanced line search: a novel method to accelerate PARAFAC. SIAM J. Matrix Anal. Appl. 2008; 30(3): 1148-1171.
14. Mitchell BC, Burdick DS. Slowly converging Parafac sequences: Swamps and two-factor degeneracies. J. Chemom. 1994; 8: 155-168.
15. Rayens WS, Mitchell BC. Two-factor degeneracies and a stabilization of Parafac. Chemom. Intell. Lab. Syst. 1997; 38: 173-181.
16. Paatero P. Construction and analysis of degenerate Parafac models. J. Chemom. 2000; 14: 285-299.
17. Harshman RA. An annoted bibliogrtaphy of articles on de- generate solutions or decompositions, 2004. Unpublished Work, http://publish.uwo.ca//harshman/bibdegen.pdf.
18. ten Berge JMF. Kruskal's polynomial for 2×2×2 arrays and a generalization to 2 × n × n arrays. Psychometrika 1991; 56: 631-636.
19. Stegeman A. Degeneracy in Candecomp/Parafac explained for p × p×2 arrays of rank p+1 or higher. Psychometrika 2006;71(3):483-501.
20. Stegeman A. Degeneracy in Candecomp/Parafac and Indscal explained for several three-sliced arrays with a two-valued typical rank. Psychometrika 2007; 72(4): 601-619.
21. Comon P, Golub G, Lim L-H, Mourrain B. Symmetric tensors and symmetric tensor rank. SIAM J. Matrix Anal. Appl. 2008; 30(3): 1254-1279.
22. De Silva V, Lim L-H. Tensor rank and the ill-posedness of the best low- rank approximation problem. SIAM J. Matrix Anal. Appl. 2008; 30(3): 1084-1127.
23. Krijnen WP, Dijkstra TK, Stegeman A. On the non-existence of optimal solutions and the occurrence of degeneracy in the Candecomp/Parafac model. Psychometrika 2008; 73(3): 431-439.
24. Stegeman A. Low-rank approximation of generic p × q × 2 arrays and diverging components in the Candecomp/Parafac model. SIAM J. Matrix Anal. Appl. 2008; 30(3): 988-1007.
25. Bini D. Border rank of a p × q × 2 tensor and the optimal approximation of a pair of bilinear forms. In Proceedings of the 7th Colloquium on Automata, Languages and Programming, Lecture Notes in Computer Science, vol. 85. Springer-Verlag:London, UK, 1980; 98-108.
26. Bini D. Border rank of m × n × (mn - q) tensors. Linear Algebra Appl. 1986; 79: 45-51.
27. Nion D, DeLathauwer L. An enhanced line search scheme for complex-valued tensor decompositions. application in DS-CDMA. Signal Processing 2008; 88(3): 749-755.
28. DeLathauwer L, de Moor B, Vandewalle J. A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 2000; 21(4): 1253-1278.
29. Bro R, Andersson CA. Improving the speed of multiway algorithms. part ii: Compression. Chemom. Intell. Lab. Syst. 1998; 42(1-2): 105-113.
30. Hitchcock FL. Multiple invariants and generalized rank of a p-way matrix or tensor. J. Math. Phys. 1927; 7(1): 39-79.
31. Carroll JD, Chang JJ. Analysis of individual differences in multidimensional scaling via n-way generalization of Eckart-Young decomposition. Psychometrika 1970; 35(3): 283-319.
32. Harshman RA. Foundations of the Parafac procedure: Models and con- ditions for an explanatory multimodal factor analysis. UCLA Working Papers in Phonetics 1970; 16: 1-84. http://publish.uwo.ca/harshman.
33. Harshman RA, Lundy ME. Parafac: Parallel factor analysis. Computational Statistics and Data Analysis 1994; 39-72.
34. Kroonenberg P. Three Mode Principal Component Analysis. SWO Press: Leiden, 1983.
35. Brambilla MC, Ottaviani G. On the Alexander-Hirschowitz theorem. J. Pure Appl. Algebra 2008; 212: 1229-1251.
36. Alexander J, Hirschowitz A. La methode d'Horace eclatee: application a l'interpolation en degre quatre. Invent. Math. 1992; 107(3): 585-602.
37. Cox D, Little J, O'Shea D. Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra. Undergraduate Texts in Mathematics. Springer Verlag: New York, 1992; 2nd edition in 1996.
38. Comon P, Mourrain B. Decomposition of quantics in sums of powers of linear forms. Signal Processing, Elsevier 1996; 53(2): 93-107; Special Issue on High-Order Statistics.
39. Ehrenborg R. Canonical forms of two by two matrices. Jour. of Algebra 1999; 213: 195-224.
40. Cayley A. On the theory of linear transformation. Cambridge Math. J. 1845; 4: 193-209.
41. Lim L-H, Comon P. Nonnegative approximations of nonnegative tensors. J. Chemom., 2009, this issue.
42. Navasca C, De Lathauwer L, Kindermann S. Swamp reducing technique for tensor decomposition. In 16th European Signal Processing Conference (Eusipco'08), Lausanne, 25-29 August 2008.
43. Terracini A. Sulla rappresentazione delle forme quaternarie mediante somme di potenze di forme lineari. Atti della R. Acc. delle Scienze di Torino 1916; 51: 107-117.
44. Lasker E. Kanonische formen. Math. Annal. 1904; 58: 434-440.
45. Mella M. Singularities of linear systems and the Waring problem. Trans. Am. Math. Soc. 2006; 358(12): 5523-5538.
46. Comon P, ten Berge J. Generic and typical ranks of three-way arrays. In Icassp'08, Las Vegas, 30 March-4 April 2008.
47. Comon P, ten Berge JMF, De Lathauwer L, Castaing J. Generic and typical ranks of multi-way arrays. Linear Algebra Appl., 2009, DOI: 10.1016/j.laa.2009.01.014/
48. Lickteig T. Typical tensorial rank. Linear Algebra Appl. 1985; 69: 95-120.
49. Abo H, Ottaviani G, Peterson C. Induction for secant varieties of Segre varieties, August 2006. arXiv:math/0607191.
50. ten Berge JMF. Partial uniqueness in CANDECOMP/PARAFAC. Jour. Chemometrics 2004; 18: 12-16.
51. ten Berge JMF. The typical rank of tall three-way arrays. Psychometrika 2000; 65(5): 525-532.
52. ten Berge JMF, Stegeman A. Symmetry transformations for square sliced three way arrays, with applications to their typical rank. Linear Algebra Appl. 2006; 418: 215-224.
53. ten Berge JMF, Kiers HAL. Simplicity of core arrays in three-way principal component analysis and the typical rank of p × q × 2 arrays. Linear Algebra Appl. 1999; 294: 169-179.
54. Chiantini L, Ciliberto C. On the concept of k-sectant order of a variety. J. London Math. Soc. 2006; 73(2): 436-454.
55. Stegeman A, Sidiropoulos ND. On Kruskal's uniqueness condition for the Candecomp/Parafac decomposition. Linear Algebra Appl. 2007; 420: 540-552.
56. Landsberg JM. Kruskal's Theorem, 2008, private communication.
57. Sidiropoulos ND, Bro R. On the uniqueness of multilinear decomposition of N-way arrays. J. Chemom. 2000; 14: 229-239.
58. DeLathauwer L. A link between canonical decomposition in multilinear algebra and simultaneous matrix diagonalization. SIAM J. Matrix Anal. 2006; 28(3): 642-666.
59. Rissanen J. Stochastic Complexity in Statistical Inquiry, volume 15 of Series in Computer Science. World Scientific Publication: London, 1989.
60. DeLathauwer L, de Moor B, Vandewalle J. On the best rank-1 and rank- (R1,R2,... RN) approximation of high-order tensors. SIAM J. Matrix Anal. Appl. 2000; 21(4): 1324-1342.
61. Rao CR. Linear Statistical Inference and its Applications. Probability and Statistics. Wiley: New York, 1965.
62. Rao CR, Mitra S. Generalized Inverse of Matrices and Its Applications. New York: Wiley, 1971.
63. Tucker LR. Some mathematical notes for three-mode factor analysis. Psychometrika 1966; 31: 279-311.
64. Paatero P. The multilinear engine: A table-driven, least squares program for solving multilinear problems, including the n-way parallel factor analysis model. J. Comput. Graph. Stat. 1999; 8(4): 854-888.
65. Tomasi G, Bro R. A comparison of algorithms for fitting the Parafac model. Comp. Stat. Data Anal. 2006; 50: 1700-1734.
66. Madsen K, Nielsen HB, Tingleff O. Methods for non-linear least squares problems. Informatics and mathematical Modelling. Technical University of Denmark: Lyngby, 2004.
67. Rajih M, Comon P. Enhanced line search: A novel method to accelerate Parafac. In Eusipco'05, Antalya, Turkey, 4-8 September 2005.
68. Golub GH, Van Loan CF. Matrix Computations. Hopkins Univ. Press: Baltimore, 1989.
69. Chan TF. An improved algorithm for computing the singular value decomposition. ACM Trans. Math. Soft. 1982; 8(1): 72-83.
70. Comon P, Golub GH. Tracking of a few extreme singular values and vectors in signal processing. Proceedings of the IEEE 1990; 78(8): 1327-1343 (published from Stanford report 78NA-89-01, February 1989).