Local convergence of the alternating least squares algorithm for canonical tensor approximation

SIAM Journal on Matrix Analysis and Applications

Volumn 33, Issue 2, 2012, Pages 639-652

  Uschmajew, André ^a  

^a TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

Author keywords

ALS; Low rank approximation; Nonlinear Gauss Seidel; PARAFAC

Indexed keywords

ALS; ALTERNATING LEAST SQUARES; CANDECOMP; GAUSS-SEIDEL; HESSIAN MATRICES; LOCAL CONVERGENCE; LOW RANK APPROXIMATIONS; PARAFAC; POSITIVE DEFINITE; TENSOR APPROXIMATION;

APPROXIMATION ALGORITHMS; TENSORS;

LEAST SQUARES APPROXIMATIONS;

EID: 84865499908     PISSN: 08954798     EISSN: 10957162     Source Type: Journal    
DOI: 10.1137/110843587     Document Type: Article

References (27)

1. J.C. Bezdek and R.J. Hathaway, Convergence of alternating optimization, Neural Parallel Sci. Comput., 11 (2003), pp. 351-368. (Pubitemid 37506735)
2. G. Beylkin and M.J. Mohlenkamp, Algorithms for numerical analysis in high dimensions, SIAM J. Sci. Comput., 26 (2005), pp. 2133-2159. (Pubitemid 41641365)
3. P. Comon, X. Luciani, and A.L.F. de Almeida, Tensor decompositions, alternating least squares and other tales, J. Chemometrics, 23 (2009), pp. 393-405.
4. V. De Silva and L.-H. Lim, Tensor rank and the ill-posedness of the best low-rank approximation problem, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 1084-1127.
5. W.H. Greub, Multilinear Algebra, Springer-Verlag, New York, 1967.
6. L. Grippo and M. Sciandrone, Globally convergent block-coordinate techniques for unconstrained optimization, Optim. Methods Softw., 10 (1999), pp. 587-637.
7. L. Grippo and M. Sciandrone, On the convergence of the block nonlinear Gauss-Seidel method under convex constraints, Oper. Res. Lett., 26 (2000), pp. 127-136. (Pubitemid 30564332)
8. R.A. Harshman, Determination and proof of minimum uniqueness conditions for PARAFAC1, UCLA Working Papers in Phonetics, 22 (1972), pp. 111-117.
9. H.B. Keller, On the solution of singular and semidefinite linear systems by iteration, J. Soc. Indust. Appl. Math. Ser. B Numer. Anal., 2 (1965), pp. 281-290.
10. T.G. Kolda, Orthogonal tensor decompositions, SIAM J. Matrix Anal. Appl., 23 (2001), pp. 243-255.
11. T.G. Kolda and B.W. Bader, Tensor decompositions and applications, SIAM Rev., 51 (2009), pp. 455-500.
12. J.B. Kruskal, Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics, Linear Algebra Appl., 18 (1977), pp. 95-138.
13. L.-H. Lim and P. Comon, Nonnegative approximations of nonnegative tensors, J. Chemometrics, 23 (2009), pp. 432-441.
14. Y.-J. Lee, J. Wu, J. Xu, and L. Zikatanov, On the convergence of iterative methods for semidefinite linear systems, SIAM J. Matrix. Anal. Appl., 28 (2006), pp. 634-641. (Pubitemid 47109354)
15. M.J. Mohlenkamp, Musings on multilinear fitting, Linear Algebra Appl., in press.
16. M.J. Mohlenkamp and L. Monzón, Trigonometric identities and sums of separable functions, Math. Intelligencer, 27 (2005), pp. 65-69. (Pubitemid 43026210)
17. C. Navasca, L.D. Lathauwer, and S. Kindermann, Swamp reducing technique for tensor decomposition, in Proceedings of the 16th European Signal Processing Conference, Lausanne, 2008.
18. J.M. Ortega and W.C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970. Reprinted as Classics Appl. Math. 30, SIAM, Philadelphia, 2000.
19. I.V. Oseledets, On a new tensor decomposition, Dokl. Math., 80 (2009), pp. 495-496; translated from Dokl. Akad. Nauk, 427 (2009), pp. 168-169.
20. I.V. Oseledets, Tensor-train decomposition, SIAM J. Sci. Comput., 33 (2011), pp. 2295-2317.
21. P. Paatero, Construction and analysis of degenerate PARAFAC models, J. Chemometrics, 14 (2000), pp. 285-299.
22. M. Rajih, P. Comon, and R.A. Harshman, Enhanced line search: A novel method to accelerate PARAFAC, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 1128-1147.
23. S. Schechter, Iteration methods for nonlinear problems, Trans. Amer. Math. Soc., 104 (1962), pp. 179-189.
24. J.M.F. Ten Berge and J.N. Tendeiro, The link between sufficient conditions by Harshman and by Kruskal for uniqueness in Candecomp/Parafac, J. Chemometrics, 33 (2009), pp. 321-323.
25. G. Tomasi and R. Bro, A comparison of algorithms for fitting the PARAFAC model, Comput. Statist. Data Anal., 50 (2006), pp. 1700-1734. (Pubitemid 43052351)
26. J. Wu, Y.-J. Lee, J. Xu, and L. Zikatanov, Convergence analysis on iterative methods for semidefinite systems, J. Comput. Math., 26 (2008), pp. 797-815.
27. T. Zhang and G.H. Golub, Rank-one approximation to high order tensors, SIAM J. Matrix Anal. Appl., 23 (2001), pp. 534-550.