Low complexity damped gauss-newton algorithms for CANDECOMP/PARAFAC

SIAM Journal on Matrix Analysis and Applications

Volumn 34, Issue 1, 2012, Pages 126-147

  Phan, Anh Huy ^a   Tichavský, Petr ^b   Cichocki, Andrzej ^a  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^b INSTITUTE OF INFORMATION THEORY AND AUTOMATION   (Czech Republic)

Author keywords

Alternating least squares; Approximation; CANDECOMP PARAFAC; Canonical decomposition; Complexvalued tensor factorization; Gauss Newton; Inverse problems; Levenberg Marquardt; Line search; Low rank; Tensor factorization

Indexed keywords

ALTERNATING LEAST SQUARES; APPROXIMATION; CANDECOMP; CANONICAL DECOMPOSITION; GAUSS- NEWTON; LEVENBERG-MARQUARDT; LINE SEARCHES; LOW-RANK; TENSOR FACTORIZATION;

ALGORITHMS; BENCHMARKING; FACTORIZATION; GAUSSIAN DISTRIBUTION; INVERSE PROBLEMS; LEAST SQUARES APPROXIMATIONS;

TENSORS;

EID: 84875661105     PISSN: 08954798     EISSN: 10957162     Source Type: Journal    
DOI: 10.1137/100808034     Document Type: Article

References (33)

1. E. Acar, D. M. Dunlavy, and T. G. Kolda, A scalable optimization approach for fitting canonical tensor decompositions, J. Chemometrics, 25 (2011), pp. 67-86.
2. C. A. Andersson and R. Bro, The N-way toolbox for MATLAB, Chemom. Intell. Lab. Syst., 52 (2000), pp. 1-4.
3. R. Bro, PARAFAC. Tutorial and applications, Chemom. Intell. Lab. Syst. 38 (1997), pp. 149- 171.
4. J. D. Carroll and J. J. Chang, Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart-Young decomposition, Psychometrika, 35 (1970), pp. 283-319.
5. A. Cichocki, R. Zdunek, A.-H. Phan, and S. Amari, Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation, Wiley, Chichester, 2009.
6. P. Comon, X. Luciani, and A. L. F. de Almeida, Tensor decompositions, alternating least squares and other tales, J. Chemometrics, 23 (2009), pp. 393-405.
7. L. De Lathauwer, B. De Moor, and J. Vandewalle, On the best rank-1 and rank- (R1,R2,. . ., RN) approximation of higher-order tensors, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1324-1342.
8. P. Guillaume and R. Pintelon, A Gauss-Newton-like optimization algorithm for weighted nonlinear nonlinear least squares problems, IEEE Trans. Signal Processing, 44 (1996), pp. 2222-2228.
9. X. Guo, S. Miron, D. Brie, and A. Stegeman, Uni-mode and partial uniqueness conditions for CANDECOMP/PARAFAC of three-way arrays with linearly dependent loadings, SIAM J. Matrix Anal. Appl., 33 (2012), pp. 111-129.
10. R. A. Harshman, Foundations of the PARAFAC procedure: Models and conditions for an explanatory multimodal factor analysis, UCLA Working Papers in Phonetics, 16 (1970), pp. 1-84.
11. C. Hayashi and F. Hayashi, A new algorithm to solve PARAFAC-model, Behaviormetrika, 11 (1982), pp. 49-60.
12. F. L. Hitchcock, Multiple invariants and generalized rank of a p-way matrix or tensor, J. Math. Phys., 7 (1927), pp. 39-79.
13. R. A. Horn and C. R. Johnson, Matrix Analysis, Cambridge University Press, Cambridge, UK, 1990.
14. T. G. Kolda and B. W. Bader, Tensor decompositions and applications, SIAM Rev., 51 (2009), pp. 455-500.
15. Z. Koldovský, P. Tichavský, and A.-H. Phan, Stability analysis and fast damped Gauss- Newton algorithm for INDSCAL tensor decomposition, in Statistical Signal Processing Workshop (SSP), IEEE, 2011, pp. 581-584.
16. X. Q. Liu and N. D. Sidiropoulos, Cramer-Rao lower bounds for low-rank decomposition of multidimensional arrays, IEEE Trans. Signal Process., 49 (2001), pp. 2074-2086.
17. B. C. Mitchell and D. S. Burdick, Slowly converging PARAFAC sequences: Swamps and two-factor degeneracies, J. Chemometrics, 8 (1994), pp. 155-168.
18. H. B. Nielsen, Damping Parameter in Marquardt's Method, Tech. report, Department of Mathematical Modelling, DTU, Lyngby, Denmark, 1999.
19. P. Paatero, Least-squares formulation of robust nonnegative factor analysis, Chemom. Intell. Lab. Syst., 37 (1997), pp. 23-35.
20. P. Paatero, A weighted non-negative least squares algorithm for three-way PARAFAC factor analysis, Chemom. Intell. Lab. Syst., 38 (1997), pp. 223-242.
21. P. Paatero, The multilinear engine: A table-driven, least squares program for solving multilinear problems, including the n-way parallel factor analysis model, J. Comput. Graph. Statist., 8 (1999), pp. 854-888.
22. A.-H. Phan, P. Tichavský, and A. Cichocki, Damped Gauss-Newton algorithm for nonnegative Tucker decomposition, in Statistical Signal Processing Workshop (SSP), IEEE, 2011, pp. 665 -668.
23. H. W. Sorenson, Parameter Estimation: Principles and Problems, Marcel Dekker, New York, 1980.
24. P. Tichavský and Z. Koldovský, Simultaneous search for all modes in multilinear models, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP10), 2010, pp. 4114-4117.
25. P. Tichavský and Z. Koldovský, Stability of CANDECOMP-PARAFAC tensor decomposition, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP11), 2011, pp. 4164-4167.
26. P. Tichavský and Z. Koldovský, Weight adjusted tensor method for blind separation of underdetermined mixtures of nonstationary sources, IEEE Trans. Signal Process., 59 (2011), pp. 1037-1047.
27. P. Tichavský, A.-H. Phan, and Z. Koldovský, Cramér-Rao Induced Bounds for CANDECOMP/ PARAFAC Tensor Decomposition, ArXiv e-prints, 2012, available online at http://arxiv.org/abs/1209.3215.
28. G. Tomasi, INDAFAC and PARAFAC3W, http://www.models.kvl/dk/source/ indafac/ index.asp, 2003.
29. G. Tomasi, Practical and Computational Aspects in Chemometric Data Analysis, Ph.D. thesis, Department of Food Science, The Royal Veterinary and Agricultural University, Frederiksberg, Denmark, 2006.
30. G. Tomasi, Recent developments in fast algorithms for fitting the PARAFAC model, in TRICAP, Crete, Greece, 2006, available online at http://www.telecom. tuc.gr/~nikos/ TRICAP2006main/TomasiTRICAP2006.ppt.
31. G. Tomasi and R. Bro, PARAFAC and missing values, Chemom. Intell. Lab. Syst., 75 (2005), pp. 163-180.
32. G. Tomasi and R. Bro, A comparison of algorithms for fitting the PARAFAC model, Comput. Statist. Data Anal., 50 (2006), pp. 1700-1734.
33. A. Uschmajew, Local convergence of the alternating least squares algorithm for canonical tensor approximation, SIAM J. Matrix Anal. Appl., 33 (2012), pp. 639-652.