Quasi-Newton methods on Grassmannians and multilinear approximations of tensors

SIAM Journal on Scientific Computing

Volumn 32, Issue 6, 2010, Pages 3352-3393

  Savas, Berkant ^a,c   Lim, Lek Heng ^b  

^a LINKÖPING UNIVERSITY   (Sweden)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

Approximations; Grassmann BFGS; Grassmann limited memory BFGS; Grassmann manifold; Grassmann quasi Newton; Grassmannian; Multilinear rank; Product of Grassmannians; Symmetric multilinear rank; Symmetric tensor; Tensor

Indexed keywords

APPROXIMATION ALGORITHMS; NEWTON-RAPHSON METHOD;

APPROXIMATION; GRASSMANN BFGS; GRASSMANN LIMITED MEMORY BFGS; GRASSMANN MANIFOLD; GRASSMANN QUASI-NEWTON; GRASSMANNIAN; LIMITED MEMORY; MULTILINEAR RANK; PRODUCT OF GRASSMANNIANS; QUASI-NEWTON; SYMMETRIC MULTILINEAR RANK; SYMMETRIC TENSORS; SYMMETRICS;

TENSORS;

EID: 79251508042     PISSN: 10648275     EISSN: None     Source Type: Journal    
DOI: 10.1137/090763172     Document Type: Article

References (55)

1. P.-A. Absil, C. G. Baker, and K. A. Gallivan, Trust-region methods on Riemannian manifolds, Found. Comput. Math., 7 (2007), pp. 303-330.
2. P.-A. Absil, R. Mahony, and R. Sepulchre, Optimization Algorithms on Matrix Manifolds, Princeton University Press, Princeton, NJ, 2008.
3. E. Anderson, Z. Bai, C. H. Bischof, J. W. Demmel, J. J. Dongarra, J. J. Du Croz, A. Greenbaum, S. J. Hammarling, A. McKenney, S. Ostrouchov, and D. C. Sorensen, LAPACK Users' Guide, 3rd ed., SIAM, Philadelphia, 1999.
4. B. W. Bader and T. G. Kolda, Algorithm 862: MATLAB tensor classes for fast algorithm prototyping, ACM Trans. Math. Software, 32 (2006), pp. 635-653.
5. W. M. Boothby, An Introduction to Differentiable Manifolds and Riemannian Geometry, 2nd ed., Academic Press, Orlando, FL, 1986.
6. R. H. Byrd, J. Nocedal, and R. B. Schnabel, Representations of quasi-Newton matrices and their use in limited memory methods, Math. Program., 63 (1994), pp. 129-156.
7. S. H. Cheng and N. J. Higham, A modified Cholesky algorithm based on a symmetric indefinite factorization, SIAM J. Matrix Anal. Appl., 19 (1998), pp. 1097-1110.
8. P. Comon, G. Golub, L.-H. Lim, and B. Mourrain, Symmetric tensors and symmetric tensor rank, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 1254-1279.
9. P. Comon, G. Golub, L.-H. Lim, and B. Mourrain, Genericity and rank deficiency of high order symmetric tensors, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '06), 31 (2006), pp. 125-128.
10. P. Comon and B. Mourrain, Decomposition of quantics in sums of powers of linear forms, Signal Process., 53 (1996), pp. 93-107.
11. L. Conlon, Differentiable Manifolds, 2nd ed., Birkh?auser, Boston, MA, 2001.
12. L. De Lathauwer, Tucker compression, parallel factor analysis and block term decompositions: New results, European Meeting on Challenges in Modern Massive Data Sets (EMMDS '09), Technical University of Denmark, Copenhagen, Denmark, 2009.
13. L. De Lathauwer, Signal Processing Based on Multilinear Algebra, Ph.D. thesis, Department of Electrical Engineering, Katholieke Universiteit Leuven, Leuven, Belgium, 1997.
14. L. De Lathauwer, B. De Moor, and J. Vandewalle, An introduction to independent component analysis, J. Chemometrics, 14 (2000), pp. 123-149.
15. L. De Lathauwer, B. De Moor, and J. Vandewalle, A multilinear singular value decomposition, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1253-1278.
16. L. De Lathauwer, B. De Moor, and J. Vandewalle, On the best rank-1 and rank- (R1,R2, . . . ,RN) approximation of higher-order tensors, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1324-1342.
17. L. De Lathauwer and J. Vandewalle, Dimensionality reduction in higher-order signal processing and rank-(r1, r2, . . . , rn) reduction in multilinear algebra, Linear Algebra Appl., 391 (2004), pp. 31-55.
18. L. De Lathauwer and J. Vandewalle, Dimensionality reduction in ICA and rank- (r1, r2, . . . , rn) reduction in multilinear algebra, in Proceedings of the International Conference on Independent Component Analysis and Blind Signal Separation (ICA '04), 5 (2004), pp. 295-302.
19. V. De Silva and L.-H. Lim, Tensor rank and the ill-posedness of the best low-rank approximation problem, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 1084-1127.
20. J. E. Dennis and J. Moré, Quasi-Newton methods, motivation and theory, SIAM Rev., 19 (1977), pp. 46-89.
21. J. E. Dennis and R. B. Schnabel, A new derivation of symmetric positive definite secant updates, Nonlinear Program., 4 (1981), pp. 167-199.
22. J. E. Dennis and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, SIAM, Philadelphia, 1996.
23. A. Edelman, T. A. Arias, and S. T. Smith, The geometry of algorithms with orthogonality constraints, SIAM J. Matrix Anal. Appl., 20 (1998), pp. 303-353.
24. L. Eldén and B. Savas, A Newton-Grassmann method for computing the best multilinear rank- (r1, r2, r3) approximation of a tensor, SIAM J. Matrix Anal. Appl., 31 (2009), pp. 248-271.
25. H.-J. Flad, B. N. Khoromskij, D. Savostianov, and E. Tyrtyshnikov, Verification of the cross 3D algorithm on quantum chemistry data, Russian J. Numer. Anal. Math. Modelling, 4 (2008), pp. 1-16.
26. D. Gabay, Minimizing a differentiable function over a differential manifold, J. Optim. Theory Appl., 37 (1982), pp. 177-219.
27. D. Goldfarb, Factorized variable metric methods for unconstrained optimization, Math. Comp., 30 (1976), pp. 796-811.
28. G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, MD, 1996.
29. W. Greub, Multilinear Algebra, 2nd ed., Springer-Verlag, New York, NY, 1978.
30. F. L. Hitchcock, The expression of a tensor or a polyadic as a sum of products, J. Math. Phys., 6 (1927), pp. 164-189.
31. M. Ishteva, L. De Lathauwer, P.-A. Absil, and S. Van Huffel, Dimensionality reduction for higher-order tensors: Algorithms and applications, Int. J. Pure Appl. Math., 42 (2008), pp. 337-343.
32. M. Ishteva, L. De Lathauwer, P.-A. Absil, and S. Van Huffel, Best Low Multilinear Rank Approximation of Higher-order Tensors, Based on the Riemannian Trust-region Scheme, Technical report 09-142, ESAT-SISTA, Katholieke Universiteit Leuven, Leuven, Belgium, 2009.
33. M. Ishteva, L. De Lathauwer, P.-A. Absil, and S. Van Huffel, Differential-geometric Newton algorithm for the best rank-(r1, r2, r3) approximation of tensors, Numer. Algorithms, 51 (2009), pp. 179-194.
34. B. N. Khoromskij and V. Khoromskaia, Low rank Tucker-type tensor approximation to classical potentials, Cent. Eur. J. Math., 5 (2007), pp. 523-550.
35. B. N. Khoromskij and V. Khoromskaia, Multigrid accelerated tensor approximation of function related multidimensional arrays, SIAM J. Sci. Comput., 31 (2009), pp. 3002-3026.
36. E. Kofidis and P. A. Regalia, On the best rank-1 approximation of higher-order supersymmetric tensors, SIAM J. Matrix Anal. Appl., 23 (2002), pp. 863-884.
37. T. G. Kolda and B. W. Bader, Tensor decompositions and applications, SIAM Rev., 51 (2009), pp. 455-500.
38. S. Lang, Algebra, rev. 3rd ed., Springer-Verlag, New York, 2002.
39. R. Lehoucq, D. Sorensen, and C. Yang, ARPACK Users' Guide, SIAM, Philadelphia, 1998.
40. L.-H. Lim and J. Morton, Cumulant component analysis: A simultaneous generalization of PCA and ICA, in Proceedings of Computational Algebraic Statistics, Theories and Applications (CASTA '08), Kyoto University, Kyoto, Japan, Intitute of Statistical Mathematics, Tokyo, Japan, 2008.
41. E. Lundström and L. Eldén, Adaptive eigenvalue computations using Newton's method on the Grassmann manifold, SIAM J. Matrix Anal. Appl., 23 (2002), pp. 819-839.
42. J. Morton, Scalable Implicit Symmetric Tensor Approximation, preprint, Pennsylvania State University, University Park, PA, 2010.
43. J. Morton and L.-H. Lim, Principal Cumulant Components Analysis, preprint, University of Chicago, Chicago, IL, 2010.
44. M. Mørup, L. K. Hansen, and S. M. Arnfred, ERPWAVELAB: A toolbox for multi-channel analysis of time-frequency transformed event related potentials, J. Neurosci. Methods, 161 (2007), pp. 361-368.
45. J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed., Springer-Verlag, New York, 2006.
46. L. Omberg, G. H. Golub, and O. Alter, A tensor higher-order singular value decomposition for integrative analysis of DNA microarray data from different studies, Proc. Natl. Acad. Sci. USA, 104 (2007), pp. 18371-18376.
47. I. V. Oseledets, Compact Matrix Form of the d-dimensional Tensor Decomposition, Technical report, Institute of Numerical Mathematics, Russian Academy of Science, Moscow, Russia, 2009.
48. I. V. Oseledets, D. V. Savostianov, and E. E. Tyrtyshnikov, Tucker dimensionality reduction of three-dimensional arrays in linear time, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 939-956.
49. B. Savas, Algorithm Package Manual: Best Low Rank Tensor Approximation, Department of Mathematics, Link?oping Univeristy, Link?oping, Sweden, 2008, . se/ http:// besav/soft.html.
50. B. Savas, Toolbox for Grassmann Manifold Computations, Department of Mathematics, Link?oping Univeristy, Link?oping, Sweden, 2008, http://www.mai.liu.se/besav/soft.html.
51. L. Simonsson, Subspace Computations via Matrix Decompositions and Geometric Optimization, Link?oping Stud. Sci. Tech., 1052, Linköping University, Link?oping, Sweden, 2007.
52. A. Smilde, R. Bro, and P. Geladi, Multi-way Analysis: Applications in the Chemical Sciences, John Wiley, West Sussex, England, 2004.
53. L. R. Tucker, Some mathematical notes on three-mode factor analysis, Psychometrika, 31 (1966), pp. 279-311.
54. M. A. O. Vasilescu and D. Terzopoulos, Multilinear subspace analysis of image ensembles, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Conference (CVPR '03), 2 (2003), pp. 93-99.
55. T. Yokonuma, Tensor Spaces and Exterior Algebra, AMS, Providence, RI, 1992.