On the best low multilinear rank approximation of higher-order tensors

Recent Advances in Optimization and its Applications in Engineering

Volumn , Issue , 2010, Pages 145-164

  Ishteva, Mariya ^a   Absil, P A ^a   Van Huffel, Sabine ^b   De Lathauwer, Lieven ^b,c  

^a UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

^b UNIVERSITY OF LEUVEN   (Belgium)

^c Group Science   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 80051813983     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-3-642-12598-0_13     Document Type: Chapter

References (75)

1. P.-A. Absil, C. G. Baker, K. A. Gallivan. Trust-region methods on Riemannian manifolds. Found. Comput. Math., 7(3):303-330, 2007.
2. P.-A. Absil, M. Ishteva, L. De Lathauwer, S. Van Huffel. A geometric Newton method for Oja's vector field. Neural Comput., 21(5):1415-1433, 2009.
3. P.-A. Absil, R. Mahony, R. Sepulchre. Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ, 2008.
4. E. Acar, C. A. Bingol, H. Bingol, R. Bro, B. Yener. Multiway analysis of epilepsy tensors. ISMB 2007 Conference Proc., Bioinformatics, 23(13):i10-i18, 2007.
5. R. L. Adler, J.-P. Dedieu, J. Y. Margulies, M. Martens, M. Shub. Newton's method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal., 22(3):359-390, 2002.
6. C. A. Andersson, R. Bro. The N-way toolbox for matlab. Chemometrics and Intelligent Laboratory Systems, 52(1):1-4, 2000. See also http://www.models.kvl. dk/source/nwaytoolbox/.
7. R. Badeau, R. Boyer. Fast multilinear singular value decomposition for structured tensors. SIAM J. Matrix Anal. Appl., 30(3):1008-1021, 2008.
8. C. Caiafa, A. Cichocki. Reconstructing matrices and tensors from few rows and columns. In Proc. of 2009 International Symposium on Nonlinear Theory and its Applications, 2009. In press.
9. J. Carroll, J. Chang. Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart-Young" decomposition. Psychometrika, 35(3):283-319, 1970.
10. J. Chen, Y. Saad. On the tensor SVD and the optimal low rank orthogonal approximation of tensors. SIAM J. Matrix Anal. Appl., 30(4):1709-1734, 2009.
11. A. Cichocki, R. Zdunek, A. H. Phan, and S.-I. Amari. Nonnegative Matrix and Tensor Factorizations. Wiley, 2009.
12. P. Comon. Independent component analysis, a new concept? Signal Processing, 36(3):287-314, 1994.
13. P. Comon. Tensor decompositions. In J. G. McWhirter, I. K. Proudler (eds), Mathematics in Signal Processing V, pp. 1-24. Clarendon Press, Oxford, 2002.
14. P. Comon, G. Golub, L.-H. Lim, B. Mourrain. Symmetric tensors and symmetric tensor rank. SIAM J. Matrix Anal. Appl., 30(3):1254-1279, 2008.
15. A. R. Conn, N. I. M. Gould, P. L. Toint. Trust-Region Methods. MPS-SIAM Series on Optimization. SIAM, Philadelphia, PA, 2000.
16. L. De Lathauwer. Signal Processing Based on Multilinear Algebra. PhD thesis, Dept. of Electrical Engineering, Katholieke Universiteit Leuven, 1997.
17. L. De Lathauwer. A link between the canonical decomposition in multilinear algebra and simultaneous matrix diagonalization. SIAM J. Matrix Anal. Appl., 28(3):642-666, 2006.
18. L. De Lathauwer. Decompositions of a higher-order tensor in block terms - Part I: Lemmas for partitioned matrices. SIAM J. Matrix Anal. Appl., 30(3): 1022-1032, 2008.
19. L. De Lathauwer. Decompositions of a higher-order tensor in block terms - Part II: Definitions and uniqueness. SIAM J. Matrix Anal. Appl., 30(3):1033- 1066, 2008.
20. L. De Lathauwer. A survey of tensor methods. In Proc. of the 2009 IEEE International Symposium on Circuits and Systems (ISCAS 2009), pp. 2773- 2776, Taipei, Taiwan, 2009.
21. L. De Lathauwer, J. Castaing. Tensor-based techniques for the blind separation of DS-CDMA signals. Signal Processing, 87(2):322-336, 2007.
22. L. De Lathauwer, B. De Moor, J. Vandewalle. A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl., 21(4):1253-1278, 2000.
23. L. De Lathauwer, B. De Moor, J. Vandewalle. On the best rank-1 and rank- (R1;R2;::: ;RN) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl., 21(4):1324-1342, 2000.
24. L. De Lathauwer, B. De Moor, J. Vandewalle. Independent component analysis and (simultaneous) third-order tensor diagonalization. IEEE Trans. Signal Process., 49(10):2262-2271, 2001.
25. L. De Lathauwer, B. De Moor, J. Vandewalle. Computation of the canonical decomposition by means of a simultaneous generalized Schur decomposition. SIAM J. Matrix Anal. Appl., 26(2):295-327, 2004.
26. L. De Lathauwer, D. Nion. Decompositions of a higher-order tensor in block terms - Part III: Alternating least squares algorithms. SIAM J. Matrix Anal. Appl., 30(3):1067-1083, 2008.
27. L. De Lathauwer, J. Vandewalle. Dimensionality reduction in higher-order signal processing and rank-(R1;R2;::: ;RN) reduction in multilinear algebra. Linear Algebra Appl., 391:31-55, 2004.
28. V. de Silva, L.-H. Lim. Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM J. Matrix Anal. Appl., 30(3):1084-1127, 2008.
29. M. De Vos, L. De Lathauwer, B. Vanrumste, S. Van Huffel, W. Van Paesschen. Canonical decomposition of ictal scalp EEG and accurate source localization: Principles and simulation study. Journal of Computational Intelligence and Neuroscience, 2007(Article ID 58253):1-10, 2007.
30. M. De Vos, A. Vergult, L. De Lathauwer, W. De Clercq, S. Van Huffel, P. Dupont, A. Palmini, W. Van Paesschen. Canonical decomposition of ictal scalp EEG reliably detects the seizure onset zone. NeuroImage, 37(3):844-854, 2007.
31. M. Elad, P. Milanfar, G. H. Golub. Shape from moments - an estimation theory perspective. IEEE Trans. on Signal Processing, 52(7):1814-1829, 2004.
32. L. Eldén, B. Savas. A Newton-Grassmann method for computing the best multi-linear rank-(r1; r2; r3) approximation of a tensor. SIAM J. Matrix Anal. Appl., 31(2):248-271, 2009.
33. R. Fletcher, C. M. Reeves. Function minimization by conjugate gradients. Comput. J., 7:149-154, 1964.
34. G. H. Golub, C. F. Van Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, Maryland, 3rd edition, 1996.
35. M. Haardt, F. Roemer, G. Del Galdo. Higher-order SVD-based subspace estimation to improve the parameter estimation accuracy in multidimensional harmonic retrieval problems. IEEE Trans. on Signal Processing, 56(7):3198- 3213, 2008.
36. W. W. Hager, H. Zhang. A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM Journal on Optimization, 16(1):170- 192, 2005.
37. R. A. Harshman. Foundations of the PARAFAC procedure: Model and conditions for an "explanatory" multi-mode factor analysis. UCLA Working Papers in Phonetics, 16(1):1-84, 1970.
38. U. Helmke, J. B. Moore. Optimization and Dynamical Systems. Springer- Verlag, 1993.
39. M. R. Hestenes, E. Stiefel. Methods of conjugate gradients for solving linear systems. J. Research Nat. Bur. Standards, 49:409-436 (1953), 1952.
40. F. L. Hitchcock. The expression of a tensor or a polyadic as a sum of products. Journal of Mathematical Physics, 6(1):164-189, 1927.
41. F. L. Hitchcock. Multiple invariants and generalized rank of a p-way matrix or tensor. Journal of Mathematical Physics, 7(1):39-79, 1927.
42. M. Ishteva. Numerical methods for the best low multilinear rank approximation of higher-order tensors. PhD thesis, Dept. of Electrical Engineering, Katholieke Universiteit Leuven, 2009.
43. M. Ishteva, P.-A. Absil, S. Van Huffel, L. De Lathauwer. Best low multilinear rank approximation with conjugate gradients. Tech. Rep. 09-246, ESAT-SISTA, K.U.Leuven, Belgium, 2009.
44. M. Ishteva, P.-A. Absil, S. Van Huffel, L. De Lathauwer. Tucker compression and local optima. Tech. Rep. UCL-INMA-2010.012, Université catholique de Louvain and 09-247, ESAT-SISTA, K.U.Leuven, Belgium, 2010.
45. M. Ishteva, L. De Lathauwer, P.-A. Absil, S. Van Huffel. Dimensionality reduction for higher-order tensors: algorithms and applications. International Journal of Pure and Applied Mathematics, 42(3):337-343, 2008.
46. M. Ishteva, L. De Lathauwer, P.-A. Absil, S. Van Huffel. Best low multilinear rank approximation of higher-order tensors, based on the Riemannian trustregion scheme. Tech. Rep. 09-142, ESAT-SISTA, K.U.Leuven, Belgium, 2009.
47. M. Ishteva, L. De Lathauwer, P.-A. Absil, S. Van Huffel. Differential-geometric Newton method for the best rank-(R1;R2;R3) approximation of tensors. Numerical Algorithms, 51(2):179-194, 2009.
48. M. Ishteva, L. De Lathauwer, S. Van Huffel. Comparison of the performance of matrix and tensor based multi-channel harmonic analysis. In 7th International Conf. on Mathematics in Signal Processing, Cirencester, UK, pp. 77-80, 2006.
49. E. Kofidis, P. A. Regalia. On the best rank-1 approximation of higher-order supersymmetric tensors. SIAM J. Matrix Anal. Appl, 23(3):863-884, 2002.
50. T. Kolda. Orthogonal tensor decompositions. SIAM J. Matrix Anal. Appl., 23:243-255, 2001.
51. T. G. Kolda, B. W. Bader. Tensor decompositions and applications. SIAM Review, 51(3):455-500, 2009.
52. P. M. Kroonenberg. Applied Multiway Data Analysis. Wiley, 2008.
53. P. M. Kroonenberg, J. de Leeuw. Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika, 45(1):69- 97, 1980.
54. M. W. Mahoney, M. Maggioni, P. Drineas. Tensor-CUR decompositions for tensor-based data. SIAM J. Matrix Anal. Appl., 30(3):957-987, 2008.
55. J. H. Manton. Optimization algorithms exploiting unitary constraints. IEEE Trans. Signal Process., 50(3):635-650, 2002.
56. C. D. Moravitz Martin, C. F. Van Loan. A Jacobi-type method for computing orthogonal tensor decompositions. SIAM J. Matrix Anal. Appl., 30(3):1219- 1232, 2008.
57. J. J. Moré, D. C. Sorensen. Computing a trust region step. SIAM J. Sci. Statist. Comput., 4(3):553-572, 1983.
58. J. Nocedal, S. J. Wright. Numerical Optimization. Springer Verlag, New York, 2nd edition, 2006. Springer Series in Operations Research.
59. I. V. Oseledets, D. V. Savostianov, E. E. Tyrtyshnikov. Tucker dimensionality reduction of three-dimensional arrays in linear time. SIAM J. Matrix Anal. Appl., 30(3):939-956, 2008.
60. J.-M. Papy, L. De Lathauwer, S. Van Huffel. Exponential data fitting using multilinear algebra: The single-channel and the multichannel case. Numer. Linear Algebra Appl., 12(8):809-826, 2005.
61. J.-M. Papy, L. De Lathauwer, S. Van Huffel. Exponential data fitting using multilinear algebra: The decimative case. J. Chemometrics, 23(7-8):341-351, 2009.
62. E. Polak, G. Ribière. Note sur la convergence de méthodes de directions conjugu ées. Rev. Française Informat. Recherche Opérationnelle, 3(16):35-43, 1969.
63. B. Savas, L. Eldén. Krylov subspace methods for tensor computations. Tech. Rep. LITH-MAT-R-2009-02-SE, Dept. of Mathematics, Linkôping University, 2009.
64. B. Savas, L.-H. Lim. Best multilinear rank approximation of tensors with quasi- Newton methods on Grassmannians. Tech. Rep. LITH-MAT-R-2008-01-SE, Dept. of Mathematics, Linkôping University, 2008.
65. M. Shub. Some remarks on dynamical systems and numerical analysis. In L. Lara-Carrero, J. Lewowicz (eds), Proc. VII ELAM., pp. 69-92. Equinoccio, U. Simón Bolívar, Caracas, 1986.
66. N. Sidiropoulos, R. Bro, G. Giannakis. Parallel factor analysis in sensor array processing. IEEE Trans. Signal Process., 48:2377-2388, 2000.
67. A. Smilde, R. Bro, P. Geladi. Multi-way Analysis. Applications in the Chemical Sciences. John Wiley and Sons, Chichester, U.K., 2004.
68. S. T. Smith. Geometric Optimization Methods for Adaptive Filtering. PhD thesis, Division of Applied Sciences, Harvard University, Cambridge, MA, 1993.
69. S. T. Smith. Optimization techniques on Riemannian manifolds. In A. Bloch (ed), Hamiltonian and gradient flows, algorithms and control, volume 3 of Fields Inst. Commun., pp. 113-136. Amer. Math. Soc., Providence, RI, 1994.
70. T. Steihaug. The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal., 20(3):626-637, 1983.
71. P. L. Toint. Towards an efficient sparsity exploiting Newton method for minimization. In I. S. Duff (ed), Sparse Matrices and Their Uses, pp. 57-88. Academic Press, London, 1981.
72. L. R. Tucker. The extension of factor analysis to three-dimensional matrices. In H. Gulliksen, N. Frederiksen (eds), Contributions to mathematical psychology, pp. 109-127. Holt, Rinehart & Winston, NY, 1964.
73. L. R. Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31:279-311, 1966.
74. L. Vanhamme, S. Van Huffel. Multichannel quantification of biomedical magnetic resonance spectroscopy signals. In F. Luk (ed), Proc. of SPIE, Advanced Signal Processing Algorithms, Architectures, and Implementations VIII, volume 3461, pp. 237-248, San Diego, California, 1998.
75. T. Zhang, G. H. Golub. Rank-one approximation to high order tensors. SIAM J. Matrix Anal. Appl., 23:534-550, 2001.