A new truncation strategy for the higher-order singular value decomposition

SIAM Journal on Scientific Computing

Volumn 34, Issue 2, 2012, Pages

  Vannieuwenhoven, Nick ^a   Vandebril, Raf ^a   Meerbergen, Karl ^a  

^a KU Leuven   (Belgium)

Author keywords

Higherorder singular value decomposition; Multilinear orthogonal projection; Multilinear singular value decomposition; Sequentially truncated higher order singular value decomposition; Tensor

Indexed keywords

CHARACTER RECOGNITION; ERRORS; SINGULAR VALUE DECOMPOSITION;

ERROR EXPRESSIONS; HIGH-ORDER; HIGHER ORDER SINGULAR VALUE DECOMPOSITION; HIGHER-ORDER; MULTILINEAR ORTHOGONAL PROJECTIONS; MULTILINEAR SINGULAR VALUE DECOMPOSITIONS; ORTHOGONAL ITERATION; SEQUENTIALLY TRUNCATED HIGHER-ORDER SINGULAR VALUE DECOMPOSITIONS; SPEED-UP FACTORS; TRUNCATION STRATEGY;

ITERATIVE METHODS;

EID: 84861396243     PISSN: 10648275     EISSN: None     Source Type: Journal    
DOI: 10.1137/110836067     Document Type: Article

References (63)

1. E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen, LAPACK Users' Guide, Third Edition, SIAM, Philadelphia, PA, 1999.
2. C. Andersson and R. Bro, Improving the speed of multi-way algorithms: Part I. Tucker3, Chemometrics and Intelligent Laboratory Systems, 42 (1998), pp. 93-103. (Pubitemid 28440364)
3. B. Bader and T. Kolda, Algorithm 862: MATLAB tensor classes for fast algorithm prototyping, ACM Trans. Math. Software, 32 (2006), pp. 635-653. (Pubitemid 44927149)
4. B. Bader and T. Kolda, MATLAB Tensor Toolbox Version 2.4, Mar. 2010, current version available online at http://www.sandia.gov/~tgkdda/TensorToolbox/ .
5. F. Bell, D. Lambrecht, and M. Head-Gordon, Higher order singular value decomposition in quantum chemistry, Molec. Phys., 108 (2010), pp. 2759-2773.
6. R. Bro and C. Andersson, Improving the speed of multiway algorithms: Part II: Compression, Chemometrics and Intelligent Laboratory Systems, 42 (1998), pp. 105-113. (Pubitemid 28440365)
7. J. Chen and Y. Saad, On the tensor SVD and the optimal low rank orthogonal approximation of tensors, SIAM J. Matrix Anal. Appl., 30 (2009), pp. 1709-1734.
8. P. Comon, G. Golub, L.-H. Lim, and B. Mourrain, Symmetric tensors and symmetric tensor rank, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 1254-1279.
9. R. Costantini, L. Sbaiz, and S. Süsstrunk, Higher order SVD analysis for dynamic texture synthesis, IEEE Trans. Image Process., 17 (2008), pp. 42-52.
10. L. De Lathauwer, B. De Moor, and J. Vandewalle, A multilinear singular value decomposition, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1253-1278. (Pubitemid 33217068)
11. n) approximation of higher-order tensors, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1324-1342. (Pubitemid 33217072)
12. L. De Lathauwer and J. Vandewalle, Dimensionality reduction in higher-order signal processing and rank-(r1, r2, . . . , rn) reduction in multilinear algebra, Linear Algebra Appl., 391 (2004), pp. 31-55.
13. V. de Silva and L.-H. Lim, Tensor rank and the ill-posedness of the best low-rank approximation problem, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 1084-1127.
14. J. W. Demmel, Applied Numerical Linear Algebra, SIAM, Philadelphia, PA, 1997.
15. 3) approximation of a tensor, SIAM J. Matrix Anal. Appl., 31 (2009), pp. 248-271.
16. F. Estienne, N. Matthijs, D. Massart, P. Ricoux, and D. Leibovici, Multi-way modelling of high-dimensionality electroencephalographic data, Chemometrics and Intelligent Laboratory Systems, 58 (2001), pp. 59-72. (Pubitemid 32824189)
17. G. Golub and C. Van Loan, Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, MD, 1996.
18. L. Grasedyck, Hierarchical singular value decomposition of tensors, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 2029-2054.
19. M. Haardt, F. Roemer, and G. Del Galdo, Higher-order SVD-based subspace estimation to improve the parameter estimation accuracy in multidimensional harmonic retrieval problems, IEEE Trans. Signal Process., 56 (2008), pp. 3198-3213.
20. W. Hackbusch and B. Khoromskij, Tensor-product approximation to operators and functions in high dimensions, J. Complexity, 23 (2007), pp. 697-714. (Pubitemid 350122963)
21. W. Hackbusch and S. Kühn, A new scheme for the tensor representation, J. Fourier Anal. Appl., 15 (2009), pp. 706-722.
22. F. Hitchcock, Multiple invariants and generalized rank of a p-way matrix or tensor, J.Math. Phys., 7 (1927), pp. 39-79.
23. M. Ishteva, P.-A. Absil, S. Van Huffel, and L. De Lathauwer, Tucker compression and local optima, Chemometrics and Intelligent Laboratory Systems, 106 (2010), pp. 57-64.
24. M. Ishteva, P.-A. Absil, S. Van Huffel, and L. De Lathauwer, Best low multilinear rank approximation of higher-order tensors, based on the Riemannian trust-region scheme, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 115-135.
25. M. Ishteva, L. De Lathauwer, P.-A. Absil, and S. Van Huffel, Dimensionality reduction for higher-order tensors: Algorithms and applications, Int. J. Pure Appl. Math., 42 (2008), pp. 337-343.
26. 3) approximation of tensors, Numer. Algorithms, 51 (2009), pp. 179-194.
27. d, Comput. Methods Appl. Math., 6 (2006), pp. 194-220.
28. E. Kofidis and P. A. Regalia, On the best rank-1 approximation of higher-order supersymmetric tensors, SIAM J. Matrix Anal. Appl., 23 (2002), pp. 863-884.
29. T. G. Kolda and B. W. Bader, Tensor decompositions and applications, SIAM Rev., 51 (2009), pp. 455-500.
30. P. Kroonenberg and J. de Leeuw, Principal component analysis of three-mode data by means of alternating least squares algorithms, Psychometrika, 45 (1980), pp. 69-97.
31. C. Liu, K. Nakashima, H. Sako, and H. Fujisawa, Handwritten digit recognition: Benchmarking of state-of-the-art techniques, Pattern Recognition, 36 (2003), pp. 2271-2285. (Pubitemid 36953768)
32. C. Liu, J. Zhou, and K. He, Image compression based on truncated HOSVD, in Proceedings of the International Conference on Information Engineering and Computer Science, IEEE Press, Piscataway, NJ, 2009, pp. 1-4.
33. N. Liu, B. Zhang, J. Yan, Z. Chen, W. Liu, F. Bai, and L. Chien, Text representation: From vector to tensor, Proceedings of the IEEE International Conference on Data Mining, J. Han, B. W. Wah, V. Raghavan, X. Wu, and R. Rastogi, eds., IEEE Computer Society, Los Alamitos, CA, 2005, pp. 725-728.
34. X. Liu, L. De Lathauwer, F. Janssens, and B. De Moor, Hybrid clustering of multiple information sources via HOSVD, in Advances in Neural Networks - ISNN 2010, L. Zhang, B.-L. Lu, and J. Kwok, eds., Lecture Notes in Comput. Sci. 6064, Springer, Berlin, Heidelberg, 2010, pp. 337-345.
35. L. Lorente, J. Vega, and A. Velazquez, Compression of aerodynamic databases using highorder singular value decomposition, Aerospace Sci. Tech., 14 (2010), pp. 168-177.
36. M. Markaki and Y. Stylianou, Discrimination of speech from nonspeech in broadcast news based on modulation frequency features, Speech Communication, 53 (2011), pp. 726-735.
37. N. Mesgarani, M. Slaney, and S. Shamma, Discrimination of speech from nonspeech based on multiscale spectro-temporal modulations, IEEE Trans. Audio Speech, Language Process., 14 (2006), pp. 920-930. (Pubitemid 46547653)
38. D. Muti and S. Bourennane, Survey on tensor signal algebraic filtering, Signal Process., 87 (2007), pp. 237-249. (Pubitemid 44602251)
39. J. Nagy and M. Kilmer, Kronecker product approximation for preconditioning in threedimensional imaging applications, IEEE Trans. Image Process., 15 (2006), pp. 604-613. (Pubitemid 43257559)
40. L. Omberg, G. Golub, and O. Alter, A tensor higher-order singular value decomposition for integrative analysis of DNA microarray data from different studies, Proc. Natl. Acad. Sci. USA, 104 (2007), pp. 18371-18376. (Pubitemid 350210703)
41. L. Omberg, J. Meyerson, K. Kobayashi, L. Drury, J. Diffley, and O. Alter, Global effects of DNA replication and DNA replication origin activity on eukaryotic gene expression, Molec. Systems Biol., 5 (2009), 312.
42. I. V. Oseledets, Tensor-train decomposition, SIAM J. Sci. Comput., 33 (2011), pp. 2295- 2317.
43. I. V. Oseledets and E. E. Tyrtyshnikov, Breaking the curse of dimensionality, or how to use SVD in many dimensions, SIAM J. Sci. Comput., 31 (2009), pp. 3744-3759.
44. I. Oseledets and E. Tyrtyshnikov, Recursive decomposition of multidimensional tensors, Dokl. Math., 80 (2009), pp. 460-462.
45. J. M. Papy, L. De Lathauwer, and S. Van Huffel, Exponential data fitting using multilinear algebra: The single-channel and multi-channel case, Numer. Linear Algebra Appl., 12 (2005), pp. 809-826. (Pubitemid 41441205)
46. Y. Saad, Numerical Methods for Large Eigenvalue Problems, Manchester University Press, Manchester, UK, 1992.
47. Y. Saad, Iterative Methods for Sparse Linear Systems, Second Edition, SIAM, Philadelphia, PA, 2003.
48. F. Samaria and A. Harter, Parameterisation of a stochastic model for human face identification, in Proceedings of the Second IEEE Workshop on Applications of Computer Vision, IEEE Press, Piscataway, NJ, pp. 138-142.
49. B. Savas, Analyses and Tests of Handwritten Digit Recognition Algorithms, examensarbete (Master's thesis), Department of Mathematics, Tekniska Högskolan i Linköping, Linköping, Sweden, 2003.
50. B. Savas and L. Eldén, Handwritten digit classification using higher order singular value decomposition, Pattern Recognition, 40 (2007), pp. 993-1003. (Pubitemid 44667764)
51. B. Savas and L.-H. Lim, Quasi-Newton methods on Grassmannians and multilinear approximations of tensors, SIAM J. Sci. Comput., 32 (2010), pp. 3352-3393.
52. J.-T. Sun, H.-J. Zeng, H. Liu, Y. Lu, and Z. Chen, CubeSVD: A novel approach to personalized web search, in Proceedings of the 14th international conference on World Wide Web (WWW '05), ACM, New York, 2005, pp. 382-390.
53. P. Symeonidis, A. Nanopoulos, and Y. Manolopoulos, Tag recommendations based on tensor dimensionality reduction, in Proceedings of the 2008 ACM Conference on Recommender Systems, New York, 2008, ACM, New York, 2008, pp. 43-50.
54. L. Tucker, Implications of factor analysis of three-way matrices for measurement of change, in Problems in Measuring Change, C. Harris, ed., University of Wisconsin Press, Madison, WI, 1963, pp. 122-137.
55. L. Tucker, The extension of factor analysis to three-dimensional matrices, in Contributions to Mathematical Psychology, H. Gulliksen and N. Frederiksen, eds., Holt, Rinehardt, & Winston, New York, 1964, pp. 110-127.
56. L. Tucker, Some mathematical notes on three-mode factor analysis, Psychometrika, 31 (1966), pp. 279-311.
57. C. Van Loan, The ubiquitous Kronecker product, J. Comput. Appl. Math., 123 (2000), pp. 85- 100.
58. N. Vannieuwenhoven, R. Vandebril, and K. Meerbergen, On the Truncated Multilinear Singular Value Decomposition, Technical Report TW589, Department of Computer Science, Katholieke Universiteit Leuven, Leuven, Belgium, 2011.
59. M. Vasilescu and D. Terzopoulos, Multilinear image analysis for facial recognition, in Proceedings of the 16th International Conference on Pattern Recognition, IEEE Computer Society, Los Alamitos, CA, 2002, Vol. 2, pp. 511-514. (Pubitemid 44844789)
60. M. Vasilescu and D. Terzopoulos, Multilinear subspace analysis of image ensembles, IEEE Computer Soc. Conf. Computer Vision and Pattern Recognition, 2 (2003), pp. 93-99.
61. M. Vasilescu and D. Terzopoulos, TensorTextures: Multilinear image-based rendering, ACM Trans. Graph., 23 (2004), pp. 336-342.
62. H. Wang and N. Ahuja, Facial expression decomposition, in Proceedings of the Ninth IEEE International Conference on Computer Vision., 2003, Vol. 2, pp. 958-965.
63. T. Zhang and G. H. Golub, Rank-one approximation to high order tensors, SIAM J. Matrix Anal. Appl., 23 (2001), pp. 534-550.