A randomized algorithm for a tensor-based generalization of the singular value decomposition

Linear Algebra and Its Applications

Volumn 420, Issue 2-3, 2007, Pages 553-571

  Drineas, Petros ^a   Mahoney, Michael W ^b  

^a Rensselaer Polytechnic Institute   (United States)

^b YALE UNIVERSITY   (United States)

Author keywords

CUR decomposition; Randomized algorithms; Tensor decomposition

Indexed keywords

APPROXIMATION THEORY; MATHEMATICAL MODELS; MATRIX ALGEBRA; MEASUREMENT ERRORS; PARAMETER ESTIMATION; TENSORS;

CUR DECOMPOSITION; RANDOMIZED ALGORITHMS; TENSOR DECOMPOSITION;

ALGORITHMS;

EID: 33751113879     PISSN: 00243795     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.laa.2006.08.023     Document Type: Article

References (51)

1. D. Achlioptas, F. McSherry, Fast computation of low rank matrix approximations, J. ACM, in press.
2. D. Achlioptas, F. McSherry, Fast computation of low rank matrix approximations, in: Proceedings of the 33rd Annual ACM Symposium on Theory of Computing, 2001, pp. 611-618.
3. M.W. Berry, S.A. Pulatova, G.W. Stewart, Computing sparse reduced-rank approximations to sparse matrices, Technical Report UMIACS TR-2004-32 CMSC TR-4589, University of Maryland, College Park, MD, 2004.
4. Beylkin G., and Mohlenkamp M.J. Numerical operator calculus in higher dimensions. Proc. Natl. Acad. Sci. USA 99 16 (2002) 10246-10251
5. Carroll J.D., and Chang J.J. Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart-Young" decomposition. Psychometrika 35 3 (1970) 283-319
6. W.F. de la Vega, R. Kannan, M. Karpinski, S. Vempala, Tensor decomposition and approximation schemes for constraint satisfaction problems, in: Proceedings of the 37st Annual ACM Symposium on Theory of Computing, 2005, pp. 747-754.
7. P. Drineas, A. Frieze, R. Kannan, S. Vempala, V. Vinay, Clustering in large graphs and matrices, in: Proceedings of the 10th Annual ACM-SIAM Symposium on Discrete Algorithms, 1999, pp. 291-299.
8. P. Drineas, R. Kannan, Fast Monte-Carlo algorithms for approximate matrix multiplication, in: Proceedings of the 42nd Annual IEEE Symposium on Foundations of Computer Science, 2001, pp. 452-459.
9. P. Drineas, R. Kannan, Pass efficient algorithms for approximating large matrices, in: Proceedings of the 14th Annual ACM-SIAM Symposium on Discrete Algorithms, 2003, pp. 223-232.
10. Drineas P., Kannan R., and Mahoney M.W. Fast Monte Carlo algorithms for matrices I: Approximating matrix multiplication. SIAM J. Comput. 36 (2006) 132-157
11. Drineas P., Kannan R., and Mahoney M.W. Fast Monte Carlo algorithms for matrices II: Computing a low-rank approximation to a matrix. SIAM J. Comput. 36 (2006) 158-183
12. Drineas P., Kannan R., and Mahoney M.W. Fast Monte Carlo algorithms for matrices III: Computing a compressed approximate matrix decomposition. SIAM J. Comput. 36 (2006) 184-206
13. P. Drineas, I. Kerenidis, P. Raghavan, Competitive recommendation systems, in: Proceedings of the 34th Annual ACM Symposium on Theory of Computing, 2002, pp. 82-90.
14. P. Drineas, M.W. Mahoney. Approximating a Gram matrix for improved kernel-based learning, in: Proceedings of the 18th Annual Conference on Learning Theory, 2005, pp. 323-337.
15. Drineas P., and Mahoney M.W. On the Nyström method for approximating a Gram matrix for improved kernel-based learning. J. Mach. Learning Res. 6 (2005) 2153-2175
16. P. Drineas, M.W. Mahoney, S. Muthukrishnan, Polynomial time algorithm for column-row based relative-error low-rank matrix approximation, Technical Report 2006-04, DIMACS, March 2006.
17. 2 regression and applications, in: Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithms, 2006, pp. 1127-1136.
18. A. Frieze, R. Kannan, S. Vempala, Fast Monte-Carlo algorithms for finding low-rank approximations, in: Proceedings of the 39th Annual IEEE Symposium on Foundations of Computer Science, 1998, pp. 370-378.
19. Frieze A., Kannan R., and Vempala S. Fast Monte-Carlo algorithms for finding low-rank approximations. J. ACM 51 6 (2005) 1025-1041
20. Gonzalez T., and Ja'Ja' J. On the complexity of computing bilinear forms with {0, 1} constants. J. Comput. Syst. Sci. 20 (1980) 77-95
21. Goreinov S.A., and Tyrtyshnikov E.E. The maximum-volume concept in approximation by low-rank matrices. Contemp. Math. 280 (2001) 47-51
22. Goreinov S.A., Tyrtyshnikov E.E., and Zamarashkin N.L. A theory of pseudoskeleton approximations. Linear Algebra Appl. 261 (1997) 1-21
23. Greub W.H. Multilinear Algebra (1967), Springer-Verlag, Berlin
24. Harshman R.A., and Lundy M.E. The PARAFAC model for three-way factor analysis and multidimensional scaling. In: Law H.G., Snyder Jr. C.W., Hattie J., and McDonald R.P. (Eds). Research Methods for Multimode Data Analysis (1984), Praeger 122-215
25. Håstad J. Tensor rank is NP-complete. J. Algorithms 11 2 (1990) 644-654
26. Howell T.D. Global properties of tensor rank. Linear Algebra Appl. 22 (1978) 9-23
27. Kiers H.A.L. Towards a standardized notation and terminology in multiway analysis. J. Chemometrics 14 (2000) 105-122
28. Kolda T.G. Orthogonal tensor decompositions. SIAM J. Matrix Anal. Appl. 23 1 (2001) 243-255
29. Kolda T.G. A counterexample to the possibility of an extension of the Eckart-Young low-rank approximation theorem for the orthogonal rank tensor decomposition. SIAM J. Matrix Anal. Appl. 24 3 (2003) 762-767
30. Kroonenberg P.M., and De Leeuw J. Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika 45 1 (1980) 69-97
31. Kruskal J.B. Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear Algebra Appl. 18 (1977) 95-138
32. Kruskal J.B. Rank, decomposition, and uniqueness for 3-way and N-way arrays. In: Coppi R., and Bolasco S. (Eds). Multiway Data Analysis (1989), Elesvier Science Publishers 7-18
33. De Lathauwer L., De Moor B., and Vandewalle J. An introduction to independent component analysis. J. Chemometrics 14 (2000) 123-149
34. De Lathauwer L., De Moor B., and Vandewalle J. A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21 4 (2000) 1253-1278
35. N) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl. 21 4 (2000) 1324-1342
36. Leibovici D., and Sabatier R. A singular value decomposition of a k-way array for a principal component analysis of multiway data, PTA-k. Linear Algebra Appl. 269 (1998) 307-329
37. L.-H. Lim, unpublished results.
38. L.-H. Lim, G.H. Golub, Tensors for numerical multilinear algebra: ranks and basic decompositions. Technical Report 05-02, Stanford University SCCM, Stanford, CA, April 2005.
39. Van Loan C.F. The ubiquitous Kronecker product. J. Comput. Appl. Math. 123 (2000) 85-100
40. M.W. Mahoney, M. Maggioni, P. Drineas, Tensor-CUR decompositions for tensor-based data, in: Proceedings of the 12th Annual ACM SIGKDD Conference, 2006, pp. 327-336.
41. P. Paschou, M.W. Mahoney, J.R. Kidd, A.J. Pakstis, S. Gu, K.K. Kidd, P. Drineas, Intra- and inter-population genotype reconstruction from tagging SNPs, submitted for publication.
42. L. Rademacher, S. Vempala, G. Wang, Matrix approximation and projective clustering via iterative sampling. Technical Report MIT-LCS-TR-983, Massachusetts Institute of Technology, Cambridge, MA, March 2005.
43. Stewart G.W. Four algorithms for the efficient computation of truncated QR approximations to a sparse matrix. Numer. Math. 83 (1999) 313-323
44. G.W. Stewart, Error analysis of the quasi-Gram-Schmidt algorithm, Technical Report UMIACS TR-2004-17 CMSC TR-4572, University of Maryland, College Park, MD, 2004.
45. Tucker L.R. Some mathematical notes on three-mode factor analysis. Psychometrika 31 3 (1966) 279-311
46. M.A.O. Vasilescu, D. Terzopoulos, Multilinear analysis of image ensembles: TensorFaces, in: Proceedings of the 7th European Conference on Computer Vision, 2002, pp. 447-460.
47. M.A.O. Vasilescu, D. Terzopoulos, Multilinear subspace analysis of image ensembles, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2003, pp. 93-99.
48. S. Vempala, Random projection: a new approach to VLSI layout, in: Proceedings of the 39th Annual IEEE Symposium on Foundations of Computer Science, 1998, pp. 389-395.
49. C.K.I. Williams, M. Seeger, Using the Nyström method to speed up kernel machines, in: Annual Advances in Neural Information Processing Systems 13: Proceedings of the 2000 Conference, 2001, pp. 682-688.
50. D. Zeimpekis, E. Gallopoulos, CLSI: a flexible approximation scheme from clustered term-document matrices, Technical Report HPCLAB-SCG 2/10-04, University of Patras, Patras, Greece, October 2004.
51. Zhang T., and Golub G.H. Rank-one approximation to high order tensors. SIAM J. Matrix Anal. Appl. 23 2 (2001) 534-550