Efficient MATLAB computations with sparse and factored tensors

SIAM Journal on Scientific Computing

Volumn 30, Issue 1, 2007, Pages 205-231

  Bader, Brett W ^a   Kolda, Tamara G ^b  

^a SANDIA NATIONAL LABORATORIES   (United States)

^b SANDIA NATIONAL LABORATORIES   (United States)

Author keywords

Canonical decomposition (CANDECOMP); MATLAB classes; Multilinear algebraic computations; Parallel factors (PARAFAC) model; Sparse multidimensional arrays; Tensor decompositions; Tucker model

Indexed keywords

COMPUTATIONAL EFFICIENCY;

ALGEBRAIC COMPUTATIONS; CANONICAL DECOMPOSITION; MATLAB CLASS; MULTIDIMENSIONAL ARRAYS; MULTILINEAR ALGEBRAIC COMPUTATION; PARALLEL FACTOR MODELS; SPARSE MULTIDIMENSIONAL ARRAY; TENSOR DECOMPOSITION; TUCK MODEL;

TENSORS;

EID: 49749096526     PISSN: 10648275     EISSN: None     Source Type: Journal    
DOI: 10.1137/060676489     Document Type: Article

References (58)

1. E. ACAR, S. A. ÇAMTEPE, AND B. YENER, Collective sampling and analysis of high order tensors for chatroom communications, in ISI 2006: IEEE International Conference on Intelligence and Security Informatics, Lecture Notes in Comput. Sci. 3975. Springer, Berlin, 2006, pp. 213-224.
2. C. A. ANDERSSON AND R. BRO, The N-way toolbox for MATLAB, Chemometr. Intell. Lab., 52 (2000), pp. 1-4. See also http://www.models.kvl.dk/source/nwaytoolbox/.
3. C. J. APPELLOF AND E. R. DAVIDSON, Strategies for analyzing data, from video fluorometric monitoring of liquid chromatographic effluents, Anal. Chem., 53 (1981), pp. 2053-2056.
4. B. W. BADER AND T. G. KOLDA, Algorithm. 862: MATLAB tensor classes for fast algorithm prototyping, ACM Trans. Math. Software, 32 (2006), pp. 635-653.
5. B. W. BADER AND T. G. KOLDA, Matlab Tensor Toolbox, Version 2.1, http://csmr.ca. sandia.gov/~tgkolda/TensorToolbox/ (December 2006).
6. G. BEYLKIN AND M. J. MOHLENKAMP, Algorithms for numerical analysis in high dimensions, SIAM J. Sci. Comput., 26 (2005), pp. 2133-2159.
7. R. BRO, PARAFAC. Tutorial and applications, Chemometr. Intell. Lab., 38 (1997), pp. 149-171.
8. R. BRO, Multi-way Analysis in the Food Industry: Models, Algorithms, and Applications, Ph.D. thesis, University of Amsterdam, 1998. Available at http://www.models.kvl.dk/research/theses/.
9. J. D. CARROLL AND J. J. CHANG, Analysis of individual differences in multidimensional scaling via an N-way generalization of 'Eckart-Young' decomposition, Psychometrika, 35 (1970), pp. 283-319.'
10. B. CHEN, A. PETROPOLU, AND L. DE LATHAUWER, Blind identification of convolutive MIM systems with 3 sources and 2 sensors, Applied Signal Processing, (2002), pp. 487-496 (Special Issue on Space-Time Coding and Its Applications, Part II).
11. P. COMON, Tensor decompositions: State of the art and applications, in Mathematics in Signal Processing V. J. G. McWhirter and I. K. Proudler, eds., Oxford University Press, Oxford, 2001, pp. 1-24.
12. L. DE LATHAUWER, B. DE MOOR, AND J. VANDEWALLE, A multilinear singular value decomposition, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1253-1278.
13. n) approximation of higher-order tensors, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1324-1342.
14. I. S. DUFF AND J. K. REID, Some design features of a sparse matrix code, ACM Trans. Math. Software, 5 (1979), pp. 18-35.
15. R. GARCIA AND A. LUMSDAINE, MultiArray: A C+ + library for generic programming with arrays, Software: Practice and Experience, 35 (2004), pp. 159-188.
16. J. R. GILBERT, C. MOLER, AND R. SCHREIBER, Sparse matrices in MATLAB: Design and implementation, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 333-356.
17. V. S. GRIGORASCU AND P. A. REGALIA, Tensor displacement structures and polyspectral matching, in Fast Reliable Algorithms for Matrices with Structure, T. Kaliath and A. H. Saved, eds., SIAM, Philadelphia, 1999, pp. 245-276.
18. F. G. GUSTAVSON, Some basic techniques for solving sparse systems, in Sparse Matrices and their Applications, D. J. Rose and R. A. Willoughby, eds., Plenum Press, New York, 1972, pp. 41-52.
19. R. A. HARSHMAN, Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-modal factor analysis, UCLA working papers in phonetics, 16 (1970), pp. 1-84. Available at http://publish.uwo.ca/~harshman/wpppfac0.pdf.
20. R. HENRJON, Body diagonalization of core matrices in three-way principal components analysis: Theoretical bounds and simulation, J. Chemometr., 7 (1993), pp. 477-494.
21. R. HENRION, N-way principal component analysis theory, algorithms and applications, Chemometr. Intell. Lab., 25 (1994), pp. 1-23.
22. H. A. KIERS, Joint orthomax rotation of the core and component matrices resulting from threemode principal components analysis, J. Classification, 15 (1998), pp. 245-263.
23. H. A. L. KIERS, Towards a standardized notation and terminology in multiway analysis, J. Chemometr., 14 (2000), pp. 105-122.
24. T. G. KOLDA, Orthogonal tensor decompositions, SIAM J. Matrix Anal. Appl., 23 (2001), pp. 243-255.
25. T. G. KOLDA, Multilinear Operators for Higher-Order Decompositions, Technical report SAND2006-2081, Sandia National Laboratories, Albuquerque, NM and Livermore, CA, 2006.
26. P. KROONENBERG, Applications of three-mode techniques: Overview, problems, and prospects (slides), Presentation at the AIM Tensor Decompositions Workshop, Palo Alto, CA, 2004. Available at http://csmr.ca. sandia.gov/~tgkolda/tdw2004/Kroonenberg%20-%20Talk.pdf.
27. P. M. KROONENBERG AND J. DE LEEUW, Principal component analysis of three-mode data by means of alternating least squares algorithms, Psychometrika, 45 (1980), pp. 69-97.
28. J. B. KRUSKAL, Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics, Linear Algebra. Appl., 18 (1977), pp. 95-138.
29. J. B. KRUSKAL, Rank, decomposition, and uniqueness for 3-way and N-way arrays, in Multiway Data Analysis, R. Coppi and S. Bolasco, eds., North-Holland, Amsterdam, 1989.
30. W. LANDRY, Implementing a high performance tensor library, Scientific Programming, 11 (2003), pp. 273-290.
31. S. LEURGANS AND R. T. Ross, Multilinear models: Applications in spectroscopy, Statist. Sci., 7 (1992), pp. 289-310.
32. L.-H. LIM, Singular values and eigenvalues of tensors: A variational approach, in CAMAP2005: 1st IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, 2005, pp. 129-132.
33. C.-Y. LIN, Y.-C. CHUNG, AND J.-S. LIU, Efficient data compression methods for multidimensional sparse array operations based on the ekmr scheme, IEEE Trans. Comput., 52 (2003), pp. 1640-1646.
34. C.-Y. LIN. J.-S. LIU, AND Y.-C. CHUNG, Efficient representation scheme for multidimensional array operations, IEEE Trans. Comput., 51 (2002), pp. 327-345.
35. R. P. MCDONALD, A simple comprehensive model for the analysis of covariance structures, British J. Math. Statist. Psych., 33 (1980), p. 161. Cited in [8].
36. M. MØRUP, L. HANSEN, J. PARNAS, AND S. M. ARNFRED, Decomposing the Time-Frequency Representation of EEG Using Nonnegative Matrix and Multi-way Factorization, http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/4144/pdf/imm4144.pdf (2006).
37. T. MURAKAMI, J. M. F. TEN BERGE, AND H. A. L. KIERS, A case of extreme simplicity of the core matrix in three-mode principal components analysis, Psychometrika, 63 (1998), pp. 255-261.
38. P. PAATERO, The multilinear engine - a table-driven, least squares program for solving multilinear problems, including the n-way parallel factor analysis model, J. Comput. Graph. Statist., 8 (1999), pp. 854-888.
39. U. W. POOCH AND A. NIEDER, A survey of indexing techniques for sparse matrices, ACM Computing Surveys, 5 (1973), pp. 109-133.
40. C. R. RAO AND S. MITRA, Generalized Inverse of Matrices and its Applications, Wiley, New York, 1971. Cited in [8].
41. J. R. RUÍZ-TOLOSA AND E. CASTILLO, From Vectors to Tensors, Universitext, Springer, Berlin, 2005.
42. Y. SAAD, Iterative Methods for Sparse Linear Systems, 2nd ed., SIAM, Philadelphia, 2003.
43. B. SAVAS, Analyses and Tests of Handwritten Digit Recognition Algorithms, master's thesis, Linköping University, Sweden, 2003.
44. A. SMILDE, R. BRO, AND P. GELADI, Multi-way Analysis: Applications in the Chemical Sciences, Wiley, West Sussex, England, 2004.
45. J. SUN. D. TAO. AND C. FALOUTSOS, Beyond streams and graphs: Dynamic tensor analysis, in KDD '06: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2006, pp. 374-383.
46. J.-T. SUN. H.-J. ZENG, H. LIU. Y. LU, AND Z. CHEN, CubeSVD: A novel approach to personalized web search, in WWW 2005: Proceedings of the 14th International Conference on World Wide Web, ACM Press, New York, 2005, pp. 382-390.
47. J. TEN BERGE, J. DE LEEUW, AND P. M. KROONENBERG, Some additional results on principal components analysis of three-mode data by means of alternating least squares algorithms, Psychometrika, 52 (1987), pp. 183-191.
48. J. M. F. TEN BERGE AND H. A. L. KIERS, Simplicity of core arrays in three-way principal component analysis and the typical rank of p X q X 2 arrays, Linear Algebra Appl., 294 (1999), pp. 169-179.
49. G. TOMASI, Use of the Properties of the Khatri-Rao Product for the Computation of Jacobian, Hessian, and Gradient of the PARAFAC Model under MATLAB, manuscript, 2005.
50. G. TOMASI AND R. BRO, A comparison of algorithms for fitting the PARAFAC model, Comput. Statist. Data Anal., 50 (2006), pp. 1700-1734.
51. L. R. TUCKER, Some mathematical notes on three-mode factor analysis, Psychometrika, 31 (1966), pp. 279-311.
52. M. A. O. VASILESCU AND D. TERZOPOULOS, Multilinear analysis of image ensembles: Tensor-Faces, in ECCV 2002: 7th European Conference on Computer Vision, Lecture Notes in Comput. Sci. 2350, Springer, Berlin, 2002, pp. 447-460.
53. D. VLASIC, M. BRAND, H. PFISTER, AND J. POPOVIĆ, Face transfer with multilinear models, ACM Trans. Graphics, 24 (2005), pp. 426-433.
54. H. WANG AND N. AHUJA, Facial expression decomposition, in ICCV 2003: 9th IEEE International Conference on Computer Vision, vol. 2, 2003, pp. 958-965.
55. B. M. WISE AND N. B. GALLAGHER, PLS_Toolbox 4.0, http://www.eigenvector.com (2007).
56. R. ZASS, HUJI Tensor Library, http://www.cs.huji.ac.il/ ~zass/htl/(May 2006).
57. E. ZHANG, J. HAYS, AND G. TURK, Interactive Tensor Field Design and Visualization on Surfaces, http://eecs.oregonstate.edu/library/files/2005-106/tenflddesn.pdf (2005).
58. T. ZHANG AND G. H. GOLUB, Rank-one approximation to high order tensors, SIAM J. Matrix Anal. Appl., 23 (2001), pp. 534-550.