Tensor rank and the ILL-posedness of the best low-rank approximation problem

SIAM Journal on Matrix Analysis and Applications

Volumn 30, Issue 3, 2008, Pages 1084-1127

  De Silva, Vin ^a   Lim, Lek Heng ^b  

^a POMONA COLLEGE   (United States)

^b Stanford University   (United States)

Author keywords

Bregman divergence of tensors; Candecomp; Eckart young theorem; Hyperdeterminants; Low rank tensor approximations; Multidimensional arrays; Multiway arrays; Numerical multilinear algebra; Parafac; Principal component analysis; Tensor decompositions; Tensor rank; Tensors

Indexed keywords

BREGMAN DIVERGENCE OF TENSORS; CANDECOMP; ECKART-YOUNG THEOREM; HYPERDETERMINANTS; LOW-RANK TENSOR APPROXIMATIONS; MULTIDIMENSIONAL ARRAYS; MULTIWAY ARRAYS; NUMERICAL MULTILINEAR ALGEBRA; PARAFAC; TENSOR DECOMPOSITIONS;

ALGEBRA; CONCRETES; EQUIVALENCE CLASSES; MATRIX ALGEBRA; PRINCIPAL COMPONENT ANALYSIS; SET THEORY; SIGNAL FILTERING AND PREDICTION;

TENSORS;

EID: 55349142218     PISSN: 08954798     EISSN: 10957162     Source Type: Journal    
DOI: 10.1137/06066518X     Document Type: Article

References (80)

1. W. A. ADKINS AND S. H. WEINTRAUB, Algebra: An Approach via Module Theory, Grad. Texts in Math. 136, Springer-Verlag, New York, 1992.
2. M. D. ATKINSON AND S. LLOYD, Bounds on the ranks of some 3-tensors, Linear Algebra Appl., 31 (1980), pp. 19-31.
3. M. D. ATKINSON AND N. M. STEPHENS, On the maximal multiplicative complexity of a family of bilinear forms, Linear Algebra Appl., 27 (1979), pp. 1-8.
4. G. M. BERGMAN, Ranks of tensors and change of base field, J. Algebra, 11 (1969), pp. 613-621.
5. D. BINI, Border rank of m × n × (mn-q) tensors, Linear Algebra Appl., 79 (1986), pp. 45-51.
6. D. BINI, Border rank of ap × q × 2 tensor and the optimal approximation of a pair of bilinear forms, in Automata, Languages, and Programming, J. W. de Bakker and J. van Leeuwen eds., Lecture Notes in Comput. Sci. 85, Springer-Verlag, New York, 1980, pp. 98-108.
7. 2.7799) complexity for n × n approximate matrix multiplication, Inform. Process. Lett., 8 (1979), pp. 234-235.
8. D. BINI, G. LOTTI, AND F. ROMANI, Approximate solutions for the bilinear form computational problem, SIAM J. Comput., 9 (1980), pp. 692-697.
9. N. BOURBAKI, Algebra I: Chapters 1-3, Elem. Math., Springer-Verlag, Berlin, 1998.
10. L. BRÈGMAN, A relaxation method of finding a common point of convex sets and its application to the solution of problems in convex programming, U.S.S.R. Comput. Math, and Math. Phys., 7 (1967), pp. 620-631.
11. R. BRO, Multi-Way Analysis in the Food Industry: Models, Algorithms, and Applications, Ph.D. thesis, Universiteit van Amsterdam, Amsterdam, The Netherlands, 1998.
12. P. BÜRGISSER, M. CLAUSEN, AND M.A. SHOKROLLAHI, Algebraic Complexity Theory, Grundlehren Math. Wiss. 315, Springer-Verlag, Berlin, 1996.
13. 1, J. Pure Appl. Algebra, 201 (2005), pp. 367-380.
14. M. V. CATALISANO, A. V. GERAMITA, AND A. GIMIGLIANO, Ranks of tensors, secant varieties of Segre varieties and fat points, Linear Algebra Appl., 355 (2002), pp. 263-285.
15. A. CAYLEY, On the theory of linear transformation, Cambridge Math. J., 4 (1845), pp. 193-209.
16. P. COMON, Tensor decompostions: State of the art and applications, in Mathematics in Signal Processing V (Coventry, UK, 2000), Inst. Math. Appl. Conf. Ser. 71, Oxford University Press, Oxford, UK, 2002, pp. 1-24.
17. P. COMON, G. H. GOLUB, L.-H. LIM, AND B. MOURRAIN, Genericity and rank deficiency of high order symmetric tensors, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '06), Vol. 31, IEEE, Washington, D.C., 2006, pp. 125-128.
18. P. COMON, G. H. GOLUB, L.-H. LIM, AND B. MOURRAIN, Symmetric tensors and symmetric tensor rank, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 1254-1279.
19. M. COSTE, An Introduction to Semialgebraic Geometry, preprint, 2002. Available online from http://perso.univ-rennesl.fr/michel.coste/ polyens/SAG.pdf.
20. L. DE LATHAUWER, B. DE MOOR, AND J. VANDEWALLE, A multilinear singular value decomposition, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1253-1278.
21. N) approximation of higher-order tensors, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1324-1342.
22. J. W. DEMMEL, On condition numbers and the distance to the nearest ill-posed problem, Numer. Math., 51 (1987), pp. 251-289.
23. J. W. DEMMEL, The geometry of ill-conditioning, J. Complexity, 3 (1987), pp. 201-229.
24. J. W. DEMMEL, The probability that a numerical analysis problem is difficult, Math. Comp., 50 (1988), pp. 449-480.
25. A. DEFANT AND K. FLORET, Tensor Norms and Operator Ideals, North-Holland Math. Stud. 176, North-Holland, Amsterdam, 1993.
26. I. S. DHILLON AND J. A. TROPP, Matrix nearness problems with Bregman divergences, SIAM J. Matrix Anal. Appl., 29 (2007), pp. 1120-1146.
27. D. S. DUMMIT AND R. M. FOOTE, Abstract Algebra, 3rd ed., John Wiley & Son, Hoboken, NJ, 2003.
28. C. ECKART AND G. YOUNG, The approximation of one matrix by another of lower rank, Psychometrika, 1 (1936), pp. 211-218.
29. L.C. EVANS, Partial Differential Equations, Grad. Stud. Math. 19, AMS, Providence, RI, 1998.
30. I. M. GELFAND, M. M. KAPRANOV, AND A. V. ZELEVINSKY, Discriminants, Resultants, and Multidimensional Determinants, Birkhäuser Boston, Boston, MA, 1994.
31. I. M. GELFAND, M. M. KAPRANOV, AND A. V. ZELEVINSKY, Hyperdeterminants, Adv. Math., 96 (1992), pp. 226-263.
32. R. GEROCH, Mathematical Physics, Chicago Lectures Phys., University of Chicago Press, Chicago, IL, 1985.
33. G. H. GOLUB AND C. VAN LOAN, Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, MD, 1996.
34. W. GREUB, Multilinear Algebra, 2nd ed., Springer-Verlag, New York, 1978.
35. D. GRIER, P. HUGGINS, B. STURMFELS, AND J. YU, The Hyperdeterminant and Triangulations of the 4-Cube, preprint, 2006. Available online from http://bio.math.berkeley.edu/4cube/index. html.
36. A. GROTHENDIECK, Résumé de la théorie métrique des produits tensoriels topologiques, Bol. Soc. Mat. Sao Paulo, 8 (1953), pp. 1-79.
37. J. HARRIS, Algebraic Geometry: A First Course, Grad. Texts in Math. 133, Springer-Verlag, New York, 1998.
38. R. A. HARSHMAN, Foundations of the PARAFAC Procedure: Models and Conditions for an Explanatory Multi-Modal Factor Analysis, UCLA Working Papers in Phonetics, 16 (1970), pp. 1-84.
39. F. L. HITCHCOCK, The expression of a tensor or a polyadic as a sum of products, J. Math. Phys., 6 (1927), pp. 164-189.
40. F. L. HITCHCOCK, Multiple invariants and generalized rank of a p-way matrix or tensor, 3. Math. Phys., 7 (1927), pp. 39-79.
41. T. W. HUNGERFORD, Algebra, Grad. Texts in Math. 73, Springer-Verlag, New York, 1980.
42. J. JÁJÁ, An addendum to Kronecker's theory of pencils, SIAM J. Appl. Math., 37 (1979), pp. 700-712.
43. J. JÁJÁ AND J. TAKCHE, On the validity of the direct sum conjecture, SIAM J. Comput., 15 (1986), pp. 1004-1020.
44. E. KOFIDIS AND P. A. REGALIA, On the best rank-1 approximation of higher-order supersym-metric tensors, SIAM J. Matrix Anal. Appl., 23 (2002), pp. 863-884.
45. E. KOFIDIS AND P. A. REGALIA, Tensor approximation and signal processing applications, in Structured Matrices in Mathematics, Computer Science, and Engineering I, Contemp. Math. 280, AMS, Providence, RI, 2001, pp. 103-133.
46. T. G. KOLDA, Orthogonal tensor decompositions, SIAM J. Matrix Anal. Appl., 23 (2001), pp. 243-255.
47. T. G. KOLDA, A counterexample to the possibility of an extension of the Eckart-Young low-rank approximation theorem for the orthogonal rank tensor decomposition, SIAM J. Matrix Anal. Appl., 24 (2003), pp. 762-767.
48. D. E. KNUTH, The Art of Computer Programming 2: Seminumerical Algorithms, 3rd ed., Addison-Wesley, Reading, MA, 1998.
49. W. P. KRIJNEN, T. K. DIJKSTRA, AND A. STEGEMAN, On the Non-Existence of Optimal Solutions and the Occurrence of "Degeneracy" in the CANDECOMP/PARAFAC Model, preprint, 2007.
50. J. B. KRUSKAL, Rank, decomposition, and uniqueness for 3-way and N-way arrays, in Multiway Data Analysis, R. Coppi and S. Bolasco, eds., Elsevier Science, Amsterdam, 1989, pp. 7-18.
51. J. B. KRUSKAL, R. A. HARSHMAN, AND M. E. LUNDY, HOW 3-MFA data can cause degenerate PARAFAC solutions, among other relationships, in Multiway Data Analysis, R. Coppi and S. Bolasco, eds., Elsevier Science, Amsterdam, 1989, pp. 115-122.
52. S. LANG, Algebra, rev. 3rd ed., Grad. Texts in Math. 211, Springer-Verlag, New York, 2002.
53. J. M. LANDSBERG, Private e-mail communication with the authors, August 29, 2006.
54. J. M. LANDSBERG, The border rank of the multiplication of 2×2 matrices is seven, J. Amer. Math. Soc, 19 (2006), pp. 447-459.
55. J. M. LANDSBERG AND L. MANIVEL, On the ideals of secant varieties of Segre varieties, Found. Comput. Math., 4 (2004), pp. 397-422.
56. D. LE1BOVICI AND R. SABATIER, A singular value decomposition of a k-way array for a principal component analysis of multiway data, PTA-k, Linear Algebra Appl., 269 (1998), pp. 307-329.
57. L.-H. LIM, Optimal Solutions to Non-Negative PARAFAC/multilinear NMF Always Exist, Centre International de rencontres Mathématiques, Luminy, France, 2005.
58. L.-H. LIM AND G. H. GOLUB, Nonnegative Decompositions of Nonnegative Matrices and Tensors, SCCM tech. rep. 06-01, preprint, Stanford University, Stanford, CA, 2006.
59. M. MARCUS, Finite Dimensional Multilinear Algebra, Part I, Pure and Applied Mathematics 23, Marcel Dekker, New York, 1973.
60. M. MARCUS, Finite Dimensional Multilinear Algebra, Part II, Pure and Applied Mathematics 23, Marcel Dekker, New York, 1975.
61. D. MARTIN, Manifold Theory: An Introduction for Mathematical Physicists, Ellis Horwood, NY, 1991.
62. D. G. NORTHCOTT, Multilinear Algebra, Cambridge University Press, Cambridge, UK, 1984.
63. P. PAATERO, Construction and analysis of degenerate PARAFAC models, J. Chemometrics, 14 (2000), pp. 285-299.
64. J. J. ROTMAN, Advanced Modern Algebra, Prentice-Hall, Upper Saddle River, NJ, 2002.
65. A. SEIDENBERG, A new decision method for elementary algebra, Ann. of Math., 60 (1954), pp. 365-374.
66. N. D. SIDIROPOULOS, R. BRO, AND G. B. GIANNAKIS, Parallel factor analysis in sensor array processing, IEEE Trans. Signal Process., 48 (2000), pp. 2377-2388.
67. A. SMILDE, R. BRO, AND P. GELADI, Multi-Way Analysis: Applications in the Chemical Sciences, John Wiley, West Sussex, UK, 2004.
68. A. STEGEMAN, Degeneracy in CANDECOMP/ PARAFAC explained for p × p × 2 arrays of rank p+1 or higher, Psychometrika, 71 (2006), pp. 483-501.
69. A. STEGEMAN, Low-rank approximation of generic p ×q × 2 arrays and diverging components in the CANDECOMP/ PARAFAC model, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 988-1007.
70. V. STRASSEN, Vermeidung von Divisionen, J. Reine Angew. Math., 264 (1973), pp. 184-202.
71. B. STURMFELS, Private e-mail communication with the authors, December 4, 2005.
72. A. TARSKI, A Decision Method for Elementary Algebra and Geometry, 2nd ed., University of California Press, Berkeley, CA, 1951.
73. J. M. F. TEN BERGE, Kruskal's polynomial for 2 ×2 ×2 arrays and a generalization to 2×n×n arrays, Psychometrika, 56 (1991), pp. 631-636.
74. J. M. F. TEN BERGE AND H. A. L. KIERS, Simplicity of core arrays in three-way principal component analysis and the typical rank of p × q × 2 arrays, Linear Algebra Appl., 294 (1999), pp. 169-179.
75. M. A. O. VASILESCU AND D. TERZOPOULOS, Multilinear image analysis for facial recognition, Proceedings of the International Conference on Pattern Recognition (ICPR), Quebec, Canada, 2002, pp. 511-514.
76. M. A. O. VASILESCU AND D. TERZOPOULOS, TensorTextures: Multilinear image-based rendering, in Proceedings of the ACM SIGGRAPH, ACM, New York, 2004, pp. 336-342.
77. J. H. WILKINSON, Rounding Errors in Algebraic Processes, Prentice-Hall, Englewood Cliffs, NJ, 1963.
78. T. YOKONUMA, Tensor Spaces and Exterior Algebra, Trans. Math. Monogr. 108, AMS, Providence, RI, 1992.
79. F. L. ZAK, Tangents and Secants of Algebraic Varieties, Trans. Math. Monogr. 127, AMS, Providence, RI, 1993.
80. T. ZHANG AND G. H. GOLUB, Rank-one approximation to high order tensors, SIAM J. Matrix Anal. Appl., 23 (2001), pp. 534-550.