Robust low-rank tensor recovery: Models and algorithms

SIAM Journal on Matrix Analysis and Applications

Volumn 35, Issue 1, 2014, Pages 225-253

  Goldfarb, Donald ^a   Qin, Zhiwei ^a  

^a Department of Earth and Environmental Sciences   (United States)

Author keywords

Alternating direction augmented Lagrangian; Higher order robust PCA; Low rank tensors; Tensor decompositions; Tucker rank; Variable splitting

Indexed keywords

ALGORITHMS; CONVEX OPTIMIZATION; GRADIENT METHODS; LAGRANGE MULTIPLIERS; PRINCIPAL COMPONENT ANALYSIS; RECOVERY;

AUGMENTED LAGRANGIANS; ROBUST PCA; TENSOR DECOMPOSITION; TUCKER RANK; VARIABLE-SPLITTING;

TENSORS;

EID: 84897553151     PISSN: 08954798     EISSN: 10957162     Source Type: Journal    
DOI: 10.1137/130905010     Document Type: Article

References (52)

1. E. Abdallah, A. Hamza, and P. Bhattacharya, Mpeg video watermarking using tensor singular value decomposition, in Proceedings of the International Conference on Image Analysis and Recognition, Lecture Notes in Comput. Sci. 4633, Springer-Verlag, Berlin, 2007, pp. 772-783.
2. E. Acar, D.M. Dunlavy, T.G. Kolda, and M. Mørup, Scalable tensor factorizations with missing data, SIAM International Conference on Datamining 2010 (SDM 2010), SIAM, Philadelphia, 2010, pp. 701-712.
3. B.W. Bader and T.G. Kolda, Efficient MATLAB computations with sparse and factored tensors, SIAM J. Sci. Comput., 30 (2008), pp. 205-231.
4. D. Baunsgaard, Factors Affecting Three-way Modeling (PARAFAC) of Fluorescence Landscapes, Department of Dairy and Food Technology, Royal Veterinary and Agricultural University, Frederiksberg, Denmark, 1999.
5. A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183-202.
6. A. Beck and L. Tetruashvili, On the Convergence of Block Coordinate Descent Type Methods, Technical report, Technion, Israel Institute of Technology, Haifa, Israel, 2011.
7. D.P. Bertsekas and J.N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, Prentice-Hall, Englewood Cliffs, NJ, 1989.
8. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Machine Learning, 3 (2010), pp. 1-123.
9. E.J. Candes, Y. Ma, J. Wright, and X. Li, Robust principal component analysis?, J. ACM, 58 (2011), article 11.
10. J.D. Carroll and J.J. Chang, Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart-Young decomposition, Psychometrika, 35 (1970), pp. 283-319.
11. L. De Lathauwer, B. De Moor, and J. Vandewalle, A multilinear singular value decomposition, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1253-1278.
12. L. De Lathauwer, B. De Moor, and J. Vandewalle, On the best rank-1 and rank-(R1, R2, ⋯,RN) approximation of higher-order tensors, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1324-1342.
13. C. Eckart and G. Young, The approximation of one matrix by another of lower rank, Psychometrika, 1 (1936), pp. 211-218.
14. J. Eckstein and D.P. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Programming, 55 (1992), pp. 293- 318.
15. S. Engelen, S. Frosch, and B.M. Jørgensen, A fully robust PARAFAC method for analyzing fluorescence data, J. Chemometrics, 23 (2009), pp. 124-131.
16. S. Engelen and M. Hubert, Detecting outlying samples in a PARAFAC model, Anal. Chim. Acta, 705 (2011), pp. 155-165.
17. S. Engelen, S.F. Møller, and M. Hubert, Automatically identifying scatter in fluorescence data using robust techniques, Chemometrics and Intelligent Laboratory Systems, 86 (2007), pp. 35-51.
18. T. Franz, A. Schultz, S. Sizov, and S. Staab, Triplerank: Ranking semantic web data by tensor decomposition, in Proceedings of the 8th International Semantic Web Conference, 2009, pp. 213-228.
19. D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximation, Comput. Math. Appl., 2 (1976), pp. 17-40.
20. S. Gandy, B. Recht, and I. Yamada, Tensor completion and low-n-rank tensor recovery via convex optimization, Inverse Problems, 27 (2011), 025010.
21. R. Glowinski and A. Marroco, Sur l'approximation, par elements finis d'ordre un, et la resolution, par penalisation-dualite d'une classe de problemes de dirichlet non lineares, Rev. Francaise Aut. Inf. Rech. Oper., 9 (1975), pp. 41-76.
22. R.A. Harshman, Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multimodal factor analysis, UCLA Working Papers in Phonetics, 9 (1970), pp. 1-84.
23. J. Hastad, Tensor rank is NP-complete, J. Algorithms, 11 (1990), pp. 644-654.
24. K. Hayashi, T. Takenouchi, T. Shibata, Y. Kamiya, D. Kato, K. Kunieda, K. Yamada, and K. Ikeda, Exponential family tensor factorization for missing-values prediction and anomaly detection, in Proceedings of the 10th IEEE International Conference on Data Mining, IEEE, 2010, pp. 216-225.
25. H. Huang and C. Ding, Robust tensor factorization using R1 norm, in IEEE Conference on Computer Vision and Pattern Recognition, 2008, IEEE, 2008, pp. 1-8.
26. T.G. Kolda and B.W. Bader, Tensor decompositions and applications, SIAM Rev., 51 (2009), pp. 455-500.
27. R.M. Larsen, Propack: A Software Package for the Symmetric Eigenvalue Problem and Singular Value Problems on Lanczos and Lanczos Bidiagonalization with Partial Reorthogonalization, SCCM, Stanford University, Palo Alto, CA, 2004.
28. X. Li, Compressed Sensing and Matrix Completion with Constant Proportion of Corruptions, preprint, arXiv:1104.1041, 2011.
29. Y. Li, J. Yan, Y. Zhou, and J. Yang, Optimum subspace learning and error correction for tensors, Lecture Notes in Comput. Sci. 6313, Springer-Verlag, Berlin, 2010, pp. 790-803.
30. Z. Lin, M. Chen, L. Wu, and Y. Ma, The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices, preprint, arXiv:1009.5055, 2010.
31. Z. Lin, A. Ganesh, J. Wright, L. Wu, M. Chen, and Y. Ma, Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix, in the International Workshop on Computer Advancement in Multi-Sensor Adapt Processing, Aruba, Dutch Antilles, 2009.
32. J. Liu, P. Musialski, P. Wonka, and J. Ye, Tensor completion for estimating missing values in visual data, in Proceedings of the 12th International Conference on Computer Vision, IEEE, 2009, pp. 2114-2121.
33. S. Ma, D. Goldfarb, and L. Chen, Fixed point and Bregman iterative methods for matrix rank minimization, Math. Program., 128 (2011), pp. 321-353.
34. S. Ma, L. Xue, and H. Zou, Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection, Technical report, University of Minnesota, Minneapolis, MN, 2012.
35. Y. Nesterov, Efficiency of coordinate descent methods on huge-scale optimization problems, SIAM J. Optim., 22 (2012), pp. 341-362.
36. V. Pravdova, F. Estienne, B. Walczak, and D.L. Massart, A robust version of the Tucker3 model, Chemometrics and Intelligent Laboratory Systems, 59 (2001), pp. 75-88.
37. Z. Qin and D. Goldfarb, Structured sparsity via alternating direction methods, J. Mach. Learn. Res., 13 (2012), pp. 1435-1468.
38. P. Richtárik and M. Takác, Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function, Math. Program. Ser. A, submitted.
39. J. Riu and R. Bro, Jack-knife technique for outlier detection and estimation of standard errors in PARAFAC models, Chemometrics and Intelligent Laboratory Systems, 65 (2003), pp. 35-49.
40. P.J. Rousseeuw, M. Debruyne, S. Engelen, and M. Hubert, Robustness and outlier detection in chemometrics, Critical Reviews in Analytical Chemistry, 36 (2006), pp. 221-242.
41. Y. Shen, Z. Wen, and Y. Zhang, Augmented Lagrangian Alternating Direction Method for Matrix Separation Based on Low-rank Factorization, TR11-02, Rice University, Houston, TX, 2011.
42. J.T. Sun, H.J. Zeng, H. Liu, Y. Lu, and Z. Chen, Cubesvd: a novel approach to personalized web search, in Proceedings of the 14th International Conference onWorldWideWeb, ACM, New York, 2005, pp. 382-390.
43. H. Tan, B. Cheng, J. Feng, G. Feng, and Y. Zhang, Tensor recovery via multi-linear augmented Lagrange multiplier method, in Proceedings of ICIG 2011, IEEE, 2011, pp. 141- 146.
44. K.C. Toh and S. Yun, An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems, Pac. J. Optim., 6 (2010), p. 615-640.
45. G. Tomasi and R. Bro, A comparison of algorithms for fitting the PARAFAC model, Comput. Statist. Data Anal., 50 (2006), pp. 1700-1734.
46. R. Tomioka, K. Hayashi, and H. Kashima, Estimation of Low-Rank Tensors via Convex Optimization, preprint, arXiv:1010.0789, 2010.
47. R. Tomioka, T. Suzuki, K. Hayashi, and H. Kashima, Statistical performance of convex tensor decomposition, Advances in Neural Information Processing Systems (NIPS), 2011, pp. 972-980.
48. L.R. Tucker, Some mathematical notes on three-mode factor analysis, Psychometrika, 31 (1966), pp. 279-311.
49. M.A.O. Vasilescu and D. Terzopoulos, Multilinear subspace analysis of image ensembles, in Proceedings of Computer Vision and Pattern Recognition, Vol. 2, IEEE, 2003, pp. 93-99.
50. J. Wright, A. Ganesh, S. Rao, and Y. Ma, Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization, J. ACM, submitted.
51. J. Yang and Y. Zhang, Alternating direction algorithms for l1-problems in compressive sensing, SIAM J. Sci. Comput., 33 (2011), pp. 250-278.
52. M. Yuan and Y. Lin, Model selection and estimation in regression with grouped variables, J. R. Stat. Soc. Ser. B Stat. Methodol., 68 (2006), pp. 49-67.