Sharp oracle inequalities for high-dimensional matrix prediction

IEEE Transactions on Information Theory

Volumn 57, Issue 10, 2011, Pages 6942-6957

  Gaïffas, Stéphane ^a   Lecué, Guillaume ^b  

^a SORBONNE UNIVERSITÉ   (France)

^b CNRS   (France)

Author keywords

Empirical process theory; empirical risk minimization; high dimensional matrix; matrix completion; nuclear norm; oracle inequalities; Schatten norms; sparsity

Indexed keywords

EMPIRICAL PROCESS; EMPIRICAL RISK MINIMIZATION; HIGH-DIMENSIONAL; MATRIX COMPLETION; NUCLEAR NORM; ORACLE INEQUALITIES; SCHATTEN NORMS; SPARSITY;

MATHEMATICAL OPERATORS;

FORECASTING;

EID: 80054004560     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2136318     Document Type: Article

References (39)

1. J. Abernethy, F. Bach, T. Evgeniou, and J. P. Vert, "A new approach to collaborative filtering: Operator estimation with spectral regularization" J. Mach. Learn. Res., vol. 10, pp. 803-826, 2009.
2. R. Adamczak, "A tail inequality for suprema of unbounded empirical processes with applications to Markov chains" Electron. J. Probabil., vol. 13, no. 34, pp. 1000-1034, 2008.
3. A. Argyriou, T. Evgeniou, and M. Pontil, "Convex multi-task feature learning." Mach. Learn., vol. 73, no. 3, pp. 243-272, 2008.
4. A. Argyriou, C. A. Micchelli, and M. Pontil, "On spectral learning." J. Mach. Learn. Res., vol. 11, pp. 935-953, 2010.
5. A. Argyriou, C. A. Micchelli, M. Pontil, and Y. Ying, "A spectral regularization framework for multi-task structure learning." Adv. Neural Inf. Process. Syst., vol. 20, pp. 25-32, 2008.
6. F. R. Bach, "Consistency of trace norm minimization." J. Mach. Learn. Res., vol. 9, pp. 1019-1048, 2008.
7. P. L. Bartlett, "Fast rates for estimation error and oracle inequalities for model selection." Econometric Theory, vol. 24, no. 2, 2008.
8. P. L. Bartlett and S. Mendelson, "Empirical minimization" Probabil. Theory Rel. Fields, vol. 135, no. 3, pp. 311-334, 2006.
9. P. L. Bartlett, S. Mendelson, and J. Neeman, " -regularized linear regression: Persistence and oracle inequalities" Bernoulli, to be published.
10. J.-F. Cai, E. J. Candès, and Z. Shen, "A singular value thresholding algorithm for matrix completion" SIAM J. Optim., vol. 20, no. 4, pp. 1956-1982, 2010.
11. E. J. Candès and Y. Plan, Tight Oracle Bounds for Low-Rank Matrix Recovery From a Minimal Number of Random Measurements Tech. Rep., Dept. Statist., Stanford Univ., Stanford, CA, 2009.
12. E. J. Candès and Y. Plan, "Matrix completion with noise" Proc. IEEE, vol. 98, no. 6, pp. 925-936, 2010.
13. E. J. Candès and B. Recht, "Exact matrix completion via convex optimization" Found. Comput. Math., vol. 9, no. 6, pp. 717-772, 2009.
14. E. J. Candès and T. Tao, "The power of convex relaxation: Near-optimal matrix completion" IEEE Trans. Inf. Theory, vol. 56, no. 5, pp. 2053-2080, 2010.
15. E. J. Candès and T. Tao, "Decoding by linear programming" IEEE Trans. Inf. Theory, vol. 51, no. 12, pp. 4203-4215, 2005.
16. B. Carl, "Inequalities of Bernstein-Jackson-type and the degree of compactness of operators in Banach spaces" Ann. Inst. Fourier (Grenoble), vol. 35, no. 3, pp. 79-118, 1985.
17. M. Fazel, H. Hindi, and S. Boyd, "Rank minimization and applications in system theory" in Amer. Control Conf., 2004, pp. 3273-3278, AACC.
18. M. Fazel, H. Hindi, and S. P. Boyd, "Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices" in Proc. Amer. Control Conf., 2003, vol. 3, pp. 2156-2162.
19. E. Giné and J. Zinn, "Some limit theorems for empirical processes" Ann. Probabil., vol. 12, no. 4, pp. 929-998, 1984, With discussion.
20. D. Gross, "Recovering low-rank matrices from few coefficients in any basis" CoRR 2009, abs/0910.1879.
21. O. Guédon, S. Mendelson, A. Pajor, and N. Tomczak-Jaegermann, "Subspaces and orthogonal decompositions generated by bounded orthogonal systems" Positivity, vol. 11, no. 2, pp. 269-283, 2007.
22. R. H. Keshavan, A. Montanari, and S. Oh, "Matrix completion from noisy entries" CoRR 2009, abs/0906.2027.
23. R. H. Keshavan, S. Oh, and A. Montanari, "Matrix completion from a few entries" Arxiv vol. 901, 2009.
24. V. Koltchinskii, K. Lounici, and A. B. Tsybakov, "Nuclear normpenalization and optimal rates for noisy low rank matrix completion" Arxiv Preprint arXiv:1011.6256 2010.
25. M. Ledoux and M. Talagrand, Probability in Banach Spaces, Vol. 23 of Ergebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)]. Berlin: Springer-Verlag, 1991, Isoperimetry and processes.
26. S. Mendelson and J. Neeman, "Regularization in kernel learning" Ann. Statist., vol. 38, no. 1, pp. 526-565, 2010.
27. S. Mendelson, A. Pajor, and N. Tomczak-Jaegermann, "Reconstruction and subgaussian operators in asymptotic geometric analysis" Geom. Funct. Anal., vol. 17, no. 4, pp. 1248-1282, 2007.
28. S. Mendelson, A. Pajor, and N. Tomczak-Jaegermann, "Uniform uncertainty principle for Bernoulli and subgaussian ensembles" Constr. Approx., vol. 28, no. 3, pp. 277-289, 2008.
29. S. Negahban, P. Ravikumar, M. J. Wainwright, and B. Yu, "A unified framework for high-dimensional analysis ofM-estimators with decomposable regularizers" Adv. Neural Inf. Process. Syst., 2009.
30. S. Negahban and M. J. Wainwright, "Estimation of (near) low-rank matrices with noise and high-dimensional scaling" Arxiv Preprint arXiv:0912.5100 2009.
31. S. Negahban and M. J. Wainwright, "Restricted strong convexity and weighted matrix completion: Optimal bounds with noise" CoRR 2010, abs/1009.2118.
32. A. Pajor and N. Tomczak-Jaegermann, "Remarques sur les nombres d'entropie d'un opérateur et de son transposé" C. R. Acad. Sci. Paris Sér. I Math., vol. 301, no. 15, pp. 743-746, 1985.
33. B. Recht, "A simpler approach to matrix completion" CoRR 2009, abs/0910.0651.
34. B. Recht, M. Fazel, and P. A. Parrilo, Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization 2007, preprint.
35. A. Rohde and A. B. Tsybakov, Estimation of High-Dimensional Low Rank Matrices Tech. Rep., Univ. Hamburg and Université Paris 6, 2009.
36. M. Rudelson, "Random vectors in the isotropic position" J. Funct. Anal., vol. 164, no. 1, pp. 60-72, 1999.
37. M. Rudelson and R. Vershynin, "On sparse reconstruction from Fourier and Gaussian measurements" Commun. Pure Appl. Math., vol. 61, no. 8, pp. 1025-1045, 2008.
38. C. Schütt, "Entropy numbers of diagonal operators between symmetric Banach spaces" J. Approx. Theory, vol. 40, no. 2, pp. 121-128, 1984.
39. N. Srebro, J. D. M. Rennie, and T. S. Jaakkola, "Maximum-margin matrix factorization" Adv. Neural Inf. Process. Syst., vol. 17, pp. 1329-1336, 2005.