Estimation of high-dimensional low-rank matrices

Annals of Statistics

Volumn 39, Issue 2, 2011, Pages 887-930

  Angelika, Rohde ^a   Tsybakov, Alexandre B ^b  

^a UNIVERSITY OF HAMBURG   (Germany)

^b CREST   (France)

Author keywords

Empirical process; High dimensional low rank matrices; Penalized least squares estimator; Quasi convex Schatten class embeddings; Schatten norm; Sparse recovery

Indexed keywords

EID: 79952902758     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/10-AOS860     Document Type: Article

References (51)

1. ABERNETHY, J., BACH, F., EVGENIOU, T. andVERT, J.-P. (2009). A new approach to collaborative filtering: Operator estimation with spectral regularization. J.Mach. Learn. Res. 10 803-826.
2. AMINI, A. andWAINWRIGHT, M. (2009). High-dimensional analysis of semidefinite relaxations for sparse principal components. Ann. Statist. 37 2877-2921. MR2541450
3. ARGYRIOU, A., EVGENIOU, T. and PONTIL, M. (2008). Convex multi-task feature learning. Mach. Learn. 73 243-272.
4. ARGYRIOU, A.,MICCHELLI, C. A. and PONTIL, M. (2010). On spectral learning. J. Mach. Learn. Res. To appear. MR2600635
5. ARGYRIOU, A., MICCHELLI, C. A., PONTIL, M. and YING, Y. (2008). A spectral regularization framework for multi-task structure learning. In Advances in Neural Information Processing Systems 20 (J.C. Platt, et al., eds.) 25-32. MIT Press, Cambridge, MA.
6. BACH, F. R. (2008). Consistency of trace norm minimization. J. Mach. Learn. Res. 9 1019-1048. MR2417263
7. BICKEL, P. J. and LEVINA, E. (2008). Regularized estimation of large covariance matrices. Ann. Statist. 36 199-227. MR2387969
8. BICKEL, P. J., RITOV, Y. and TSYBAKOV, A. B. (2009). Simultaneous analysis of lasso and Dantzig selector. Ann. Statist. 37 1705-1732. MR2533469
9. BUNEA, F., SHE, Y. andWEGKAMP, M. H. (2010). Optimal selection of reduced rank estimators of high-dimensional matrices. Available at arXiv:1004.2995.
10. BUNEA, F., TSYBAKOV, A. B. andWEGKAMP, M. H. (2007). Aggregation for Gaussian regression. Ann. Statist. 35 1674-1697. MR2351101
11. CAI, T., ZHANG, C.-H. and ZHOU, H. H. (2010). Optimal rates of convergence for covariance matrix estimation. Ann. Statist. 38 2118-2144. MR2676885
12. Candèe, E. J. and PLAN, Y. (2010a). Matrix completion with noise. Proc. IEEE 98 925-936.
13. Candèe, E. J. and PLAN, Y. (2010b). Tight oracle bounds for low-rank matrix recovery from a mininal number of noisy random measurements. Available at arXiv:1001.0339.
14. Candèe, E. J. and RECHT, B. (2009). Exact matrix completion via convex optimization. Found. Comput. Math. 9 717-772. MR2565240
15. Candèe, E. J. and TAO, T. (2005). Decoding by linear programming. IEEE Trans. Inform. Theory 51 4203-4215. MR2243152
16. Candèe, E. J. and TAO, T. (2009). The power of convex relaxation: Near-optimal matrix completion. Unpublished manuscript.
17. MCCARTHY, C. A. (1967). Cp. Israel J. Math. 5 249-272. MR0225140
18. DALALYAN, A. and TSYBAKOV, A. (2008). Aggregation by exponential weighting, sharp oracle inequalities and sparsity. Mach. Learn. 72 39-61.
19. DONOHO, D. L., JOHNSTONE, I. M., HOCH, J. C. and STERN, A. S. (1992). Maximum entropy and the nearly black object. J. Roy. Statist. Soc. Ser. B 54 41-81. MR1157714
20. EDMUNDS, D. E. and TRIEBEL, H. (1996). Function Spaces, Entropy Numbers, Differential Operators. Cambridge Univ. Press, Cambridge. MR1410258
21. EDMUNDS, D. E. and TRIEBEL, H. (1989). Entropy numbers and approximation numbers in function spaces. Proc. London Math. Soc. 58 137-152. MR0969551
22. FOSTER, D. P. and GEORGE, E. I. (1994). The risk inflation criterion for multiple regression. Ann. Statist. 22 1947-1975. MR1329177
23. Guédon, O. and LITVAK, A. E. (2000). Euclidean projections of a p-convex body. In Geometric Aspects of Functional Analysis, Israel Seminar (GAFA) 1996-2000 (V. D. Milman and G. Schechtman, eds.). Lecutre Notes in Mathematics 1745 95-108. Springer, Berlin. MR1796715
24. GROSS, D. (2009). Recovering low-rank matrices from few coefficients in any basis. Available at arXiv:0910.1879.
25. KESHAVAN, R. H., MONTANARI, A. and OH, S. (2009). Matrix completion from noisy entries. Available at arXiv:0906.2027.
26. KOLMOGOROV, A. N. and TIKHOMIROV, V. M. (1959). The ε-entropy and ε-capacity of sets in function spaces. Uspekhi Matem. Nauk 14 3-86. MR0112032
27. KOLTCHINSKII, V. (2008). Oracle inequalities in empirical risk minimization and sparse recovery problems. Ecole d'Eté de Probabilités de Saint-Flour, Lecture Notes. Preprint.
28. LOUNICI, K., PONTIL, M., TSYBAKOV, A. B. and VAN DE GEER, S. (2009). Taking advantage of sparsity in multi-task learning. In Proceedings of COLT-2009.
29. LOUNICI, K., PONTIL, M., TSYBAKOV, A. B. and VAN DE GEER, S. (2010). Oracle inequalities and optimal inference under group sparsity. Available at arXiv:1007.1771.
30. MEINSHAUSEN, N. and BüHLMANN, P. (2006). High-dimensional graphs and variable selection with the lasso. Ann. Statist. 34 1436-1462. MR2278363 (Pubitemid 44231168)
31. MENDELSON, S., PAJOR, A. and TOMCZAK-JAEGERMANN, N. (2007). Reconstruction and subgaussian operators in asymptotic geometric analysis. Geom. Funct. Anal. 17 1248-1282. MR2373017
32. NEGAHBAN, S., RAVIKUMAR, P., WAINWRIGHT, M. J. and YU, B. (2009). A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers. In Advances in Neural Information Processing Systems, NIPS-2009.
33. NEGAHBAN, S. andWAINWRIGHT, M. J. (2011). Estimation of (near) low rank matrices with noise and high-dimensional scaling. Ann. Statist. To appear. Available at arXiv:0912.5100.
34. NEMIROVSKI, A. (2004). Regular Banach spaces and large deviations of random sums. Unpublished manuscript.
35. PAJOR, A. (1998). Metric entropy of the Grassmann manifold. Convex Geom. Anal. 34 181-188. MR1665590
36. PAULSEN, V. I. (1986). Completely Bounded Maps and Dilations. In Pitman Research Notes in Mathematics 146. Longman, New York. MR0868472
37. PIETSCH, A. (1980). Operator Ideals. Elsevier, Amsterdam. MR0582655
38. PINELIS, I. F. and SAKHANENKO, A. I. (1985). Remarks on inequalities for the probabilities of large deviations. Theory Probab. Appl. 30 143-148. MR0779438
39. RAVIKUMAR, P., WAINWRIGHT, M., RASKUTTI, G. and YU, B. (2008). High-dimensional covariance estimation by minimizing l1-penalized log-determinant divergence. Unpublished manuscript.
40. RECHT, B. (2009). A simpler approach to matrix completion. Available at arXiv:0910.0651.
41. RECHT, B., FAZEL, M. and PARRILO, P. A. (2010). Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52 471-501. MR2680543
42. RIGOLLET, P. and TSYBAKOV, A. B. (2010). Exponential screening and optimal rates of sparse estimation. Available at arXiv:1003.2654.
43. ROTFELD, S. Y. (1969). The singular numbers of the sum of completely continuous operators. In Topics in Mathematical Physics (M. S. Birman, ed.). Spectral Theory 3 73-78. English version published by Consultants Bureau, New York.
44. SREBRO, N., RENNIE, J. and JAAKKOLA, T. (2005). Maximum margin matrix factorization. In Advances in Neural Information Processing Systems 17 (L. Saul, Y. Weiss and L. Bottou, eds.) 1329-1336. MIT Press, Cambridge, MA.
45. SREBRO, N. and SHRAIBMAN, A. (2005). Rank, trace-norm and max-norm. In Learning Theory, Proceedings of COLT-2005. Lecture Notes in Comput. Sci. 3559 545-560. Springer, Berlin. MR2203286
46. TROPP, J. A. (2010). User-friendly tail bounds for sums of random matrices. Available at arXiv:1004.4389.
47. TSYBAKOV, A. (2009). Introduction to Nonparametric Estimation. Springer, New York. MR2724359
48. TSYBAKOV, A. and VAN DE GEER, S. (2005). Square root penalty: Adaptation to the margin in classification and in edge estimation. Ann. Statist. 33 1203-1224. MR2195633 (Pubitemid 41137880)
49. VAN DE GEER, S. (2000). Empirical Processes in M-estimation. Cambridge Univ. Press, Cambridge.
50. VERSHYNIN, R. (2007). Some problems in asymptotic convex geometry and random matrices motivated by numerical algorithms. In Banach Spaces and Their Applications in Analysis (B. Randrianantoanina and N. Randrianantoanina, eds.) 209-218. de Gruyter, Berlin. MR2374708
51. ZHAO, P. andYU, B. (2006). On model selection consistency of lasso. J. Mach. Learn. Res. 7 2541-2563. MR2274449 (Pubitemid 44866738)