Matrix concentration inequalities via the method of exchangeable pairs

Annals of Probability

Volumn 42, Issue 3, 2014, Pages 906-945

  Mackey, Lester ^a   Jordan, Michael I ^b   Chen, Richard Y ^c   Farrell, Brendan ^c   Tropp, Joel A ^c  

^a STANFORD UNIVERSITY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c California Institute of Technology   (United States)

Author keywords

Concentration inequalities; Exchangeable pairs; Moment inequalities; Noncommutative; Random matrix; Stein's method

Indexed keywords

EID: 84919813328     PISSN: 00911798     EISSN: None     Source Type: Journal    
DOI: 10.1214/13-AOP892     Document Type: Article

References (50)

1. AHLSWEDE, R. andWINTER, A. (2002). Strong converse for identification via quantum channels.IEEE Trans. Inform. Theory 48 569-579. MR1889969.
2. BHATIA, R. (1997). Matrix Analysis. Graduate Texts in Mathematics 169. Springer, New York.MR1477662
3. BUCHHOLZ, A. (2001). Operator Khintchine inequality in noncommutative probability. Math.319 1-16. MR1812816
4. BURKHOLDER, D. L. (1973). Distribution function inequalities for martingales. Ann. Probab.1 19-42. MR0365692
5. CARLEN, E. (2010). Trace inequalities quantum entropy: An introductory course. In Entropy and the Quantum. Contemp. Math. 529 73-140. Amer. Math. Soc., Providence, RI.MR2681769
6. CHATTERJEE, S. (2007). Stein's method for concentration inequalities. Probab. Theory Related Fields 138 305-321. MR2288072
7. CHATTERJEE, S. (2008). Concentration inequalities with exchangeable pairs. Ph.D. thesis,Stanford Univ., Palo Alto.
8. CHEN, R. Y., GITTENS, A. and TROPP, J. A. (2012). The masked sample covariance estimator: An analysis using matrix concentration inequalities. Information and Inference 12-20.
9. CHEUNG, S. S., SO, A. M.-C. and WANG, K. (2011). Chance-constrained linear matrix inequalities with dependent perturbations: A safe tractable approximation approach. Available at http://www.optimization-online.org/DB_FILE/2011/01/2898.pdf.
10. CHIU, J. and DEMANET, L. (2011). Sublinear randomized algorithms for skeleton decomposition. Available at arXiv:1110.4193.
11. CHIU, J. andDEMANET, L. (2012).Matrix probing and its conditioning. SIAM J. Numer. Anal. 50 171-193. MR2888309
12. COHEN, A., DAVENPORT, M. and LEVIATAN, D. (2011). On the stability and accuracy of least-squares approximation. Available at arXiv:1111.4422.
13. FOYGEL, R. and SREBRO, N. (2011). Concentration-based guarantees for low-rank matrix reconstruction. J. Mach. Learn. Res.,19:315-340
14. GITTENS, A. (2011). The spectral norm error of the naïve Nyström extension. Available at arXiv:1110.5305.
15. GITTENS, A. and TROPP, J. A. (2011). Tail bounds for all eigenvalues of a sum of random matrices. Available at arXiv:1104.4513.
16. GROSS, D. (2011). Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inform. Theory 57 1548-1566. MR2815834
17. HIGHAM, N. J. (2008). Functions of Matrices: Theory and Computation. SIAM, Philadelphia, PA. MR2396439
18. HOEFFDING, W. (1951). A combinatorial central limit theorem. Ann. Math. Statistics 22 558- 566. MR0044058
19. HOEFFDING, W. (1963). Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13-30. MR0144363
20. HSU, D., KAKADE, S. M. and ZHANG, T. (2012). Tail inequalities for sums of random matrices that depend on the intrinsic dimension. Electron. Commun. Probab. 17 13. MR2900355
21. JUNGE, M. and XU, Q. (2003). Noncommutative Burkholder/Rosenthal inequalities. Ann. Probab. 31 948-995. MR1964955
22. JUNGE, M. and XU, Q. (2008). Noncommutative Burkholder/Rosenthal inequalities. II. Applications. Israel J. Math. 167 227-282. MR2448025
23. JUNGE, M. and ZHENG, Q. (2011). Noncommutative Bennett and Rosenthal inequalities. Available at arXiv:1111.1027.
24. KOLTCHINSKII, V. (2011). Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems. Lecture Notes in Math. 2033. Springer, Heidelberg. MR2829871
25. LEDOUX, M. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Amer. Math. Soc., Providence, RI. MR1849347
26. LIEB, E. H. (1973). Convex trace functions and the Wigner-Yanase-Dyson conjecture. Adv. Math. 11 267-288. MR0332080
27. LUGOSI, G. (2009). Concentration-of-measure inequalities. Available at http://www.econ.upf. edu/~lugosi/anu.pdf.
28. LUST-PIQUARD, F. (1986). Inégalités de Khintchine dans Cp (1
29. LUST-PIQUARD, F. and PISIER, G. (1991). Noncommutative Khintchine and Paley inequalities. Ark. Mat. 29 241-260. MR1150376
30. MACKEY, L., TALWALKAR, A. and JORDAN, M. I. (2011). Divide-and-conquer matrix factorization. In Advances in Neural Information Processing Systems 24 (J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. C. N. Pereira and K. Q. Weinberger, eds.) 1134-1142.
31. MC DIARMID, C. (1989). On the method of bounded differences. In Surveys in Combinatorics, 1989 (Norwich, 1989). London Mathematical Society Lecture Note Series 141 148-188. Cambridge Univ. Press, Cambridge. MR1036755
32. MINSKER, S. (2011). Some extensions of Bernstein's inequality for self-adjoint operators. Available at arXiv:1112.5448.
33. NAGAEV, S. V. and PINELIS, I. F. (1977). Some inequalities for the distributions of sums of independent random variables. Theory Probab. Appl. 22 248-256.
34. NEGAHBAN, S. and WAINWRIGHT, M. J. (2012). Restricted strong convexity and weighted matrix completion: Optimal bounds with noise. J. Mach. Learn. Res. 13 1665-1697. MR2930649
35. OLIVEIRA, R. I. (2009). Concentration of the adjacency matrix and of the Laplacian in random graphs with independent edges. Available at arXiv:0911.0600.
36. OLIVEIRA, R. I. (2010). Sums of random Hermitian matrices and an inequality by Rudelson. Electron. Commun. Probab. 15 203-212. MR2653725
37. PAULSEN, V. (2002). Completely Bounded Maps and Operator Algebras. Cambridge Studies in Advanced Mathematics Cambridge Univ. Press, Cambridge.177. MR1976867
38. PETZ, D. (1994). A survey of certain trace inequalities. In Functional Analysis and Operator Theory (Warsaw, 1992). Banach Center Publ. 30 287-298. Polish Acad. Sci., Warsaw. MR1285615
39. PINELIS, I. (1994). Optimum bounds for the distributions of martingales in Banach spaces. Ann. Probab. 22 1679-1706. MR1331198
40. PISIER, G. and XU, Q. (1997). Noncommutative martingale inequalities. Comm. Math. Phys. 189 667-698. MR1482934
41. RAUHUT, H. (2010). Compressive sensing and structured random matrices. In Theoretical Foundations and Numerical Methods for Sparse Recovery. Radon Ser. Comput. Appl. Math. 9 1-92. de Gruyter, Berlin. MR2731597
42. RECHT, B. (2011). A simpler approach to matrix completion. J. Mach. Learn. Res. 12 3413-3430. MR2877360
43. ROSENTHAL, H. P. (1970). On the subspaces of Lp (p > 2) spanned by sequences of independent random variables. Israel J. Math. 8 273-303. MR0271721
44. RUDELSON, M. (1999). Random vectors in the isotropic position. J. Funct. Anal. 164 60-72. MR1694526
45. RUDELSON, M. and VERSHYNIN, R. (2007). Sampling from large matrices: An approach through geometric functional analysis. J. ACM 54 Art. 21, 19 pp. (electronic). MR2351844
46. SO, A. M.-C. (2011). Moment inequalities for sums of random matrices and their applications in optimization. Math. Program. 130 125-151. MR2853163
47. STEIN, C. (1972). A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability (Univ. California, Berkeley, Calif., 1970/1971), Vol. II: Probability Theory 583-602. Univ. California Press, Berkeley, CA. MR0402873
48. TROPP, J. A. (2011). Freedman's inequality for matrix martingales. Electron. Commun. Probab. 16 262-270. MR2802042
49. TROPP, J. A. (2012). User-friendly tail bounds for sums of random matrices. Found. Comput. Math. 12 389-434. MR2946459
50. WIGDERSON, A. and XIAO, D. (2008). Derandomizing the Ahlswede-Winter matrix-valued Chernoff bound using pessimistic estimators, and applications. Theory Comput. 4 53-76. MR2403380