Computational and statistical tradeoffs via convex relaxation

Proceedings of the National Academy of Sciences of the United States of America

Volumn 110, Issue 13, 2013, Pages

  Chandrasekaran, Venkat ^a   Jordan, Michael I ^b  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Convex geometry; Convex optimization; High dimensional statistics

Indexed keywords

ALGORITHM; ARTICLE; COMPUTER MODEL; CONVEX RELAXATION; MATHEMATICAL ANALYSIS; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; PERFORMANCE; PRIORITY JOURNAL; PROCESS OPTIMIZATION; SEQUENTIAL ANALYSIS; STATISTICAL ANALYSIS; TIME SERIES ANALYSIS;

EID: 84875499797     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1302293110     Document Type: Article

References (54)

1. Lai TL (2001) Sequential analysis: Some classical problems and new challenges. Statistica Sinica 11(2):303-408.
2. Arora S, Barak B (2009) Computational Complexity: A Modern Approach (Cambridge Univ Press, New York).
3. Bühlmann P, van de Geer S (2011) Statistics for High-Dimensional Data: Methods, Theory and Applications (Springer, Berlin).
4. Vazirani V (2004) Approximation Algorithms (Springer, Berlin).
5. Johnstone IM (2011) Gaussian estimation: Sequence and wavelet models. Available at www-stat.stanford.edu/~imj/. Last accessed February 26, 2013.
6. Donoho DL (1995) Denoising by soft thresholding. IEEE Trans Inf Theory 41:613-627.
7. Donoho DL, Johnstone IM (1998) Minimax estimation via wavelet shrinkage. Annals of Statistics 26(3):879-921.
8. Boyd SP, Vandenberghe L (2004) Convex Optimization (Cambridge Univ Press, New York).
9. Shalev-Shwartz S, Shamir O, Tromer E (2012) Using more data to speed up training time. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, April 21-23, 2012, La Palma, Canary Islands, pp 1019-1027.
10. Decatur S, Goldreich O, Ron D (1998) Computational sample complexity. SIAM J Sci Comput 29:854-879.
11. Servedio R (2000) Computational sample complexity and attribute-efficient learning. Journal of Computer and Systems Sciences 60(1):161-178.
12. Birnbaum A, Shalev-Shwartz S (2012) Learning halfspaces with the zero-one loss: time-accuracy tradeoffs. Advances in Neural Information Processing Systems 25, eds Bartlett P, Pereira FCN, Burges CJC, Bottou L, Weinberger KQ, pp 935-943.
13. Bottou L, Bousquet O (2008) The tradeoffs of large scale learning. Advances in Neural Information Processing Systems 20, eds Platt JC, Koller D, Singer Y, Roweis S (MIT Press, Cambridge, MA), pp 161-168.
14. Shalev-Schwartz S, Srebro N (2008) SVM optimization: Inverse dependence on training set size. Proceedings of the 25th Annual International Conference on Machine Learning, eds McCallum A, Roweis S (Omnipress, Helsinki, Finland), pp 928-935.
15. Perkins TJ, Hallett MT (2010) A trade-off between sample complexity and computational complexity in learning Boolean networks from time-series data. IEEE/ACM Trans Comput Biol Bioinformatics 7(1):118-125.
16. Amini A, Wainwright M (2009) High-dimensional analysis of semidefinite programming relaxations for sparse principal component analysis. Ann Stat 37:2877-2921.
17. Kolda T (2001) Orthogonal tensor decompositions. SIAM Journal on Matrix Analysis 23:243-255.
18. Johnstone IM, Lu AY (2009) On consistency and sparsity for principal components analysis in high dimensions. J Am Stat Assoc 104(486):682-693.
19. Agarwal A, Bartlett P, Duchi J (2012) Oracle inequalities for computationally adaptive model selection. arXiv:1208.0129.
20. Nesterov Y, Nemirovskii A (1995) Interior-Point Polynomial Algorithms in Convex Programming (Society for Industrial and Applied Mathematics, Philadelphia).
21. Candès EJ, Romberg J, Tao T (2006) Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans Inf Theory 52:489-509.
22. Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory 52:1289-1306.
23. Ziegler G (1995) Lectures on Polytopes (Springer, Berlin).
24. Goemans M (2012) Smallest Compact Formulation for the Permutahedron. Technical Report (Department of Mathematics, Massachusetts Institute of Technology, Cambridge, MA).
25. Gouveia J, Parrilo P, Thomas R (2012) Lifts of convex sets and cone factorizations. Mathematics of Operations Research, 10.1287/moor.1120.0575.
26. Goemans M, Williamson D (1995) Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM 42(6):1115-1145.
27. Sherali HD, Adams WP (1990) A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems. SIAM Journal on Discrete Mathematics 3(3):411-430.
28. Bochnak J, Coste M, Roy M (1988) Real Algebraic Geometry (Springer, Berlin).
29. Parrilo PA (2000) Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization. PhD thesis (California Institute of Technology, Pasadena, CA).
30. Parrilo PA (2003) Semidefinite programming relaxations for semialgebraic problems. Mathematical Programming 96(2):293-320.
31. Lasserre JB (2001) Global optimization with polynomials and the problem of moments. SIAM J Optim 11:796-817.
32. Gouveia J, Parrilo P, Thomas R (2010) Theta bodies for polynomial ideals. SIAM J Optim 20:2097-2118.
33. Renegar J (2006) Hyperbolic programs and their derivative relaxations. Foundations of Computational Mathematics 6(1):59-79.
34. Srebro N, Shraibman A (2005) Learning Theory. 18th Annual Conference on Learning Theory, June 27-30, 2005, Bertinoro, Italy. Lecture Notes in Computer Science 3559, eds Auer P, Meir R (Springer).
35. Rockafellar RT (1970) Convex Analysis (Princeton Univ Press, Princeton).
36. Dudley RM (1967) The sizes of compact subsets of Hilbert space and continuity of Gaussian processes. Journal of Functional Analysis 1(3):290-330.
37. Bartlett P, Mendelson S (2002) Rademacher and Gaussian complexities: risk bounds and structural results. J Mach Learn Res 3:463-482.
38. Chandrasekaran V, Recht B, Parrilo P, Willsky A (2012) The convex geometry of linear inverse problems. Foundations of Computational Mathematics 12(6):805-849.
39. Ledoux M (2000) The Concentration of Measure Phenomenon (American Mathematical Society, Providence, RI).
40. Sanyal R, Sottile F, Sturmfels B (2011) Orbitopes. Mathematika 57:275-314.
41. James W, Stein C (1961) Estimation with quadratic loss. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, ed Neyman J (University of California Press, Berkeley, CA), pp 361-379.
42. Ben Tal A, Nemirovskii A (2001) Lectures on Modern Convex Optimization (Society for Industrial and Applied Mathematics, Philadelphia).
43. Higham N (2002) Computing the nearest correlation matrix-A problem from finance. IMA Journal of Numerical Analysis 22(3):329-343.
44. Bickel P, Levina L (2008) Regularized estimation of large covariance matrices. Annals of Statistics 36(1):199-227.
45. Garey M, Johnson D (1979) Computers and Intractability: A Guide to the Theory of NP-Completeness (Freeman, New York).
46. Kolar M, Balakrishnan S, Rinaldo A, Singh A (2011) Minimax localization of structural information in large noisy matrices. Neural Information Processing Systems 24, eds Shawe-Taylor J, Zemel RS, Bartlett P, Pereira, FCN, Weinberger, pp 909-917.
47. Alon N, Krivelevich M, Sudakov B (1998) Finding a large hidden clique in a random graph. Random Structures Algorithms 13:457-466.
48. Feige U, Krauthgamer R (2000) Finding and certifying a large hidden clique in a semirandom graph. Random Structures Algorithms 16:195-208.
49. Ames BPW, Vavasis SA (2011) Nuclear norm minimization for the planted clique and biclique problems. Mathematical Programming, Series B 129:69-89.
50. Rouvray DH, Balaban AT (1979) Chemical applications of graph theory. Applications of Graph Theory (Academic Press, London), pp 177-221.
51. Shoubridge P, Krarne M, Ray D (1999) Detection of abnormal change in dynamic networks. Proceedings of Information, Decision, and Control, pp 557-562.
52. Edmonds J (1965) Maximum matching and a polyhedron with 0-1 vertices. J Res Natl Bur Stand 69B:125-130.
53. Padberg M, Rao M (1982) Odd minimum cut-sets and b-matchings. Mathematics of Operations Research 7(1):67-80.
54. Feldman D, Langberg M (2011) A unified framework for approximating and clustering data. Proceedings of the Symposium on the Theory of Computing, pp 569-578.