Living on the edge: Phase transitions in convex programs with random data

Information and Inference

Volumn 3, Issue 3, 2014, Pages 224-294

  Amelunxen, Dennis ^a   Lotz, Martin ^a   McCoy, Michael B ^b   Tropp, Joel A ^b  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

^b California Institute of Technology   (United States)

Author keywords

1 minimization; Compressed sensing; Conic geometry; Convex optimization; Integral geometry; Morphological component analysis; Nuclear norm minimization; Phase transitions; Rank sparsity decomposition

Indexed keywords

EID: 85071577100     PISSN: None     EISSN: 20498772     Source Type: Journal    
DOI: 10.1093/imaiai/iau005     Document Type: Article

References (101)

1. Ablowitz, M. J. & Fokas, A. S. Complex Variables: Introduction and Applications, 2nd edn. Cambridge Texts in Applied Mathematics. Cambridge: Cambridge University Press, 2003.
2. Abramowitz, M. & Stegun, I. A. (1964) Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. National Bureau of Standards Applied Mathematics Series 55. Washington, D.C.: For sale by the Superintendent of Documents, U.S. Government Printing Office.
3. Affentranger, F. & Schneider, R. (1992) Random projections of regular simplices. Discrete Comput. Geom., 7, 219-226.
4. Alizadeh, F. & Goldfarb, D. (2003) Second-order cone programming. Math. Program., 95, 3-51. ISMP 2000, Part 3 (Atlanta, GA).
5. 1. Bull. Amer. Math. Soc., 54, 128-135.
6. Amelunxen, D. (2011) Geometric analysis of the condition of the convex feasibility problem. Ph.D. Thesis, University of Paderborn.
7. Amelunxen, D. & Bürgisser, P. (2012) Intrinsic volumes of symmetric cones. Available at arxiv.org/abs/1205.1863. Abridged version, 'Intrinsic volumes of symmetric cones and applications in convex programming', Math. Progam., January 2014.
8. Amelunxen, D. & Bürgisser, P. (2013) Probabilistic analysis of the Grassmann condition number. Found. Comput. Math. Also available at arxiv.org/abs/1112.2603.
9. Anderson, T. W. (1984) An Introduction to Multivariate Statistical Analysis, 2nd edn. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. New York: John Wiley & Sons Inc.
10. Artstein, S. (2002) Proportional concentration phenomena on the sphere. Israel J. Math., 132, 337-358.
11. Bai, Z. & Silverstein, J. W. (2010) Spectral Analysis of Large Dimensional Random Matrices, 2nd edn. Springer Series in Statistics. New York: Springer.
12. Bayati, M., Lelarge, M. & Montanari, A. (2013) Universality in polytope phase transitions and message passing algorithms. Ann. Applied Probab., to appear. Available at arxiv.org/abs/1207.7321.
13. Bayati, M. & Montanari, A. (2012) The LASSO risk for Gaussian matrices. IEEE Trans. Inform. Theory, 58, 1997-2017.
14. Ben-Tal, A. & Nemirovski, A. (2001) Analysis, algorithms, and engineering applications. Lectures on Modern Convex Optimization. MPS/SIAM Series on Optimization. Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM).
15. Bibikov, P. V. & Zhgoon, V. S. (2009) Angle measures of some cones associated with finite reflection groups. J. Lie Theory, 19, 767-769.
16. Bobin, J., Moudden, Y. & Starck, J.-L. (2006) Morphological diversity and source separation. IEEE Trans. Signal Process., 13, 409-412.
17. Bogachev, V. I. Gaussian measures. Mathematical Surveys and Monographs, vol. 62. Providence, RI: American Mathematical Society, 1998.
18. Böröczky, K. Jr. & Henk, M. (1999) Random projections of regular polytopes. Arch. Math. (Basel), 73, 465-473.
19. Borovik, A. V. & Borovik, A. (2010) The geometry of finite reflection groups. Mirrors and Reflections. Universitext. New York: Springer.
20. Boucheron, S., Lugosi, G. & Massart, P. (2013) Concentration Inequalities: a Nonasymptotic Theory of Independence. Oxford: Oxford University Press.
21. Boyd, S. & Vandenberghe, L. (2004) Convex Optimization. Cambridge: Cambridge University Press.
22. Candès, E. J. & Tao, T. (2006) Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inform. Theory, 52, 5406-5425.
23. Chandrasekaran, V., Recht, B., Parrilo, P. A. & Willsky, A. S. (2012) The convex geometry of linear inverse problems. Found. Comput. Math., 12, 805-849.
24. Chandrasekaran, V., Sanghavi, S., Parrilo, P. A. & Willsky, A. S. (2011) Rank-sparsity incoherence for matrix decomposition. SIAM J. Optim., 21, 572-596.
25. Chen, S. S., Donoho, D. L. & Saunders, M. A. (2001) Atomic decomposition by basis pursuit. SIAM Rev., 43, 129-159.
26. Reprinted from SIAM J. Sci. Comput., 20 (1998) 33-61 (electronic).
27. Claerbout, J. F. & Muir, F. (1973) Robust modeling of erratic data. Geophysics, 38, 826-844.
28. Coxeter, H. S. M. & Moser, W. O. J. (1972) Generators and Relations for Discrete Groups, 3rd edn. Ergebnisse der Mathematik und ihrer Grenzgebiete, Band 14. New York: Springer.
29. Donoho, D. L. (2006) Compressed sensing. IEEE Trans. Inform. Theory, 52, 1289-1306.
30. Donoho, D. L. (2006) High-dimensional centrally symmetric polytopes with neighborliness proportional to dimension. Discrete Comput. Geom., 35, 617-652.
31. Donoho, D. L., Gavish, M. & Montanari, A. (2013) The phase transition of matrix recovery from Gaussian measurements matches the minimax MSE of matrix denoising. Proc. Natl Acad. Sci. USA, 110, 8405-8410.
32. Donoho, D. L., Johnstone, I. & Montanari, A. (2013) Accurate prediction of phase transitions in compressed sensing via a connection to minimax denoising. IEEE Trans. Inform. Theory, 59, 3396-3433.
33. Donoho, D. L., Maleki, A. & Montanari, A. (2009) Message-passing algorithms for compressed sensing. Proc. Natl Acad. Sci. USA, 106, 18914-18919.
34. Donoho, D. L., Maleki, A. & Montanari, A. (2009) Supporting information to: message-passing algorithms for compressed sensing. Proc. Natl Acad. Sci. USA. http://www.pnas.org/content/106/45/18914/suppl/DCSupplemental
35. Donoho, D. L. & Tanner, J. (2005) Neighborliness of randomly projected simplices in high dimensions. Proc. Natl Acad. Sci. USA, 102, 9452-9457 (electronic).
36. Donoho, D. L. & Tanner, J. (2009) Counting faces of randomly projected polytopes when the projection radically lowers dimension. J. Amer. Math. Soc., 22, 1-53.
37. Donoho, D. L. & Tanner, J. (2009) Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci., 367, 4273-4293.
38. Donoho, D. L. & Tanner, J. (2010) Counting the faces of randomly-projected hypercubes and orthants, with applications. Discrete Comput. Geom., 43, 522-541.
39. Donoho, D. L. & Tanner, J. (2010) Exponential bounds implying construction of compressed sensing matrices, error-correcting codes, and neighborly polytopes by random sampling. IEEE Trans. Inform. Theory, 56, 2002-2016.
40. Drton, M. & Klivans, C. (2010) A geometric interpretation of the characteristic polynomial of reflection arrangements. Proc. Amer. Math. Soc., 138, 2873-2887.
41. Elad, M., Starck, J.-L., Querre, P. & Donoho, D. L. (2005) Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmon. Anal., 19, 340-358.
42. Eldar, Y. C. & Kutyniok, G. eds. Theory and Applications. Compressed Sensing. Cambridge: Cambridge University Press, 2012.
43. Fazel, M. (2002) Matrix rank minimization with applications. Ph.D. Thesis, Stanford University.
44. Foucart, S. & Rauhut, H. (2013) A Mathematical Introduction to Compressive Sensing. Applied and Numerical Harmonic Analysis. New York: Birkhäuser/Springer.
45. Foygel, R. & Mackey, L. (2014) Corrupted sensing: Novel guarantees for separating structured signals. IEEE Trans. Inf. Theory, 60, 1223-1247.
46. Glasauer, S. (1995) Integralgeometrie konvexer Körper im sphärischen Raum. Ph.D. Thesis, Freiburg im Breisgau.
47. Glasauer, S. (1996) Integral geometry of spherically convex bodies. Diss. Summ. Math., 1, 219-226.
48. Gordon, Y. (1985) Some inequalities for Gaussian processes and applications. Israel J. Math., 50, 265-289.
49. n. Geometric Aspects of Functional Analysis (1986/87). Lecture Notes in Mathematics, vol. 1317. Berlin: Springer, pp. 84-106.
50. Grant, M. C. & Boyd, S. P. (2013) The CVX user's guide, release 2.0 (beta). User manual. Available at cvxr.com/cvx/doc.
51. Grünbaum, B. (1968) Grassman angles of convex polytopes. Acta Math., 121, 293-302.
52. Herglotz, G. (1943) Über die Steinersche Formel für Parallelflächen. Abh. Math. Sem. Hansischen Univ., 15, 165-177.
53. Hiriart-Urruty, J.-B. & Lemaréchal, C. (1993) Convex Analysis and Minimization Algorithms. I: Fundamentals I, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 305. Berlin: Springer.
54. Hohlweg, C., C. E. Lange, M. C. & Thomas, H. (2011) Permutahedra and generalized associahedra. Adv. Math., 226, 608-640.
55. Horn, R. A. & Johnson, C. R. (1990) Matrix analysis. Cambridge: Cambridge University Press. Corrected reprint of the 1985 original.
56. Hotelling, H. (1939) Tubes and spheres in n-spaces, and a class of statistical problems. Amer. J. Math., 61, 440-460.
57. Jegou, H., Furon, T. & Fuchs, J.-J. (2012) Anti-sparse coding for approximate nearest neighbor search. 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2029-2032.
58. 1 minimization for sparse recovery with nonuniform sparse models. IEEE Trans. Signal Process., 59, 1985-2001.
59. Klain, D. A. & Rota, G.-C. (1997) Introduction to Geometric Probability. Lezioni Lincee. [Lincei Lectures]. Cambridge: Cambridge University Press.
60. Klivans, C. J. & Swartz, E. (2011) Projection volumes of hyperplane arrangements. Discrete Comput. Geom., 46, 417-426.
61. Ledoux, M. (2001) The Concentration of Measure Phenomenon, Mathematical Surveys and Monographs 89. Providence, RI: American Mathematical Society.
62. Marčenko, V. A. & Pastur, L. A. (1967) Distribution of eigenvalues in certain sets of random matrices. Mat. Sb. (N.S.), 72, 507-536.
63. McCoy, M. B. (2013) A geometric analysis of convex demixing. Ph.D. Thesis, California Institute of Technology.
64. McCoy, M. B. (2014) Snowmaker: Matlab code for statistical dimension computations. Available at users.cms.caltech.edu/∼mccoy/code/snow.m, Jan.
65. McCoy, M. B. & Tropp, J. A. (2013) The achievable performance of convex demixing. Available at arxiv.org/abs/1309.7478.
66. McCoy, M. B. & Tropp, J. A. (2014) From Steiner formulas for cones to concentration of intrinsic volumes. Discrete Comput. Geom., 51, 926-963.
67. McCoy, M. B. & Tropp, J. A. (2014) Sharp recovery bounds for convex demixing, with applications. Found. Comput. Math., 14, 503-567.
68. McMullen, P. (1993) Valuations and dissections. Handbook of Convex Geometry, Vol. A, B. Amsterdam: North-Holland, pp. 933-988.
69. Mesbahi, M. & Papavassilopoulos, G. P. (1997) On the rank minimization problem over a positive semidefinite linear matrix inequality. IEEE Trans. Automat. Control, 42, 239-243.
70. Mezzadri, F. (2007) How to generate random matrices from the classical compact groups. Notices Amer. Math. Soc., 54, 592-604.
71. Mu, C., Huang, B., Wright, J. & Goldfarb, D. (2013) Square deal: lower bounds and improved relaxations for tensor recovery. Available at http://arxiv.org/abs/1307.5870.
72. Olver, F. W. J., Lozier, D. W., Boisvert, R. F. & Clark, C. W. eds. (2010) NIST Handbook of Mathematical Functions. Washington, D.C.: U.S. Department of Commerce, National Institute of Standards and Technology.
73. Oymak, S. & Hassibi, B. (2010) New null space results and recovery thresholds for matrix rank minimization. Partial results presented at ISIT 2011. Available at arxiv.org/abs/1011.6326.
74. Oymak, S. & Hassibi, B. (2013) Sharp MSE bounds for proximal denoising. Partial results presented at Allerton 2012. Available at arxiv.org/abs/1305.2714.
75. Oymak, S., Khajehnejad, M. A. & Hassibi, B. (2011) Improved thresholds for rank minimization. 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 5988-5991.
76. Recht, B., Fazel, M. & Parrilo, P. A. (2010) Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev., 52, 471-501.
77. Recht, B., Xu, W. & Hassibi, B. (2011) Null space conditions and thresholds for rank minimization. Math. Program., 127, 175-202.
78. Rockafellar, R. T. (1970) Convex Analysis. Princeton Mathematical Series, vol. 28. Princeton, NJ: Princeton University Press.
79. Rockafellar, R. T. & Wets, R. J.-B. Variational Analysis. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 317. Berlin: Springer, 1998.
80. Rudelson, M. & Vershynin, R. (2008) On sparse reconstruction from Fourier and Gaussian measurements. Comm. Pure Appl. Math., 61, 1025-1045.
81. Santaló, L. A. (1950) On parallel hypersurfaces in the elliptic and hyperbolic n-dimensional space. Proc. Amer. Math. Soc., 1, 325-330.
82. Santaló, L. A. (1976) Integral Geometry and Geometric Probability. Reading, MA, London, Amsterdam: Addison-Wesley Publishing Co.
83. Santosa, F. & Symes, W. W. (1986) Linear inversion of band-limited reflection seismograms. SIAM J. Sci. Statist. Comput., 7, 1307-1330.
84. Schläfli, L. (1950) Gesammelte mathematische Abhandlungen. Band I. Basel: Birkhäuser.
85. Schläfli, L. (1950) Theorie der Vielfachen Kontinuität (1852), Gesammelte mathematische Abhandlungen. Band I. Basel: Birkhäuser.
86. Schneider, R. (1993) Convex Bodies: the Brunn-Minkowski Theory. Encyclopedia of Mathematics and its Applications, vol. 44. Cambridge: Cambridge University Press.
87. Schneider, R. & Weil, W. (2008) Stochastic and Integral Geometry. Springer Series in Statistics: Probability and its Applications. Springer.
88. Shephard, G. C. & Todd, J. A. (1954) Finite unitary reflection groups. Canadian J. Math., 6, 274-304.
89. Starck, J.-L., Donoho, D. L. & Candès, E. J. (2003) Astronomical image representation by the curvelet transform. Astronom. Astrophys., 398, 785-800.
90. Starck, J.-L., Elad, M. & Donoho, D. L. (2005) Image decomposition via the combination of sparse representations and a variational approach. IEEE Trans. Image Process., 14, 1570-1582.
91. Steinberg, R. (1959) Finite reflection groups. Trans. Amer. Math. Soc., 91, 493-504.
92. 1 -optimization in compressed sensing. Available at arxiv.org/abs/0907.3666.
93. Stojnic, M. (2013) Regularly random duality. Available at arXiv.org/abs/1303.7295.
94. van de Ven, R. & Weber, N. C. (1993) Bounds for the median of the negative binomial distribution. Metrika, 40, 185-189.
95. Vershik, A. M. & Sporyshev, P. V. (1986) An asymptotic estimate for the average number of steps in the parametric simplex method. USSR Comput. Maths. Math. Phys., 26, 104-113.
96. Vershik, A. M. & Sporyshev, P. V. (1992) Asymptotic behavior of the number of faces of random polyhedra and the neighborliness problem. Selecta Math. Soviet., 11, 181-201. Selected translations.
97. Watson, G. A. (1992) Characterization of the subdifferential of some matrix norms. Linear Algebra Appl., 170, 33-45.
98. Weyl, H. (1939) On the volume of tubes. Amer. J. Math., 61, 461-472.
99. 1 minimization: a unified geometric framework. IEEE Trans. Inform. Theory, 57, 6894-6919.
100. Xu, W. & Hassibi, B. (2012) Fundamental thresholds in compressed sensing: a high-dimensional geometry approach. Compressed Sensing. Cambridge: Cambridge University Press, pp. 305-347.
101. Ziegler, G. M. (1995) Lectures on Polytopes. Graduate Texts in Mathematics 152. New York: Springer.