High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: Risk underestimation

Annals of Statistics

Volumn 38, Issue 6, 2010, Pages 3487-3566

  El Karoui, Noureddine ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Concentration of measure; Convex optimization; Covariance matrices; Elliptical distributions; High dimensional inference; Markowitz problem; Multivariate statistical analysis; Quadratic programs; Random matrix theory; Wishart matrices

Indexed keywords

EID: 78650151035     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/10-AOS795     Document Type: Article

References (43)

1. ANDERSON, T. W. (2003). An Introduction toMultivariate Statistical Analysis, 3rd ed.Wiley, Hoboken, NJ. MR1990662
2. BAI, Z. D. (1999). Methodologies in spectral analysis of large-dimensional random matrices, a review. Statist. Sinica 9 611-677. With comments by G. J. Rodgers and Jack W. Silverstein; and a rejoinder by the author. MR1711663
3. BAI, Z., LIU, H. and WONG, W.-K. (2009). Enhancement of the applicability of Markowitz's portfolio optimization by utilizing random matrix theory. Math. Finance 19 639-667. MR2583523
4. BICKEL, P. J. and LEVINA, E. (2008a). Covariance regularization by thresholding. Ann. Statist. 36 2577-2604. MR2485008
5. BICKEL, P. J. and LEVINA, E. (2008b). Regularized estimation of large covariance matrices. Ann. Statist. 36 199-227. MR2387969
6. BIROLI, G., BOUCHAUD, J.-P. and POTTERS, M. (2007). The student ensemble of correlation matrices: Eigenvalue spectrum and Kullback-Leibler entropy. Acta Phys. Polon. B 38 4009-4026. MR2372771
7. BLACK, F. and LITTERMAN, R. (1990). Asset allocation: Combining investor views with market equilibrium. Golman Sachs Fixed Income Research.
8. BOYD, S. and VANDENBERGHE, L. (2004). Convex Optimization. Cambridge Univ. Press, Cambridge. MR2061575
9. CAMPBELL, J., LO, A. and MACKINLAY, C. (1996). The Econometrics of Financial Markets. Princeton Univ. Press, Princeton, NJ.
10. CHIKUSE, Y. (2003). Statistics on Special Manifolds. Lecture Notes in Statistics 174. Springer, New York. MR1960435
11. CHOW, Y. S. and TEICHER, H. (1997). Probability Theory: Independence, Interchangeability, Martingales, 3rd ed. Springer, New York. MR1476912
12. DAVIDSON, K. R. and SZAREK, S. J. (2001). Local operator theory, random matrices and Banach spaces. In Handbook of the Geometry of Banach Spaces, Vol. I 317-366. North-Holland, Amsterdam. MR1863696
13. EATON, M. L. (1983). Multivariate Statistics: A Vector Space Approach. Wiley, New York. MR0716321
14. EL KAROUI, N. (2007). Tracy-Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices. Ann. Probab. 35 663-714. MR2308592
15. EL KAROUI, N. (2008). Operator norm consistent estimation of large dimensional sparse covariance matrices. Ann. Statist. 36 2717-2756. MR2485011
16. EL KAROUI, N. (2009a). Concentration of measure and spectra of random matrices: Applications to correlation matrices, elliptical distributions and beyond. Ann. Appl. Probab. 19 2362-2405. MR2588248
17. EL KAROUI, N. (2009b). On the realized risk of high-dimensional Markowitz portfolios. Technical Report No. 784, Dept. Statistics, Univ. California, Berkeley.
18. FANG, K. T., KOTZ, S. and NG, K. W. (1990). Symmetric Multivariate and Related Distributions. Monographs on Statistics and Applied Probability 36. Chapman and Hall, London. MR1071174
19. FRAHM, G. and JAEKEL, U. (2005). Random matrix theory and robust covariance matrix estimation for financial data. Available at arXiv:physics/0503007.
20. HORN, R. A. and JOHNSON, C. R. (1990). Matrix Analysis. Cambridge Univ. Press, Cambridge. Corrected reprint of the 1985 original. MR1084815
21. HORN, R. A. and JOHNSON, C. R. (1994). Topics in Matrix Analysis. Cambridge Univ. Press, Cambridge. Corrected reprint of the 1991 original. MR1288752
22. JOBSON, J. D. and KORKIE, B. (1980). Estimation for Markowitz efficient portfolios. J. Amer. Statist. Assoc. 75 544-554. MR0590686
23. JOHNSTONE, I. (2001). On the distribution of the largest eigenvalue in principal component analysis. Ann. Statist. 29 295-327. MR1863961
24. KAN, R. and SMITH, D. R. (2008). The distribution of the sample minimum-variance frontier. Management Science 54 1364-1380.
25. LAI, T. L. and XING, H. (2008). Statistical Models and Methods for Financial Markets. Springer Texts in Statistics. Springer, New York. MR2434025
26. LALOUX, L., CIZEAU, P., BOUCHAUD, J.-P. and POTTERS, M. (2000). Random matrix theory and financial correlations. Internat. J. Theoret. Appl. Finance 3 391-397.
27. LEDOIT, O. and WOLF, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 365-411. MR2026339
28. LEDOUX, M. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Amer. Math. Soc., Providence, RI. MR1849347
29. LUGOSI, G. (2006). Concentration of measure inequalities. Lecture notes available online.
30. MARČENKO, V. A. and PASTUR, L. A. (1967). Distribution of eigenvalues in certain sets of random matrices. Mat. Sb. (N.S.) 72 507-536. MR0208649
31. MARDIA, K. V., KENT, J. T. and BIBBY, J. M. (1979). Multivariate Analysis. Academic Press, London. MR0560319
32. MARKOWITZ, H. (1952). Portfolio selection. J. Finance 7 77-91.
33. MCNEIL, A. J., FREY, R. and EMBRECHTS, P. (2005). Quantitative Risk Management: Concepts, Techniques and Tools. Princeton Univ. Press, Princeton, NJ. MR2175089
34. MEUCCI, A. (2005). Risk and Asset Allocation. Springer Finance. Springer, Berlin. MR2155219
35. MEUCCI, A. (2008). Enhancing the Black-Litterman and related approaches: Views and stress-test on risk factors. Available at SSRN, http://ssrn.com/ abstract=1213323.
36. MICHAUD, R. O. (1998). Efficient Asset Management: A Practical Guide to Stock Portfolio Optimization and Asset Allocation. Oxford Univ. Press.
37. PAFKA, S. and KONDOR, I. (2003). Noisy covariance matrices and portfolio optimization. II. Phys. A 319 487-494. MR1965597
38. ROTHMAN, A. J., BICKEL, P. J., LEVINA, E. and ZHU, J. (2008). Sparse permutation invariant covariance estimation. Electron. J. Statist. 2 494-515 (electronic). MR2417391
39. RUPPERT, D. (2006). Statistics and Finance: An Introduction. Springer, New York. Corrected second printing of the 2004 original. MR2060918
40. SILVERSTEIN, J. W. (1995). Strong convergence of the empirical distribution of eigenvalues of largedimensional random matrices. J. Multivariate Anal. 55 331-339. MR1370408
41. TYLER, D. E. (1987). A distribution-free M-estimator of multivariate scatter. Ann. Statist. 15 234-251. MR0885734
42. VAN DER VAART, A. W. (1998). Asymptotic Statistics. Cambridge Univ. Press, Cambridge. MR1652247
43. WACHTER, K. W. (1978). The strong limits of random matrix spectra for sample matrices of independent elements. Ann. Probab. 6 1-18. MR0467894