Large covariance estimation by thresholding principal orthogonal complements

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Volumn 75, Issue 4, 2013, Pages 603-680

  Fan, Jianqing ^a   Liao, Yuan ^b   Mincheva, Martina ^a  

^a PRINCETON UNIVERSITY   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

Approximate factor model; Cross sectional correlation; Diverging eigenvalues; High dimensionality; Low rank matrix; Principal components; Sparse matrix; Thresholding; Unknown factors

Indexed keywords

EID: 84881566884     PISSN: 13697412     EISSN: 14679868     Source Type: Journal    
DOI: 10.1111/rssb.12016     Document Type: Article

References (147)

1. Agarwal, A., Negahban, S. and Wainwright, M. J. (2012) Noisy matrix decomposition via convex relaxation: optimal rates in high dimensions. Ann. Statist., 40, 1171-1197.
2. Ahn, S., Lee, Y. and Schmidt, P. (2001) GMM estimation of linear panel data models with time-varying individual effects. J. Econmetr., 101, 219-255.
3. Alessi, L., Barigozzi, M. and Capassoc, M. (2010) Improved penalization for determining the number of factors in approximate factor models. Statist. Probab. Lett., 80, 1806-1813.
4. Amini, A. A. and Wainwright, M. J. (2009) High-dimensional analysis of semidefinite relaxations for sparse principal components. Ann. Statist., 37, 2877-2921.
5. Antoniadis, A. and Fan, J. (2001) Regularized wavelet approximations. J. Am. Statist. Ass., 96, 939-967.
6. d'Aspremont, A., Bach, F. and El Ghaoui, L. (2008) Optimal solutions for sparse principal component analysis. J. Mach. Learn. Res., 9, 1269-1294.
7. Athreya, K. and Lahiri, S. (2006) Measure Theory and Probability Theory. New York: Springer.
8. Bai, J. (2003) Inferential theory for factor models of large dimensions. Econometrica, 71, 135-171.
9. Bai, J. and Ng, S. (2002) Determining the number of factors in approximate factor models. Econometrica, 70, 191-221.
10. Bai, J. and Ng, S. (2008) Large dimensional factor analysis. Found. Trends Econmetr., 3, 89-163.
11. Bai, J. and Shi, S. (2011) Estimating high dimensional covariance matrices and its applications. Ann. Econ. Finan., 12, 199-215.
12. Bickel, P. and Levina, E. (2008) Covariance regularization by thresholding. Ann. Statist., 36, 2577-2604.
13. Birnbaum, A., Johnstone, I., Nadler, B. and Paul, D. (2012) Minimax bounds for sparse PCA with noisy high-dimensional data. Ann. Statist., to be published.
14. Boivin, J. and Ng, S. (2006) Are more data always better for factor analysis? J. Econmetr., 132, 169-194.
15. Cai, J., Candès, E. and Shen, Z. (2008) A singular value thresholding algorithm for matrix completion. SIAM J. Optimizn, 20, 1956-1982.
16. Cai, T. and Liu, W. (2011) Adaptive thresholding for sparse covariance matrix estimation. J. Am. Statist. Ass., 106, 672-684.
17. Cai, T. and Zhou, H. (2012) Optimal rates of convergence for sparse covariance matrix estimation. Ann. Statist., 40, 2389-2420.
18. Candès, E., Li, X., Ma, Y. and Wright, J. (2011) Robust principal component analysis? J. Ass. Comput. Mach., 58, 3.
19. Carvalho, C., Chang, J., Lucas, J., Nevins, J., Wang, Q. and West, M. (2008) High-dimensional sparse factor modeling: applications in gene expression genomics. J. Am. Statist. Ass., 103, 1438-1456.
20. Chamberlain, G. and Rothschild, M. (1983) Arbitrage, factor structure and mean-variance analysis in large asset markets. Econometrica, 51, 1305-1324.
21. Davis, C. and Kahan, W. (1970) The rotation of eigenvectors by a perturbation III. SIAM J. Numer. Anal., 7, 1-46.
22. Doz, C., Giannone, D. and Reichlin, L. (2011) A two-step estimator for large approximate dynamic factor models based on Kalman filtering. J. Econmetr., 164, 188-205.
23. Efron, B. (2007) Correlation and large-scale simultaneous significance testing. J. Am. Statist. Ass., 102, 93-103.
24. Efron, B. (2010) Correlated z-values and the accuracy of large-scale statistical estimates. J. Am. Statist. Ass., 105, 1042-1055.
25. Fama, E. and French, K. (1992) The cross-section of expected stock returns. J. Finan., 47, 427-465.
26. Fan, J., Fan, Y. and Lv, J. (2008) High dimensional covariance matrix estimation using a factor model. J. Econmetr., 147, 186-197.
27. Fan, J., Han, X. and Gu, W. (2012) Control of the false discovery rate under arbitrary covariance dependence (with discussion). J. Am. Statist. Ass., 107, 1019-1048.
28. Fan, J., Liao, Y. and Mincheva, M. (2011a) High dimensional covariance matrix estimation in approximate factor models. Ann. Statist., 39, 3320-3356.
29. Fan, J., Liao, Y. and Mincheva, M. (2011b) Large covariance estimation by thresholding principal orthogonal complements. Preprint arxiv.org/pdf/1201.0175.pdf.
30. Fan, J., Zhang, J. and Yu, K. (2012) Vast portfolio selection with gross-exposure constraints. J. Am. Statist. Ass., 107, 592-606.
31. Forni, M., Hallin, M., Lippi, M. and Reichlin, L. (2000) The generalized dynamic factor model: identification and estimation. Rev. Econ. Statist., 82, 540-554.
32. Forni, M., Hallin, M., Lippi, M. and Reichlin, L. (2004) The generalized dynamic factor model consistency and rates. J. Econmetr., 119, 231-255.
33. Forni, M. and Lippi, M. (2001) The generalized dynamic factor model: representation theory. Econmetr. Theor., 17, 1113-1141.
34. Fryzlewicz, P. (2012) High-dimensional volatility matrix estimation via wavelets and thresholding. Manuscript. London School of Economics and Political Science, London.
35. Hallin, M. and Liška, R. (2007) Determining the number of factors in the general dynamic factor model. J. Am. Statist. Ass., 102, 603-617.
36. Hallin, M. and Liška, R. (2011) Dynamic factors in the presence of blocks. J. Econmetr., 163, 29-41.
37. Hastie, T. J., Tibshirani, R. and Friedman, J. (2009) The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. New York: Springer.
38. James, W. and Stein, C. (1961) Estimation with quadratic loss. In Proc. 4th Berkeley Symp. Mathematical Statistics and Probability, vol., pp. 361-379. Berkeley: University of California Press.
39. Johnstone, I. M. (2001) On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist., 29, 295-327.
40. Johnstone, I. M. and Lu, A. Y. (2009) On consistency and sparsity for principal components analysis in high dimensions. J. Am. Statist. Ass., 104, 682-693.
41. Jung, S. and Marron, J. S. (2009) PCA consistency in high dimension, low sample size context. Ann. Statist., 37, 4104-4130.
42. Kapetanios, G. (2010) A testing procedure for determining the number of factors in approximate factor models with large datasets. J. Bus. Econ. Statist., 28, 397-409.
43. Lam, C. and Fan, J. (2009) Sparsistency and rates of convergence in large covariance matrix estimation. Ann. Statist., 37, 4254-4278.
44. Lawley, D. and Maxwell, A. (1971) Factor Analysis as a Statistical Method, 2nd edn. London: Butterworth.
45. Leek, J. and Storey, J. (2008) A general framework for multiple testing dependence. Proc. Natn. Acad. Sci. USA, 105, 18718-18723.
46. Lin, Z., Ganesh, A., Wright, J., Wu, L., Chen, M. and Ma, Y. (2009) Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix. Int. Wrkshp Computational Advances in Multi-Sensor Adaptive Processing, Aruba.
47. Luo, X. (2011) High dimensional low rank and sparse covariance matrix estimation via convex minimization. Manuscript. University of Pennsylvania, Philadelphia.
48. Ma, Z. (2013) Sparse principal components analysis and iterative thresholding. Ann. Statist., to be published.
49. Meinshausen, N. and Bühlmann, P. (2006) High dimensional graphs and variable selection with the Lasso. Ann. Statist., 34, 1436-1462.
50. Onatski, A. (2010) Determining the number of factors from empirical distribution of eigenvalues. Rev. Econ. Statist., 92, 1004-1016.
51. Pati, D., Bhattacharya, A., Pillai, N. and Dunson, D. (2012) Posterior contraction in sparse Bayesian factor models for massive covariance matrices. Manuscript. Duke University, Durham.
52. Paul, D. (2007) Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sin., 17, 1617-1642.
53. Pesaran, M. H. (2006) Estimation and inference in large heterogeneous panels with a multifactor error structure. Econometrica, 74, 967-1012.
54. Pesaran, M. H. and Yamagata, T. (2012) Testing CAPM with a large number of assets. American Finance Association San Diego Meetings Paper. (Available from http://ssrn.com/abstracts.)
55. Phan, Q. (2012) On the sparsity assumption of the idiosyncratic errors covariance matrix-Support from the FTSE 100 stock returns. Manuscript. University of Warwick, Coventry.
56. Ross, S. A. (1976) The arbitrage theory of capital asset pricing. J. Econ. Theor., 13, 341-360.
57. Rothman, A., Levina, E. and Zhu, J. (2009) Generalized thresholding of large covariance matrices. J. Am. Statist. Ass., 104, 177-186.
58. Sentana, E. (2009) The econometrics of mean-variance efficiency tests: a survey. Econometr. J., 12, 65-101.
59. Sharpe, W. (1964) Capital asset prices: a theory of market equilibrium under conditions of risks. J. Finan., 19, 425-442.
60. Shen, H. and Huang, J. (2008) Sparse principal component analysis via regularized low rank matrix approximation. J. Multiv. Anal., 99, 1015-1034.
61. Stock, J. and Watson, M. (1998) Diffusion Indexes. Working Paper 6702. National Bureau of Economic Research, Cambridge.
62. Stock, J. and Watson, M. (2002) Forecasting using principal components from a large number of predictors. J. Am. Statist. Ass., 97, 1167-1179.
63. Witten, D. M., Tibshirani, R. and Hastie, T. (2009) A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics, 10, 515-534.
64. Wright, J., Peng, Y., Ma, Y., Ganesh, A. and Rao, S. (2009) Robust principal component analysis: exact recovery of corrupted low-rank matrices by convex optimization. New. Inform. Process. Syst.
65. Xiong, H., Goulding, E. H., Carlson, E. J., Tecott, L. H., McCulloch, C. E. and Sen, S. (2011) A flexible estimating equations approach for mapping function-valued traits. Genetics, 189, 305-316.
66. Yap, J. S., Fan, J. and Wu, R. (2009) Nonparametric modeling of longitudinal covariance structure in functional mapping of quantitative trait loci. Biometrics, 65, 1068-1077.
67. Zhang, Y. and El Ghaoui, L. (2011) Large-scale sparse principal component analysis with application to text data. New. Inform. Process. Syst.
68. Abadir, K. M., Distaso, W. and Zikes, F. (2010) Model-free estimation of large variance matrices. Working Paper 10-17. Rimini Centre For Economic Analysis, Rimini.
69. Agarwal, A., Negahban, S. and Wainwright, M. J. (2012) Noisy matrix decomposition via convex relaxation: optimal rates in high dimensions. Ann. Statist., 40, 1171-1197.
70. Alon, U., Barkai, N., Notterman, D., Gish, K., Ybarra, S., Mack, D. and Levine, A. J. (1999) Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc. Natn. Acad. Sci. USA, 96, 6745-6750.
71. Amini, A. A. and Wainwright, M. J. (2009) High-dimensional analysis of semidefinite relaxations for sparse principal components. Ann. Statist., 37, 2877-2921.
72. Antoniadis, A. (2007) Wavelet methods in statistics: some recent developments and their application. Statist. Surv., 1, 16-55.
73. Bai, J. and Li, K. (2012) Statistical analysis of factor models of high dimension. Ann. Statist., 40, 436-465.
74. Bai, J. and Ng, S. (2002) Determining the number of factors in approximate factor models. Econometrica, 70, 191-221.
75. Bai, Z. D. and Yao, J. F. (2008) Central limit theorems for eigenvalues in a spiked population model. Ann. Inst. H. Poincaré, 44, 447-474.
76. Baik, J. and Silverstein, J. W. (2006) Eigenvalues of large sample covariance matrices of spiked population models. J. Multiv. Anal., 97, 1382-1408.
77. Bickel, P. and Levina, E. (2008) Covariance regularization by thresholding. Ann. Statist., 36, 2577-2604.
78. Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009) Simultaneous analysis of Lasso and Dantzig selector. Ann. Statist., 37, 1705-1732.
79. Boehm, H. and von Sachs, R. (2008) Structural shrinkage of nonparametric spectral estimators for multivariate time series. Electron. J. Statist., 2, 696-721.
80. Boehm, H. and von Sachs, R. (2009) Shrinkage estimation in the frequency domain of multivariate time series. J. Multiv. Anal., 100, 913-935.
81. Bouveyron, C., Girard, S. and Schmid, C. (2007) High-dimensional data clustering. Computnl Statist. Data Anal., 52, 502-519.
82. Boyd, S. and Dattorro, J. (2003) Alternating projections. (Available from http://www.stanford.edu/class/ee392o/alt_proj.pdf.)
83. Boyd, S. and Vandenberghe, L. (2004) Convex Optimization. Cambridge: Cambridge University Press.
84. Brillinger, D. R. (1981) Time Series: Data Analysis and Theory. San Francisco: Holden-Day.
85. Byers, S. and Raftery, A. E. (1998) Nearest-neighbor clutter removal for estimating features in spatial point processes. J. Am. Statist. Ass., 93, 577-584.
86. Cai, T. and Liu, W. (2011) Adaptive thresholding for sparse covariance matrix estimation. J. Am. Statist. Ass., 106, 672-684.
87. Carvalho, C., Chang, J., Lucas, J., Nevins, J., Wang, Q. and West, M. (2008) High-dimensional sparse factor modeling: applications in gene expression genomics. J. Am. Statist. Ass., 103, 1438-1456.
88. Chamberlain, G. and Rothschild, M. (1983) Arbitrage, factor structure and mean-variance analysis in large asset markets. Econometrica, 51, 1305-1324.
89. Chen, B. B. and Pan, G. M. (2012) Convergence of the largest eigenvalue of normalized sample covariance matrices when p and n both tend to infinity with their ratio converging to zero. Bernoulli, 18, 1405-1420.
90. Chudik, A., Pesaran, M. H. and Tosetti, E. (2011) Weak and strong cross-section dependence and estimation of large panels. Econmetr. J., 14, 45-90.
91. Collins, L. B. and Cressie, N. (2001) Analysis of spatial point patterns using bundles product LISA function. J. Agric. Biol. Environ. Statist., 6, 118-135.
92. Eaton, J. W. (2002) GNU Octave Manual. Bristol: Network Theory Limited.
93. Engelen, S., Hubert, M. and Vanden Branden, K. (2005) A comparison of three procedures for Robust PCA in high dimensions. Austn J. Statist., 34, 117-126.
94. Fama, E. and French, K. R. (1993) Common risk factors in the returns on stocks and bonds. J. Finan. Econ., 33, 3-56.
95. Fan, J., Fan, Y. and Lv, J. (2008) High dimensional covariance matrix estimation using a factor model. J. Econmetr., 147, 186-197.
96. Fan, J. and Han, X. (2013) Estimation of false discovery proportion with unknown dependence. Manuscript.
97. Fan, J., Liao, Y. and Mincheva, M. (2011) High dimensional covariance matrix estimation in approximate factor models. Ann. Statist., 39, 3320-3356.
98. Fan, J. and Lv, J. (2008) Sure independence screening for ultrahigh dimensional feature space (with discussion). J. R. Statist. Soc. B, 70, 849-911.
99. Fan, J., Zhang, J. and Yu, K. (2012) Vast portfolio selection with gross-exposure constraints. J. Am. Statist. Ass., 107, 592-606.
100. Farnè, M. and Montanari, A. (2013) Different estimators of the spectral matrix: an empirical comparison; testing a new shrinkage estimator. Communs Statist. Theor. Meth., to be published.
101. Forni, M., Hallin, M., Lippi, M. and Reichlin, L. (2000) The generalized dynamic factor model: identification and estimation. Rev. Econ. Statist., 82, 540-554.
102. Forni, M., Hallin, M., Lippi, M. and Reichlin, L. (2004) The generalized dynamic factor model consistency and rates. J. Econmetr., 119, 231-255.
103. Forni M. and Lippi, M. (2001) The generalized dynamic factor model: representation theory. Econmetr. Theor., 17, 1113-1341.
104. Golub, T. R., Slonim, D. K., Tamayo, P., Huard, C., Gassenbeek, M., Mesirov, J. P., Coller, H., Loh, M., Downing, J. R., Caligiuri, M. A., Bloomfield, C. D. and Lander, E. S. (1999) Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science, 286, 531-537.
105. Hall, P., Marron, J. S. and Neeman, A. (2005) Geometric representation of high dimension, low sample size data. J. R. Statist. Soc. B, 67, 427-444.
106. Huang, H., Liu, Y., Yuan, M. and Marron, J. S. (2013) Statistical significance of clustering using soft thresholding. Preprint arXiv:1305.5879.
107. Jagannathan, R. and Ma, T. (2003) Risk reduction in large portfolios: why imposing the wrong constraints helps. J. Finan., 58, 1651-1683.
108. Johnstone, I. M. and Lu, A. Y. (2009) On consistency and sparsity for principal components analysis in high dimensions. J. Am. Statist. Ass., 104, 682-693.
109. Jolliffe, I. T. (2002) Principal Component Analysis, 2nd edn. New York: Springer.
110. Jung, S. and Marron, J. S. (2009) PCA consistency in high dimension, low sample size context. Ann. Statist., 37, no. 6B, 4104-4130.
111. Jung, S., Sen A. and Marron, J. (2012) Boundary behavior in High Dimension, Low Sample Size asymptotics of PCA. J. Multiv. Anal., 109, 196-203.
112. Kaufman, C. G., Bingham, D., Habib, S., Heitmann, K. and Frieman, J. A. (2011) Efficient emulators of computer experiments using compactly supported correlation functions with an application to cosmology. Ann. Appl. Statist., 5, 2470-2492.
113. Kaufman, C. G., Schervish, M. J. and Nychka, D. W. (2008) Covariance tapering for likelihood-based estimation in large spatial data sets. J. Am. Statist. Ass., 103, 1545-1555.
114. Ke, T., Jin, J. and Fan, J. (2012) Covariance assisted screening and estimation. Preprint arXiv:1205.4645v2.
115. Kourtis, A., Dotsis, G. and Markellos, R. (2012) Parameter uncertainty in portfolio selection: shrinking the inverse covariance matrix. J. Bankng Finan., 36, 2522-2531.
116. Lam, C. and Yao, Q. (2012) Factor modeling for high-dimensional time series: inference for the number of factors. Ann. Statist., 40, 694-726.
117. Lam, C., Yao, Q. and Bathia, K. (2011) Estimation of latent factors for high-dimensional time series. Biometrika, 98, 901-918.
118. Lansangan, J. R. G. and Barrios, E. B. (2009) Principal components analysis of nonstationary time series data. Statist. Comput., 19, 173-187.
119. Lawley, D. and Maxwell, A. (1971) Factor Analysis as a Statistical Method, 2nd edn. London: Butterworth.
120. Ledoit, O. and Wolf, M. (2003) Improved estimation of the covariance matrix of stock returns with an application to portfolio selection. J. Empir. Finan., 10, 603-621.
121. Ledoit, O. and Wolf, M. (2004) A well-conditioned estimator for large-dimensional covariance matrices. J. Multiv. Anal., 88, 365-411.
122. Liu, H., Han, F., Yuan, M., Lafferty, J. and Wasserman, L. (2012) High dimensional semiparametric gaussian copula graphical models. Ann. Statist., 40, 2293-2326.
123. Liu, Y., Hayes, D. N., Nobel, A. and Marron, J. S. (2008) Statistical significance of clustering for high-dimension, low-sample size data. J. Am. Statist. Ass., 103, 1281-1293.
124. Liu, H. and Wang, L. (2012) Tiger: a tuning-insensitive approach for optimally estimating gaussian graphical models. Princeton University, Princeton. (Available from http://arxiv.org/abs/1209.2437.)
125. Markowitz, H. (1952) Portfolio selection. J. Finan., 7, 77-91.
126. Onatski, A. (2009) Testing hypotheses about the number of factors in large factor models. Econometrica, 77, 1447-1479.
127. Onatski, A. (2010) Determining the number of factors from empirical distribution of eigenvalues. Rev. Econ. Statist., 92, 1004-1016.
128. Pati, D., Bhattacharaya, A., Pillai, N. and Dunson, D. (2012) Posterior contraction in sparse Bayesian factor models for massive covariance matrices. Manuscript. Duke University, Durham.
129. Paul, D. (2007) Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sin., 17, 1617-1642.
130. Pesaran, M. H. and Yamagata, T. (2012) Testing CAPM with a large number of assets. American Finance Association San Diego Meetings Paper. (Available from http://ssrn.com/abstracts.)
131. Pourahmadi, M. (2013) High-dimensional Covariance Estimation. New York: Wiley.
132. Schaefer, J. and Strimmer, K. (2005) A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Statist. Applic. Genet. Molec. Biol., 4, article 32.
133. Shen, D., Shen, H. and Marron, J. (2012) A general framework for consistency of principal component analysis. Preprint arXiv:1211.2671.
134. Shen, D., Shen, H. and Marron, J. S. (2013) Consistency of sparse PCA in high dimension, low sample size contexts. J. Multiv. Anal., 115, 317-333.
135. Stock, J. H. and Watson, M. W. (2002a) Macroeconomic forecasting using diffusion indexes. J. Bus. Econ. Statist., 20, 147-162.
136. Stock, J. H. and Watson, M. W. (2002b) Forecasting using principal components from a large number of predictors. J. Am. Statist. Ass., 97, 1167-1179.
137. Stock, J. H. and Watson, M. W. (2005) Implications of dynamic factor models for VAR analysis. Working Paper 11467. National Bureau of Economic Research, Cambridge.
138. Tang, C. Y. and Fan, Y. (2013) Precision matrix estimation by inverse principle orthogonal decomposition. Manuscript.
139. Tyler, D. E., Critchley, F., Dümbgen, L. and Oja, H. (2009) Invariant co-ordinate selection (with discussion). J. R. Statist. Soc. B, 71, 549-592.
140. Wang, H. (2012) Factor profiled sure independence screening. Biometrika, 99, 15-28.
141. Won, J.-H., Lim, J., Kim, S.-J. and Rajaratnam, B. (2013) Condition-number-regularized covariance estimation. J. R. Statist. Soc. B, 75, 427-450.
142. Xue, L., Ma, S. and Zou, H. (2012) Positive definite l1 penalized estimation of large covariance matrices. J. Am. Statist. Ass., 107, 1480-1491.
143. Yen, Y. M. (2011) Sparse weighted norm minimum variance portfolio. Preprint.
144. Zehetmayer, S., Bauer, P. and Posch, M. (2005) Two-stage designs for experiments with a large number of hypotheses. Bioinformatics, 21, 3771-3777.
145. Zehetmayer, S., Bauer, P. and Posch, M. (2008) Optimized multi-stage designs controlling the false discovery or the family-wise error rate. Statist. Med., 27, 4145-4160.
146. Zhang, J. (2012) Source screening for neuroimaging. Technical Report. School of Mathematics, Statistics and Actuarial Science, University of Kent, Canterbury.
147. Zhang, J., Liu, C. and Green, G. (2012) Source localization with MEG data. Technical Report. School of Mathematics, Statistics and Actuarial Science, University of Kent, Canterbury.