Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach

Annual Review of Economics

Volumn 7, Issue 1, 2015, Pages 649-688

  Chernozhukov, Victor ^a   Hansen, Christian ^b   Spindler, Martin ^c  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b UNIVERSITY OF CHICAGO   (United States)

^c MAX PLANCK INSTITUTE FOR SOCIAL LAW AND SOCIAL POLICY   (Germany)

Author keywords

C( ) statistics; efficiency; Neyman; optimal instrument; optimal moment; optimal score; optimality; orthogonalization

Indexed keywords

EID: 84938561538     PISSN: 19411383     EISSN: 19411391     Source Type: Journal    
DOI: 10.1146/annurev-economics-012315-015826     Document Type: Article

References (65)

1. Belloni A, Chen D, Chernozhukov V, Hansen C. 2012. Sparse models and methods for optimal instruments with an application to eminent domain. Econometrica 80:2369-429
2. Belloni A, Chernozhukov V. 2011. High-dimensional sparse econometric models: an introduction. In Inverse Problems and High-Dimensional Estimation: Stats in the Château Summer School, August 31-September 2, 2009, ed. P Alquier, E Gautier, G Stoltz, pp. 121-56. New York: Springer
3. Belloni A, Chernozhukov V. 2013. Least squares after model selection in high-dimensional sparse models. Bernoulli 19:521-47
4. Belloni A, Chernozhukov V, Fernández-Val I, Hansen C. 2013a. Program evaluation with high-dimensional data. ArXiv:1311. 2645 [math. ST]
5. Belloni A, Chernozhukov V, Hansen C. 2010. LASSO methods for Gaussian instrumental variables models. ArXiv:1012. 1297 [stat. ME]
6. Belloni A, Chernozhukov V, Hansen C. 2013b. Inference for high-dimensional sparse econometric models. In Advances in Economics and Econometrics: 10th World Congress, Vol. 3: Econometrics, ed. DAcemoglu, M Arellano, E Dekel, pp. 245-95. Cambridge, UK: Cambridge Univ. Press
7. Belloni A, Chernozhukov V, Hansen C. 2014a. Inference on treatment effects after selection amongst highdimensional controls. Rev. Econ. Stud. 81:608-50
8. Belloni A, Chernozhukov V, Hansen C, Kozbur D. 2014b. Inference in high dimensional panel models with an application to gun control. ArXiv:1411. 6507 [stat. ME]
9. Belloni A, Chernozhukov V, Kato K. 2013c. Robust inference in approximately sparse quantile regression models (with an application to malnutrition). ArXiv:1312. 7186 [math. ST]
10. Belloni A, Chernozhukov V, Kato K. 2013d. Uniform post selection inference for LAD regression models and other Z-estimation problems. ArXiv:1304. 0282 [math. ST]
11. Belloni A, Chernozhukov V, Wang L. 2011. Square-root-LASSO: pivotal recovery of sparse signals via conic programming. Biometrika 98:791-806
12. Belloni A, Chernozhukov V, Wei Y. 2013e. Honest confidence regions for logistic regression with a large number of controls. ArXiv:1304. 3969 [stat. ME]
13. Berk R, Brown L, Buja A, Zhang K, Zhao L. 2013. Valid post-selection inference. Ann. Stat. 41:802-37
14. Berry S, Levinsohn J, Pakes A. 1995. Automobile prices in market equilibrium. Econometrica 63:841-90
15. Bickel PJ. 1982. On adaptive estimation. Ann. Statist. 10:647-71
16. Bickel PJ, Ritov Y, Tsybakov AB. 2009. Simultaneous analysis of Lasso and Dantzig selector. Ann. Stat. 37:1705-32
17. Bühlmann P, van de Geer S. 2011. Statistics for High-Dimensional Data: Methods, Theory and Applications. New York: Springer
18. Candès E, Tao T. 2007. The Dantzig selector: statistical estimation when p is much larger than n. Ann. Stat. 35:2313-51
19. Carrasco M. 2012. A regularization approach to the many instruments problem. J. Econom. 170:383-98
20. Carrasco M, Tchuente G. 2015. Regularized LIML with many instruments. J. Econom. 186:427-42
21. Chamberlain G. 1987. Asymptotic efficiency in estimation with conditional moment restrictions. J. Econom. 34:305-34
22. Chamberlain G, Imbens G. 2004. Random effects estimators with many instrumental variables. Econometrica 72:295-306
23. Chao JC, Swanson NR, Hausman JA, Newey WK, Woutersen T. 2012. Asymptotic distribution of JIVE in a heteroskedastic IV regression with many instruments. Econom. Theory 28:42-86
24. Chen LHY, Fang X. 2011. Multivariate normal approximation by Stein's method: the concentration inequality approach. ArXiv:1111. 4073 [math. PR]
25. Chen X, Linton O, Keilegom IV. 2003. Estimation of semiparametric models when the criterion function is not smooth. Econometrica 71:1591-608
26. Chernozhukov V, Chetverikov D, Kato K. 2013. Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors. Ann. Stat. 41:2786-819
27. Chernozhukov V, Liu H, Lu J, Ning Y. 2014. Statistical inference in high-dimensional sparse models using generalized method of moments. Unpublished manuscript, Mass. Inst. Technol., Cambridge, MA, Princeton Univ., Princeton, NJ
28. Dudley RM. 2002. Real Analysis and Probability. Cambridge, UK: Cambridge Univ. Press
29. Fan J, Li R. 2001. Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96:1348-60
30. Fan J, Lv J. 2010. Aselective overviewof variable selection in high dimensional feature space. Stat. Sin. 20:101-48
31. Farrell MH. 2014. Robust inference on average treatment effects with possibly more covariates than observations. ArXiv:1309. 4686 [math. ST]
32. Fithian W, Sun D, Taylor J. 2014. Optimal inference after model selection. ArXiv:1410. 2597v1 [math. ST]
33. Frank IE, Friedman JH. 1993. A statistical view of some chemometrics regression tools. Technometrics 35:109-35
34. Gautier E, Tsybakov AB. 2011. High-dimensional instrumental variables regression and confidence sets. ArXiv:1105. 2454v4 [math. ST]
35. Gillen BJ, Shum M, Moon HR. 2014. Demand estimation with high-dimensional product charateristics. Adv. Econom. 34:301-23
36. G'Sell MG, Taylor J, Tibshirani R. 2013. Adaptive testing for the graphical lasso. ArXiv:1307. 4765 [math. ST]
37. Hansen C, Kozbur D. 2014. Instrumental variables estimation with many weak instruments using regularized JIVE. J. Econom. 182:290-308
38. Hastie T, Tibshirani R, Friedman J. 2009. Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer
39. Huber PJ. 1964. The behavior of maximum likelihood estimates under nonstandard conditions. Proc. 5th Berkeley Symp., ed. J Neyman, pp. 221-23. Berkeley: Univ. Calif. Press
40. Javanmard A, Montanari A. 2014. Confidence intervals and hypothesis testing for high-dimensional regression. ArXiv:1306. 3171v2 [stat. ME]
41. Jing B-Y, Shao Q-M,Wang Q. 2003. Self-normalized Cramer-type large deviations for independent random variables. Ann. Probab. 31:2167-215
42. Kozbur D. 2014. Inference in nonparametric models with a high-dimensional component. Work. Pap., ETH Zürich
43. Lee JD, Sun DL, Sun Y, Taylor JE. 2013. Exact post-selection inference, with application to the lasso. ArXiv: 1311. 6238 [math. ST]
44. Lee JD, Taylor JE. 2014. Exact post model selection inference for marginal screening. ArXiv:1402. 5596 [stat. ME]
45. Leeb H, Pötscher BM. 2008a. Recent developments in model selection and related areas. Econom. Theory 24:319-22
46. Leeb H, Pötscher BM. 2008b. Sparse estimators and the oracle property, or the return of Hodges' estimator. J. Econom. 142:201-11
47. Lockhart R, Taylor JE, Tibshirani RJ, Tibshirani R. 2014. A significance test for the lasso. Ann. Stat. 42:413-68
48. Loftus JR, Taylor JE. 2014. A significance test for forward stepwise model selection. ArXiv:1405. 3920 [stat. ME]
49. Meinshausen N, Yu B. 2009. Lasso-type recovery of sparse representations for high-dimensional data. Ann. Stat. 37:2246-70
50. Neyman J. 1959. Optimal asymptotic tests of composite statistical hypotheses. In Probability and Statistics: The Harald Cramer Volume, ed. U Grenander, pp. 213-34. New York: Wiley
51. Neyman J. 1979. Ca tests and their use. Sankhya 41:1-21
52. Ning Y, Liu H. 2014. SPARC: optimal estimation and asymptotic inference under semiparametric sparsity. ArXiv:1412. 2295 [stat. ML]
53. Okui R. 2011. Instrumental variable estimation in the presence of many moment conditions. J. Econom. 165:70-86
54. Pakes A, Pollard D. 1989. Simulation and asymptotics of optimization estimators. Econometrica 57:1027-57
55. Robins JM, Rotnitzky A. 1995. Semiparametric efficiency in multivariate regression models with missing data. J. Am. Stat. Assoc. 90:122-29
56. Rudelson M, Vershynin R. 2008. On sparse reconstruction from Fourier and Gaussian measurements. Commun. Pure Appl. Math. 61:1025-45
57. Rudelson M, Zhou S. 2011. Reconstruction from anisotropic random measurements. ArXiv:1106. 1151 [math. ST]
58. Taylor J, Lockhart R, Tibshirani RJ, Tibshirani R. 2014. Exact post-selection inference for forward stepwise and least angle regression. ArXiv:1401. 3889 [stat. ME]
59. Tibshirani R. 1996. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 58:267-88
60. van de Geer S, Bühlmann P, Ritov Y, Dezeure R. 2014. On asymptotically optimal confidence regions and tests for high-dimensional models. Ann. Stat. 42:1166-202
61. van de Geer S, Nickl R. 2013. Confidence sets in sparse regression. Ann. Stat. 41:2852-76
62. van der Vaart AW. 1998. Asymptotic Statistics. Cambridge, UK: Cambridge Univ. Press
63. Voorman A, Shojaie A, Witten D. 2014. Inference in high dimensions with the penalized score test. ArXiv: 1401. 2678 [stat. ME]
64. Yang Z, Ning Y, Liu H. 2014. On semiparametric exponential family graphical models. ArXiv:1412. 8697 [stat. ML]
65. Zhang C-H, Zhang SS. 2014. Confidence intervals for low dimensional parameters in high dimensional linear models. J. R. Stat. Soc. B 76:217-42