On measuring and correcting the effects of data mining and model selection

Journal of the American Statistical Association

Volumn 93, Issue 441, 1998, Pages 120-131

  Ye, Jianming ^a  

^a UNIVERSITY OF CHICAGO   (United States)

Author keywords

Data mining; Degrees of freedom; Effect of model selection; Goodness of fit; Half normal plot; Nonparametric regression; Sensitivity

Indexed keywords

EID: 0032351389     PISSN: 01621459     EISSN: 1537274X     Source Type: Journal    
DOI: 10.1080/01621459.1998.10474094     Document Type: Article

References (28)

1. Akaike, H. (1973), "Information Theory and the Maximum Likelihood Principle," in International Symposium on Information Theory, eds. V. Petrov and F. Csâki, Budapest: Akademiai Kiàdo.
2. Bickel, P. and Zhang, P. (1992), "Variable Selection in Nonparametric Regression with Categorical Covariates." Journal of the American Statistical Asociation, 87, 90-97.
3. Brciman, L. (1992), "The Little Bootstrap and Other Methods for Dimensionality Selection in Regression: X-Fixed Prediction Error," Journal of the American Statistical Association, 87, 738-754.
4. Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J. (1984), Classification and Regression Trees, Belmont, CA: Wadsworth.
5. Chipman, H., George, E. I., and McCullogh, R. E. (1996). "Bayesian CART," Technical Report, University of Chicago, Graduate School of Business.
6. Clark, L. A., and Pregibon, D. (1992), 'Tree-Based Models," in Statistical Models in S, eds. J. M. Chambers, and T. J. Hastie, New York: Chapman and Hall.
7. Craven, P., and Wahba, G. (1979), "Smoothing Noisy Data With Spline Functions," Numerical Mathematics, 31, 377-403.
8. Donoho, D. L., and Johnstone, I. M. (1994), "Ideal Spatial Adaptation by Wavelet Shrinkage," Biometrika, 81, 425-456.
9. Draper, D. (1995), "Assessment and Propagation of Model Uncertainty" (with discussion). Journal of the Royal Statistical Society, Ser. B, 57, 45-98.
10. Efron, B. (1983), "Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation," Journal of the American Statistical Association, 78, 316-331.
11. Efron, B. (1986), "How Biased is the Apparent Error Rate of a Prediction Rule?" Journal of the Americati Statistical Association, 81, 461-470.
12. Eubank, R. L., and Spiegelman, C. H. (1990), "Testing the Goodness of Fit of a Linear Model via Nonparametric Regression Techniques," Journal of the American Statistical Association, 85, 387-392.
13. Faraway, J. J. (1992), "On the Cost of Data Analysis," Journal of Computational and Graphical Statistics, 1, 213-229.
14. Friedman, J. H. (1991), "Multivariate Adaptive Regression Splines," (with discussion), The Annals of Statistics, 19, 1-67.
15. George, E. I., and Foster, D. P. (1994), 'The Risk Inflation Criterion for Multiple Regression," The Annals of Statistics, 22, 1947-1975.
16. Gu, C. (1996), "Model Indexing and Smoothing Parameter Selection in Nonparametric Function Estimation," Technical Report, Purdue University, Dept, of Statistics.
17. Hastie, T. J., and Tibshirani, R. J. (1990), Generalized Additive Models, London: Chapman and Hall.
18. Mallows, C. L. (1973), "Some Comments on Cp," Technics, 15, 661-675.
19. Miller, A. J. (1990), Subset Selection in Regression, London: Chapman and Hall.
20. Owen, A. (1991), "Comment on 'Multivariate Adaptive Regression Splines'," by J. H. Friedman, The Annals of Statistics, 19, 102-112.
21. Schwarz, G. (1978), "Estimating the Dimension of a Model," The Annals of Statistics, 6, 461-464.
22. Staniswalis, J. G., and Severini, T. A. (1991), "Diagnostics for Assessing Regression Models," Journal of the American Statistical Association, 86. 684-692.
23. Venables, W. N., and Ripley, B. D. (1994), Modem Applied Statistics with S-Plus, New York: Springer-Verlag.
24. Wahba, G. (1983), "Bayesian 'Confidence Intervals' for the Cross-Validated Smoothing Spline," Journal of Royal Statistical Society, Ser. B, 45, 133-150.
25. Zahn, D. (1975a), "Modifications of and Revised Critical Values for the Half-Normal Plot," Technics, 17, 189-200.
26. Zahn, D. (1975b), "An Empirical Study of the Half-Normal Plot" Technics, 17,201-211.
27. Zhang, P. (1991). "Variable Selection in Nonparametric Regression With Continuous Covariates," The Annals of Statistics, 19, 1869-1882.
28. Zhang, P. (1992), "On the Distribution Properties of Model Selection Criteria," Journal of the American Statistical Association, 87, 732-737.