Reducing variability of crossvalidation for smoothing-parameter choice

Biometrika

Volumn 96, Issue 1, 2009, Pages 175-186

  Hall, Peter ^a   Robinson, Andrew P ^a  

^a UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

Bagging; Bandwidth; Bootstrap aggregation; Half sampling; Kernel estimation; Nonparametric density estimation; Nonparametric regression; Statistical smoothing; Subsampling

Indexed keywords

EID: 60449092039     PISSN: 00063444     EISSN: 14643510     Source Type: Journal    
DOI: 10.1093/biomet/asn068     Document Type: Article

References (46)

1. BORRA, S. & DI CIACCIO, A. (2001). Performance evaluation of bagging and boosting in nonparametric regression. Metron 59, 141-56.
2. BORRA, S. & DI CIACCIO, A. (2002). Improving nonparametric regression methods by bagging and boosting. Comput. Statist. Data Anal. 38, 407-20.
3. BOWMAN, A. W. (1984). An alternative method of cross-validation for the smoothing of density estimates. Biometrika 71, 353-60.
4. BREIMAN, L. (1996). Bagging predictors. Mach. Learn. 24, 123-40.
5. BÜHLMANN, P. (2004). Bagging, boosting and ensemble methods. In Handbook of Computational Statistics, Ed. J. E. Gentle, W. Hardle and Y. Mori, pp. 877-907. Berlin: Springer.
6. BÜHLMANN, P. & YU, B. (2002). Analyzing bagging. Ann. Statist. 30, 927-61.
7. BUJA, A. & STUETZLE, W (2006). Observations on bagging. Statist. Sinica 16, 323-51.
8. CAO, R., CUEVAS, A. & GONZÁLEZ- MANTEIGA, W. (1994). A comparative study of several smoothing methods in density estimation. Comput. Statist. Data Anal. 17, 153-76.
9. CHEN, S. X. & HALL, P. (2003). Effects of bagging and bias correction on estimators defined by estimating equations. Statist. Sinica 13, 97-109.
10. CHIU, S. T. (1991). Bandwidth selection for kernel density estimation. Ann. Statist. 19, 1883-905.
11. CRAVEN, P. & WAHBA, G. (1978). Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized cross-validation. Numer. Math. 31, 377-403.
12. DELAIGLE, A. & HALL, P. (2008). Using SIMEX for smoothing-parameter choice in errors-in-variables problems. J. Am. Statist. Assoc. 103, 280-7.
13. DEY, A. K., RUYMGAART, F. H. & MAIR, B. A. (1996). Cross-validation, for parameter selection in inverse estimation problems. Scand. J. Statist. 23, 609-20.
14. FAN, J. (1993). Local linear regression estimators and their minimax efficiencies. Ann. Statist. 21, 196-216.
15. FARAWAY, J. J. & JHUN, M. (1990). Bootstrap choice of bandwidth for density estimation. J. Am. Statist. Assoc. 85, 1119-22.
16. FELUCH, W. & KORONACKI, J. (1992). A note on modified cross-validation in density estimation. Comp. Statist. Data Anal. 13, 143-51.
17. FRIEDMAN, J. H. & HALL, P. (2007). On bagging and nonlinear estimation. J. Statist. Plan. Infer. 137, 669-83.
18. HALL, P. (1984). Asymptotic properties of integrated square error and cross-validation for kernel estimation of a regression function. Z. Wahr. verw. Geb. 67, 175-96.
19. HALL, P. & JOHNSTONE, I. M. (1992). Empirical functional and efficient smoothing parameter selection. J. R. Statist. Soc. B 54, 475-530.
20. HALL, P. & MARRON, J. S. (1987). Extent to which least-squares cross-validation minimises integrated square error in nonparametric density estimation. Prob. Theory Rel. Fields 74, 567-81.
21. HALL, P., MARRON, J. S. & PARK, B. U. (1992). Smoothed cross-validation. Prob. Theory Rel. Fields 92, 1-20.
22. HALL, P. & SAMWORTH, R. J. (2005). Properties of bagged nearest neighbour classifiers. J. R. Statist. Soc. B 67, 363-79.
23. HALL, P., SHEATHER, J., JONES, M. C & MARRON, J. S. (1991). On optimal data-based bandwidth selection in kernel density estimation. Biometrika 78, 263-9.
24. HÄR.DLE, W., HALL, P. & MARRON, J. S. (1988). How far are automatically chosen regression smoothing parameters from their optimum? J. Am. Statist. Assoc. 83, 86-101.
25. HÄRDLE, W., HALL, P. & MARRON, J. S. (1992). Regression smoothing parameters that are not far from their optimum. J. Am. Statist. Assoc. 87, 227-33.
26. HESSE, C. H. (1999). Data-driven deconvolution. Nonparam. Statist. 10, 343-73.
27. HOTHORN, T. & LAUSEN, B. (2005). Bundling classifiers by bagging trees. Comp. Statist. Data Anal. 49, 1068-78.
28. JONES, M. C., MARRON, J. S. & PARK, B. U. (1991). A simple root n bandwidth selector. Ann. Statist. 19, 1919-32.
29. KAUERMANN, G. & OPSOMER, J. D. (2004). Generalized cross-validation for bandwidth selection of backfitting estimates in generalized additive models. J. Comp. Graph. Statist. 13, 66-89.
30. LOADER, C. R. (1999). Bandwidth selection: Classical, or plug-in? Ann. Statist. 27, 415-38.
31. LUKAS, M. A. (2006). Robust generalized cross-validation for choosing the regularization parameter. Inverse Problems 22, 1883-902.
32. MARRON, J. S. & WAND, M. P. (1992). Exact mean-integrated squared error. Ann. Statist. 20, 712-36.
33. PARK, B. U. & MARRON, J. S. (1990). Comparison of data-driven bandwidth selectors. J. Am. Statist. Assoc. 85, 66-72.
34. R DEVELOPMENT CORE TEAM . (2008). R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing.
35. RICE, J. (1984). Bandwidth choice for nonparametric regression. Ann. Statist. 12, 1215-30.
36. RIDGEWAY, G. (2002). Looking for lumps: Boosting and bagging for density estimation. Comp. Statist. Data Anal. 38, 379-92.
37. RUDEMO, M. (1982). Empirical choice of histograms and kernel density estimators. Scand. J. Statist. 9, 65-78.
38. SCOTT, D. W. & TERRELL, G. R. (1987). Biased and unbiased cross-validation in density estimation. J. Am. Statist. Assoc. 82, 1131-46.
39. SINISI, S. E., NEUGEBAUER, R. & VAN DER LAAN, M. J. (2006). Cross-validated bagged prediction of survival. Statist. Appl. Genet. Molec. Biol. 5, 26 pp.
40. STUTE, W. (1992). Modified cross-validation in density estimation. J. Statist. Plan. Infer. 30, 293-305.
41. TAYLOR, C. C. (1989). Bootstrap choice of the smoothing parameter in kernel, density estimation. Biometrika 76, 705-12.
42. WAHBA, G. & WOLD, S. (1975). A completely automatic French curve: Fitting spline functions by cross validation. Commun. Statist. 4, 1-17.
43. WAND, M. P. & JONES, M. C. (1994). Multivariate plug-in bandwidth selection. Comp. Statist. 9, 97-116.
44. WONG, W. H. (1983). On the consistency of cross-validation in kernel, nonparametric regression. Ann. Statist. 11, 1136-41.
45. WU, C. O. (1997). A cross-validation bandwidth choice for kernel density estimates with selection biased data. J. Mult. Anal. 61, 38-60.
46. ZHANG, H. (1998). Classification trees for multiple binary responses. J. Am. Statist. Assoc. 93, 180-93.