Universal smoothing factor selection in density estimation" theory and practice

Test

Volumn 6, Issue 2, 1997, Pages 223-320

  Devroye, Luc ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

asymptotic optimality; convergence; Density estimation; kernel estimate; minimum distance estimate; simulation study; smoothing factor

Indexed keywords

EID: 0001791608     PISSN: 11330686     EISSN: 18638260     Source Type: Journal    
DOI: 10.1007/BF02564701     Document Type: Article

References (123)

1. Akaike H. (1954). An approximation to the density flmction. Annals of the Institute of Statistical Mathematics, 6, 127-132.
2. Bartlett M.S. (1963). Statistical estimation of density fimctions. Sankhya Series A, 25,245-254.
3. Bean S.J. and C.P. Tsokos (1980). Developments in nonparametric density estimation. International Statistical Review, 48, 267-287.
4. Berlinet A. and L. Dcvroye (1994). A comparison of kernel density estimates Publications de l'Institut de, qtatistique de l'Universitd de Paris, 38(3), 3-59.
5. Bosq D. and J.P. Lecoutre (1987). Thdorie de l' estimation fonctionnelle, Economica, Paris.
6. Bowman A.W. (1984). An alternative method of cross-validation for the smoothing of density estimates. Biometrika, 71,353-360.
7. Bowman A.W. (1985). A comparative study of some kernel-based non-parametric density estimators. Journal of Statistical Computation and Simulation, 21, 313-327.
8. Bretagnolle J. and C. Huber (1979). Estimation des densités: risque minimax. Zeitschrift fiir Wahrscheinlichkeitstheorie und verwandte Gebiete, 47, 119-137.
9. Broniatowski M., P. Deheuvels and L. Devroye (1989). On the relationship between stability of extreme order statistics and convergence of the maximum likelihood kernel density estimate. Annals of Statistics, 17, 1070-1086.
10. Burman P. (1985). A data dependent approach to density estimation. Zeitschrift f(ir Wahrscheinlichkeitstheorie und verwandte Gebiete, 69,609-628.
11. Cao R., A. Cuevas and W. Gonzs (1994). A comparative study of several smoothing methods in density estimation. Computational Statistics and Data Analysis, 17, 153-176.
12. Cao-Abad R. (1990). Aplicaciones y nuevos resultados del mdtodo bootstrap en la estimaci6n no paramdtrica de curvas. Ph.D. Dissertation, University of Santiago de Compostela, Spain.
13. Cheng R.C.H. and N.A.K. Amin (1983). Estimating parameters in continuous univariate distributions with a shifted origin. Journal of the Royal Statistical Society, B45, 394-403.
14. Chiu S.T. (1991). Bandwidth selection for kernel density estimation. Annals of Statistics, 19, 1883-1905.
15. Chow Y.S., S. Geman and L.D. Wu (1983). Consistent cross-validated density estimation. Annals of Statistics, 11, 25-38.
16. Cline D.B.H. (1988). Admissible kernel estimators of a multivariate density. Annals of Statistics, 16, 1421-1427.
17. Cline D.B.H. (1990). Optimal kernel estimation of densities. Annals of the Institute of Statistical Mathematics, 42,287-303.
18. Deheuvels P. (1977a). Estimation non paramétrique de la densité par histogrammes generalisés. Revue de Statistique Appliqude, 25, 5-42.
19. Deheuvels P. (1977b). Estimation nonparamétrique de la densitg par histogrammes generalisés. Publications de l'Institut de Statistique de l'Universitg de Paris, 22, 1-23.
20. Deheuveis P. and P. Hominal (1980). Estimation automatique de la densitY. Revue de Statistique Appliqude, 28, 25-55.
21. Dcvroye L. (1983). The equivalence of weak, strong and complete convergence in Ll for kernel density estimates. Annals of Statistics, 11,896-904.
22. Devroye L. (1986). Non-Uniform Random Variate Generation, Springer-Verlag, New York.
23. Devroye L. (1987). A Course in Density Estimation, Birkhguser, Boston.
24. Devroye L. (1988). Asymptotic performance bounds for the kernel estimate. Annals of Statistics, 16, 1162-1179.
25. Devroye L. (1988a). The kernel estimate is relatively stable. Probability Theory and Related Fields, 77, 521-536.
26. Devroye L. (1988b). Asymptotic performance bounds for the kernel estimate. Annals of Statistics, 16, 1162-1179.
27. Devroye L. (1989a). Nonparametric density estimates with improved performance on given sets of densities. Statistws (Mathematische Operationsforschung und Statistik), 20, 357-376.
28. Devroye L. (1989b). The double kernel method in density estimation. Annales de l'Institut Henri Poincaré, 25, 533-580.
29. Devroye L. (1989c). A universal lower bound for the kernel estimate. Statistics and Probability Letters, 8,419-423.
30. Devroye L. (1989d). On the non-consistency of the L2 cross-validated kernel density estimate. Statistics and Probability Letters, 8,425-433.
31. Devroye L. (1991). Exponential inequalities in nonparametric estimation. In: Nonparametric Functional Estimatwn and Related Topics, ed. G. Roussas, 31-44, NATO ASI Series, Kluwer Academic Publishers, Dordrecht.
32. Devroye L. (1992). A note on the usefulness of superkernels in density estimation. Annals of Statistzcs, 20, 2037-2056.
33. Devroye L. (1994). On the non-consistency of an estimate of Chiu. Statistics and Probability Letters, 20, 183-188.
34. Dcvroye L. (1994). On good deterministic smoothing sequences for kernel density estimates. Annals of Statistics, 22, 886-889.
35. Devroye L. and L. lySrfi (1985). Nonparametric Density Estimation: The L1 View, John Wiley, New York.
36. Devroye L., L. GySrfi and G. Lugosi (1996). A Probabilistic Theory of Pattern Recognitzon, Springer-Verlag, New York.
37. Devroye L. and G. Lugosi (1996). A universally acceptable smoothing factor for kernel density estimation. Annals of Statistics, 24, 2499-2512.
38. Devroye L. and G. Lugosi (1997). Nonasymptotic universal smoothing factors, kernel complexity, and Yatracos classes. Annals of Statistics, 25, to appear.
39. Devroye L. and M. P. Wand (1993). On the effect of density shape on the performance of its kernel estimate. Statistics, 24, 215-233.
40. Duin, R.P.W. (1976). On the choice of smoothing parameters for Parzen estimators of probability density functions. IEEE Transactions on Computers, Cé 1175-1179.
41. Eggermont P.P.B. and V.N. LaRiccia (1995). Maximum smoothed likelihood density estimation for inverse problems. Annals of Statistics, 23(1), 199-220.
42. Eggermont P.P.B. and V.N. LaRiccia (1996). A simple and effective bandwidth selector for kernel density estimation. Scandinavian Journal of Statistics, 23, 285-301.
43. Engel J., E. Herrmann and T. Gasser (1992). An iterative bandwidth selector for kernel estimation of densities and derivatives. Technical Report, University of Heidelberg.
44. Epanechnikov V.A. (1969). Nonparametric estimation of a multivariate probability density. Theory of Probability and its Applications, 14, 153-158.
45. Falk M. (1992). Bootstrap optimal bandwidth selection for kernel density estimation. Journal of Statistical Planning and Inference, 30, 13-22.
46. Fan J. and J. S. Marron (1992). Best possible constant for bandwidth selection. Annals of Statistics, 20, 2057-2070.
47. Faraway J. and M. Jhun (1990). Bootstrap choice of bandwidth for density estimation. Journal of the American Statistical Association, 85, 1ll9-1122.
48. Gasser T., H.G. Miiller and V. Mammitzsch (1985). Kernels for nonparametric curve estimation. Journal of the Royal Statistical Society, Series B, 47, 238-252.
49. Habbema J.D.F., J. Hermans and K. Vandenbroek (1974). A stepwise discriminant analysis program using density estimation. In: COMPSTAT 1974, ed. G. Bruckmann, 101-110, Physica Verlag, Wien.
50. Hall P. (1982). Cross-validation in density estimation. Biometrika, f69, 383-390.
51. Hall P. (1983). Large-sample optimality of least squares cross-validation in density estimation. Annals of Statistics, 11, 1156-1174.
52. Hall P. (1985). Asymptotic theory of minimum integrated square error for multivariate density estimation, in: Multivariate Analysis VI, ed. P. R. Krishnaiah, 289-309, North-Holland, Amsterdam.
53. Hall P. (1989). On convergence rates in nonparametric problems. International Statistical Review, 57, 45-58.
54. Hall P. (1990). Using the bootstrap to estimate mean squared error and select smoothing parameter in nonparametric problems. Journal of Multivariate Analysis, 32, 177-203.
55. Hall P., T.J. DiCiceio and J.P. Romano (1989). On smoothing and the bootstrap. Annals of Statistics, 17,692-704.
56. Hall P. and I.M. Johnstone (1992). Empirical functionals and efficient smoothing parameter selection. Journal of the Royal Statistical Society, B54, 475-530.
57. Hall P. and J.S. Marron (1987a). On the amount of noise inherent in bandwidth selection of a kernel density estimator. Annals of Statistics, 15, 163-181.
58. Hall P. and J.S. Marron (1987b). Extent to which least-squares cross-validation minimises integrated square error in nonparametric density estimation. Probability Theory and Related Fields, 74, 567-581.
59. Hall P. and J.S. Marron (1990). Lower bounds for bandwidth selection in density estimation. Probability Theory and Related Fields, 90, 149-173.
60. Hall P. and J.S. Marron (1991). Local minima in cross-validation functions. Journal of the Royal Statistical Association, B53,245-252.
61. Hall P., J.S. Matron and B. Park (1992). Smoothed cross-validation. Probability Theory and Related Fields, 92, 1-20.
62. Hall P., S.J. Sheather, M.C. Jones and J.S. Marron (1992). On optimal data-based bandwidth selection in kernel density estimation. Biometrika, 78, 263-269.
63. Hall P. and M.P. Wand (1988). Minimizing L1 distance in nonparametric density estimation. Journal of Multivariate Analysis, 26, 59-88.
64. Hoeffding W. (1963). Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58, 13-30.
65. Izenman A.J. (1991). Recent developments in nonparametric density estimation. Journal of the American Statistical Association, 86,205-225.
66. Janssen P., J.S. Marron, N. Veravereke and W. Sarle (1992). Scale measures for bandwidth selection. Technical Report, University of Limburg, Belgium.
67. Jones M.C. (1990). Changing kernels' orders. Technical Report, Department of Statistics, The Open University, Milton Keynes, U.K.
68. Jones M.C. (1991a). The roles of ISE and MISE in density estimation. Statistics and Probability Letters, 12, 51-56.
69. Jones M.C. (1991b). Prospects for automatic bandwidth selection in extensions to basic kernel density estimation. In: Nonparametric Functional Estimation and Related Topics, ed. G. Roussas, 241-250, NATO ASI Series, Kluwer Academic Publishers, Dordrecht.
70. Jones M.C. and R.F. Kappenman (1992). On a class of kernel density estimate bandwidth selectors. Scandinavian Journal of Statistics, 19, 337-349.
71. Jones M.C., J.S. Marron and B.U. Park (1991). selector. Annals of Statistics, 19, 1919-1932. A simple root n bandwidth
72. Jones, M.C., J.S. Marron and S. J. Sheather (1992). Progress in data-based bandwidth selection for kernel density estimation. Mimeo series 2088, Department of Statistics, University of North Carolina.
73. Jones M.C. and S.J. Sheather (1991). Using non-stochastic terms to advantage in kernel-based estimation of integrated squared density derivatives. Statistics and Probability Letters, 11,511-514.
74. Kim W.C., B.U. Park and J.S. Marron (1994). Asymptotically best bandwidth selectors in kernel density estimation. Statistics and Probability Letters, 19, 119-127..
75. Mammen E. (1990). A short note on optimal bandwidth selection for kernel estimators. Statistics and Probability Letters, 9, 23-25.
76. Marron J.S. (1985). An asymptotically efficient solution to the bandwidth problem of kernel density estimation. Annals of Statistics, 13, 1011-1023.
77. Marron J.S. (1986). Will the art of smoothing ever become a science? Contemporary Mathematics, 59, 169-178.
78. Marron J.S. (1987). A comparison of cross-validation techniques in density estimation. Annals of Statistics, 15, 152-162.
79. Marron J.S. (1988). Automatic smoothing parameter selection: a survey. Empirical Economics, 13, 187-208.
80. Marron J.S. (1989). Comments on a data based bandwidth selector. Computational Statistics and Data Analysis, 8, 155-170.
81. Marron J.S. (1991). Root n bandwidth selection. In: Nonparametric Functional Estimation and Related Topics, ed. G. Roussas, 251-260, NATO ASI Series, Kluwer Academic Publishers, Dordrecht.
82. Marron J.S. (1992). Bootstrap bandwidth selection. In: Exploring the Limits of the Bootstrap, ed. R. LePage and L. Billard, 249-262, John Wiley, New York.
83. Matron J.S. and M.P. Wand (1992). Exact mean integrated square error. Annals of Statistics, 20,712-736.
84. McDiarmid C. (1989). On the method of bounded differences. In: Surveys in Combinatorics 1989, 141, 148-188, London Mathematical Society Lecture Notes Series, Cambridge University Press, Cambridge.
85. Mihoubi A. (1992). Bootstrap et validation croisde en estimation non paramdtrique de la densitd. Thèse de doctorat; Universitè Paris VI.
86. Miller H.G. (1984). Smooth optimum kernel estimators of densities, regression curves and modes. Annals of Statistics, 12, 766-774.
87. Nadaraya E.A. (1974). On the integral mean square error of some nonparametric estimates for the density function. Theory of Probability and its Applications, 19, 133-141.
88. Park B.U. (1989). On the plug-in bandwidth selectors in kernel density estimation. Journal of the Korean Statistical Society, 18, 107-117.
89. Park B.U. and J.S. Matron (1990). Comparison of data-driven bandwidth selectors. Journal of the American Statistical Association, 85, 66-72.
90. Park B.U. and J.S. Marron (1992). On the use of pilot estimators in bandwidth selection. Journal of Nonparametrie Statistics, 1,231-240.
91. Park B.U. and B.A. Turlach (1992). Practical performance of several data driven bandwidth selectors (with discussion). Computational Statistics, 7, 251-270.
92. Parzen E. (1962). On the estimation of a probability density function and the mode. Annals of Mathematical Statistics, 33, 1065-1076.
93. Ranneby B. (1984). The maximum spacings method: an estimation method. Scandinavian Journal of Statistics, 11, 93-112.
94. Roeder K. (1990). Density estimation with confidence sets exemplified by superclusters and voids in the galaxies. Journal of the American Statistical Association, 85, 617-624.
95. Rosenblatt M. (1956). Remarks on some nonparametric estimates of a density function. Annals of Mathematical Statistics, 27, 832-837.
96. Rosenblatt M. (1979). Global measures of deviation for kernel and nearest neighbor density estimates. In: Proceedings of the Heidelberg Workshop, 181-190, Springer Lecture Notes in Mathematics 757, Springer-Verlag, Berlin.
97. Rudemo M. (1982) Empirical choice of histograms and kernel density estimators. Scandinavian Journal of Statistics, 9, 65-78.
98. Schueany W.R. and J.P. Sommers (1977). Improvement of kernel type density estimators. Journal of the American Statistical Association, 72, 420-423.
99. Schuster E.F. and G.G. Gregory (1981). On the nonconsistency of maximum likelihood nonparametric density estimators, in: Computer Science and Statistics: Proceedings of the 13th Symposium n the Interface, ed. W. F. Eddy, 295-298, Springer-Verlag, New York.
100. Scott D.W. (1992). Multivariate Density Estimation, John Wiley, New York.
101. Scott D.W., Tapia R.A. and J.R. Thompson (1977). Kernel density estimation revisited. Nonlinear Analysis, 1,339-372.
102. Scott D.W. and G.R. Terrell (1987). Biased and unbiased cross-validation in density estimation. Journal of the American Statistical Association, 82, 1131-1146.
103. Sheather S.J. (1993). The performance of six popular bandwidth selection methods on some real data sets (with discussion). Computational Statistics, 11, 180-215.
104. Sheather S.J. and M.C. Jones (1991). A reliable data-based bandwidth selection method for kernel density stimation. Journal of the Royal Statistical Society, B53,683-60.
105. Stone C.J. (1984). An asymptotically optimal window selection rule for kernel density estimates. Annals of Statistics, 12, 1285-1297.
106. Stuetzle W. and Mittal Y. (1979). Some comments on the asymptotic behavior of robust smoothers. In: Proceedings of the Heidelberg Workshop, ed. T. Gasser and M. Rosenblatt, 191-195, Springer Lecture Notes in Mathematics 757, Springer-Verlag, Heidelberg.
107. Stute W. (1992). Modified cross-validation in density estimation. Journal of Statistical Planning and Inference, 30, 293-305.
108. Su-Wong H.Y., B. Prasad and R.S. Singh (1982). A comparison between two kernel estimators of a probability density function and its derivatives. Scandinavian Actuarial Journal, O, 216-222.
109. Terrell G.R. (1990). The maximal smoothing principle in density estimation. Journal of the American Statistical Association, 85,470-477.
110. Terrell G.R. and D.W. Scott (1985). Oversmoothed nonparametric density estimates. Journal of the American Statistical Association, 80, 209-214.
111. Titterington D.M. (1985). Common structure of smoothing techniques in statistics. International Statistical Review, 53, 141-170.
112. Turlach B.A. (1993). Bandwidth selection in kernel density estimation: a review. Technical Report, Université Catholique de Louvain.
113. van Es B. (1988). Aspects of Nonparametric Density Estimation. Ph.D. Dissertation, University of Amsterdam, The Netherlands.
114. van Es B. (1989). Likelihood cross-validation bandwidth selection for nonpara-metric kernel density estimators. Technical Report 89-10, Faculty of Technical Mathematics and Informatics, University of Delft, The Netherlands.
115. Vapnik V.N. and A.Ya. Chervonenkis (1971). On the uniform convergence of relative frequencies of events to their robabilities. Theory of Probability and
116. its Applications, 16,264-280.
117. Wand M.P. and L. Devroye (1993). How easy is a given density to estimate? Computational Statistics and Data Analysis, 16, 311-323.
118. Wand M.P. and M.C. Jones (1995). An Introduction to Kernel Smoothing, Chapman and Hall, London.
119. Wand M.P. and W.R. Sehucany (1990). Gaussian-based kernels. Canadian Journal of Statistics, 18, 197-204.
120. Watson G.S. and M.R. Leadbetter (1963). On the estimation of the probability density. Annals of Mathematical Statistics, 34, 480-491.
121. Woodroofe M. (1970). On choosing a delta sequence. Annals of Mathematical Statistics, 41, 1665-1671.
122. Y'atracos Y.G. (1985). Rates of convergence of minimum distance estimators and Kolmogorov's entropy. Annals of Statistics, 13,768-774.
123. Young G.A. (1990). Alternative smoothed bootstraps. Journal of the Royal Statistical Society, B52,477-484.