Density Estimation by Total Variation Penalized Likelihood Driven by the Sparsity ℓ1 Information Criterion

Scandinavian Journal of Statistics

Volumn 37, Issue 2, 2010, Pages 321-337

  Sardy, S ^a   Tseng, Paul ^b  

^a UNIVERSITY OF GENEVA   (Switzerland)

^b UNIVERSITY OF WASHINGTON   (United States)

Author keywords

1 penalization; Convex programme; Dual block coordinate relaxation; Extreme value theory; Smoothing; Total variation; Universal penalty parameter

Indexed keywords

EID: 77953963579     PISSN: 03036898     EISSN: 14679469     Source Type: Journal    
DOI: 10.1111/j.1467-9469.2009.00672.x     Document Type: Article

References (50)

1. Akaike H. Information theory and an extension of the maximum likelihood principle. 2nd International Symposium on Information Theory 1973, 267-281. Petrov B NCsaki F. eds, Akademiai Kiado, Budapest.
2. Berlinet A, Devroye L. A comparison of kernel density estimates. Publ. l'Inst. Stat. l'Univ. Paris 1994, 38:3-59.
3. Bertsekas D P. Nonlinear programming 1999, Athena Scientific, Belmont, MA
4. Besag J. On the statistical analysis of dirty pictures (with discussion). J. Roy. Statist. Soc. Ser. B 1986, 48:192-236.
5. Coles S. An introduction to statistical modeling of extreme values 2001, Springer-Verlag Inc., London
6. Craven P, Wahba G. Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the method of generalized cross-validation. Numer. Math. 1979, 31:377-403.
7. Davies P L, Kovac A. Densities, spectral densities and modality. Ann. Statist. 2004, 32:1093-1136.
8. Donoho D L, Johnstone I M. Ideal spatial adaptation via wavelet shrinkage. Biometrika 1994, 81:425-455.
9. Donoho D L, Johnstone I M, Hoch J C, Stern A S. Maximum entropy and the nearly black object (with discussion). J. Roy. Statist. Soc. Ser. B 1992, 54:41-81.
10. Donoho D L, Johnstone I, Kerkyacharian G, Picard D. Wavelet shrinkage: asymptotia? (with discussion). J. Roy. Statist. Soc. Ser. B 1995, 57:301-369.
11. Eggermont P P B, LaRiccia V N. Optimal convergence rates for Good's nonparametric maximum likelihood density estimator. Ann. Statist. 1999, 27:1600-1615.
12. Fu W J. Penalized regressions: the bridge versus the lasso. J. Comput. Graph. Statist. 1998, 7:397-416.
13. Geman S, Geman D. Stochastic relaxation. Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell 1984, 61:721-741.
14. Good I. The estimation of probabilities: an essay on modern Bayesian methods 1965, MIT Press, Cambridge, MA
15. Good I. A nonparametric roughness penalty for probability densities. Nature 1971, 229:29-30.
16. Good I J, Gaskins R A. Nonparametric roughness penalties for probability densities. Biometrika 1971, 58:255-277.
17. Hall W J, Wellner J A. Confidence bands for a survival curve from censored data. Biometrika 1980, 67:133-143.
18. Hsing T, Hüsler J, Leadbetter M R. On the exceedance point process for a stationary sequence. Probab. Theory Related Fields 1988, 78:97-112.
19. Izenman A J, Sommer C J. Philatelic mixtures and multimodal densities. J. Amer. Statist. Assoc. 1988, 83:941-953.
20. Koenker R, Mizera I. Density estimation by total variation regularization. Advances in statistical modeling and inference. Essays in honor of Kjell A. Doksum 2006, 613-633. Nair V, ed, World Scientific, Singapore
21. Koenker R, Ng P, Portnoy S. Quantile smoothing splines. Biometrika 1994, 81:673-680.
22. Kooperberg C, Stone C J. A study of logspline density estimation. Comput. Statist. Data Anal. 1991, 12:327-347.
23. Kooperberg C, Stone C J. Logspline density estimation with free knots. Comput. Statist. Data Anal. 2002, 12:327-347.
24. Mallows C L. Some comments on Cp. Technometrics 1973, 15:661-675.
25. Mammen E, van de Geer S. Locally adaptive regression splines. Ann. Statist. 1997, 25:387-413.
26. Marron J S, Wand M P. Exact mean integrated squared error. Ann. Statist. 1992, 20:712-736.
27. O'Sullivan F. Fast computation of fully automated log-density and log-hazard estimators. SIAM J. Sci. Statist. Comput. 1988, 9:363-379.
28. Penev S, Dechevsky L. On non-negative wavelet-based density estimators. J. Nonparametr. Statist. 1997, 7:365-394.
29. Pinheiro A, Vidakovic B. Estimating the square root of a density via compactly supported wavelets. Comput. Statist. Data Anal. 1997, 25:399-415.
30. Renaud O. Sensitivity and other properties of wavelet regression and density estimators. Statist. Sinica 2002, 12:1275-1290.
31. Rockafellar R T. Convex analysis 1970, Princeton University Press, Princeton
32. Rockafellar R T. Network flows and monotropic programming 1984, Wiley-Interscience, New-York, republished by Athena Scientific, Belmont, 1998
33. Roeder K. Density estimation with confidence sets exemplified by superclusters and voids in the galaxies. J. Amer. Statist. Assoc. 1990, 85:617-624.
34. Sardy S. Adaptive posterior mode estimation of a sparse sequence for model selection. Scand. J. Statist. 2009, 36:577-601.
35. Sardy S, Tseng P. On the statistical analysis of smoothing by maximizing dirty Markov random field posterior distributions. J. Amer. Statist. Assoc. 2004, 99:191-204.
36. Sardy S, Bruce A, Tseng P. Block coordinate relaxation methods for nonparametric wavelet denoising. J. Comput. Graph. Statist. 2000, 9:361-379.
37. Schwarz G. Estimating the dimension of a model. Ann. Statist. 1978, 6:461-464.
38. Scott D W. Multivariate density estimation: theory, practice, and visualization 1992, Wiley-Interscience, New York
39. Sheather S J, Jones M C. A reliable data-based bandwidth selection method for kernel density estimation. J. Roy. Statist. Soc. Ser. B 1991, 53:683-690.
40. Silverman B W. On the estimation of a probability density function by the maximum penalized likelihood method. Ann. Statist. 1982, 10:795-810.
41. Silverman B W. Density estimation for statistics and data analysis 1986, Chapman & Hall, London
42. Simonoff J S. Smoothing methods in statistics 1996, Springer-Verlag, New York
43. Stone M. Cross-validatory choice and assessment of statistical predictions (with discussion). J. Roy. Statist. Soc. Ser. B 1974, 36:111-147.
44. Stone C J. Large sample inference for logspline model. Ann. Statist. 1990, 18:717-741.
45. Tibshirani R. Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 1996, 58:267-288.
46. Tikhonov A N. Solution of incorrectly formulated problems and the regularization method. Soviet Math. Dokl. 1963, 4:1035-1038.
47. Tseng P. Convergence of block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 2001, 109:475-494.
48. Vidakovic B. Statistical modeling by wavelets 1999, Wiley, New York
49. Wahba G. Spline models for observational data 1990, Society for Industrial and Applied Mathematics, Philadelphia, PA
50. Willett R, Nowak R D. Multiscale density estimation 2003, Technical Report, Rice University