Bayesian Methods for Function Estimation

Handbook of Statistics

Volumn 25, Issue , 2005, Pages 373-414

  Choudhuri, Nidhan ^a   Ghosal, Subhashis ^a   Roy, Anindya ^a  

^a NONE

Author keywords

consistency; convergence rate; density estimation; Dirichlet process; Markov chain Monte Carlo; posterior distribution; regression function; spectral density; transition density

Indexed keywords

EID: 33845604510     PISSN: 01697161     EISSN: None     Source Type: Book Series    
DOI: 10.1016/S0169-7161(05)25013-7     Document Type: Review

References (137)

1. Albert J., and Chib S. Bayesian analysis of binary and polychotomous response data. J. Amer. Statist. Assoc. 88 (1993) 669-679
2. Amewou-Atisso M., Ghosal S., Ghosh J.K., and Ramamoorthi R.V. Posterior consistency for semiparametric regression problems. Bernoulli 9 (2003) 291-312
3. Ansley C.F., Kohn R., and Wong C. Nonparametric spline regression with prior information. Biometrika 80 (1993) 75-88
4. Antoniak C. Mixtures of Dirichlet processes with application to Bayesian non-parametric problems. Ann. Statist. 2 (1974) 1152-1174
5. Barron, A.R. (1988). The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions. Unpublished manuscript
6. Barron A.R. Information-theoretic characterization of Bayes performance and the choice of priors in parametric and nonparametric problems. In: Bernardo J.M., et al. (Ed). Bayesian Statistics, vol. 6 (1999), Oxford University Press, New York 27-52
7. Barron A., Schervish M., and Wasserman L. The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 (1999) 536-561
8. Barry D. Nonparametric Bayesian regression. Ann. Statist. 14 (1986) 934-953
9. Basu S., and Mukhopadhyay S. Bayesian analysis of binary regression using symmetric and asymmetric links. Sankhyā, Ser. B 62 (2000) 372-387
10. Belitser E.N., and Ghosal S. Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution. Ann. Statist. 31 (2003) 536-559
11. Berger J.O., and Guglielmi A. Bayesian and conditional frequentist testing of a parametric model versus nonparametric alternatives. J. Amer. Statist. Assoc. 96 453 (2001) 174-184
12. Berk R. Limiting behavior of the posterior distribution when the model is incorrect. Ann. Math. Statist. 37 (1966) 51-58
13. Birgé L. Robust testing for independent non-identically distributed variables and Markov chains. In: Florens J.P., et al. (Ed). Specifying Statistical Models. From Parametric to Non-Parametric. Using Bayesian or Non-Bayesian Approaches. Lecture Notes in Statistics vol. 16 (1983), Springer-Verlag, New York 134-162
14. Blackwell D. Discreteness of Ferguson selection. Ann. Statist. 1 (1973) 356-358
15. Blackwell D., and Dubins L.E. Merging of opinions with increasing information. Ann. Math. Statist. 33 (1962) 882-886
16. Blackwell D., and MacQueen J.B. Ferguson distributions via Polya urn schemes. Ann. Statist. 1 (1973) 353-355
17. Blum J., and Susarla V. On the posterior distribution of a Dirichlet process given randomly right censored observations. Stochastic Process. Appl. 5 (1977) 207-211
18. Brunner L.J. Bayesian nonparametric methods for data from a unimodal density. Statist. Probab. Lett. 14 (1992) 195-199
19. Brunner L.J. Bayesian linear regression with error terms that have symmetric unimodal densities. J. Nonparametr. Statist. 4 (1995) 335-348
20. Brunner L.J., and Lo A.Y. Bayes methods for a symmetric unimodal density and its mode. Ann. Statist. 17 (1989) 1550-1566
21. Carter C.K., and Kohn R. Semiparametric Bayesian inference for time series with mixed spectra. J. Roy. Statist. Soc., Ser. B 59 (1997) 255-268
22. Choudhuri N., Ghosal S., and Roy A. Bayesian estimation of the spectral density of a time series. J. Amer. Statist. Assoc. 99 (2004) 1050-1059
23. Choudhuri, N., Ghosal, S., Roy, A. (2004b). Bayesian nonparametric binary regression with a Gaussian process prior. Preprint
24. Choudhuri N., Ghosal S., and Roy A. Contiguity of the Whittle measure in a Gaussian time series. Biometrika 91 (2004) 211-218
25. Cifarelli D.M., and Regazzini E. Distribution functions of means of a Dirichlet process. Ann. Statist. 18 (1990) 429-442
26. Cox D.D. An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 (1993) 903-923
27. Dalal S.R. Dirichlet invariant processes and applications to nonparametric estimation of symmetric distribution functions. Stochastic Process. Appl. 9 (1979) 99-107
28. Denison D.G.T., Mallick B.K., and Smith A.F.M. Automatic Bayesian curve fitting. J. Roy. Statist. Soc., Ser. B Stat. Methodol. 60 (1998) 333-350
29. Diaconis P., and Freedman D. On the consistency of Bayes estimates (with discussion). Ann. Statist. 14 (1986) 1-67
30. Diaconis P., and Freedman D. On inconsistent Bayes estimates. Ann. Statist. 14 (1986) 68-87
31. DiMatteo I., Genovese C.R., and Kass R.E. Bayesian curve-fitting with free-knot splines. Biometrika 88 (2001) 1055-1071
32. Doksum K.A. Tail free and neutral random probabilities and their posterior distributions. Ann. Probab. 2 (1974) 183-201
33. Doob, J.L. (1948). Application of the theory of martingales. Coll. Int. du CNRS, Paris, pp. 22-28
34. Doss H. Bayesian nonparametric estimation of the median. I. Computation of the estimates. Ann. Statist. 13 (1985) 1432-1444
35. Doss H. Bayesian nonparametric estimation of the median. II. Asymptotic properties of the estimates. Ann. Statist. 13 (1985) 1445-1464
36. Doss H., and Sellke T. The tails of probabilities chosen from a Dirichlet prior. Ann. Statist. 10 (1982) 1302-1305
37. Dykstra R.L., and Laud P.W. A Bayesian nonparametric approach to reliability. Ann. Statist. 9 (1981) 356-367
38. Escobar M. Estimating normal means with a Dirichlet process prior. J. Amer. Statist. Assoc. 89 (1994) 268-277
39. Escobar M., and West M. Bayesian density estimation and inference using mixtures. J. Amer. Statist. Assoc. 90 (1995) 577-588
40. Escobar M., and West M. Computing nonparametric hierarchical models. Practical Nonparametric and Semiparametric Bayesian Statistics. Lecture Notes in Statistics vol. 133 (1998), Springer, New York 1-22
41. Ferguson T.S. A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 (1973) 209-230
42. Ferguson T.S. Prior distribution on the spaces of probability measures. Ann. Statist. 2 (1974) 615-629
43. Ferguson, T.S. (1983). Bayesian density estimation by mixtures of Normal distributions. In: Rizvi, M., Rustagi, J., Siegmund, D. (Eds.), Recent Advances in Statistics, pp. 287-302
44. Ferguson T.S., and Phadia E.G. Bayesian nonparametric estimation based on censored data. Ann. Statist. 7 (1979) 163-186
45. Freedman D. On the asymptotic distribution of Bayes estimates in the discrete case I. Ann. Math. Statist. 34 (1963) 1386-1403
46. Freedman D. On the asymptotic distribution of Bayes estimates in the discrete case II. Ann. Math. Statist. 36 (1965) 454-456
47. Freedman D. On the Bernstein-von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 (1999) 1119-1140
48. Fristedt B. Sample function behavior of increasing processes with stationary independent increments. Pacific J. Math. 21 (1967) 21-33
49. Fristedt B., and Pruitt W.E. Lower functions for increasing random walks and subordinators. Z. Wahsch. Verw. Gebiete 18 (1971) 167-182
50. Gangopadhyay A.K., Mallick B.K., and Denison D.G.T. Estimation of spectral density of a stationary time series via an asymptotic representation of the periodogram. J. Statist. Plann. Inference 75 (1998) 281-290
51. Gasparini M. Bayesian density estimation via Dirichlet density process. J. Nonparametr. Statist. 6 (1996) 355-366
52. Gelfand A.E., and Kuo L. Nonparametric Bayesian bioassay including ordered polytomous response. Biometrika 78 (1991) 657-666
53. Ghosal S. Asymptotic normality of posterior distributions for exponential families with many parameters. J. Multivariate Anal. 74 (2000) 49-69
54. Ghosal S. Convergence rates for density estimation with Bernstein polynomials. Ann. Statist. 29 (2001) 1264-1280
55. Ghosal S., and van der Vaart A.W. Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. 29 (2001) 1233-1263
56. Ghosal, S., van der Vaart, A.W. (2003a). Convergence rates for non-i.i.d. observations. Preprint
57. Ghosal, S., van der Vaart, A.W. (2003b). Posterior convergence rates of Dirichlet mixtures at smooth densities. Preprint
58. Ghosal S., Ghosh J.K., and Samanta T. On convergence of posterior distributions. Ann. Statist. 23 (1995) 2145-2152
59. Ghosal S., Ghosh J.K., and Ramamoorthi R.V. Noniformative priors via sieves and consistency. In: Panchapakesan S., and Balakrishnan N. (Eds). Advances in Statistical Decision Theory and Applications (1997), Birkhäuser, Boston 119-132
60. Ghosal S., Ghosh J.K., and Ramamoorthi R.V. Consistency issues in Bayesian nonparametrics. In: Ghosh S. (Ed). Asymptotics, Nonparametrics and Time Series: A Tribute to Madan Lal Puri (1999), Marcel Dekker, New York 639-668
61. Ghosal S., Ghosh J.K., and Ramamoorthi R.V. Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 (1999) 143-158
62. Ghosal S., Ghosh J.K., and Ramamoorthi R.V. Consistent semiparametric Bayesian inference about a location parameter. J. Statist. Plann. Inference 77 (1999) 181-193
63. Ghosal S., Ghosh J.K., and van der Vaart A.W. Convergence rates of posterior distributions. Ann. Statist. 28 (2000) 500-531
64. Ghosal S., Lember Y., and van der Vaart A.W. On Bayesian adaptation. Acta Appl. Math. 79 (2003) 165-175
65. Ghosh J.K., and Ramamoorthi R.V. Consistency of Bayesian inference for survival analysis with or without censoring. In: Koul H. (Ed). Analysis of Censored Data. IMS Lecture Notes Monograph Series vol. 27 (1995), Inst. Math. Statist., Hayward, CA 95-103
66. Ghosh J.K., and Ramamoorthi R.V. Bayesian Nonparametrics (2003), Springer-Verlag, New York
67. Ghosh, S.K., Ghosal, S. (2003). Proportional mean regression models for censored data. Preprint
68. Ghosh J.K., Ghosal S., and Samanta T. Stability and convergence of posterior in non-regular problems. In: Gupta S.S., and Berger J.O. (Eds). Statistical Decision Theory and Related Topics V (1994), Springer-Verlag, New York 183-199
69. Green P. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82 (1995) 711-732
70. Grenander U. Abstract Inference (1981), Wiley, New York
71. Hanson T., and Johnson W.O. Modeling regression error with a mixture of Polya trees. J. Amer. Statist. Assoc. 97 (2002) 1020-1033
72. Hjort N.L. Nonparametric Bayes estimators based on beta processes in models for life history data. Ann. Statist. 18 (1990) 1259-1294
73. Hjort, N.L. (1996). Bayesian approaches to non- and semiparametric density estimation. In: Bernardo, J., et al. (Eds.), Bayesian Statistics, vol. 5, pp. 223-253
74. Hjort, N.L. (2000). Bayesian analysis for a generalized Dirichlet process prior. Preprint
75. Hjort N.L. Topics in nonparametric Bayesian statistics (with discussion). In: Green P.J., Hjort N., and Richardson S. (Eds). Highly Structured Stochastic Systems (2003), Oxford University Press 455-487
76. Holmes C.C., and Mallick B.K. Generalized nonlinear modeling with multivariate free-knot regression splines. J. Amer. Statist. Assoc. 98 (2003) 352-368
77. Huang T.Z. Convergence rates for posterior distributions and adaptive estimation. Ann. Statist. 32 (2004) 1556-1593
78. Ibragimov I.A., and Has'minskii R.Z. Statistical Estimation: Asymptotic Theory (1981), Springer-Verlag, New York
79. Iswaran H., and Zarepour M. Exact and approximate sum representation for the Dirichlet process. Canad. J. Statist. 26 (2002) 269-283
80. Iswaran, H., Zarepour, M. (2002b). Dirichlet prior sieves in finite normal mixture models. Statistica Sinica, 269-283
81. Kim Y. Nonparametric Bayesian estimators for counting processes. Ann. Statist. 27 (1999) 562-588
82. Kim Y., and Lee J. On posterior consistency of survival models. Ann. Statist. 29 (2001) 666-686
83. Kim Y., and Lee J. A Bernstein-von Mises theorem in the nonparametric right-censoring model. Ann. Statist. 32 (2004) 1492-1512
84. Kleijn, B., van der Vaart, A.W. (2002). Misspecification in infinite-dimensional Bayesian statistics. Preprint
85. Kottas A., and Gelfand A.E. Bayesian semiparametric median regression modeling. J. Amer. Statist. Assoc. 96 (2001) 1458-1468
86. Kraft C.H. A class of distribution function processes which have derivatives. J. Appl. Probab. 1 (1964) 385-388
87. Lavine M. Some aspects of Polya tree distributions for statistical modeling. Ann. Statist. 20 (1992) 1222-1235
88. Lavine M. More aspects of Polya tree distributions for statistical modeling. Ann. Statist. 22 (1994) 1161-1176
89. Le Cam L. Asymptotic Methods in Statistical Decision Theory (1986), Springer-Verlag, New York
90. Le Cam L., and Yang G.L. Asymptotics in Statistics. second ed (2000), Springer-Verlag
91. Lenk P.J. The logistic normal distribution for Bayesian, nonparametric, predictive densities. J. Amer. Statist. Assoc. 83 (1988) 509-516
92. Lenk P.J. Towards a practicable Bayesian nonparametric density estimator. Biometrika 78 (1991) 531-543
93. Leonard T. Density estimation, stochastic processes, and prior information. J. Roy. Statist. Soc., Ser. B 40 (1978) 113-146
94. Liseo B., Marinucci D., and Petrella L. Bayesian semiparametric inference on long-range dependence. Biometrika 88 (2001) 1089-1104
95. Lo A.Y. Bayesian nonparametric statistical inference for Poisson point process. Z. Wahsch. Verw. Gebiete 59 (1982) 55-66
96. Lo A.Y. Weak convergence for Dirichlet processes. Sankhyā, Ser. A 45 (1983) 105-111
97. Lo A.Y. On a class of Bayesian nonparametric estimates I: Density estimates. Ann. Statist. 12 (1984) 351-357
98. Lo A.Y. A remark on the limiting posterior distribution of the multiparameter Dirichlet process. Sankhyā, Ser. A 48 (1986) 247-249
99. MacEachern S.N., and Muller P. Estimating mixture of Dirichlet process models. J. Comput. Graph. Statist. 7 (1998) 223-228
100. Mallick B.K., and Gelfand A.E. Generalized linear models with unknown link functions. Biometrika 81 (1994) 237-245
101. Mauldin R.D., Sudderth W.D., and Williams S.C. Polya trees and random distributions. Ann. Statist. 20 (1992) 1203-1221
102. Muliere P., and Tardella L. Approximating distributions of functionals of Ferguson-Dirichlet priors. Canad. J. Statist. 30 (1998) 269-283
103. Newton M.A., Czado C., and Chappell R. Bayesian inference for semiparametric binary regression. J. Amer. Statist. Assoc. 91 (1996) 142-153
104. Nieto-Barajas L.E., and Walker S.G. Bayesian nonparametric survival analysis via Lévy driven Markov process. Statistica Sinica 14 (2004) 1127-1146
105. Petrone S. Random Bernstein polynomials. Scand. J. Statist. 26 (1999) 373-393
106. Petrone S. Bayesian density estimation using Bernstein polynomials. Canad. J. Statist. 26 (1999) 373-393
107. Petrone S., and Veronese P. Nonparametric mixture priors based on an exponential random scheme. Statist. Methods Appl. 11 (2002) 1-20
108. Petrone S., and Wasserman L. Consistency of Bernstein polynomial posteriors. J. Roy. Statist. Soc., Ser. B 64 (2002) 79-100
109. Regazzini E., Guglielmi A., and Di Nunno G. Theory and numerical analysis for exact distributions of functionals of a Dirichlet process. Ann. Statist. 30 (2002) 1376-1411
110. Rubin D. The Bayesian bootstrap. Ann. Statist. 9 (1981) 130-134
111. Schwartz L. On Bayes procedures. Z. Wahsch. Verw. Gebiete 4 (1965) 10-26
112. Sethuraman J. A constructive definition of Dirichlet priors. Statistica Sinica 4 (1994) 639-650
113. Sethuraman J., and Tiwari R. Convergence of Dirichlet measures and interpretation of their parameters. In: Gupta S.S., and Berger J.O. (Eds). Statistical Decision Theory and Related Topics. III, vol. 2 (1982), Academic Press, New York 305-315
114. Shen X. Asymptotic normality of semiparametric and nonparametric posterior distributions. J. Amer. Statist. Assoc. 97 (2002) 222-235
115. Shen X., and Wasserman L. Rates of convergence of posterior distributions. Ann. Statist. 29 (2001) 687-714
116. Shively T.S., Kohn R., and Wood S. Variable selection and function estimation in additive nonparametric regression using a data-based prior (with discussions). J. Amer. Statist. Assoc. 94 (1999) 777-806
117. Smith M., and Kohn R. A Bayesian approach to nonparametric bivariate regression. J. Amer. Statist. Assoc. 92 (1997) 1522-1535
118. Smith M., Wong C., and Kohn R. Additive nonparametric regression with autocorrelated errors. J. Roy. Statist. Soc., Ser. B 60 (1998) 311-331
119. Susarla V., and Van Ryzin J. Nonparametric Bayesian estimation of survival curves from incomplete observations. J. Amer. Statist. Assoc. 71 (1976) 897-902
120. Susarla V., and Van Ryzin J. Large sample theory for a Bayesian nonparametric survival curve estimator based on censored samples. Ann. Statist. 6 (1978) 755-768
121. Tang, Y., Ghosal, S. (2003). Posterior consistency of Dirichlet mixtures for estimating a transition density. Preprint
122. Tokdar, S.T. (2003). Posterior consistency of Dirichlet location-scale mixtures of normals in density estimation and regression. Preprint
123. van der Vaart A.W. Asymptotic Statistics (1998), Cambridge University Press
124. van der Vaart A.W., and Wellner J.A. Weak Convergence and Empirical Processes (1996), Springer-Verlag, New York
125. Wahba G. Improper priors, spline smoothing and the problem of guarding against model errors in regression. J. Roy. Statist. Soc., Ser. B 40 (1978) 364-372
126. Walker S.G. On sufficient conditions for Bayesian consistency. Biometrika 90 (2003) 482-490
127. Walker S.G. New approaches to Bayesian consistency. Ann. Statist. 32 (2004) 2028-2043
128. Walker S.G., and Hjort N.L. On Bayesian consistency. J. Roy. Statist. Soc., Ser. B 63 (2001) 811-821
129. Walker S.G., and Muliere P. Beta-Stacy processes and a generalization of the Polya-urn scheme. Ann. Statist. 25 (1997) 1762-1780
130. Wasserman L. Asymptotic properties of nonparametric Bayesian procedures. In: Dey D., et al. (Ed). Practical Nonparametric and Semiparametric Bayesian Statistics. Lecture Notes in Statistics vol. 133 (1998), Springer-Verlag, New York 293-304
131. Whittle P. Curve and periodogram smoothing. J. Roy. Statist. Soc., Ser. B 19 (1957) 38-63
132. Whittle P. Gaussian estimation in stationary time series. Bull. Int. Statist. Inst. 39 (1962) 105-129
133. Wood S., and Kohn R. A Bayesian approach to robust binary nonparametric regression. J. Roy. Statist. Soc., Ser. B 93 (1998) 203-213
134. Wood S., Kohn R., Shively T., and Jiang W. Model selection in spline nonparametric regression. J. Roy. Statist. Soc., Ser. B 64 (2002) 119-139
135. Wood S., Jiang W., and Tanner M. Bayesian mixture of splines for spatially adaptive nonparametric regression. Biometrika 89 (2002) 513-528
136. Yau P., Kohn R., and Wood S. Bayesian variable selection and model averaging in high-dimensional multinomial nonparametric regression. J. Comput. Graph. Statist. 12 (2003) 23-54
137. Zhao L.H. Bayesian aspects of some nonparametric problems. Ann. Statist. 28 (2000) 532-552