A general framework for updating belief distributions

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Volumn 78, Issue 5, 2016, Pages 1103-1130

  Bissiri, P G ^a   Holmes, C C ^b   Walker, S G ^c  

^a UNIVERSITY OF MILANO BICOCCA   (Italy)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

Decision theory; General Bayesian updating; Generalized estimating equations; Gibbs posteriors; Information; Loss function; Maximum entropy; Provably approximately correct Bayes methods; Self information loss function

Indexed keywords

EID: 84959423086     PISSN: 13697412     EISSN: 14679868     Source Type: Journal    
DOI: 10.1111/rssb.12158     Document Type: Article

References (58)

1. Ali, S. M. and Silvey, S. D. (1966) A general class of coefficients of divergence of one distribution from another. J. R. Statist. Soc. B, 28, 131–142.
2. Alquier, P. (2008) PAC-Bayesian bounds for randomized empirical risk minimizers. Math. Meth. Statist., 17, 279–304.
3. Bar-Hillel, M. and Falk, R. (1982) Some teasers concerning conditional probabilities. Cognition, 11, 109–122.
4. Berger, J. O. (1993) Statistical Decision Theory and Bayesian Analysis. New York: Springer.
5. Bernardo, J. M. (1979) Expected information as expected utility. Ann. Statist., 7, 686–690.
6. Bernardo, J. M. and Smith, A. F. M. (1994) Bayesian Theory. Chichester: Wiley.
7. Bissiri, P. G. and Walker, S. G. (2010) On Bayesian learning from Bernoulli observations. J. Statist. Planng Inf., 140, 3520–3530.
8. Bissiri, P. G. and Walker, S. G. (2012a) On Bayesian learning from loss functions. J. Statist. Planng Inf., 142, 3167–3173.
9. Bissiri, P. G. and Walker, S. G. (2012b) Converting information into probability measures via the Kullback–Leibler divergence. Ann. Inst. Statist. Math., 64, 1139–1160.
10. Catoni, O. (2003) A PAC Bayesian approach to adaptive classification. Preprint 840. Laboratoire de Probabilités et Modèles Aléatoires, Université Paris 6, Paris.
11. Cesa-Bianchi, N. and Lugosi, G. (2006) Prediction, Learning, and Games. Cambridge: Cambridge University Press.
12. Cheng, Y. and Church, G. M. (2000) Biclustering of expression data. In Proc. 8th Int. Conf. Intelligent Systems for Molecular Biology (eds P. Bourne, M. Gribskov, R. Altman, N. Jensen, D. Hope, T. Lengauer, J. Mitchell, E. Scheeff, C. Smith, S. Strande and H. Weissig), pp. 93–103. Menlo Park: American Association for Artificial Intelligence Press.
13. Cooley, D., Ribatet, M. and Davison, A. C. (2009) Bayesian inference from composite likelihoods, with an application to spatial extremes. Preprint arXiv:0911.5357. Department of Statistics, University of Colorado, Boulder.
14. Cox, D. R. (1972) Regression models and life-tables (with discussion). J. R. Statist. Soc. B, 34, 187–220.
15. Datta, G. S. and Sweeting, T. J. (2005) Probability matching priors. In Handbook of Statistics (eds D. Dey and C. R. Rao), vol. 25, pp. 91–114. Amsterdam: Elsevier.
16. Diaconis, P. and Zabell, S. L. (1982) Updating subjective probability. J. Am. Statist. Ass., 77, 822–830.
17. Doucet, A. and Shephard, N. (2012) Robust inference on parameters via particle filters and sandwich covariance matrices. Report 606. Department of Economics, University of Oxford, Oxford.
18. Fan, J and Li, R. (2002) Variable selection for Cox's proportional hazards model and frailty model. Ann. Statist., 30, 74–99.
19. Faraggi, D. and Simon, R. (1998) Bayesian variable selection method for censored survival data. Biometrics, 54, 1475–1485.
20. de Finetti, B. (1937) La prévision: ses lois logiques, ses sources subjectives. Ann. Inst. H. Poincaré, 7, 1–68.
21. Freund, J. E. (1965) Puzzle or paradox? Ann. Statist., 19, 29–44.
22. Gardner, M. (1959) The Scientific American Book of Mathematical Puzzles and Diversions. New York: Simon and Schuster.
23. Goldstein, M. (1981) Revising previsions: a geometric interpretation. J. R. Statist. Soc. B, 43, 105–130.
24. Goldstein, M. and Wooff, D. (2007) Bayes Linear Statistics, Theory & Methods. Chichester: Wiley.
25. Hartigan, J. A. (1972) Direct clustering of a data matrix. J. Am. Statist. Ass., 67, 123–129.
26. Hastie, T., Tibshirani, R. and Friedman, J. (2009) Elements of Statistical Learning. New York: Springer.
27. Heard, N. A., Holmes, C. C., Stephens, D. A., Hand, D. J. and Dimopoulos, G. (2005) Bayesian coclustering of Anopheles gene expression time series: study of immune defence response to multiple experimental challenges. Proc. Natn. Acad. Sci. USA, 102, 16939–16944.
28. Hirshleifer, J. and Riley, J. G. (1992) The Analytics of Uncertainty and Information. Cambridge: Cambridge University Press.
29. Hoff, P. and Wakefield, J. C. (2013) Bayesian sandwich posteriors for pseudo-true parameters. J. Statist. Planng Inf., 143, 1638–1642.
30. Hüber, P. (1964) Robust estimation of a location parameter. Ann. Math. Statist., 35, 73–101.
31. Hüber, P. (2009) Robust Statistics, 2nd edn. Hoboken: Wiley.
32. Hutchison, K. (1999) What are conditional probabilities conditional upon? Br. J. Philos. Sci., 50, 665–695.
33. Hutchison, K. (2008) Resolving some puzzles of conditional probability. Adv. Sci. Lett., 1, 212–221.
34. Ibrahim, J. G. and Chen, M. H. (2000) Power prior distributions for regression models. Statist. Sci., 15, 46–60.
35. Ibrahim, J. G., Chen, M. H. and MacEachern, S. N. (1999) Bayesian variable selection for proportional hazards models. Can. J. Statist., 27, 701–711.
36. Ibrahim, J. G., Chen, M. H. and Sinha, D. (2001) Bayesian Survival Analysis. New York: Springer.
37. Jasra, A., Holmes, C. C. and Stephens, D. A. (2005) Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modeling. Statist. Sci., 20, 50–67.
38. Jiang, W. and Tanner, M. A. (2008) Gibbs posterior for variable selection in high-dimensional classification and data mining. Ann. Statist., 36, 2207–2231.
39. Kass, R. E. and Wasserman, L. A. (1996) The selection of prior distributions by formal rules. J. Am. Statist. Ass., 91, 1343–1370.
40. Key, J. T., Pericchi, L. R. and Smith, A. F. M. (1999) Bayesian model choice: what and why (with discussion)? In Bayesian Statistics 6 (eds J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith), pp. 343–370. Oxford: Oxford University Press.
41. Kullback, S. and Leibler, R. A. (1951) On information and sufficiency. Ann. Math. Statist., 22, 79–86.
42. Langford, J. (2005) Tutorial on practical prediction theory for classification. J. Mach. Learn. Res., 6, 273–306.
43. Marin, J. M., Pudlo, P., Robert, C. P. and Ryder, R. J. (2012) Approximate Bayesian computational methods. Statist. Comput., 22, 1167–1180.
44. McAllester, D. (1998) Some PAC Bayes theorems. In Proc. 11th A. Conf. Computational Learning Theory, pp. 164–170. New York: Association for Computing Machinery.
45. Merhav, N. and Feder, M. (1998) Universal prediction. IEEE Trans. Inform. Theor., 44, 2124–2147.
46. Müller, U. (2012) Risk of Bayesian inference in misspecified models, and the sandwich covariance matrix. Department of Economics, Princeton University, Princeton.
47. von Neumann, J. and Morgenstern, O. (1944) Theory of Games and Economic Behaviour. Princeton: Princeton University Press.
48. Royall, R., and Tsou, T.-S. (2003) Interpreting statistical evidence by using imperfect models: robust adjusted likelihood functions. J. R. Statist. Soc. B, 65, 391–404.
49. Savage, L. J. (1954) The Foundations of Statistics. New York: Wiley.
50. Seldin, Y. and Tishby, N. (2010) PAC Bayesian analysis of co-clustering and beyond. J. Mach. Learn. Res., 11, 3595–3646.
51. Shawe-Taylor, J. and Williamson, R. (1997) A PAC analysis of a Bayesian estimator. In Proc. 10th A. Conf. Computational Learning Theory, pp. 2–9. New York: Association for Computing Machinery.
52. Tanay, A., Sharan, R. and Shamir, R. (2002) Discovering statistically significant biclusters in gene expression data. Bioinformatics, 18, 136–144.
53. Tibshirani, R. J. (1997) The lasso method for variable selection in the Cox model. Statist. Med., 16, 385–395.
54. Volinsky, C. T., Madigan, D., Raftery, A. E. and Kronmal, R. A. (1997) Bayesian model averaging in proportional hazard models: assessing the risk of a stroke. Appl. Statist., 46, 433–448.
55. Walker, S. and Hjort, N. L. (2001) On Bayesian consistency. J. R. Statist. Soc. B, 63, 811–821.
56. Zellner, A. (1988) Optimal information processing and Bayes's theorem. Am. Statistn, 42, 278–284.
57. Zhang, T. (2006a) From ε-entropy to KL-entropy: analysis of minimum information complexity density estimation. Ann. Statist., 34, 2180–2210.
58. Zhang, T. (2006b) Information theoretical upper and lower bounds for statistical estimation. IEEE Trans. Inform. Theor., 52, 1307–1321.