Information optimality and Bayesian modelling

Journal of Econometrics

Volumn 138, Issue 2, 2007, Pages 405-429

  Clarke, Bertrand ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Bayesian method of moments; Data dependent priors; Entropy; Reference priors; Stochastic complexity

Indexed keywords

DATA PROCESSING; METHOD OF MOMENTS; PROBLEM SOLVING; STATISTICAL METHODS; STOCHASTIC MODELS;

BAYESIAN METHOD OF MOMENTS; DATA DEPENDENT PRIORS; REFERENCE PRIORS; STOCHASTIC COMPLEXITY;

BAYESIAN NETWORKS;

EID: 34247402213     PISSN: 03044076     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jeconom.2006.05.003     Document Type: Article

References (40)

1. Arimoto S. An algorithm for computing the capacity of arbitrary discrete memoryless channels. Transactions on Information Theory 18 (1972) 14-20
2. Barron, A.R., 1985, Logically smooth density estimation. Ph.D. Dissertation, Stanford University, Stanford, CA.
3. Barron A.R., and Cover T. Minimum complexity density estimation. Transactions on Information Theory 37 (1990) 1034-1054
4. Barron A.R., Rissanen J., and Yu B. The minimum description length principle in coding and modelling. Transactions on Information Theory 44 (1998) 2743-2760
5. Berger J.O., and Bernardo J.M. Estimating a product of means: Bayesian analysis with reference priors. Journal of the American Statistical Association 84 (1989) 200-207
6. Bernardo J.M. Reference posterior distributions for Bayesian inference. Journal of the Royal Statistical Society, Series B 41 (1979) 113-147
7. Bernardo J.M. Reference decisions, Istituto Nazionale di Alta Matematica. Symposia Mathematica 25 (1981) 85-94
8. Bernardo J.M., and Smith A.F.M. Bayesian Theory (1994), Wiley, Chichester, UK
9. Blahut R.E. Computation of channel capacity and rate distortion functions. Transactions on Information Theory 18 (1972) 460-473
10. Bryant P.G., and Cordero-Brana O.I. Model selection using the MDL principle. American Statistician 54 (2000) 257-268
11. Burnham K.P., and Anderson D.R. Model Selection and Inference: A Practical Information-theoretic Approach (1998), Springer, New York
12. Cesa-Bianchi N., and Lugosi G. Worst Case Bound for the Logarithmic Loss of Predictors (2000), Kluwer Academic Publishers, Netherlands
13. Clarke, B., Barron, A.R., 1989. Information theoretic asymptotics of Bayes methods. University of Illinois, Department of Statistics Technical Report # 26.
14. Clarke B., and Barron A.R. Information theoretic asymptotics of Bayes methods. Transactions on Information Theory 36 (1990) 453-471
15. Clarke B., and Barron A.R. Jeffreys prior is asymptotically least favorable under entropy risk. Journal of Statistical Planning and Inference 41 (1994) 37-61
16. Clarke B., and Yuan A. Partial information reference priors: derivation and interpretations. Journal of Statistical Planning and Inference 123 (2003) 313-345
17. Cover T.M., and Thomas J.A. Elements of Information Theory (1991), Wiley, New York
18. Csiszar I. I-divergence geometry of probability distributions and minimization problems. Annals of Probability 3 (1975) 146-158
19. Csiszar I., and Tusnady G. Information geometry and alternating minimization procedures. Statistics and Decisions: Supplement Issue 1 (1984) 205-237
20. Freedman D., and Purves R.A. Bayes method for bookies. Annals of Mathematics and Statistics 40 (1969) 1177-1186
21. Ghosh J.K., and Mukerjee R. Noninformative priors. In: Berger J.O., Bernardo J.M., Dawid A.P., and Smith A.F.M. (Eds). Bayesian Statistics vol. 4 (1992), Oxford University Press, Oxford 195-203
22. Hansen M., and Yu B. Model selection and minimum description length principle. Journal of the American Statistical Association 96 (2001) 746-774
23. Hansen, M., Yu, B., 2002. Minimum description length model selection criteria for generalized linear models. In: Goldstein, D. (Ed.), Science and Statistics: Festschrift for Terry Speed. Lecture Notes Monograph Series, vol. 40. Institute of Mathematical Statistics, Hayward, pp. 145-164.
24. Lindley D. On a measure of information provided by an experiment. Annals in Mathematics and Statistics 27 (1956) 986-1005
25. References to the properties and applications of MDL can be found at 〈http://www.csse.monash.edu.au/∼dld/MML.html〉 and 〈http://www.mdl-research.org/〉. Other websites such as 〈http://www.bayesian.org〉 have references for other topics covered here.
26. Reid, N., Mukerjee, R., Fraser, D., 2002. Some aspects of matching priors. In: Moore, M., Froda, S., Léger, C. (Eds.), Mathematical Statistics and Applications: Festschrift for C. VanEeden Lecture Notes Monograph Series, vol. 42. Institute of Mathematical Statistics, Hayward, pp. 31-44.
27. Rissanen J. Universal coding, information, prediction and estimation. Transactions on Information Theory 30 (1984) 629-636
28. Rissanen J. Fisher information and stochastic complexity. Transactions on Information Theory 42 (1996) 40-47
29. Rissanen J., Speed T., and Yu B. Density estimation by stochastic complexity. Transactions on Information Theory 38 (1992) 315-323
30. Shtarkov Y. Universal sequential coding of single messages. Transactions from Problems in Information Transmission 23 (1988) 3-17
31. Soofi E.S. Capturing the intangible concept of information. Journal of the American Statistical Association 89 (1994) 1243-1254
32. Soofi E.S., Ebrahimi N., and Habibullah M. Information distinguishability with application to analysis of failure data. Journal of the American Statistical Association 90 (1995) 657-668
33. Sun D., and Berger J.O. Reference priors with partial information. Biometrika 85 (1998) 55-71
34. Tobias, J., Zellner, A., 1998. Further results on BMOM analysis of the multiple regression model. See: 〈http://www-gsb.uchicago.edu/fac/arnold.zellner/more/CURRENT-PAPERS/paper.ps〉.
35. Wasserman L.A. Asymptotic inference for mixture models using data-dependent priors. Journal of the Royal Statistical Society Series B 61 (2000) 159-180
36. Xie Q., and Barron A.R. Minimax redundancy for the class of memoryless sources. Transactions on Information Theory 43 (1997) 646-657
37. Xie Q., and Barron A.R. Asymptotic minimax regret for data compression, gambling, and prediction. Transactions on Information Theory 46 (2000) 431-445
38. Yuan A., and Clarke B. An information criterion for likelihood selection. Transactions on Information Theory 45 (1999) 562-571
39. Yuan A., and Clarke B. A minimally informative likelihood for decision analysis and robustness. Canadian Journal Statistics 27 (1999) 649-665
40. Zellner, A., Tobias, J., Ryu, H., 1997. BMOM Analysis of parametric and semi-parametric regression models. See: 〈http://www-gsb.uchicago.edu/fac/arnold.zellner/more/CURRENT-PAPERS/paper2.ps〉.