Statistical inference, Occam's razor, and statistical mechanics on the space of probability distributions

Neural Computation

Volumn 9, Issue 2, 1997, Pages 349-368

  Balasubramanian, Vijay ^a  

^a PRINCETON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0000810448     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/neco.1997.9.2.349     Document Type: Article

References (23)

1. Amari, S. I. 1985. Differential Geometrical Methods in Statistics. Springer-Verlag, Berlin.
2. Amari, S. I., Barndorff-Nielsen, O. E., Kass, R. E., Lauritzen, S. L., and Rao, C. R. 1987. Differential Geometry in Statistical Inference, vol. 10. Lecture Note-Monograph Series. Institute of Mathematical Statistics, Hayward, CA.
3. Atick, J. J. 1991. Could information theory provide an ecological theory of sensory processing? In Princeton Lectures in Biophysics, W. Bialek, ed., pp. 223-291. World Scientific, Singapore.
4. Balasubramanian, V. 1996. A geometric formulation of Occam's razor for inference of parameteric distributions. Available as preprint number adaporg/9601001 from http://xyz.lanl.gov/ and as Princeton University Physics Preprint PUPT-1588.
5. Barron, A. R. 1985. Logically smooth density estimation. Ph.D. thesis, Stanford University.
6. Barron, A. R., and Cover, T. M. 1991. Minimum complexity density estimation. IEEE Transactions on Information Theory 37(4), 1034-1054.
7. Bialek, W. 1991. Optimal signal processing in the nervous system. In Princeton Lectures in Biophysics, W. Bialek, ed., pp. 321-401. World Scientific, Singapore.
8. Clarke, B. S., and Barron, A. R. 1990. Information-theoretic asymptotics of Bayes methods. IEEE Transactions on Information Theory 36(3), 453-471.
9. Cover, T. M., and Thomas, J. A. 1991. Elements of Information Theory. Wiley, New York.
10. Itzykson, C., and Drouffe, J. M. 1989. Statistical Field Theory. Cambridge University Press, Cambridge.
11. Jeffreys, H. 1961. Theory of Probability. 3d ed. Oxford University Press, New York.
12. Lee, P. M. 1989. Bayesian Statistics: An Introduction. Oxford University Press, New York.
13. Ma, S. K. 1985. Statistical Mechanics. World Scientific, Singapore.
14. MacKay, D. J. C. 1992a. Bayesian interpolation. Neural Computation 4(3), 415-447.
15. MacKay, D. J. C. 1992b. A practical Bayesian framework for backpropagation networks. Neural Computation 4(3), 448-472.
16. Moody, J. 1992. The effective number of parameters: An analysis of generalization and regularization in nonlinear learning systems. Proceedings of NIPS 4, 847-854.
17. Murata, N., Yoshizawa, S., and Amari, S. 1994. Network information criterion - determining the number of hidden units for artificial neural network models. IEEE Trans. Neural Networks 5, 865-872.
18. Potters, M., and Bialek, W. 1994. Statistical mechanics and visual signal processing. Journal de Physique I 4(11), 1755-1775.
19. Rissanen, J. 1984. Universal coding, information, prediction and estimation. IEEE Transactions on Information Theory 30(4), 629-636.
20. Rissanen, J. 1986. Stochastic complexity and modelling. Annals of Statistics 14(3), 1080-1100.
21. de Ruyter van Steveninck, R. R., Bialek, W., Potters, M., Carlson, R. H., and Lewen, G. D. 1995. Adaptive movement computation by the blowfly visual system. Princeton factures in Biophysics.
22. Wallace, C. S., and Freeman, P. R. 1987. Estimation and inference by compact coding. Journal of the Royal Statistical Society 49(3), 240-265.
23. Yamanishi, K. 1995. A decision-theoretic extension of stochastic complexity and its applications to learning. Submitted for publication.