Information Geometry of Boltzmann Machines

IEEE Transactions on Neural Networks

Volumn 3, Issue 2, 1992, Pages 260-271

  Amari, Shunichi ^a   Kurata, Koji ^b   Nagaoka, Hiroshi ^c  

^a UNIVERSITY OF TOKYO   (Japan)

^b OSAKA UNIVERSITY   (Japan)

^c HOKKAIDO UNIVERSITY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING SYSTEMS; MATHEMATICAL TECHNIQUES - GEOMETRY; PROBABILITY - RANDOM PROCESSES;

BOLTZMANN MACHINES; INFORMATION GEOMETRY; STOCHASTIC NEURONS;

NEURAL NETWORKS;

EID: 0026835133     PISSN: 10459227     EISSN: 19410093     Source Type: Journal    
DOI: 10.1109/72.125867     Document Type: Article

References (23)

1. S. Amari, N. Fujita, and S. Shinomoto, “Four types of learning curves,” Neural Computation, to appear.
2. D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, “A learning algorithm for Boltzmann machines,” Cognitive Science, vol. 9, pp. 147–169, 1985.
3. e. Aarts and J. Korst, Simulated Annealing and Boltzmann Machines. New York: Wiley, 1989.
4. S. Amari, “Differential geometry of curved exponential families—Curvatures and information loss,” Annals of Statistics, vol. 10, no. 2, pp. 357–385, 1982.
5. S. Amari, Differential-Geometrical Methods in Statistics ‘ Springer Lecture Notes in Statistics vol. 28’. New York: Springer, 1985.
6. H. Nagaoka and S. Amari, “Differential geometry of smooth families of probability distributions,” METR 82-7, Univ. Tokyo, 1982.
7. S. Amari, O. E. Barndorff-Hielsen, R. E. Kass, S. L. Lauritzen, and C. R. Rao, Differential Geometry in Statistical Inferences ‘IMS Lecture Notes Monograph Series, vol. 10’. IMS CA: Hayward, 1987.
8. N.N. Chentsov, Optimal Decission Rules and Optimal Inference ‘in Russian’. Moscow: Nauka, 1972. Translated into English, AMS: Rhode Island, 1982.
9. M. Kumon and S. Amari, “Geometrical theory of higher-order asymptotics of test, interval estimator and conditional inference,” in Proc. R. Soc. London, A387, pp. 429–458, 1983.
10. S. Amari and M. Kumon, “Estimation in the presence of infinitely many nuisance parameters—Geometry of estimating functions,” Annals of Statistics, vol. 16, pp. 1044–1068, 1988.
11. I. Okamoto, S. Amari and K. Takeuchi, “Asymptotic theory of sequential estimation: Differential geometrical approach,” Annals of Statistics, vol. 19, pp. 961–981, 1991.
12. S. Amari “Differential geometry of a parametric family of invertible linear systems—Riemannian metric, dual affine connections and divergence,” Mathematical Systems Theory, vol. 20, pp. 53-82, 1987.
13. S. Amari and T. S. Han, “Statistical inference under multiterminal rate restrictions—A differential geometrical approach,” IEEE Trans. Inform. Theory, vol. IT-35, pp. 217–227, 1989.
14. S. Amari, “Fisher information under restriction of Shannon information in multiterminal situations,” Ann. Inst. Statist. Math. 41, no. 4, pp. 623–648, 1989.
15. S. Geman and D. Geman, “Stochastic relaxation, Gibbs distribution, and the Bayestan restoration of images,” IEEE Trans., Pattern Anal. Machine Intell., vol. PAMI-6, pp. 721–741, 1984.
16. A. Amit, Modeling Brain Function. Cambridge University Press, 1989.
17. E. Levin, N. Tishby and S. A. Solla, “A statistical approach to learning and generalization in layered neural networks,” Proc. IEEE, vol. 78, pp. 1568–1574, 1990.
18. H. Seung and H. E. Sompolinski, “Statistical mechanics of learning from examples,” Phys. Rev. A, to appear.
19. S. Amari, “Mathematical foundations of neurocomputing,” Proc. IEEE, vol. 78, pp. 1443–1463, 1990.
20. S. Amari, “Dualistic geometry of the manifold of higher-order neurons,” Neural Networks, vol. 4, pp. 443–451, 1991.
21. C. R. Rao, “Information and accuracy attainable in the estimation of statistical parameters,” Bull. Calcutta Math. Soc., vol. 37, pp. 81–91, 1945.
22. O. Barndorf-Nielsen, D. R. Cox, and N. Reid, “The role of differential geometry in statistical theory,” Int. Statist. Rev., vol. 54, pp. 83–96, 1986.
23. R. E. Kass, “The geometry of asymptotic inference,” Statistical Science, vol. 4, pp. 188–234, 1989.