Information geometry of the EM and em algorithms for neural networks

Neural Networks

Volumn 8, Issue 9, 1995, Pages 1379-1408

  Amari, Shun ichi ^a  

^a RIKEN   (Japan)

Author keywords

e Projection; EM algorithm; Hidden variable; Identification of neural network; Information geometry; Learning; m Projection; Stochastic model of neural networks

Indexed keywords

ALGORITHMS; COMPUTER SIMULATION; EQUIVALENCE CLASSES; IDENTIFICATION (CONTROL SYSTEMS); ITERATIVE METHODS; LEARNING SYSTEMS; RANDOM PROCESSES; STATISTICAL METHODS;

HIDDEN VARIABLES; INFORMATION GEOMETRY; INPUT OUTPUT RELATIONSHIP; ITERATIVE STATISTICAL TECHNIQUE; STOCHASTIC MODEL;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; LEARNING; PRIORITY JOURNAL;

EID: 0029587111     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/0893-6080(95)00003-8     Document Type: Article

References (53)

1. Differential geometry of curved exponential families—curvatures and information loss
2. Differential geometrical methods in statistics
3. Differential geometry of a parametric family of invertible linear systems—Riemannian metric, dual affine connections and divergence
4. Differential geometrical theory of statistics
5. Fisher information under restriction of Shannon information in multiterminal situations
6. Mathematical foundations of neurocomputing
7. Dualistic geometry of the manifold of higher-order neurons
8. The EM algorithm and information geometry in neural network learning
9. Differential geometry in statistical inferences
10. Statistical inference under multiterminal rate restrictions—a differential geometrical approach
11. Information geometry of estimating functions in semiparametric statistical models
12. Information geometry of Boltzmann machines
13. Smooth on-line learning algorithm for hidden Markov models
14. Parametric statistical model and likelihood
15. The role of differential geometry in statistical theory
16. Approximating exponential models
17. A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains
18. Spatial statistics and Bayesian computation
19. Alternating minimization and Boltzmann machine learning
20. Neural networks—a review from statistical perspectives—comments and rejoinders
21. I-divergence geometry of probability distributions and minimization problems
22. Information geometry and alternating minimization procedures
23. Maximum likelihood from incomplete data via the EM algorithm
24. Dualistic dynamical systems in the framework of information geometry
25. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images
26. Noisy reinforcement training for pRAM nets
27. Identifiability of hidden Markov information sources and their minimum degrees of freedom
28. Higherarchial mixtures of experts an the EM-algorithm
29. Convergence results for the EM approach to mixtures of experts architectures
30. Adaptive mixtures of local experts
31. The geometry of asymptotic inference (with discussions)
32. Estimation of network parameters in semiparametric stochastic perceptron
33. Dual connections and affine geometry
34. Enhanced MLP performance and fault tolerance resulting from synaptic weight noise during training
35. Differential geometry of smooth families of probability distributions
36. Differential geometric structures of stable feedback systems with dual connections
37. Asymptotic theory of sequential estimation Differential geometrical approach
38. Information and accuracy attainable in the estimation of statistical parameters
39. Neural networks and related method for classification
40. A new criterion for selecting models from partially observed data
41. Maximum likelihood training of probabilistic neural networks
42. Learning in artificial networks A statistical perspective
43. Statistical physics mixtures of distributions and the EM algorithm
44. Piecewise-linear division of signal space by a multilayer neural networks with the maximum detector