An efficient EM-based training algorithm for feedforward neural networks

Neural Networks

Volumn 10, Issue 2, 1997, Pages 243-256

  Ma, Sheng ^a   Ji, Chuanyi ^b   Farmer, James ^b  

^a RENSSELAER POLYTECHNIC INSTITUTE   (United States)

^b NONE

Author keywords

EM algorithm; fast training; feedforward networks; hidden representation; linear weighted regression; probability model of feedforward networks

Indexed keywords

FEEDFORWARD NEURAL NETWORKS; LEARNING SYSTEMS; MATHEMATICAL MODELS; PROBABILITY; REGRESSION ANALYSIS;

HIDDEN REPRESENTATION; LINEAR WEIGHTED REGRESSION; TRAINING ALGORITHM;

LEARNING ALGORITHMS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; PRIORITY JOURNAL; TRAINING;

EID: 0031105693     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(96)00049-4     Document Type: Article

References (21)

1. Aman, S. (1995). Information geometry of EM algorithm. Neural Computation, 7, 13-18.
2. Amari, S., Kurata, K., & Nagaoka, H. (1992). Information geometry of Boltzmann machines. IEEE Transactions on Neural Networks, 3, 260-277.
3. Barron, A. (1993). Universal approximation bounds for artificial neural networks. IEEE Transactions on Information Theory, 39, 930-944.
4. Breiman, L. E., & Friedman, J. H. (1993). Function approximation using ramps. Neural Networks for computing. Utah: Snowbird.
5. Byrne, W. (1992). Alternating minimization and Boltzmann machine learning. IEEE Transactions on Neural Networks, 3, 612-620.
6. Cybenko, G. (1989). Approximation by superpositions of a sigmoid function. Mathematics, Control Signals and Systems, 2, 303-314.
7. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via EM algorithm, Journal of Royal Statistical Society, B39, 1-33.
8. Fahlman, S. E., & Lebiere, C. (1990). The cascade-correlation learning architecture. In: D. Touretzky (Ed.), Advances in neural information processing systems 2 (pp. 524-532). San Mateo: Morgan Kaufmann.
9. Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721-741.
10. Golden, R. M. (1988). A unified framework for connectionist systems. Biological Cybernetics, 59, 109-120.
11. Grossman, T. (1992). Learning, capacity and implementation of neural network models. PhD Thesis, Weizmann Institute of Science.
12. Jordan, M., & Jacobs, R. A. (1994). Hierarchical mixture of experts and the EM algorithm. Neural Computation, 6, 181-214.
13. Krough, A., Thorberaso, G. I., & Hertz, J. A. (1990). A cost function for internal representations. In: D. Touretzky (Ed.), Advances in neural information processing systems 2 (pp. 733-745). San Mateo, CA: Morgan Kaufmann.
14. Levin, E., Tishby, N., & Solla, S. A. (1990). A statistical approach to learning and generalization in layered neural network. Proceedings of the IEEE, 78, 1568-1574.
15. Mackay, D. J. (1992). A practical Bayesian framework for backpropagation networks. Neural Computation, 4, 448-472.
16. Nadas, A. (1995). Binary classification by stochastic neural nets. IEEE Transactions on Neural Networks, 6, 488-491.
17. Nguyen, D., & Widrow, B. (1990). Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights. Proceedings of International Joint Conference on Neural Networks, Vol. 3, San Diego, pp. 21-26. New York: IEEE.
18. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In: D. E. Rumelhart, J. L. McClelland and PDP Reserch Group (Eds.) Parallel distributed processing: explorations in the microstructure of cognition. Vol. 1: Foundations. Cambridge, Mass: MIT Press.
19. Saad, D., & Marom, E. (1990). Learning by choice of internal representations: an energy minimization approach. Complex systems, 4, 107-118.
20. Shadafan, R. S., & Niranjan, M. (1994). A dynamic neural network architecture by sequential partitioning of the input space. Neural Computation, 6, 1203-1222.
21. White, H. (1989). Learning in artificial neural networks: a statistical perspective. Neural Computation, 1, 425-464.