The Interchangeability of Learning Rate and Gain in Backpropagation Neural Networks

Neural Computation

Volumn 8, Issue 2, 1996, Pages 451-460

  Thimm, Georg ^a   Moerland, Perry ^a   Fiesler, Emile ^a  

^a IDIAP RESEARCH INSTITUTE   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; KINETICS;

ALGORITHMS; KINETICS; NEURAL NETWORKS (COMPUTER);

EID: 0030584163     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/neco.1996.8.2.451     Document Type: Article

References (23)

1. Bachmann, C. M. 1990. Learning and generalization in neural networks. Ph.D. thesis, Department of Physics, Brown University, Providence, RI.
2. Brown, M., An, P. C., Harris, C. J., and Wang H. 1993. How biased is your multi-layer perceptron. World Congress Neural Networks 3, 507-511.
3. Brown, M., and Harris, C. 1994. Neurofuzzy adaptive modelling and control. In Prentice Hall International Series in Systems and Control Engineering, M. J. Grimble, ed. Prentice-Hall, Englewood Cliffs, NJ.
4. Cho, T.-H., Conners, R. W., and Araman, P. A. 1991. Fast back-propagation learning using steep activation functions and automatic weight reinitialization. Proc. 1991 IEEE Int. Conf. Systems, Man, Cybernetics: Decision Aiding for Complex Syst. 3, 1587-1592.
5. Codrington, C., and Tenorio, M. 1994. Adaptive gain networks. Proc. IEEE Int. Conf. Neural Networks (ICNN94), 1, 339-344.
6. Corwin, E., Logar, A., and Oldham, W. 1994. An iterative method for training multilayer networks with threshold functions. IEEE Trans. Neural Networks 5(3), 507-508.
7. Fahlman, S. E. 1988. An Empirical Study of Learning Speed in Backpropagation Networks. Tech. Rep. CMU-CS-88-162, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA.
8. Fiesler, E. 1994. Neural network classification and formalization. In Computer Standards & Interfaces, special issue on Neural Network Standardization, J. Fulcher, ed., Vol. 16, No. 3, pp. 231-239. North-Holland/Elsevier Science Publishers, Amsterdam, The Netherlands.
9. Fiesler, E., Choudry, A., and Caulfield, H. J. 1996. A universal weight discretization method for multi-layer neural networks. IEEE Transactions on Systems, Man, and Cybernetics (IEEE-SMC) (conditionally accepted for publication). See also Fiesler, E., Choudry, A., and Caulfield, H. J. 1990. A weight discretization paradigm for optical neural networks. Proc. Int. Cong. Optical Sci. Eng. SPIE 1281, 164-173.
10. Fiesler, E., Choudry, A., and Caulfield, H. J. 1996. A universal weight discretization method for multi-layer neural networks. IEEE Transactions on Systems, Man, and Cybernetics (IEEE-SMC) (conditionally accepted for publication). See also Fiesler, E., Choudry, A., and Caulfield, H. J. 1990. A weight discretization paradigm for optical neural networks. Proc. Int. Cong. Optical Sci. Eng. SPIE 1281, 164-173.
11. Izui, Y., and Pentland, A. 1990. Analysis of neural networks with redundancy. Neural Comp. 2, 226-238.
12. Jia, Q., Hagiwara, K., Toda, N., and Usui, S. 1994. Equivalence relation between the backpropagation learning process of an FNN and that of an FNNG. Neural Networks 7(2), 411.
13. Kruschke, J. K., and Movellan, J. R. 1991. Benefits of gain: Speeded learning and minimal hidden layers in backpropagation networks. IEEE Trans. Syst. Man Cybern. 21(1), 273-280.
14. Mundie, D. B., and Massengill, L. W. 1992. Threshold non-linearity effects on weight-decay tolerance in analog neural networks. Proc. Int. Joint Conf. Neural Networks (IJCNN92) 2, 583-587.
15. Plaut, D., Nowlan, S., and Hinton, G. 1986. Experiments on Learning by Back Propagation. Tech. Rep. CMU-CS-86-126, Carnegie Mellon University, Pittsburgh.
16. Rumelhart, D., Hinton, G., and Williams, R. 1986. Learning internal representations by error propagation. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1: Foundations, pp. 318-362. MIT Press, Cambridge, MA.
17. Saxena, I., and Fiesler, E. 1995. Adaptive multilayer optical neural network with optical thresholding. In Optical Engineering, special on Optics in Switzerland, P. Rastogi, ed., Vol. 34(8), pp. 2435-2440.
18. Sperduti, A., and Starita, A. 1993. Speed up learning and network optimization with extended backpropagation. Neural Networks 6, 365-383.
19. Thimm, G., and Fiesler, E. 1996. Weight initialization for high order and multilayer perceptrons. IEEE Trans. Neural Networks. (conditionally accepted for publication). See also Thimm, G., and Fiesler, E. 1994. Weight initialization for high order and multilayer perceptrons. In Proceedings of the '94 SIPAR-Workshop on Parallel and Distributed Computing, M. Aguilar, ed., pp. 91-94. Institute of Informatics, University Pérolles, Chemin du Musée 3, Fribourg, Switzerland. SI Group for Parallel Systems.
20. Thimm, G., and Fiesler, E. 1996. Weight initialization for high order and multilayer perceptrons. IEEE Trans. Neural Networks. (conditionally accepted for publication). See also Thimm, G., and Fiesler, E. 1994. Weight initialization for high order and multilayer perceptrons. In Proceedings of the '94 SIPAR-Workshop on Parallel and Distributed Computing, M. Aguilar, ed., pp. 91-94. Institute of Informatics, University Pérolles, Chemin du Musée 3, Fribourg, Switzerland. SI Group for Parallel Systems.
21. Wessels, L. F. A., and Barnard, E. 1992. Avoiding false local minima by proper initialization of connections. IEEE Trans. Neural Networks 3, 899-905.
22. Yu, X., Loh, N., and Miller, W. 1994. Training hard-limiting neurons using backpropagation algorithm by updating steepness factors. Proc. IEEE Int. Conf. Neural Networks (ICNN94) 1, 526-530.
23. Zurada, J. M. 1992. Introduction to Artificial Neural Systems. West Publishing Company, St. Paul, MN.