Nonlinear backpropagation: Doing backpropagation without derivatives of the activation function

IEEE Transactions on Neural Networks

Volumn 8, Issue 6, 1997, Pages 1321-1327

  Hertz, John ^a   Krogh, Anders ^b   Lautrup, Benny ^c   Lehmann, Torsten ^b  

^a NORDITA   (Denmark)

^b TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

^c UNIVERSITY OF COPENHAGEN   (Denmark)

Author keywords

Backpropagation; Neural networks; Neural network implementation; Recurrent backpropagation

Indexed keywords

COMPUTER SIMULATION; FEEDFORWARD NEURAL NETWORKS; LEARNING ALGORITHMS; MATHEMATICAL MODELS; NONLINEAR EQUATIONS; RECURSIVE FUNCTIONS; VLSI CIRCUITS;

ACTIVATION FUNCTION; NONLINEAR BACKPROPAGATION; RECURRENT BACKPROPAGATION;

BACKPROPAGATION;

EID: 0031274112     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.641455     Document Type: Article

References (14)

1. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by backpropagating errors," Nature, vol. 323, pp. 533-536, 1986.
2. L. B. Almeida, "A learning rule for asynchronous perceptrons with feedback in a combinatorial environment," in IEEE Int. Conf. Neural Networks, M. Caudill and C. Butler, Eds. San Diego, CA, vol. 2, 1987, pp. 609-618.
3. F. J. Pineda, "Generalization of backpropagation to recurrent neural networks," Phys. Rev. Letters, vol. 59, pp. 2229-2232, 1987.
4. T. J. Sejnowski and C. R. Rosenberg, "Parallel networks that learn to pronounce English text," Complex Syst., vol. 1, pp. 145-168, 1987.
5. J. A. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computation. Reading, MA: Addison-Wesley, 1991.
6. A. Krogh, G. I. Thorbergsson, and J. A. Hertz, "A cost function for internal representations," in Advances in Neural Inform. Processing Syst., D. S. Touretzky, Ed., Denver, CO, 1989, vol. 2, pp. 733-740.
7. S. A. Solla, E. Levin, and M. Fleisher, "Accelerated learning in layered neural networks," Complex Syst., vol. 2, pp. 625-639, 1988.
8. L. Tarassenko, J. Tombs, and G. Cairns, "On-chip learning with analog VLSI neural networks," Int. J. Neural Syst., vol. 4, no. 4, pp. 419-426, 1993.
9. T. Lehmann and E. Bruun, "Analog VLSI implementation of backpropagation learning in artificial neural networks," in Proc. 11th European Conf. Circuit Theory and Design, 1993, pp. 491-496.
10. T. Shima, T. Kimura, Y. Kamatani, T. Itakura, Y. Fujita, and T. Iida, "Neuro chips with on-chip backpropagation and/or Hebbian learning," IEEE J. Solid-State Circuits, vol. 27, pp. 1868-1876, 1992.
11. G. Bogason, "Generation of a neuron transfer function and its derivative," Electron. Lett., vol. 29, pp. 1867-1869, 1993.
12. T. Morie and Y. Amemiya, "An all-analog expandable neural-network LSI with on-chip backpropagation learning," IEEE J. Solid-State Circuits, vol. 29, pp. 1086-1093, 1994.
13. A. J. Montalvo, P. W. Hollis, and J. J. Paulos, "On-chip learning in the analog domain with limited precision circuits," in Int. Symp. Circuits Syst., 1992, pp. I-196-I-201.
14. S. R. Skinner, J. E. Steck, and E. C. Behrman, "Optical neural-network using Kerr-type nonlinear materials," in 4th Int. Conf. Microelectron. Neural Networks Fuzzy Syst., 1994, pp. 12-15.