A signal-flow-graph approach to on-line gradient calculation

Neural Computation

Volumn 12, Issue 8, 2000, Pages 1901-1927

  Campolucci, Paolo ^a   Uncini, Aurelio ^a   Piazza, Francesco ^a  

^a UNIVERSITÀ POLITECNICA DELLE MARCHE   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; FEEDBACK SYSTEM; INFORMATION PROCESSING; NONLINEAR SYSTEM;

ALGORITHMS; AUTOMATIC DATA PROCESSING; FEEDBACK; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS;

EID: 0034241899     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976600300015196     Document Type: Article

References (32)

1. Almeida, L. B. (1987). A learning rule for asynchronous perceptrons with feedback in combinatorial environment. In Proc. Int. Conf. on Neural Networks (Vol. 2, pp. 609-618).
2. Back, A. D., & Tsoi, A. C. (1991). FIR and IIR synapses, a new neural network architecture for time series modelling. Neural Computation, 3, 375-385.
3. Beaufays, F., & Wan, E. (1994). Relating real-time backpropagation and backpropagation-through-time: An application of flow graph interreciprocity. Neural Computation, 6, 296-306.
4. Campolucci, P. (1998). A circuit theory approach to recurrent neural network architectures and learning methods. Doctoral dissertation in English, University of Bologna, Italy. PDF available online at http://nnsp.eealab.unian.it/ campolucci_P or requested from campoluc@tiscalinet.it.
5. Campolucci, P., Marchegiani, A., Uncini, A., & Piazza, F. (1997). Signal-flow-graph derivation of on-line gradient learning algorithms. In Proc. ICNN-97, IEEE Int. Conference on Neural Networks (Houston, TX).
6. Campolucci, P., Piazza, F., & Uncini, A. (1995). On-line learning algorithms for neural networks with IIR synapses. In Proc. IEEE International Conference of Neural Networks (Perth).
7. Campolucci, P., Uncini, A., & Piazza, F. (1998). Dynamical systems learning by a circuit theoretic approach. Proc. ISCAS-98, IEEE Int. Symposium on Circuits and Systems.
8. Campolucci, P., Uncini, A., Piazza, F., & Rao, B. D. (1999). On-line learning algorithms for locally recurrent neural networks. IEEE Trans. on Neural Networks, 10, 253-271.
9. Director, S. W., & Rohrer, R. A. (1969). The generalized adjoint network and network sensitivities. IEEE Trans. on Circuit Theory, CT-16, 318-323.
10. Gherrity, M. (1989). A learning algorithm for analog, fully recurrent neural networks. In Proc. Int. Joint Conference Neural Networks, (Vol. 1, pp. 643-644).
11. Haykin, S. (1994). Neural networks: A comprehensive foundation. New York: IEEE Press-Macmillan.
12. Horne, B. G., & Giles, C. L. (1995). An experimental comparison of recurrent neural networks. In G. Tasauro, D. Touretzky, & T. Leen (Eds.), Advances in neural information processing systems, 7 Cambridge, MA: MIT Press.
13. Lee, A. Y. (1974). Signal flow graphs - Computer-aided system analysis and sensitivity calculations. IEEE Transactions on Circuits and Systems, cas-21, 209-216.
14. Martinelli, G., & Perfetti, R. (1991). Circuit theoretic approach to the backpropagation learning algorithm. Proc. IEEE Int. Symposium on Circuits and Systems.
15. Mason, S. J. (1953). Feedback theory - Some properties of signal-flow graphs. Proc. Institute of Radio Engineers, 41, 1144-1156.
16. Mason, S. J. (1956). Feedback theory - Further properties of signal-flow graphs. Proc. Institute of Radio Engineers, 44, 920-926.
17. Narendra, K. S., Parthasarathy, K. (1991). Gradient methods for the optimization of dynamical systems containing neural networks. IEEE Trans. on Neural Networks, 2, 252-262.
18. Nerrand, O., Roussel-Ragot, P., Personnaz, L., Dreyfus, G., & Marcos, S. (1993). Neural networks and nonlinear adaptive filtering: Unifying concepts and new algorithms. Neural Computation, 5 165-199.
19. Oppenheim, A. V., & Schafer, R. W. (1975). Digital signal processing. Englewood Cliffs, NJ: Prentice Hall.
20. Osowski, S. (1994). Signal flow graphs and neural networks. Biological Cybernetics, 70, 387-395.
21. Pearlmutter, B. A. (1995). Gradient calculations for dynamic recurrent neural networks: A survey. IEEE Trans. on Neural Networks, 6, 1212-1228.
22. Penfield, P., Spence, R., & Duiker, S. (1970). Tellegen's theorem and electrical networks. Cambridge, MA: MIT Press.
23. Srinivasan, B., Prasad, U. R., & Rao, N. J. (1994). Backpropagation through adjoints for the identification of non linear dynamic systems using recurrent neural models. IEEE Trans. on Neural Networks, 5, 213-228.
24. Tellegen, B. D. H. (1952). A general network theorem, with applications. Philips Res. Rep., 7, 259-269.
25. Tsoi, A. C., & Back, A. D. (1994). Locally recurrent globally feedforward networks: A critical review of architectures. IEEE Transactions on Neural Networks, 5, 229-239.
26. Uncini, A., Vecci, L., Campolucci, P., & Piazza, F. (1999). Complex-valued neural networks with adaptive spline activation function for digital radio links nonlinear equalization. IEEE Transactions on Signal Processing, 47, 505-514.
27. Wan, E. A., & Beaufays, F. (1996). Diagrammatic derivation of gradient algorithms for neural networks. Neural Computation, 8, 182-201.
28. Wan, E. A., & Beaufays, F. (1998). Diagrammatic methods for deriving and relating temporal neural networks algorithms. In M. Gori & C. L. Giles (Eds.), Adaptive processing of sequences and data structures. Berlin: Springer-Verlag.
29. Werbos, P. J. (1990). Backpropagation through time: What it does and how to do it. Proc. of IEEE, 78, 1550-1560.
30. Williams, R. J., & Peng, J. (1990). An efficient gradient-based algorithm for on line training of recurrent network trajectories. Neural Computation, 2, 490-501.
31. Williams, R. J., & Zipser, D. (1989). A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1, 270-280.
32. Williams, R. J., & Zipser, D. (1994). Gradient-based learning algorithms for recurrent networks and their computational complexity. In Y. Chauvin & D. E. Rumelhart (Eds.), Backpropagation: Theory, architectures and applications (pp. 433-486). Hillsdale, NJ: Erlbaum.