A note on the equivalence of NARX and RNN

Neural Computing and Applications

Volumn 8, Issue 1, 1999, Pages 33-39

  Sum, John P F ^a   Kan, W K ^b   Young, G H ^c  

^a HONG KONG BAPTIST UNIVERSITY   (Hong Kong)

^b CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^c HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

Author keywords

Extended Kalman filter; Model equivalence; NARX model; Output feedback control; Pruning; Recurrent neural networks; Recursive least square; Training complexity

Indexed keywords

EID: 0033242068     PISSN: 09410643     EISSN: None     Source Type: Journal    
DOI: 10.1007/s005210050005     Document Type: Article

References (34)

1. Chen S, Billings SA, Grant PM. Non-linear system identification using neural networks. Int J Control 1990; 51(6): 1191-1214
2. Chen S, Cowan C, Billings SA, Grant PM. Parallel recursive prediction error algorithm for training layered neural networks. Int J Control 1990; 51(6): 1215-1228
3. Lin T, Lee Giles C, Horne BG, Kung SY. A delay damage model selection algorithm for NARX neural networks. IEEE Trans Signal Processing, Special Issue on 'Neural Networks for Signal Processing' 1997; 45(11): 2719
4. Narendra KS, Parthasarathy K. Neural networks and dynamical systems. Int J Approximate Reasoning 1992; 6: 109-131
5. Siegelmann HT, Horne BG, Lee Giles C. Computational capabilities of recurrent NARX neural networks. IEEE Trans Systems, Man and Cybernetics - Part B: Cybernetics 1997; 27(2): 208
6. Rumelhart DE et al. Learning internal representations by error propagation. In Parallel Distributed Processing Volume 1: Foundations, DE Rumelhart et al. (ed), MIT Press, 1986, pp 318-362
7. Jin L et al. Absolute stability conditions for discrete-time recurrent neural network. IEEE Trans Neural Networks 1994; 5(6): 954-964
8. Jin L et al. Approximation of discrete-time stage-space trajectories using dynamic recurrent neural networks. IEEE Trans Automatic Control 1995; 40(7): 1266-1270
9. Jin L, Gupta MM. Globally asymptotical stability of discrete-time analog neural network. IEEE Trans Neural Networks 1996; 7(4): 1024-1031
10. With Pedersen M, Hansen LK. Recurrent networks: Second order properties and pruning. Advances in Neural Information Processing Systems 7 (G. Tesauro et al. (ed), MIT Press, 1995, pp 673-680
11. Puskorius GV, Feldkamp LA. Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks. IEEE Trans Neural Networks 1994; 5(2): 279-297
12. Sontag ED. Recurrent neural networks: Some systems-theoretic aspects. In: Dealing with Complexity: a Neural Network Approach (M Karny, K Warwick, V Kurkova, eds.), Springer-Verlag, London, 1998, pp 1-11
13. Williams RJ. Training recurrent networks using the extended Kalman filter. Proc IJCNN'92, Baltimore, Vol IV, 1992 pp 241-246.
14. Connor JT, Martin D, Atlas LE. Recurrent neural networks and robust time series prediction. IEEE Trans Neural Networks 1994; 5(2): 240-254
15. Olurotimi O. Recurrent neural network training with feedforward complexity. IEEE Trans Neural Networks 1994; 5(2): 185-197
16. Cybenko G. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Syst 1989; 2: 303-314
17. Funahashi K. On the approximate realization of continuous mappings by neural networks. Neural Networks 1989; 2(3): 183-192
18. Sontag ED. Neural network for control. In Essays on Control: Perspectives in the Theory and its Applications (HL Trentelman, JC Willems, eds), Birkhauser, Boston, 1993, pp 339-380
19. Sum J, Chan LW. On the approximation property of recurrent neural network. Proc World Multiconference on Systemics, Cybernetics and Informatics, Caracas, Venezuela, July 7-11 1997
20. Leung CS et al. On-line training and pruning for RLS algorithms. Elec Lett 1996; 7: 2152-2153
21. Puskorius GV, Feldkamp LA. Decoupled extended Kalman filter training of feedforward layered networks. Proc IJCNN'91, Vol I, 1991, pp 771-777
22. Sum JPF et al. Extended Kalman filter in recurrent neural network training and pruning. Trechnical report CS-TR-96-05, Department of Computer Science and Engineering, CUHK, June 1996
23. Suykens J, De Moor B, Vandewalle J. Nonlinear system identification using neural state space models, applicable to robust control design. Int J Control 1995; 82(1): 129-152
24. Sum J et al. Extended Kalman filter-based pruning method for recurrent networks. Neural Computation 1998; 10(6): 1481-1505
25. With Pedersen et al. Pruning with generalization based weight saliencies: γOBD and γOBS. Advances in Neural Information Processing Systems 8 (DS Touretzky et al. (eds), MIT Press, 1996
26. LeCun Y et al. Optimal brain damage. Advances in Neural Information Processing Systems 2 (DS Touretzky, ed.) 1990, pp 396-404
27. Reed R. Pruning algorithms - A survey. IEEE Trans Neural Networks 1993; 4(5): 740-747
28. Hassibi B, Stork DG. Second order derivatives for network pruning: Optimal brain surgeon. In: Hanson et al. (eds) Advances in Neural Information Processing Systems 1993, pp 164-171
29. Moody J. Prediction risk and architecture selection for neural networks. In From Statistics to Neural Networks: Theory and Pattern Recognition Application, V. Cherkassky et al. (eds.), Springer-Verlag, 1994
30. Cottrell M et al. Neural modeling for time series: A statistical stepwise method for weight elimination. IEEE Trans Neural Networks 1995; 6(6): 1355-1362.
31. Finnoff W, Hergert F, Zimmermann HG. Improving model selection by nonconvergent methods. Neural Networks 1993; 6: 771-783
32. Ku C, Lee K. Diagonal recurrent neural networks for dynamic systems control. IEEE Trans Neural Networks 1995; 6(1): 144-156
33. Narendra KS, Parthasarathy K. Identification and control of dynamical systems using neural networks. IEEE Trans Neural Networks 1990; 1(1): 4-27
34. Shah S et al. Optimal filtering algorithms for fast learning in feedforward neural networks. Neural Networks. 1992; 5: 779-787