Nonlinear Bayesian filters for training recurrent neural networks

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 5317 LNAI, Issue , 2008, Pages 12-33

  Arasaratnam, Ienkaran ^a   Haykin, Simon ^a  

^a MCMASTER UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATE BAYESIAN; BAYESIAN FILTERS; DYNAMIC SYSTEMS; EMPIRICAL RESULTS; LINEAR ESTIMATIONS; NUMERICAL STABILITIES; OPTIMAL PERFORMANCES; SPECIAL EMPHASES;

ARTIFICIAL INTELLIGENCE; BAYESIAN NETWORKS; BIONICS; CONTROL THEORY; KALMAN FILTERS; NEURAL NETWORKS; REINFORCEMENT LEARNING; WAVE FILTERS;

RECURRENT NEURAL NETWORKS;

EID: 57049186443     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-88636-52     Document Type: Conference Paper

References (28)

1. Anderson, B.D.O., Moore, J.B.: Optimal Filtering. Prentice Hall, New Jersey (1979)
2. Arasaratnam, I., Haykin, S.: Cubature Kalman Filters. IEEE Trans. Automatic Cont. (2008) (under 3rd review)
3. Bengio, Y., Simard, P., Frasconi, P.: Learning Long-term Dependencies with Gradient Descent is Difficult. IEEE Trans. Neural Netw. 5(2), 157-166 (1994)
4. Cappe, O., Godsil, S. J., Moulines, E.: An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo. Proc. IEEE 95(5) (2007)
5. Gordon, N.J., Salmond, D.J., Smith, A.F.M.: Novel Approach to Nonlinear/Non-Gaussian Bayesian State Estimation. IEE Proc. -F. 140, 107-113 (1993)
6. Hanson, S. J., Burr, D.J.: Minkowski-r Back-Propagation: Learning in Connectionist Models with Non-Euclidian Error Signals. In: Anderson, D. (ed.) Neural inform. proces, sys., pp. 348-357. American Inst, of Physics, NY (1988)
7. Haykin, S., Li, L.: Nonlinear Adaptive Prediction of Nonstationary Signals. IEEE Trans. Signal Process 43(2), 526-535 (1995)
8. Haykin, S.: Neural Networks: A Comprehensive Foundation. Prentice Hall, New Jersey (1999)
9. Haykin, S. (ed.): Kalman Filtering and Neural Networks. Wiley, New York (2001)
10. Julier, S.J., Ulhmann, J.K., Durrant-Whyte, H.F.: A New Method for Nonlinear Transformation of Means and Covariances in Filters and Estimators. IEEE Trans. Automatic Cont. 45, 472-482 (2000)
11. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-Based Learning Applied to Document Recognition. Proc. IEEE 86(11), 2278-2324 (1998)
12. Liu, J.S.: Monte Carlo Strategies in Scientific Computing. Springer, Heidelberg (2001)
13. Narendra, K.S., Parthasarathy, K.: Identification and Control of Dynamical Systems Using Neural Networks. IEEE Trans. Neural Netw. 1(1), 4-27 (1990)
14. Nørgaard, M.N., Poulsen, N.K., Ravn, O.: New Developments in State Estimation of Nonlinear Systems. Automatica 36, 1627-1638 (2000)
15. Ott, E.: Chaos in Dynamical Systems, 2nd edn. Cambridge Univ. Press, Cambridge (2002)
16. Peterka, V.: Bayesian Approach to System Identification. In: Eykhoff, P. (ed.) Trends and progress in system identification, pp. 239-304. Pergamon Press, Oxford (1981)
17. Pineda, F.: Generalization of Backpropagation to Recurrent Neural Networks. Physical Review Letters 59(19) (1987)
18. Plumer, E.S.: Training Neural Networks Using Sequential Extened Kalman Filters. In: Proc. World Congress on Neural Netw., Washigton, DC, pp. I-764-I-769 (1995)
19. Prokhorov, D.: Training Recurrent Neurocontrollers for Robustness with Derivative-Free Kalman Filter. IEEE Trans. Neural Netw. 17(6), 1606-1616 (2006)
20. Puskorious, G., Feldkamp, L.: Neurocontrol of Nonlinear Dynamical Systems with Kalman Filter Trained Neural Networks. IEEE Trans. Neural Netw. 5(2) (1994)
21. Ruck, D.W., Rogers, S.K., Kabrisky, M., Maybeck, P., Oxle, M.E.: Comparative Analysis of Backpropgation and the Extended Kaiman Filter for Training Multilayer Perceptions. IEEE Trans. Patt. Anal. & Mach. Intell. 14(6), 686-691 (1992)
22. Sarangapani, J.: Neural Network Control of Nonlinear Discrete-Time Systems. CRC Press, Florida (2006)
23. Schottky, B., Saad, D.: Statistical Mechanics of EKF Learning in Neural Networks. J. Physics A 32(9), 1605-1621 (1999)
24. Shah, S., Palmieri, F., Datum, M.: Optimal Filtering Algorithms for Fast Learning in Feedforward Neural Networks. Neural Net. 5(5), 779-787 (1992)
25. Sum, J., Leung, C., Young, G.H., Kan, W.: On the Kalman Filtering Method in Neural Network Training and Pruning. IEEE Trans. Neural Netw. 10(1) (1999)
26. Yamada, T., Yabuta, T.: Dynamic System Identification Using Neural Networks. IEEE Trans. Systems, Man and Cyber. 23(1), 204-211 (1993)
27. Werbos, P.: Backpropagation Through Time: What It Does and How to Do It. Proc. IEEE 78(10), 1550-1560 (1990)
28. West, M., Harrison, J.: Bayesian Forecasting and Dynamic Linear Models. Springer, Heidelberg (1996)