Noisy recurrent neural networks: The continuous-time case

IEEE Transactions on Neural Networks

Volumn 9, Issue 5, 1998, Pages 913-936

  Das, Soumitra ^a   Olurotimi, Oluseyi ^a  

^a Krasnow Institute for Advanced Study   (United States)

Author keywords

Noise; Nonlinear control and identification; Recurrent neural networks; Stochastic dynamical systems; Trajectory learning

Indexed keywords

IDENTIFICATION (CONTROL SYSTEMS); LEARNING SYSTEMS; MATHEMATICAL MODELS; NONLINEAR CONTROL SYSTEMS; RANDOM PROCESSES; SPURIOUS SIGNAL NOISE;

RECURRENT NEURAL NETWORKS;

NEURAL NETWORKS;

EID: 0032164439     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.712164     Document Type: Article

References (44)

1. O. Olurotimi and S. Das, "Noisy Recurrent Neural Networks: The discrete-time case," IEEE Trans. Neural Networks, vol. 9, pp. XXX, 1998.
2. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Networks, vol. 2, pp. 359-366, 1989.
3. _, "Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks," Neural Networks, vol. 3, pp. 551-560, 1990.
4. K.-I. Funahashi and Y. Nakamura, "Approximation of dynamical systems by continuous-time RNN's," Neural Networks, vol. 6, pp. 801-806, 1993.
5. "NIPS'95 workshop: Neural networks for signal processing," in Advances in Neural Information Processing Syst. 7, Dec. 1995.
6. S.-Z. Qin, H.-T. Su, and T. J. McAvoy, "Comparison of four neural net learning methods for dynamic system identification," IEEE Trans. Neural Networks, vol. 3, pp. 122-130, Jan. 1992.
7. C. L. Giles, C. B. Miller, D. Chen, H. H. Chen, G. Z. Sun, and Y. C. Lee, "Learning and extracting finite state automata with second-order recurrent neural networks," Neural Comput., vol. 4, pp. 393-405, 1992.
8. P. J. Werbos, "Generalization of backpropagation with application to a recurrent gas market model," Neural Networks, vol. 1, pp. 339-356, 1988.
9. R. J. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks," Neural Comput., vol. 1, pp. 270-280, 1989.
10. F. J. Pineda, "Generalization of backpropagation to recurrent and higher order neural networks," in Neural Inform. Processing Syst., D. Z. Anderson, Ed. New York: Amer. Inst. Phys., 1988, pp. 602-611.
11. B. A. Pearlmutter, "Learning state-space trajectories in recurrent neural networks," Neural Comput., vol. 1, pp. 263-269, 1989.
12. N. B. Toomarian and J. Barhen, "Learning a trajectory using adjoint functions and teacher forcing," Neural Networks, vol. 5, pp. 473-484, 1992.
13. O. Olurotimi, "RNN training with feedforward complexity," IEEE Trans. Neural Networks, vol. 5, pp. 185-197, Mar. 1994.
14. H. Bersini, M. Saerens, and L. G. Sotelino, "Hopfield net generation, encoding and classification of temporal trajectories," IEEE Trans. Neural Networks, vol. 5, pp. 945-953, Nov. 1994.
15. K. Jim, B. G. Horne, and C. L. Giles, "Effects of noise on convergence and generalization in recurrent networks," in Advances in Neural Inform. Processing Syst. 7. San Mateo, CA: Morgan Kaufmann, 1995, pp. 649-656.
16. R. Reed, R. J. Marks, II, and S. Oh, "Similarities of error regularization, sigmoid gain scaling, target smoothing, and training with jitter," IEEE Trans. Neural Networks, vol. 6, pp. 529-538, May 1995.
17. E. Wong, "Stochastic neural networks," Algorithmica, vol. 6, pp. 466-478, 1992.
18. R. A. Iltis and P.-Y. Ting, "Computing association probabilities using parallel Boltzmann machines," IEEE Trans. Neural Networks, vol. 4, pp. 221-233, Mar. 1993.
19. B. C. Levy and M. B. Adams, "Global optimization with stochastic neural networks," in Proc. 1st IEEE Int. Conf. Neural Networks, June 1987.
20. P.-Y. Ting and R. A. Iltis, "An analog diffusion network architecture for solving data association problems in multitarget tracking," SPIE Proc., vol. 1709, pp. 326-337, Apr. 1992.
21. K. Jim, C. L. Giles, and B. G. Home, "Synaptic noise in dynamically-driven recurrent neural networks: Convergence and generalization," IEEE Trans. Neural Networks, to appear.
22. B. Kosko, "Unsupervised learning in noise," IEEE Trans. Neural Networks, vol. 1, pp. 44-57, Mar. 1990.
23. _, "Structural stability of unsupervised learning in feedback neural networks," IEEE Trans. Automat. Contr., vol. 36, pp. 785-792, July 1991.
24. D. A. Castanon, "A counter example in stochastic stability of feedback neural networks," IEEE Trans. Automat. Contr., vol. 38, pp. 378-380, Feb. 1993.
25. B. Gidas, "Optimization via Langevin equation," in Proc. 24th IEEE Conf. Decision Contr., Dec. 1985.
26. E. B. Kosmatopoulos and M. A. Christodoulou, "Filtering, prediction, and learning properties of ECE neural networks," IEEE Trans. Syst., Man, Cybern., vol. 34, pp. 971-981, July 1994.
27. E. B. Kosmatopoulos and M. A. Christodoulou, "Structural properties of gradient recurrent high-order neural networks," IEEE Trans. Circuits Syst. II, vol. 42, pp. 592-603, Sept. 1995.
28. A. Dembo, O. Olurotimi (Farotimi), and T. Kailath, "High-order absolutely stable neural networks," IEEE Trans. Circuits Syst., vol. 38, Jan. 1991.
29. S. Grossberg, "Nonlinear neural networks: Principles, mechanisms, and architectures," Neural Networks, vol. 1, pp. 17-61, 1988.
30. E. Wong and B. Hajek, Stochastic Processes in Engineering Systems. New York: Springer-Verlag, 1985.
31. D. Rumelhart and J. McClelland, Parallel Distributed Processing #1. Cambridge, MA: MIT Press, 1987.
32. M. A. Cohen and S. Grossberg, "Absolute stability of global pattern formation and parallel memory storage by competitive neural networks," IEEE Trans. Syst., Man, Cybern, vol. SMC-13, pp. 815-826, 1983.
33. I. I. Gibman and A. V. Skorohod, Stochastic Differential Equations. New York: Springer-Verlag, 1972.
34. T. C. Gard, Introduction to Stochastic Differential Equations. New York: Marcel Dekker, 1988.
35. H. Khalil, Nonlinear Systems. Englewood Cliffs, NJ: Prentice-Hall, Inc., 1996.
36. W. J. Rugh, Linear System Theory. Englewood Cliffs, NJ: Prentice-Hall, 1996.
37. T. Kailath, Linear Systems. Englewood Cliffs, NJ: Prentice-Hall, 1980.
38. B. W. Lindgren, Statistical Theory. New York: Macmillan, 1976.
39. A. Papoulis, Probability, Random Variables, and Stochastic Processes. New York: McGraw-Hill, 1965.
40. W. Rudin, Principles of Mathematical Analysis. New York: McGraw-Hill, 1976.
41. E. Kreyszig, Advanced Engineering Mathematics. New York: Wiley, 1993.
42. B. A. Pearlmutter, "Gradient calculations for dynamic recurrent neural networks: A survey," IEEE Trans. Neural Networks, vol. 6, pp. 1212-1228, Sept. 1995.
43. K. S. Narendra and K. Parthasarathy, "Identification and control of dynamical systems using neural networks," IEEE Trans. Neural Networks, vol. 1, pp. 4-27, Mar. 1990.
44. B. A. Pearlmutter, "Temporally continuous vs. clocked networks," in Neural Networks in Robotics, K. Goldberg and G. A. Bekey, Eds. Boston, MA: Kluwer, 1992, pp. 237-252.