Robust initialization of a Jordan network with recurrent constrained learning

IEEE Transactions on Neural Networks

Volumn 22, Issue 12 PART 2, 2011, Pages 2460-2473

  Song, Qing ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Generalization; real time rucurrent learning; recurrent sensitivity analysis; robust initialization; time series prediction; weight convergence proof

Indexed keywords

GENERALIZATION; REAL-TIME RUCURRENT LEARNING; ROBUST INITIALIZATION; TIME SERIES PREDICTION; WEIGHT CONVERGENCE;

BENCHMARKING; CONVERGENCE OF NUMERICAL METHODS; LEARNING ALGORITHMS; SENSITIVITY ANALYSIS; TIME SERIES;

RECURRENT NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; THEORETICAL MODEL;

ALGORITHMS; COMPUTER SIMULATION; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER);

EID: 83655163812     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2011.2168423     Document Type: Article

References (19)

1. M. I. Jordan, "Constrained supervised learning," J. Math. Psychol., vol. 36, no. 3, pp. 396-425, Sep. 1992.
2. M. I. Jordan, "Serial order: A parallel distributed processing approach," Adv. Psychol., vol. 121, pp. 471-495, May 1997.
3. G. Castellano, A. Fanelli, and M. Pelillo, "An iterative pruning algorithm for feedforward neural networks," IEEE Trans. Neural Netw., vol. 8, no. 3, pp. 519-531, May 1997. (Pubitemid 127767797)
4. E. Castillo, O. Fontenla-Romero, B. Guijarro-Berdiñas, and A. Alonso-Betanzos, "A global optimum approach for one-layer neural networks," Neural Comput., vol. 14, no. 6, pp. 1429-1449, Jun. 2002.
5. G. Drago and S. Ridella, "Statistically controlled activation weight initialization (SCAWI)," IEEE Trans. Neural Netw., vol. 3, no. 4, pp. 627-631, Jul. 1992.
6. D. Erdogmus, O. Fontenla-Romero, J. Principe, A. Alonso-Betanzos, and E. Castillo, "Linear-least-squares initialization of multilayer perceptrons through backpropagation of the desired response," IEEE Trans. Neural Netw., vol. 16, no. 2, pp. 325-337, Mar. 2005. (Pubitemid 40390816)
7. S. Haykin, Neural Networks: A Comprehensive Foundation. Englewood Cliffs, NJ: Prentice-Hall, 1999.
8. D. Nguyen and B. Widrow, "Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights," in Proc. Int. Joint Conf. Neural Netw., vol. 3. San Diego, CA, Jun. 1990, pp. 21-26.
9. Y. Yam, T. Chow, and C. Leung, "A new method in determining initial weights of feedforward neural networks for training enhancement," Neurocomputing, vol. 16, no. 1, pp. 23-32, 1997. (Pubitemid 27286289)
10. B. Hammer and P. Tino, "Recurrent neural networks with small weights implement definite memory machines," Neural Comput., vol. 15, no. 8, pp. 1897-1929, Aug. 2003. (Pubitemid 37035552)
11. G. Goodwin and K. Sin, Adaptive Filtering Prediction and Control. New York: Dover, 2009.
12. C. Kuan, K. Hornik, and H. White, "A convergence result for learning in recurrent neural networks," Neural Comput., vol. 6, no. 3, pp. 420-440, May 1994.
13. K. Narendra and K. Parthasarathy, "Adaptive identification and control of dynamical systems using neural networks," in Proc. 28th IEEE Conf. Decis. Control, vol. 2. Tampa, FL, Dec. 1989, pp. 1737-1738. (Pubitemid 20701059)
14. Y. Park, T. Murray, and C. Chen, "Predicting sun spots using a layered perceptron neural network," IEEE Trans. Neural Netw., vol. 7, no. 2, pp. 501-505, Mar. 1996. (Pubitemid 126774440)
15. Q. Song, Y. Wu, and Y. Soh, "Robust adaptive gradient-descent training algorithm for recurrent neural networks in discrete time domain," IEEE Trans. Neural Netw., vol. 19, no. 11, pp. 1841-1853, Nov. 2008.
16. Q. Song, "On the weight convergence of Elman networks," IEEE Trans. Neural Netw., vol. 21, no. 3, pp. 463-480, Mar. 2010.
17. D. Mandic and J. Chambers, Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability. New York: Wiley, 2001.
18. X. Yu and Y. Huang, "Convergence study in extended Kalman filterbased training of recurrent neural networks," IEEE Trans. Neural Netw., vol. 22, no. 4, pp. 588-600, Apr. 2011.
19. A. S. Wigend and N. A. Gershenfeld, Time Series Prediction: Foresacting the Future and Understanding the Past. London, U.K.: Addison-Wesley, 1993.