Nonlinear system modeling with random matrices: Echo state networks revisited

IEEE Transactions on Neural Networks and Learning Systems

Volumn 23, Issue 1, 2012, Pages 175-182

  Zhang, Bai ^a   Miller, David J ^b   Wang, Yue ^a  

^a Virginia Polytechnic Institute and State University   (United States)

^b The Pennsylvania State University   (United States)

Author keywords

Circular law; concentration of measure; echo state networks; echo state property; random matrix theory; recurrent neural networks

Indexed keywords

CIRCULAR LAW; CONCENTRATION OF MEASURE; ECHO STATE NETWORKS; ECHO STATE PROPERTY; RANDOM MATRIX THEORY;

NONLINEAR DYNAMICAL SYSTEMS; RANDOM VARIABLES; RECURRENT NEURAL NETWORKS;

MATRIX ALGEBRA;

ARTIFICIAL NEURAL NETWORK; NONLINEAR SYSTEM; RANDOMIZATION;

NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; RANDOM ALLOCATION;

EID: 84866504536     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2011.2178562     Document Type: Article

References (31)

1. H. Jaeger and H. Haas, "Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication," Science, vol. 304, no. 5667, pp. 78-80, Apr. 2004.
2. H. Jaeger, "The 'echo state' approach to analysing and training recurrent neural networks," German National Research Center Information Technology, St. Augustin, Germany, Tech. Rep. 148, 2001.
3. W. Maass, T. Natschläger, and H. Markram, "Real-time computing without stable states: A new framework for neural computation based on perturbations," Neural Comput., vol. 14, no. 11, pp. 2531-2560, Nov. 2002.
4. Z. Shi and M. Han, "Support vector echo-state machine for chaotic timeseries prediction," IEEE Trans. Neural Netw., vol. 18, no. 2, pp. 359-372, Mar. 2007.
5. M. C. Ozturk and J. C. Príncipe, "An associative memory readout for ESNs with applications to dynamical pattern recognition," Neural Netw., vol. 20, no. 3, pp. 377-390, Apr. 2007.
6. M. D. Skowronski and J. G. Harris, "Automatic speech recognition using a predictive echo state network classifier," Neural Netw., vol. 20, no. 3, pp. 414-423, Apr. 2007.
7. B. Zhang and Y. Wang, "Echo state networks with decoupled reservoir states," in Proc. IEEE Workshop Mach. Learn. Signal Process., Oct. 2008, pp. 444-449.
8. Z. Deng and Y. Zhang, "Collective behavior of a small-world recurrent neural system with scale-free distribution," IEEE Trans. Neural Netw., vol. 18, no. 5, pp. 1364-1375, Sep. 2007.
9. Y. Xue, L. Yang, and S. Haykin, "Decoupled echo state networks with lateral inhibition," Neural Netw., vol. 20, no. 3, pp. 365-376, 2007.
10. M. C. Ozturk, D. Xu, and J. C. Príncipe, "Analysis and design of echo state networks," Neural Comput., vol. 19, no. 1, pp. 111-138, Jan. 2007.
11. Y. Xia, B. Jelfs, M. Van Hulle, J. Príncipe, and D. Mandic, "An augmented echo state network for nonlinear adaptive filtering of complex noncircular signals," IEEE Trans. Neural Netw., vol. 22, no. 1, pp. 74-83, Jan. 2011.
12. S. Chatzis and Y. Demiris, "Echo state Gaussian process," IEEE Trans. Neural Netw., vol. 22, no. 9, pp. 1435-1445, Sep. 2011.
13. A. Rodan and P. Tino, "Minimum complexity echo state network," IEEE Trans. Neural Netw., vol. 22, no. 1, pp. 131-144, Jan. 2011.
14. M. Lukosevicius and H. Jaeger, "Reservoir computing approaches to recurrent neural network training," Comput. Sci. Rev., vol. 3, no. 3, pp. 127-149, Aug. 2009.
15. J. L. Elman, "Finding structure in time," Cognit. Sci., vol. 14, no. 2, pp. 179-211, 1990.
16. M. I. Jordan, "Attractor dynamics and parallelism in a connectionist sequential machine," in Artificial Neural Networks: Concept Learning (IEEE Computer Society Neural Networks Technology), J. Diederich, Ed. Los Alamitos, CA: IEEE Computer Society Press, Jan. 1990, pp. 112-127.
17. M. Buehner and P. Young, "A tighter bound for the echo state property," IEEE Trans. Neural Netw., vol. 17, no. 3, pp. 820-824, May 2006.
18. H. Jaeger, "Echo state network," Scholarpedia, vol. 2, no. 9, p. 2330, 2007.
19. M. Mehta, Random Matrices and the Statistical Theory of Energy Levels. New York: Academic, 1967.
20. A. Edelman, "The probability that a random real Gaussian matrix has k real eigenvalues, related distributions, and the circular law," J. Multivar. Anal., vol. 60, no. 2, pp. 203-232, Feb. 1997.
21. V. L. Girko, "Circular law," Theory Probab. Appl., vol. 29, no. 4, pp. 694-706, 1984.
22. Z. D. Bai, "Circular law," Ann. Probab., vol. 25, no. 1, pp. 494-529, 1997.
23. T. Tao and V. Vu, "Random matrices: The circular law," Commun. Contemp. Math., vol. 10, no. 2, pp. 261-307, 2008.
24. T. Tao, V. Vu, and M. Krishnapur, "Random matrices: Universality of ESDs and the circular law," Ann. Prob., vol. 38, no. 5, pp. 2023-2065, 2010.
25. T. Tao and V. Vu, "Random matrices: The distribution of the smallest singular values," Geometric Funct. Anal., vol. 20, no. 1, pp. 1-43, Mar. 2010.
26. V. A. Marcenko and L. A. Pastur, "Distribution of eigenvalues for some sets of random matrices," Math. USSR-Sbornik, vol. 1, no. 4, pp. 457-483, Apr. 1967.
27. Y. Q. Yin, "Limiting spectral distribution for a class of random matrices," J. Multivar. Anal., vol. 20, no. 1, pp. 50-68, Oct. 1986.
28. Z. D. Bai and Y. Q. Yin, "Limiting behavior of the norm of products of random matrices and two problems of Geman-Hwang," Probab. Theory Rel. Fields, vol. 73, no. 4, pp. 555-569, 1986.
29. Y. Q. Yin, Z. D. Bai, and P. R. Krishnaiah, "On the limit of the largest eigenvalue of the large dimensional sample covariance matrix," Prob. Theory Rel. Fields, vol. 78, no. 4, pp. 509-521, 1988.
30. M. Ledoux, The Concentration of Measure Phenomenon. Providence, RI: AMS, 2001.
31. D. Achlioptas, "Database-friendly random projections," in Proc. 20th ACM SIGMOD-SIGACT-SIGART Symp. Principles Database Syst., 2001, pp. 274-281.