Simple deterministically constructed cycle reservoirs with regular jumps

Neural Computation

Volumn 24, Issue 7, 2012, Pages 1822-1852

  Rodan, Ali ^a   Tiňo, Peter ^a  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; BIOLOGICAL MODEL; COMPUTER SIMULATION; FEMALE; HUMAN; PHYSIOLOGY; SPEECH PERCEPTION;

ALGORITHMS; COMPUTER SIMULATION; FEMALE; HUMANS; MODELS, NEUROLOGICAL; SPEECH PERCEPTION;

EID: 84868229207     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00297     Document Type: Article

References (48)

1. Ajdari Rad, A., Jalili, M., and Hasler, M. (2008). Reservoir optimization in recurrent neural networks using kronecker kernels. In Proceedings of the IEEE Symposium on Circuits and Systems (pp. 868-871). Piscataway, NJ: IEEE.
2. Atiya, A., and Parlos, A. (2000). New results on recurrent network training: Unifying the algorithms and accelerating convergence. IEEE Transactions on Neural.
3. Bertschinger, N., and Natschlager, T. (2004). Real-time computation at the edge of chaos in recurrent neural networks. Neural Computation, 16(7), 1413-1436.
4. Bush, K., and Anderson, C. (2005). Modeling reward functions for incomplete state representations via echo state networks. In Proceedings of the IEEE International Joint Conference on Neural Networks (pp. 2995-3000). Piscataway, NJ: IEEE.
5. Busing, L., Schrauwen, B., and Legenstein, R. A. (2010). Connectivity, dynamics, and memory in reservoir computing with binary and analog neurons. Neural Computation, 22(5), 1272-1311.
6. Cernansky, M., and Makula, M. (2005). Feed-forward echo state networks. In Proceedings of the IEEE International Joint Conference on Neural Networks (pp. 1479-1482). Piscataway, NJ: IEEE.
7. Cernansky, M., and Tino, P. (2008). Predictive modelling with echo state networks. In Proceedings of the 18th International Conference on Artificial Neural Networks (pp. 778-787). New York: Springer-Verlag.
8. Deng, Z., and Zhang, Y. (2007). Collective behavior of a small-world recurrent neural system with scale-free distribution. IEEE Transactions on Neural Networks, 18(5), 1364-1375.
9. Dockendorf, K., Park, I., Ping, H., Principe, J. C., and DeMarse, T. (2009). Liquid state machines and cultured cortical networks: The separation property. Biosystems, 95(2), 90-97.
10. Dutoit, X., Schrauwen, B., Van Campenhout, J., Stroobandt, D., Van Brussel, H., and Nuttin, M. (2009). Pruning and regularization in reservoir computing. Neurocomputing, 72, 1534-1546.
11. Fette, G., and Eggert, J. (2005). Short term memory and pattern matching with simple echo state networks. In Proc. of the 15th International Conference on Artificial Neural Networks (pp. 13-18). New York: Springer-Verlag.
12. Hausler, S.,Markram,M., andMaass,W. (2003). Perspectives of the high-dimensional dynamics of neural microcircuits from the point of view of low-dimensional readouts. Complexity, 8(4), 39-50.
13. Holzmann, G., and Hauser, H. (2009). Echo state networks with filter neurons and a delay and sum readout. Neural Networks, 32(2), 244-256.
14. Ishii, K., van der Zant, T., Becanovic, V. and Ploger, P. (2004). Identification of motion with echo state network. In Proceedings of the OCEANS 2004 MTS/IEEE-TECHNOOCEAN Conference (Vol. 3, pp. 1205-1210).
15. Jaeger, H. (2001). The "echo state" approach to analysing and training recurrent neural networks (Tech. Rep. No. 148). Sankt Augustin: GermanNational Research Center for Information Technology.
16. Jaeger,H. (2002a). Short term memory in echo state networks (Tech. Rep. No. 152). Sankt Augustin: German National Research Center for Information Technology.
17. Jaeger,H. (2002b). Atutorial on training recurrent neural networks, covering BPPT,RTRL, EKF and the "echo state network" approach (Tech. Rep. No. 159). Sankt Augustin: German National Research Center for Information Technology.
18. Jaeger,H. (2003). Adaptive nonlinear systems identification with echo state network. In S. Becker, S. Thrün, and K. Obermayer (Eds.), Advances in neural information processing systems, 15 (pp. 593-600). Cambridge, MA: MIT Press.
19. Jaeger, H. (2005). Reservoir riddles: Suggestions for echo state network research. In Proceedings of the IEEE International Joint Conference on Neural Networks (pp. 1460-1462). New York: Springer-Verlag.
20. Jaeger, H. (2007). Discovering multiscale dynamical features with hierarchical echo state networks (Tech. Rep. No. 10). Bremen: Jacobs University.
21. Jaeger, H., and Hass, H. (2004). Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless telecommunication. Science, 304, 78-80.
22. Jaeger,H., Lukosevicius,M., Popovici,D.,andSiewert, U. (2007). Optimisation and applications of echo state networks with leaky-integrator neurons. Neural Networks, 20(3), 335-352.
23. Jones, B., Stekel, D., Rowe, J., and Fernando, C. (2007). Is there a liquid state machine in the bacterium Escherichia coli? In Proceedings of the 2007 IEEE Symposium on Artificial Life (pp. 187-191). Piscataway, NJ: IEEE.
24. Legenstein, R., and Maass, W. (2007). Edge of chaos and prediction of computational performance for neural circuit models. Neural Networks, 20(3), 323-334.
25. Lukosevicius,M., and Jaeger, H. (2009). Reservoir computing approaches to recurrent neural network training. Computer Science Review, 3(3), 127-149.
26. Lyon, R. F. (1982). A computational model of filtering, detection and compression in the cochlea. In Proceedings of the IEEE ICASSP (pp. 1282-1285). Piscataway, NJ: IEEE. Maass, W., Natschlager, T., and Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11), 2531-2560.
27. Maass, W., Natschlager, T., and Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11), 2531-2560.
28. Maass,W., Natschlager, T., andMarkram, H. (2004). Fading memory and kernel properties of generic cortical microcircuit models. Journal of Physiology, 98(4-6), 315-330.
29. Ozturk, M. C., Xu, D., and Principe, J. C. (2007). Analysis and design of echo state network. Neural Computation, 19(1), 111-138. Rodan, A., and Ti? no, P. (2011). Minimum complexity echo state network. IEEE Transactions on Neural Networks, 22(1), 131-144.
30. Rodan, A., and Tiňo, P. (2011). Minimum complexity echo state network. IEEE Transactions on Neural Networks, 22(1), 131-144.
31. Schmidhuber, J., Wierstra, D., Gagliolo, M., and Gomez, F. (2007). Training recurrent networks by Evolino. Neural Computation, 19, 757-779.
32. Schrauwen, B., Busing, L., and Legenstein, R. A. (2008). On computational power and the order-chaos phase transition in reservoir computing. In D. Koller, D. Shuurmans, Y., Bengio, and L. Bottou (Eds.), Neural information processing systems (pp. 425-1432). Cambridge, MA: MIT Press.
33. Schrauwen, B., Defour, J., Verstraeten, D., and Van Campenhout, J. M. (2007). The introduction of time-scales in reservoir computing, applied to isolated digits recognition. In Proceedings of the 17th International Conference on Artificial Neural Networks (pp. 471-479). New York: Springer-Verlag.
34. Schrauwen, B., Wardermann, M., Verstraeten, D., Steil, J., and Stroobandt, D. (2008). Improving reservoirs using intrinsic plasticity. Neurocomputing, 71(7-9), 1159-1171.
35. Skowronski, M. D., and Harris, J. G. (2006). Minimum mean squared error time series classification using an echo state network prediction model. In IEEE International Symposium on Circuits and Systems (pp. 3153-3156). Piscataway, NJ: IEEE.
36. Steil, J. (2004). Backpropagation-decorrelation: Recurrent learning with o(n) complexity. In Proc. of the International Joint Conference on Neural Networks (pp. 843-848). New York: Springer-Verlag.
37. Steil, J. (2007). Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning. Neural Networks, 20, 353-364.
38. Tabor, W. (2002). The value of symbolic computation. Ecological Psychology, 14(1-2), 21-51.
39. Tiňo, P.,andDorffner, G. (2001). Predicting the future of discrete sequences from fractal representations of the past. Machine Learning, 45(2), 187-218.
40. Tong, M. H., Bicket, A. D., Christiansen, E. M., and Cottrell, G. W. (2007). Learning grammatical structure with echo state network. Neural Networks, 20, 424-432.
41. Triefenbach, F., Jalalvand, A., Schrauwen, B., and Martens, J. (2010). Phoneme recognition with large hierarchical reservoirs. In J. Lafferty, C.K.I. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta (Eds.), Advances in neural information processing systems, 23 (pp. 2307-2315). Red Hook, NY: Curran Associates.
42. Verstraeten, D., Dambre, J., Dutoit, X., and Schrauwen, B. (2010). Memory versus nonlinearity in reservoirs. In The 2010 International Joint Conference on Neural Networks (pp. 1-8). Piscataway, NJ: IEEE Press.
43. Verstraeten, D., Schrauwen, B., D'Haene, M., and Stroobandt, D. (2006). The unified reservoir computing concept and its digital hardware implementations. In Proc. of the EPFL-LATSIS Symposium 2006 (pp. 139-140). Swiss Federal Institute of Technology at Lausanne.
44. Verstraeten,D., Schrauwen, B., D'Haene, M.,andStroobandt,D. (2007). An experimental unification of reservoir computing methods. Neural Networks, 20, 391-403.
45. Watts,D. J.,andStrogatz, S. H. (1998). Collective dynamics of "small-world" networks. Nature, 393 (6684), 409-10.1998.
46. Wyffels, F., Schrauwen, B., and Stroobandt, D. (2008). Stable output feedback in reservoir computing using ridge regression. In Proceedings of the 18th International Conference on Artificial Neural Networks (pp. 808-817). New York: Springer-Verlag.
47. Xue, Y. Yang, L., and Haykin, S. (2007). Decoupled echo state networks with lateral inhibition. Neural Networks, 20, 365-376.
48. Zhang, B., and Wang, Y. (2008). Echo state networks with decoupled reservoir states. In 18th IEEE International Workshop on Machine Learning for Signal Processing. Piscataway, NJ: IEEE.