Iterative temporal learning and prediction with the sparse online echo state gaussian process

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2012, Pages

  Soh, Harold ^a   Demiris, Yiannis ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ACTION RECOGNITION; BENCH-MARK PROBLEMS; COMPUTATIONAL COSTS; ECHO STATE NETWORKS; GAUSSIAN PROCESSES; GRAPHICAL MODEL; HIGH-DIMENSIONAL; ONLINE METHODS; PREDICTION TASKS; PREDICTIVE DISTRIBUTIONS; REGRESSION METHOD; SPARSE APPROXIMATIONS; TEMPORAL DYNAMICS; TEMPORAL LEARNING;

GAUSSIAN DISTRIBUTION; NEURAL NETWORKS;

GAUSSIAN NOISE (ELECTRONIC);

EID: 84865085819     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2012.6252504     Document Type: Conference Paper

References (34)

1. F. Kianifard and W. H. Swallow, "A review of the development and application of recursive residuals in linear models," Journal of the American Statistical Association, vol. 91, no. 433, pp. 391-400, 1996.
2. M. Lukosevicius and H. Jaeger, "Reservoir computing approaches to recurrent neural network training," Computer Science Review, vol. 3, no. 3, pp. 127-149, 2009.
3. H. Jaeger, "The "echo state" approach to analysing and training recurrent neural networks," German National Research Center for Information Technology, Bremen, Tech. Rep. 148, 2001.
4. Z. Deng and Y. Zhang, "Collective behavior of a small-world recurrent neural system with scale-free distribution," IEEE Trans. on Neural Networks, vol. 18, no. 5, pp. 1364 -1375, Sept. 2007.
5. A. Rodan and P. Tino, "Minimum complexity echo state network," IEEE Trans. on Neural Networks, vol. 22, no. 1, pp. 131 -144, jan. 2011.
6. S. Vijayakumar, A. D'souza, and S. Schaal, "Incremental online learning in high dimensions," Neural Computation, vol. 17, no. 12, pp. 2602- 2634, 2005.
7. A. Kapoor, K. Grauman, R. Urtasun, and T. Darrell, "Active learning with gaussian processes for object categorization," in IEEE 11th International Conference on Computer Vision (ICCV), 2007, pp. 1-8.
8. L. Csató and M. Opper, "Sparse on-line gaussian processes," Neural Computation, vol. 14, no. 3, pp. 641-668, March 2002.
9. H. Jaeger, "Adaptive nonlinear system identification with echo state networks," in Advances in Neural Information Processing Systems, 2003, pp. 593-600.
10. P. Kountouriotis, D. Obradovic, S. Goh, and D. Mandic, "Multi-step forecasting using echo state networks," in The International Conference on Computer as a Tool, vol. 2. IEEE, 2005, pp. 1574-1577.
11. S. Chatzis and Y. Demiris, "Echo state gaussian process," IEEE Trans. on Neural Networks, vol. 22, no. 9, pp. 1435 -1445, Sept. 2011.
12. Y. Engel, S. Mannor, and R. Meir, "The kernel recursive least-squares algorithm," IEEE Trans. on Signal Processing, vol. 52, no. 8, pp. 2275-2285, Aug. 2004.
13. D. Rumelhart, G. Hinton, and R. Williams, "Learning representations by back-propagating errors," Nature, vol. 323, no. 6088, pp. 533-536, 1986.
14. P. J. and Werbos, "Generalization of backpropagation with application to a recurrent gas market model," Neural Networks, vol. 1, no. 4, pp. 339 - 356, 1988.
15. R. J. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks," Neural Computation, vol. 1, no. 2, pp. 270-280, June 1989.
16. C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning. The MIT Press, 2006.
17. L. Csató, "Gaussian processes: iterative sparse approximations," Ph.D. dissertation, Aston University, 2002.
18. J. Quiñonero Candela and C. E. Rasmussen, "A unifying view of sparse approximate gaussian process regression," J. Mach. Learn. Res., vol. 6, pp. 1939-1959, December 2005.
19. M. Opper, A Bayesian approach to on-line learning. Cambridge University Press, 1998, pp. 363-378.
20. M. Seeger, C. Williams, and N. Lawrence, "Fast forward selection to speed up sparse gaussian process regression," in Ninth Intl. Workshop on Artificial Intelligence and Statistics, vol. 9, 2003.
21. S. Keerthi and W. Chu, "A matching pursuit approach to sparse gaussian process regression," in Advances in Neural Information Processing Systems 18. Cambridge, MA: MIT Press, 2006, pp. 643-650.
22. G. Guennebaud, B. Jacob et al., "Eigen v3," http://eigen.tuxfamily.org.
23. D. Grollman, "Sparse online gaussian process c++ library." [Online]. Available: http://lasa.epfl.ch/dang/code.shtml
24. H. Soh, "Online Temporal Learning (OTL) C++ Library." [Online]. Available: http://www.haroldsoh.com/
25. S. Vijayakumar, A. D'Souza, and S. Schaal, "LWPR: Locally Weighted Projection Regression library." [Online]. Available: http://www.ipab.inf. ed.ac.uk/slmc/software/lwpr/
26. S. Klanke and S. Vijayakumar, "LWPR supplementary documentation," Tech. Rep., 2008.
27. A. S. Weigend, Time Series Prediction: Forecasting The Future And Understanding The Past, ser. Sante Fe Institute Series, N. A. Gershenfeld, Ed. Westfield Press, Nov 1993.
28. C. Richard, J. Bermudez, and P. Honeine, "Online prediction of time series data with kernels," IEEE Trans. on Signal Processing, vol. 57, no. 3, pp. 1058 -1067, March 2009.
29. J. Kivinen, A. Smola, and R. Williamson, "Online learning with kernels," IEEE Trans. on Signal Processing, vol. 52, no. 8, pp. 2165 - 2176, 2004.
30. V. Kadirkamanathan and M. Niranjan, "A function estimation approach to sequential learning with neural networks," Neural Computation, vol. 5, no. 6, pp. 954-975, Nov 1993.
31. W. Li, Z. Zhang, and Z. Liu, "Action recognition based on a bag of 3d points," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 2010, pp. 9-14.
32. F. Triefenbach, A. Jalalvand, B. Schrauwen, and J.-P. Martens, "Phoneme recognition with large hierarchical reservoirs," in Advances in Neural Information Processing Systems 23, 2010, pp. 2307-2315.
33. Y. Demiris and B. Khadhouri, "Hierarchical attentive multiple models for execution and recognition of actions," Robotics and Autonomous Systems, vol. 54, no. 5, pp. 361-369, 2006.
34. H. Soh and Y. Demiris, "Evolving policies for multi-reward partially observable markov decision processes (MR-POMDPs)," in Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, 2011, pp. 713-720.