Variational Bayesian learning of nonlinear hidden state-space models for model predictive control

Neurocomputing

Volumn 72, Issue 16-18, 2009, Pages 3704-3712

  Raiko, Tapani ^a   Tornio, Matti ^a  

^a AALTO UNIVERSITY   (Finland)

Author keywords

Model predictive control; Neural network; Nonlinear system; Partially observable Markov decision process; State space method; Stochastic optimal control; Variational methods

Indexed keywords

CART-POLE SYSTEM; CONTROL SCHEMES; HIDDEN STATE; HIGH NOISE; INDIRECT CONTROL; NON-LINEARITY; PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; PROBABILISTIC INFERENCE; STATE-SPACE METHOD; STOCHASTIC APPROACH; STOCHASTIC OPTIMAL CONTROL; SYSTEM STATE; VARIATIONAL BAYESIAN LEARNING; VARIATIONAL METHODS;

BAYESIAN NETWORKS; IDENTIFICATION (CONTROL SYSTEMS); MARKOV PROCESSES; NEURAL NETWORKS; NONLINEAR SYSTEMS; OPTIMAL CONTROL SYSTEMS; PREDICTIVE CONTROL SYSTEMS; ROBUSTNESS (CONTROL SYSTEMS); SPEECH ENHANCEMENT; STATE SPACE METHODS; STOCHASTIC CONTROL SYSTEMS; STOCHASTIC MODELS;

MODEL PREDICTIVE CONTROL;

ACCURACY; ARTICLE; ARTIFICIAL NEURAL NETWORK; BAYESIAN LEARNING; CONTROL; CONTROLLED STUDY; INFERENTIAL STATISTICS; NOISE; NONLINEAR SYSTEM; PREDICTION; PRIORITY JOURNAL; PROBABILITY; SIMULATION; STATISTICAL MODEL; VELOCITY;

EID: 69249203563     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2009.06.009     Document Type: Article

References (37)

1. Anderson B., and Moore J. Optimal Filtering (1979), Prentice-Hall, Englewood Cliffs, NJ
2. Åstrüm K., Albertos P., Blamke M., Isidori A., Schaufelberger W., and Sanz R. Control of complex systems (2001), Springer, Berlin
3. Åstrüm K., and Wittenmark B. Adaptive Control. second ed. (1995), Addison-Wesley, Reading, MA
4. Atkeson C.G. Using local trajectory optimizers to speed up global optimization in dynamic programming. In: Cowan J.D., Tesauro G., and Alspector J. (Eds). Advances in Neural Information Processing Systems vol. 6 (1994), Morgan Kaufmann Publishers Inc., Los Altos, CA
5. Atkeson C.G., Moore A.W., and Schaal S. Locally weighted learning for control. Artificial Intelligence Review 11 1-5 (1997) 75-113
6. H. Attias, Planning by probabilistic inference, in: Proceedings of the AI-Stats 2003, 2003.
7. Bar-Shalom Y. Stochastic dynamic programming: caution and probing. IEEE Transactions on Automatic Control 26 5 (1981) 1184-1195
8. Bemporad A., Ferrari-Trecate G., and Morari M. Observability and controllability of piecewise affine and hybrid systems. IEEE Transactions on Automatic Control 45 10 (2000) 1864-1876
9. Dayan P., and Hinton G.E. Using expectation-maximization for reinforcement learning. Neural Computation 9 2 (1997) 271-278
10. Farrell J.A., and Polycarpou M.M. Adaptive Approximation Based Control: Unifying Neural, Fuzzy and Traditional Adaptive Approximation Approaches (2006), Wiley, New York
11. Haykin S. Neural Networks-A Comprehensive Foundation. second ed. (1999), Prentice-Hall, Englewood Cliffs, NJ
12. M. Hoffman, A. Doucet, N. de Freitas, A. Jasra, Bayesian policy learning with trans-dimensional MCMC, in: Advances in Neural Information Processing Systems (NIPS), 2008.
13. Honkela A., and Valpola H. Unsupervised variational Bayesian learning of nonlinear models. In: Saul L., Weiss Y., and Bottou L. (Eds). Advances in Neural Information Processing Systems 17 (2005), MIT Press, Cambridge, MA, USA 593-600
14. H. Kappen, A linear theory for control of non-linear stochastic systems, Physical Review Letters 95 (20) (2005).
15. H. Kimura, S. Kobayashi, Efficient non-linear control by combining Q-learning with local linear controllers, in: Proceedings of the ICML, 1999.
16. J. Kocijan, R. Murray-Smith, C. Rasmussen, A. Girard, Gaussian process model based predictive control, in: American Control Conference, Boston, MA, 2004.
17. J. Kocijan, R. Murray-Smith, C.E. Rasmussen, B. Likar, Predictive control with Gaussian process models, in: Proceedings of IEEE Region 8 Eurocon 2003: Computer as a Tool, 2003.
18. Mayne D., Rawlings J., Rao C., and Scokaert P. Constrained model predictive control: stability and optimality. Automatica 36 (2000) 789-814
19. A. Moore, Acquisition of dynamic control knowledge for a robotic manipulator, in: M.B. Morgan (Ed.), Proceedings of the 7th International Conference on Machine Learning, Morgan Kaufmann, San Francisco, CA 94104, 1990.
20. Murray R., Åstrüm K.J., Boyd S.P., Brockett R.W., and Stein G. Future directions in control in an information-rich world. IEEE Control Systems Magazine 23 2 (2003) 20-33
21. Nocedal J., and Wright S.J. Numerical Optimization (1999), Springer-Verlag, New York
22. Nørgaard M., Ravn O., Poulsen N.K., and Hansen L.K. Neural Networks for Modelling and Control of Dynamic Systems (2001), Springer-Verlag London Limited
23. J. Peters, S. Schaal, Policy learning for motor skills, in: Proceedings of 14th International Conference on Neural Information Processing (ICONIP 2007), Springer, Berlin, 2007.
24. Peters J., and Schaal S. Learning to control in operational space. The International Journal of Robotics Research 27 2 (2008) 197-212
25. T. Raiko, M. Tornio, Learning nonlinear state-space models for control, in: Proceedings of the IJCNN'05, Montreal, Canada, 2005.
26. T. Raiko, M. Tornio, A. Honkela, J. Karhunen, State inference in variational Bayesian nonlinear state-space models, in: Proceedings of the ICA 2006, Charleston, South Carolina, USA, 2006.
27. C.E. Rasmussen, M.P. Deisenroth, Probabilistic inference for fast learning in control, in: Recent Advances in Reinforcement Learning, Lecture Notes in Computer Science, vol. 5323, Springer, Berlin, 2008, pp. 229-242.
28. Richter S., Aberdeen D., and Yu J. Natural actor-critic for road traffic optimisation. Advances in Neural Information Processing Systems (2007), MIT Press, Cambridge, MA
29. Rosenqvist F., and Karlstrüm A. Realisation and estimation of piecewise-linear output-error models. Automatica 41 3 (2005) 545-551
30. Schaal S., Atkeson C.G., and Vijayakumar S. Scalable techniques from nonparameteric statistics for real-time robot learning. Applied Intelligence 1 (2002) 49-60
31. Sutton R. Learning to predict by the methods of temporal differences. Machine Learning 3 (1988) 9-44
32. Thrun S.B. The role of exploration in learning control. Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches (1992), Van Nostrand Reinhold, New York 527-559
33. M. Toussaint, Bayesian inference for motion control and planning, Technical Report 22, Technische Universität Berlin, 2007.
34. M. Toussaint, A. Storkey, Probabilistic inference for solving discrete and continuous state Markov decision processes, in: ICML '06: Proceedings of the 23rd International Conference on Machine Learning, ACM, New York, NY, USA, 2006.
35. Valpola H., and Karhunen J. An unsupervised ensemble learning method for nonlinear dynamic state-space models. Neural Computation 14 11 (2002) 2647-2692
36. Vijayakumar S., D'Souza A., Shibata T., Conradt J., and Schaal S. statistical learning for humanoid robots. Autonomous Robots 1 (2002) 59-72
37. P. Wawrzynski, A. Pacut, Model-free off-policy reinforcement learning in continuous environment, in: Proceedings of the International Joint Conference on Neural Networks, Budapest, Hungary, 2004.