A survey of inverse reinforcement learning techniques

International Journal of Intelligent Computing and Cybernetics

Volumn 5, Issue 3, 2012, Pages 293-311

  Zhifei, Shao ^a   Joo, Er Meng ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Artificial intelligence; Inverse reinforcement learning; Learning methods; Reinforcement learning; Reward function

Indexed keywords

COMPLEX PROBLEMS; DESIGN/METHODOLOGY/APPROACH; DYNAMIC ENVIRONMENTS; FUNDAMENTAL THEORY; INVERSE REINFORCEMENT LEARNING; LATEST DEVELOPMENT; LEARNING METHODS; REINFORCEMENT LEARNING TECHNIQUES; REWARD FUNCTION; SEQUENTIAL DECISION MAKING; SUCCINCT REPRESENTATION;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; SURVEYS;

REINFORCEMENT LEARNING;

EID: 84865146660     PISSN: 1756378X     EISSN: 17563798     Source Type: Journal    
DOI: 10.1108/17563781211255862     Document Type: Article

References (58)

1. Abbeel, P. and Ng, A. (2004), "Apprenticeship learning via inverse reinforcement learning" in Proceedings of the 21st International Conference on Machine Learning, p. 1.
2. Abbeel, P., Coates, A. and Ng, A. (2010), "Autonomous helicopter aerobatics through apprenticeship learning" in International Journal of Robotics Research, Vol. 29, No. 13, pp. 1608-39.
3. Abbeel, P., Coates, A., Hunter, T. and Ng, A. (2009), "Autonomous autorotation of an RC helicopter" in Proceedings of the International Symposium on Experimental Robotics, pp. 385-94.
4. Abbeel, P., Coates, A., Quigley, M. and Ng, A. (2007), "An application of reinforcement learning to aerobatic helicopter flight" in Advances in Neural Information Processing Systems 19: Proceedings of the 2006 Conference, p. 1.
5. Abbeel, P., Dolgov, D., Ng, A. and Thrun, S. (2008), "Apprenticeship learning for motion planning with application to parking lot navigation" in Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, pp. 1083-90.
6. Amari, S. (1998), "Natural gradient works efficiently in learning" in Neural Computation, Vol. 10, No. 2, pp. 251-76.
7. Argall, B., Chernova, S., Veloso, M. and Browning, B. (2009), "A survey of robot learning from demonstration" in Robotics and Autonomous Systems, Vol. 57, No. 5, pp. 469-83.
8. Atkeson, C. and Schaal, S. (1997), "Robot learning from demonstration" in Proceedings of the International Conference on Machine Learning (ICML'97), Morgan Kaufmann, Burlington, MA, pp. 12-20.
9. Babes, M., Marivate, V., Littman, M. and Subramanian, K. (2010), "Apprenticeship learning about multiple intentions", Proceedings of International Conference on Machine Learning (ICML 2011).
10. Boularias, A. and Chaib-Draa, B. (2011), "Bootstrapping apprenticeship learning", Proceedings of Neural Information Processing Systems, 2010.
11. Boularias, A., Kober, J. and Peters, J. (2011), "Relative entropy inverse reinforcement learning" in Proceedings of Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2011), JMLR WC&P, Vol. 15, pp. 182-9.
12. Chandramohan, S., Geist, M., Lefevre, F. and Pietquin, O. (2011), "User simulation in dialogue systems using inverse reinforcement learning", Proceedings of the 12th Annual Conference of the International Speech Communication Association (Interspeech 2011), Florence (Italy), August.
13. Choi, J. and Kim, K. (2009), "Inverse reinforcement learning in partially observable environments" in Proceedings of the 21st International Joint Conference on Artifical Intelligence (IJCAI), pp. 1028-33.
14. Chung, S. and Huang, H. (2010), "A mobile robot that understands pedestrian spatial behaviors" in Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on, pp. 5861-6.
15. Coates, A., Abbeel, P. and Ng, A. (2008), "Learning for control from multiple demonstrations" in Proceedings of the 25th International Conference on Machine Learning, pp. 144-51.
16. Cohn, R., Maxim, M., Durfee, E. and Singh, S. (2010), "Selecting operator queries using expected myopic gain" in IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, pp. 40-7.
17. Dimitrakakis, C. and Rothkopf, C. (2011), "Bayesian multitask inverse reinforcement learning", paper presented at the 9th European Workshop on Reinforcement Learning (EWRL 2011), Athens, Greece, 9-11 September.
18. Friedman, M. and Savage, L. (1952), "The expected-utility hypothesis and the measurability of utility" in The Journal of Political Economy, No. 6, pp. 463-74.
19. Grollman, D. and Billard, A. (2011), "Donut as I do: learning from failed demonstrations" in IEEE International Conference on Robotics and Automation, Shanghai, 9-13 May, pp. 9-13.
20. Grudic, G. and Lawrence, P. (1996), "Human-to-robot skill transfer using the spore approximation" in Robotics and Automation, Proceedings, 1996 IEEE International Conference on, Vol. 4, pp. 2962-7.
21. Henry, P., Vollmer, C., Ferris, B. and Fox, D. (2010), "Learning to navigate through crowded environments" in Robotics and Automation (ICRA), 2010 IEEE International Conference on, pp. 981-6.
22. Heskes, T. (1998), "Solving a huge number of similar tasks: a combination of multi-task learning and a hierarchical Bayesian approach" in Proceedings of the 15th International Conference on Machine Learning (ICML'98), pp. 233-41.
23. Jaynes, E. (1957), "Information theory and statistical mechanics" in Physical Review, Vol. 108, No. 2, p. 171.
24. Kaelbling, L., Littman, M. and Moore, A. (1996), "Reinforcement learning: a survey" in Journal of Artificial Intelligence Research, Vol. 4, pp. 237-85.
25. Kalakrishnan, M., Theodorou, E. and Schaal, S. (2010), "Inverse reinforcement learning with PI 2", The Snowbird Workshop (submitted to).
26. Keeney, R. and Raiffa, H. (1993), Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Cambridge University Press, Cambridge.
27. Kober, J. and Peters, J. (2010), "Imitation and reinforcement learning, practical algorithms for motor primitives in robotics" in Robotics and Automation Magazine, IEEE, Vol. 17, No. 2, pp. 55-62.
28. Kolter, J., Abbeel, P. and Ng, A. (2008), "Hierarchical apprenticeship learning with application to quadruped locomotion", Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA.
29. Lafferty, J., McCallum, A. and Pereira, F. (2011), "Conditional random fields: probabilistic models for segmenting and labeling sequence data" in Proceedings of the International Conference on Machine Learning (ICML 2011), Morgan Kaufmann, Burlington, MA, pp. 282-9.
30. Lee, S. and Popovi, Z. (2010), "Learning behavior styles with inverse reinforcement learning" in ACM, New York, NY, pp. 1-7, ACM SIGGRAPH 2010 papers.
31. Lopes, M., Melo, F. and Montesano, L. (2009), "Active learning for reward estimation in inverse reinforcement learning" in Machine Learning and Knowledge Discovery in Databases, Vol. 5782, No. 1, pp. 31-46.
32. Mason, M. and Lopes, M. (2011), "Robot self-initiative and personalization by learning through repeated interactions" in Proceedings of the 6th International Conference on Human-robot Interaction, pp. 433-40.
33. Melo, F. and Lopes, M. (2010), "Learning from demonstration using MDP induced metrics" in Machine Learning and Knowledge Discovery in Databases, Springer, Berlin, pp. 385-401.
34. Murphy, K. (2000), "A survey of POMDP solution techniques" in Environment, Vol. 2, p. X3.
35. Neu, G. and Szepesvári, C. (2007), "Apprenticeship learning using inverse reinforcement learning and gradient methods" in Proceedings of the Twenty-third Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-07), pp. 295-302.
36. Ng, A. and Russell, S. (2000), "Algorithms for inverse reinforcement learning" in Proceedings of the Seventeenth International Conference on Machine Learning, pp. 663-70.
37. Pomerleau, D. (1991), "Efficient training of artificial neural networks for autonomous navigation" in Neural Computation, Vol. 3, No. 1, pp. 88-97.
38. Puterman, M. (1994), Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, New York, NY.
39. Qiao, Q. and Beling, P. (2011), "Inverse reinforcement learning with Gaussian process" in American Control Conference (ACC), pp. 113-18.
40. Ramachandran, D. and Amir, E. (2007), "Bayesian inverse reinforcement learning", Proceedings of the 20th International Joint Conference on Artificial Intelligence.
41. Ratliff, N., Bagnell, J. and Zinkevich, M. (2006), "Maximum margin planning" in Proceedings of the 23rd International Conference on Machine Learning, pp. 729-36.
42. Ratliff, N., Silver, D. and Bagnell, J. (2009), "Learning to search: functional gradient techniques for imitation learning" in Autonomous Robots, No. 1, pp. 25-53.
43. Robbins, H. (1992), "An empirical Bayes approach to statistics" in Breakthroughs in Statistics: Foundations and Basic Theory, Vol. 1, p. 388.
44. Rothkopf, C. and Dimitrakakis, C. (2011), "Preference elicitation and inverse reinforcement learning" in Proceedings of 22nd European Conference on Machine Learning ECML, Part III, LNAI 6913, pp. 34-48.
45. Russell, S. (1998), "Learning agents for uncertain environments (extended abstract)" in Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pp. 101-3.
46. Schaal, S. (1999), "Is imitation learning the route to humanoid robots?" in Trends in Cognitive Sciences, Vol. 3, No. 6, pp. 233-42.
47. Shen-yi, C., Hui, Q., Jia, F., Zhuo-jun, J., Miao-liang, Z., Springer (2010), "Modified reward function on abstract features in inverse reinforcement learning" in Journal of Zhejiang University - Science C, Vol. 11, No. 9, pp. 718-23.
48. Silva, V., Costa, A. and Lima, P. (2006), "Inverse reinforcement learning with evaluation" in IEEE International Conference on Robotics and Automation (ICRA06), Orlando, FL, USA, pp. 4246-51.
49. Silver, D., Bagnell, J. and Stentz, A. (2010), "Learning from demonstration for autonomous navigation in complex unstructured terrain" in The International Journal of Robotics Research, Vol. 29, No. 12, p. 1565.
50. Silver, D., Bagnell, J. and Stentz, A. (2011), "Perceptual interpretation for autonomous navigation through dynamic imitation learning" in International Symposium on Robotics Research, pp. 433-49.
51. Sondik, E. (1971), "The optimal control of partially observable Markov processes", Stanford University, Stanford, CA, PhD thesis.
52. Sutton, R. and Barto, A. (1998), Introduction to Reinforcement Learning, MIT Press, Cambridge, MA.
53. Syed, U. and Schapire, R. (2008), "A game-theoretic approach to apprenticeship learning" in Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA, pp. 1449-56.
54. Tang, J., Singh, A., Goehausen, N. and Abbeel, P. (2010), "Parameterized maneuver learning for autonomous helicopter flight" in Robotics and Automation (ICRA), 2010 IEEE International Conference on, pp. 1142-8.
55. Theodorou, E., Buchli, J. and Schaal, S. (2010), "Reinforcement learning of motor skills in high dimensions: a path integral approach" in Robotics and Automation (ICRA), 2010 IEEE International Conference on, pp. 2397-403.
56. Zhang, H. and Parkes, D. (2008), "Enabling environment design via active indirect elicitation", Proceedings Workshop on Preference Handling, Chicago, IL.
57. Ziebart, B., Maas, A., Bagnell, J. and Dey, A. (2008), "Maximum entropy inverse reinforcement learning" in Proceedings 23rd AAAI Conference Artificial Intelligence, pp. 1433-8.
58. Leishman, J. (2006), Principles of Helicopter Aerodynamics, Cambridge University Press, New York, NY.