Autonomous helicopter aerobatics through apprenticeship learning

International Journal of Robotics Research

Volumn 29, Issue 13, 2010, Pages 1608-1639

  Abbeel, Pieter ^a   Coates, Adam ^b   Ng, Andrew Y ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b Stanford University   (United States)

Author keywords

Apprenticeship learning; autonomous flight; autonomous helicopter; helicopter aerobatics; learning from demonstrations

Indexed keywords

AIR SHOWS; APPRENTICESHIP LEARNING; AUTONOMOUS FLIGHT; AUTONOMOUS HELICOPTERS; AUTONOMOUS TRANSITIONS; CONTROL PROBLEMS; HUMAN EXPERT; HUMAN PILOTS; LEARNING FROM DEMONSTRATION; STATE OF THE ART;

APPRENTICES; DEMONSTRATIONS; HELICOPTERS;

LEARNING ALGORITHMS;

EID: 77955809093     PISSN: 02783649     EISSN: 17413176     Source Type: Journal    
DOI: 10.1177/0278364910371999     Document Type: Article

References (57)

1. Abbeel, P., Coates, A., Hunter, T. and Ng, A.Y. (2008). Autonomous auto-rotation of an RC helicopter. Proceedings of ISER.
2. Abbeel, P., Coates, A., Quigley, M. and Ng, A.Y. (2007). An application of reinforcement learning to aerobatic helicopter flight. Proceedings of NIPS 19.
3. Abbeel, P., Ganapathi, V. and Ng, A.Y. (2006 a). Learning vehicular dynamics with application to modeling helicopters. Proceedings of NIPS 18.
4. Abbeel, P. and Ng, A.Y. (2004). Apprenticeship learning via inverse reinforcement learning. Proceedings of ICML.
5. Abbeel, P. and Ng, A.Y. (2005 a). Exploration and apprenticeship learning in reinforcement learning. Proceedings of ICML.
6. Abbeel, P. and Ng, A.Y. (2005 b). Learning first order Markov models for control. Proceedings of NIPS 18.
7. Abbeel, P., Quigley, M., and Ng, A.Y. (2006 b). Using inaccurate models in reinforcement learning. Proceedings of ICML.
8. An, C.H., Atkeson, C.G. and Hollerbach, J.M. (1988). Model-Based Control of a Robot Manipulator. Cambridge, MA, MIT Press.
9. Anderson, B. and Moore, J. (1989). Optimal Control: Linear Quadratic Methods. Englewood Cliffs, NJ, Prentice-Hall.
10. Atkeson, C.G., Moore, A.W. and Schaal, S. (1997). Locally weighted learning for control. Artificial Intelligence Review, 11 ( 1-5 ).
11. Atkeson, C.G. and Schaal, S. (1997). Robot learning from demonstration. Proceedings of the 14th International Conference on Machine Learning. San Mateo, CA, Morgan Kaufmann, pp. 12-20.
12. Bagnell, J. and Schneider, J. (2001). Autonomous helicopter control using reinforcement learning policy search methods. International Conference on Robotics and Automation. Piscataway, NJ, IEEE Press.
13. Bar-Itzhack, I.Y. and Oshman, Y. (1985). Attitude determination from vector observations: Quaternion estimation. IEEE Transaction on Aerospace and Electronic Systems.
14. Bertsekas, D.P. (2001). Dynamic Programming and Optimal Control, volume 1, 2 nd edn. New York, Athena Scientific.
15. Beygelzimer, A., Kakade, S. and Langford, J. (2006). Cover trees for nearest neighbor. ICML '06: Proceedings of the 23rd International Conference on Machine Learning.
16. Boutilier, C., Friedman, N., Goldszmidt, M. and Koller, D. (1996). Context-specific independence in Bayesian networks. Proceedings of UAI.
17. Calinon, S., Guenter, F. and Billard, A. (2007). On learning, representing and generalizing a task in a humanoid robot. IEEE Transaction on Systems, Man, and Cybernetics, Part B, 37.
18. Coates, A., Abbeel, P. and Ng, A.Y. (2008). Learning for control from multiple demonstrations. Proceedings of ICML.
19. Dempster, A.P., Laird, N.M. and Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society.
20. Frazzoli, E., Dahleh, M. and Feron, E. (2002). Real-time motion planning for agile autonomous vehicles. AIAA Journal on Guidance, Control and Dynamics.
21. Frazzoli, E., Dahleh, M. and Feron, E. (2005). Maneuver-based motion planning for nonlinear systems with symmetries. IEEE Transactions on Robotics.
22. Gavrilets, V., Martinos, I., Mettler, B. and Feron, E. (2002 a). Control logic for automated aerobatic flight of miniature helicopter. AIAA Guidance, Navigation and Control Conference.
23. Gavrilets, V., Martinos, I., Mettler, B. and Feron, E. (2002 b). Flight test and simulation results for an autonomous aerobatic helicopter. AIAA/IEEE Digital Avionics Systems Conference.
24. Gavrilets, V., Martinos, M., Mettler, B. and Feron, E. (2002 c). Control logic for automated aerobatic flight of miniature helicopter. Proceedings of the AIAA Guidance, Navigation, and Control Conference.
25. Gelb, A. (ed.) (1974). Applied Optimal Estimation. Cambrdige, MA, MIT Press.
26. Jacobson, D.H. and Mayne, D.Q. (1970). Differential Dynamic Programming. Amsterdam, Elsevier.
27. Johnson, W. (1977). Helicopter Optimal Descent and Landing After Power Loss. Report NASA TM- 73244.
28. Kalman, R.E. (1960). A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering (Series D), 82: 35-45.
29. La Civita, M., Messner, W.C. and Kanade, T. (2002). Modeling of small-scale helicopters with integrated first-principles and system-identification techniques. Proceedings of the 58th Forum of the American Helicopter Society.
30. La Civita, M., Papageorgiou, G., Messner, W.C. and Kanade, T. (2003). Design and flight testing of a gain-scheduled H-infinity loop shaping controller for wide-envelope flight of a robotic helicopter. Proceedings of the American Control Conference.
31. La Civita, M., Papageorgiou, G., Messner, W.C. and Kanade, T. (2006). Design and flight testing of a high-bandwidth a loop shaping controller for a robotic helicopter. Journal of Guidance, Control, and Dynamics, 29 (2). 485-494.
32. Lee, A. (1985). Optimal Landing of a Helicopter in Autorotation. PhD Thesis, Stanford University.
33. Lefferts, E., Markley, F.L. and Shuster, M.D. (1982). Kalman filtering for spacecraft attitude estimation. Journal of Guidance, Control, and Dynamics.
34. Leishman, J. (2000). Principles of Helicopter Aerodynamics. Cambridge, Cambridge University Press.
35. Listgarten, J. (2006). Analysis of Sibling Time Series Data: Alignment and Difference Detection. PhD Thesis, University of Toronto.
36. Listgarten, J., Neal, R.M., Roweis, S.T. and Emili, A. (2005). Multiple alignment of continuous time series. Proceedings of NIPS 17.
37. Mettler, B., Tischler, M. and Kanade, T. (1999). System identification of small-size unmanned helicopter dynamics. American Helicopter Society, 55th Forum.
38. Moore, A., Schneider, J. and Deng, K. (1997). Efficient locally weighted polynomial regression predictions. Proceedings of ICML.
39. Neal, R. and Hinton, G. (1999). A view of the EM algorithm that justifies incremental, sparse, and other variants. Learning in Graphical Models. Cambrdige, MA, MIT Press, pp. 355-368.
40. Needleman, S. and Wunsch, C. (1970). A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology.
41. Neu, G. and Szepesvari, C. (2007). Apprenticeship learning using inverse reinforcement learning and gradient methods. Proceedings of UAI.
42. Ng, A.Y., Coates, A., Diel, M., Ganapathi, V., Schulte, J., Tse, B., Berger, E. and Liang, E. (2004 a). Autonomous inverted helicopter flight via reinforcement learning. Proceedings of ISER.
43. Ng, A.Y., Kim, H.J., Jordan, M., and Sastry, S. (2004 b). Autnonomous helicopter flight via reinforcement learning. Proceedings of NIPS 16.
44. Ng, A.Y. and Russell, S. (2000). Algorithms for inverse reinforcement learning. Proceedings of ICML.
45. Preparata, F.P. and Shamos, M. (1985). Computational Geometry. Berlin, Springer-Verlag.
46. Ramachandran, D. and Amir, E. (2007). Bayesian inverse reinforcement learning. Proceedings of IJCAI.
47. Randlov, J. and Alstrom, P. (1998). Learning to drive a bicycle using reinforcement learning and shaping. Proceedings of ICML.
48. Ratliff, N., Bagnell, J. and Zinkevich, M. (2006). Maximum margin planning. Proceedings of ICML.
49. Roberts, J.M., Corke, P.I. and Buskey, G. (2003). Low-cost flight control system for a small autonomous helicopter. IEEE International Conference on Robotics and Automation.
50. Sakoe, H. and Chiba, S. (1978). Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing.
51. Saripalli, S., Montgomery, J. and Sukhatme, G. (2003). Visually-guided landing of an unmanned aerial vehicle.
52. Seddon, J. (1990). Basic Helicopter Aerodynamics (AIAA Education Series). America Institute of Aeronautics and Astronautics.
53. Shuster, M.D. (2003). Constraint in attitude estimation: Part II unconstrained estimation. Journal of the Astronautical Sciences.
54. Syed, U. and Schapire, R.E. (2008). A game-theoretic approach to apprenticeship learning. Proceedings of NIPS 20.
55. Tedrake, R., Zhang, T.W. and Seung, H.S. (2004). Stochastic policy gradient reinforcement learning on a simple 3D biped. Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS).
56. Tischler, M. and Cauffman, M. (1992). Frequency response method for rotorcraft system identification: flight application to BO-105 couple rotor/fuselage dynamics. Journal of the American Helicopter Society.
57. Zanetti, R. and Bishop, R.H. (2006). Quaternion estimation and norm constrained Kalman filtering. AIAA/AAS Astrodynamics Specialist Conference.