Model-based and Model-free Reinforcement Learning for Visual Servoing

Proceedings - IEEE International Conference on Robotics and Automation

Volumn , Issue , 2009, Pages 2917-2924

  Farahmand, Amir Massoud ^a   Shademan, A Azad ^a   Jagersand M ^a   Szepesvari C ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

APRIORI; ESTIMATED MODEL; FORWARD KINEMATICS; INTERMEDIATE MODEL; LINEAR REGRESSION METHODS; MODEL FREE; MODEL-BASED; REINFORCEMENT LEARNING METHOD; ROBOT SYSTEM; SIMULATION RESULT;

LIGHT MEASUREMENT; REINFORCEMENT; REINFORCEMENT LEARNING; ROBOTICS; ROBOTS;

VISUAL SERVOING;

EID: 70350418421     PISSN: 10504729     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ROBOT.2009.5152834     Document Type: Conference Paper

References (31)

1. S. Hutchinson, G. Hager, and P. Corke, "A tutorial on visual servo control," IEEE Trans. Robotics and Automation, vol. 12, no. 5, pp. 651-670, Oct. 1996.
2. F. Chaumette and S. Hutchinson, "Visual servo control, part I: Basic approaches," IEEE Robotics and Automation Magazine, vol. 13, no. 4, pp. 82-90, Dec. 2006.
3. "Visual servo control, part II: Advanced approaches," IEEE Robotics and Automation Magazine, vol. 14, no. 1, pp. 109-118, Mar. 2007.
4. K. Hosoda and M. Asada, "Versatile visual servoing without knowledge of true Jacobian," in IEEE/RSJ International Conf. Intelligent Robots and Systems (IROS), vol. 1, September 1994, pp. 186-193.
5. M. Jägersand, O. Fuentes, and R. Nelson, "Experimental evaluation of uncalibrated visual servoing for precision manipulation," in IEEE International Conf. Robotics and Automation (ICRA), vol. 4, April 1997, pp. 2874-2880.
6. H. Sutanto, R. Sharma, and V. Varma, "The role of exploratory movement in visual servoing without calibration," Robotics and Autonomous Systems, vol. 23, pp. 153-169, 1998.
7. J. A. Piepmeier, G. V. McMurray, and H. Lipkin, "Uncalibrated dynamic visual servoing," IEEE Trans. Robotics and Automation, vol. 20, no. 1, pp. 143-147, February 2004.
8. W. J. Wilson, C. C. W. Hulls, and G. S. Bell, "Relative end-effector control using cartesian position based visual servoing," IEEE Trans. Robotics and Automation, vol. 12, no. 5, pp. 684-696, October 1996.
9. L. Weiss, A. Sanderson, and C. Neuman, "Dynamic sensor-based control of robots with visual feedback," IEEE Journal of Robotics and Automation, vol. 3, no. 5, pp. 404-417, 1987.
10. B. Espiau, F. Chaumette, and P. Rives, "A new approach to visual servoing in robotics," IEEE Trans. Robotics and Automation, vol. 8, no. 3, pp. 313-326, June 1992.
11. E. Malis, F. Chaumette, and S. Boudet, "2-1/2-d visual servoing," IEEE Trans. Robotics and Automation, vol. 15, no. 2, pp. 238-250, April 1999.
12. A. M. Farahmand, M. Ghavamzadeh, C. Szepesvári, and S. Mannor, "Regularized fitted Q-iteration: Application to bounded resource planning," in Recent Advances in Reinforcement Learning: 8th European Workshop, EWRL 2008, Villeneuve d'Ascq, France, June 30-July 3, 2008, Revised and Selected Papers. Berlin, Heidelberg: Springer-Verlag, 2008, pp. 55-68.
13. L. Wasserman, All of Nonparametric Statistics (Springer Texts in Statistics). Secaucus, NJ, USA: Springer-Verlag New York, Inc., 2006.
14. A. M. Farahmand, A. Shademan, and M. Jägersand, "Global visualmotor estimation for uncalibrated visual servoing," in IEEE/RSJ In-ternational Conf. Intelligent Robots and Systems (IROS), 2007.
15. S. Schaal, C. G. Atkeson, and S. Vijayakumar, "real-time robot learning with locally weighted statistical learning," in international conference on robotics and automation (icra2000), 2000.
16. S. Mannor, I. Menache, and N. Shimkin, "Basis function adaptation in temporal difference reinforcement learning," Annals of Operations Research, vol. 134, pp. 215-238, 2005.
17. R. Parr, C. Painter-Wakefield, L. Li, and M. Littman, "Analyzing feature generation for value-function approximation," in ICML, 2007, pp. 737-744.
18. D. Ernst, P. Geurts, and L. Wehenkel, "Tree-based batch mode reinforcement learning," Journal of Machine Learning Research, vol. 6, pp. 503-556, 2005.
19. Y. Engel, S. Mannor, and R. Meir, "Reinforcement learning with Gaussian processes," in ICML '05: Proceedings of the 22nd international conference on Machine learning. New York, NY, USA: ACM, 2005, pp. 201-208.
20. L. Györfi, M. Kohler, A. Krzy?zak, and H. Walk, A distribution-free theory of nonparametric regression. New York: Springer-Verlag, 2002.
21. T. Jung and D. Polani, "Least squares SVM for least squares TD learning," in ECAI, 2006, pp. 499-503.
22. M. Loth, M. Davy, and P. Preux, "Sparse temporal difference learning using LASSO," in IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, 2007.
23. D. P. Bertsekas and S. Shreve, Stochastic Optimal Control (The Discrete Time Case). Academic Press, New York, 1978.
24. B. Schölkopf and A. Smola, Learning with Kernels. Cambridge, MA: MIT Press, 2002.
25. Y. Engel, S. Mannor, and R. Meir, "The kernel recursive least squares algorithm," IEEE Transaction on Signal Processing, vol. 52, no. 8, pp. 2275-2285, 2004.
26. A. Antos, R. Munos, and C. Szepesvári, "Fitted Q-iteration in continuous action-space MDPs," in Advances in Neural Information Processing Systems 20 (NIPS-2007), 2008, (in print).
27. D.-X. Zhou, "Capacity of reproducing kernel spaces in learning theory," IEEE Transactions on Information Theory, vol. 49, pp. 1743-1752, 2003.
28. R. S. Sutton, C. Szepesvári, A. Geramifard, and M. Bowling, "Dynastyle planning with linear function approximation and prioritized sweeping," in Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, 2008.
29. P. Corke, "A robotics toolbox for MATLAB," IEEE Robotics and Automation Magazine, vol. 3, no. 1, pp. 24-32, Mar. 1996.
30. G. Mariottini and D. Prattichizzo, "EGT: a toolbox for multiple view geometry and visual servoing," IEEE Robotics and Automation Magazine, vol. 3, no. 12, December 2005.
31. V. Mnih, C. Szepesvári, and J.-Y. Audibert, "Empirical bernstein stopping," Proceedings of the 25th Annual International Conference on Machine Learning (ICML 2008), pp. 672-679, 2008.