A tutorial on visual servo control

IEEE Transactions on Robotics and Automation

Volumn 12, Issue 5, 1996, Pages 651-670

  Hutchinson, Seth ^a,b   Hager, Gregory D ^a,c   Corke, Peter I ^a,d  

^a IEEE

^b University of Illinois at Urbana Champaign   (United States)

^c Yale University   (United States)

^d CSIRO   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; CONTROL SYSTEMS; IMAGE PROCESSING; MANIPULATORS; STEREO VISION;

TUTORIAL; VISUAL SERVO CONTROL;

SERVOMECHANISMS;

EID: 0030261036     PISSN: 1042296X     EISSN: None     Source Type: Journal    
DOI: 10.1109/70.538972     Document Type: Article

References (80)

1. Y. Shirai and H. Inoue, "Guiding a robot by visual feedback in assembling tasks," Pattern Recognit., vol. 5, pp. 99-108, 1973.
2. J. Hill and W. T. Park, "Real time control of a robot with a mobile camera," in Proc. 9th ISIR, Washington, D.C., Mar. 1979, pp. 233-246.
3. P. Corke, "Visual control of robot manipulators - A review," in Visual Servoing K. Hashimoto, Ed. Singapore: World Scientific, 1993, pp. 1-31. (vol. 7 of Robotics and Automated Systems).
4. G. D. Hager, "The "X-vision" system: A general purpose substrate for real-time vision-based robotics," in Proc. Workshop on Vision for Robots, 1995, pp. 56-63, 1995. Also available as Yale CS-RR-1078.
5. A. C. Sanderson and L. E. Weiss, "Image-based visual servo control using relational graph error signals," Proc. IEEE, pp. 1074-1077, 1980.
6. J. C. Latombe, Robot Motion Planning. Boston: Kluwer, 1991.
7. J. J. Craig, Introduction to Robotics. Menlo Park: Addison-Wesley, 2nd ed., 1986.
8. B. K. P. Horn, Robot Vision. Cambridge, MA: MIT Press, 1986.
9. N. Hollinghurst and R. Cipolla, "Uncalibrated stereo hand eye coordination," Image and Vision Computing, vol. 12, no. 3, pp. 187-192, 1994.
10. J. Feddema and O. Mitchell, "Vision-guided servoing with feature-based trajectory generation," IEEE Trans. Robot. Automat., vol. 5, pp. 691-700, Oct. 1989.
11. B. Espiau, F. Chaumette, and P. Rives, "A new approach to visual servoing in robotics," IEEE Trans. Robot. Automat., vol. 8, pp. 313-326, 1992.
12. M. L. Cyros, "Datacube at the space shuttle's launch pad," Datacube World Review, vol. 2, pp. 1-3, Sept. 1988. Datacube Inc., 4 Dearborn Road, Peabody, MA.
13. W. Jang, K. Kim, M. Chung, and Z. Bien, "Concepts of augmented image space and transformed feature space for efficient visual servoing of an "eye-in-hand robot"," Robotica, vol. 9, pp. 203-212, 1991.
14. A. Castano and S. A. Hutchinson, "Visual compliance: Task-directed visual servo control," IEEE Trans. Robot. Automat., vol. 10, pp. 334-342, June 1994.
15. K. Hashimoto, T. Kimoto, T. Ebine, and H. Kimura, "Manipulator control with image-based visual servo," in Proc. IEEE Int'l Conf. on Robotics and Automation, 1991, pp. 2267-2272.
16. N. P. Papanikolopoulos and P. K. Khosla, "Adaptive robot visual tracking: Theory and experiments," IEEE Trans. Automat. Contr., vol. 38, no. 3, pp. 429-445, 1993.
17. N. P. Papanikolopoulos, P. K. Khosla, and T. Kanade, "Visual tracking of a moving target by a camera mounted on a robot: A combination of vision and control," IEEE Trans. Robot. Automat., vol. 9, no. 1, pp. 14-35, 1993.
18. S. Skaar, W. Brockman, and R. Hanson, "Camera-space manipulation," Int. J. Robot. Res., vol. 6, no. 4, pp. 20-32, 1987.
19. S. B. Skaar, W. H. Brockman, and W. S. Jang, "Three-dimensional camera space manipulation," Int. J. Robot. Res., vol. 9, no, 4, pp. 22-39, 1990.
20. J. T. Feddema, C. S. G. Lee, and O. R. Mitchell, "Weighted selection of image features for resolved rate visual feedback control," IEEE Trans. Robot. Automat., vol. 7, pp. 31-47, Feb. 1991.
21. A. C. Sanderson, L. E. Weiss, and C. P. Neuman, "Dynamic sensor-based control of robots with visual feedback," IEEE Trans. Robot. Automat., vol. RA-3, pp. 404-417, Oct. 1987.
22. M. Lei and B. K. Ghosh, "Visually-guided robotic motion tracking," in Proc. Thirtieth Annu. Allerton Conf. on Communication, Control, and Computing, 1992, pp. 712-721.
23. R. L. Andersson, A Robot Ping-Pong Player. Experiment in Real-Time Intelligent Control. Cambridge, MA: MIT Press, 1988.
24. B. Yoshimi and P. K. Allen, "Active, uncalibrated visual servoing," in Proc. IEEE Int. Conf. on Robotics and Automation, San Diego, CA, May 1994, pp. 156-161.
25. B. Nelson and P. K. Khosla, "Integrating sensor placement and visual tracking strategies," in Proc. IEEE Int'l Conf. on Robotics and Automation, 1994, pp. 1351-1356.
26. I. E. Sutherland, "Three-dimensional data input by tablet," Proc. IEEE, vol. 62, pp. 453-461, Apr. 1974.
27. R. Tsai and R. Lenz, "A new technique for fully autonomous and efficient 3D robotics hand/eye calibra tion," IEEE Trans. Robot. Automat., vol. 5, pp. 345-358, June 1989.
28. R. Tsai, "A versatile camera calibration technique for high accuracy 3D machine vision m etrology using off-the-shelf TV cameras and lenses," IEEE Trans. Robot. Automat., vol. 3, pp. 323-344, Aug. 1987.
29. P. I. Corke, High-Performance Visual Closed-Loop Robot Control. Ph.D. Dissertation, University of Melbourne, Dept. Mechanical and Manufacturing Engineering, July 1994.
30. D. E. Whitney, "The mathematics of coordinated control of prosthetic arms and manipulators," J. Dyn. Syst., Meas. Control, vol. 122, pp. 303-309, Dec. 1972.
31. S. Chieaverini, L. Sciavicco, and B. Siciliano, "Control of robotic systems through singularities," in Proc. Int. Workshop on Nonlinear and Adaptive Control: Issues in Robotics C. C. de Wit, Ed. Berlin: Springer-Verlag, 1991.
32. S. Wijesoma, D. Wolfe, and R. Richards, "Eye-to-hand coordination for vision-guided robot control applications," Int. J. Robot. Res., vol. 12, no. 1, pp. 65-78, 1993.
33. G. D. Hager, W.-C. Chang, and A. S. Morse, "Robot hand-eye coordination based on stereo vision," IEEE Control Syst. Mag., vol. 15, pp. 30-39, Feb. 1995.
34. C. Samson, M. Le Borgne, and B. Espiau, Robot Control: The Task Function Approach. Oxford, England: Clarendon, 1992.
35. G. Franklin, J. Powell, and A. Emami-Naeini, Feedback Control of Dynamic Systems. Boston, MA: Addison-Wesley, 2nd ed., 1991.
36. P. K. Allen, A. Timcenko, B. Yoshimi, and P. Michelman, "Automated tracking and grasping of a moving object with a robotic hand-eye system," IEEE Trans. Robot. Automat., vol. 9, no. 2, pp. 152-165, 1993.
37. A. Rizzi and D. Koditschek, "An active visual estimator for dexterous manipulation," in Proc. IEEE Int. Conf. on Robotics and Automation, 1994.
38. T. S. Huang and A. N. Netravali, "Motion and structure from feature correspondences: A review," Proc. IEEE, vol. 82, no. 2, pp. 252-268, 1994.
39. M. A. Fischler and R. C. Bolles, "Random sample consensus: a paradigm for model fitting with applicatio ns to image analysis and automated cartography," Commun. ACM, vol. 24, pp. 381-395, June 1981.
40. R. M. Haralick, C. Lee, K. Ottenberg, and M. Nolle, "Analysis and solutions of the three point perspective pose estimation problem," in Proc. IEEE Conf. Computer Vision Pattern Recognition, pp. 592-598, 1991.
41. D. DeMenthon and L. S. Davis, "Exact and approximate solutions of the perspective-three-point problem," IEEE Trans. Pattern Anal. Machine Intell., no. 11, pp. 1100-1105, 1992.
42. R. Horaud, B. Canio, and O. Leboullenx, "An analytic solution for the perspective 4-point problem," Comput. Vision Graphics, Image Process, no. 1, pp. 33-44, 1989.
43. M. Dhome, M. Richetin, J. Lapresté, and G. Rives, "Determination of the attitude of 3-D objects from a single perspective view," IEEE Trans. Pattern Anal. Machine Intell., no. 12, pp. 1265-1278, 1989.
44. G. H. Rosenfield, "The problem of exterior orientation in photogrammetry," Photogrammetric Eng., pp. 536-553, 1959.
45. D. G. Lowe, "Fitting parametrized three-dimensional models to images," IEEE Trans. Pattern Anal. Machine Intell., no. 5, pp. 441-450, 1991.
46. R. Goldberg, "Constrained pose refinement of parametric objects," Int. J. Comput. Vision, no. 2, pp. 181-211, 1994.
47. R. Kumar, "Robust methods for estimating pose and a sensitivity analysis," CVGIP: Image Understanding, no. 3, pp. 313-342, 1994.
48. S. Ganapathy, "Decomposition of transformation matrices for robot vision," Pattern Recog. Lett., pp. 401-412, 1989.
49. M. Fischler and R. C. Bolles, "Random sample consensus: A paradigm for model fitting and automatic cartography," Commun. ACM, no. 6, pp. 381-395, 1981.
50. Y. Liu, T. S. Huang, and O. D. Faugeras, "Determination of camera location from 2-D to 3-D line and point correspondences," IEEE Trans. Pat. Anal. Machine Intell., no. 1, pp. 28-37, 1990.
51. C. Lu, E. J. Mjolsness, and G. D. Hager, "Online computation of exterior orientation with application to hand-eye calibration," DCS RR-1046, Yale University, New Haven, CT, Aug. 1994;
52. To appear in Mathematical and Computer Modeling.
53. A. Gelb, Ed., Applied Optimal Estimation. Cambridge, MA: MIT Press, 1974.
54. W. Wilson, "Visual servo control of robots using Kalman filter estimates of robot pose relative to work-pieces," in Visual Servoing, K. Hashimoto, Ed. Singapore: World Scientific, 1994, pp. 71-104.
55. C. Fagerer, D. Dickmanns, and E. Dickmanns, "Visual grasping with long delay time of a free floating object in orbit," Auton. Robots, vol. 1, no. 1, 1994.
56. J. Pretlove and G. Parker, "The development of a real-time stereo-vision system to aid robot guidance in carrying out a typical manufacturing task," in Proc. 22nd ISRR, Detroit, MI, 1991, pp. 21.1-21.23.
57. B. K. P. Horn, H. M. Hilden, and S. Negahdaripour, "Closed-form solution of absolute orientation using orthonormal matrices," J. Opt. Soc. Amer., vol. A-5, pp. 1127-1135, 198.
58. K. S. Arun, T. S. Huang, and S. D. Blostein, "Least-squares fitting of two 3-D point sets," IEEE Trans. Pattern Anal. Machine Intell., vol. 9, pp. 698-700, 1987.
59. B. K. P. Horn, "Closed-form solution of absolute orientation using unit quaternions," J. Opt. Soc. Amer., vol. A-4, pp. 629-642, 1987.
60. G. D. Hager, G. Grunwald, and G. Hirzinger, "Feature-based visual servoing and its application to telerobotics," in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Jan. 1994, pp. 164-171.
61. G. Agin, "Calibration and use of a light stripe range sensor mounted on the hand of a robot," in Proc. IEEE Int. Conf. on Robotics and Automation, 1985, pp. 680-685.
62. S. Venkatesan and C. Archibald, "Realtime tracking in five degrees of freedom using two wrist-mounted laser range finders," in Proc. IEEE Int. Conf. on Robotics and Automation, 1990, pp. 2004-2010.
63. J. Dietrich, G. Hirzinger, B. Gombert, and J. Schott, "On a unified concept for a new generation of light-weight robots," in Experimental Robotics 1, V. Hayward and O. Khatib, eds., Berling, Germany: Springer-Verlag, 1989, pp. 287-295. (vol. 139 of Lecture Notes in Control and Information Sciences).
64. J. Aloimonos and D. P. Tsakiris, "On the mathematics of visual tracking," Image and Vision Computing, vol. 9, pp. 235-251, Aug. 1991.
65. R. M. Haralick and L. G. Shapiro, Computer and Robot Vision. Reading, MA: Addison-Wesley, 1993.
66. F. W. Warner, Foundations of Differentiable Manifolds and Lie Groups. New York: Springer-Verlag, 1983.
67. W. Jang and Z. Bien, "Feature-based visual servoing of an eye-in-hand robot with improved tracking performance," in Proc. IEEE Int. Conf. on Robotics and Automation, 1991, pp. 2254-2260.
68. O. Faugeras, Three-Dimensional Computer Vision. Cambridge, MA: MIT Press, 1993.
69. G. D. Hager, "Calibration-free visual control using projective invariance," in Proc. ICCV, pp. 1009-1015, 1995. Also available as Yale CS-RR-1046.
70. D. Kim, A. Rizzi, G. D. Hager, and D. Koditschek, "A "robust" convergent visual servoing system," in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 1995, vol. I, pp. 348-353.
71. R. L. Anderson, "Dynamic sensing in a ping-pong playing robot," IEEE Trans. Robot. Automat., vol. 5, no. 6, pp. 723-739, 1989.
72. E. Dickmanns and V. Graefe, "Dynamic monocular machine vision," Mach. Vis. Applicat., vol. 1, pp. 223-240, 1988.
73. J. Foley, A. van Dam, S. Feiner, and J. Hughes, Computer Graphics. Reading, MA: Addison-Wesley, 1993.
74. D. Ballard and C. Brown, Computer Vision. Englewood Cliffs, NJ: Prentice-Hall, 1982.
75. B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. Int. Joint Conf. on Artificial Intelligence, 1981, pp. 674-679.
76. P. Anandan, "A computational framework and an algorithm for the measurement of structure from motion," Int. J. Comput. Vis., vol. 2, pp. 283-310, 1989.
77. J. Shi and C. Tomasi, "Good features to track," in Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 1994, pp. 593-600.
78. M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: active contour models," Int. J. Comput. Vis., vol. 1, no, 1, pp. 321-331, 1987.
79. K. Hosada and M. Asada, "Versatile visual servoing without knowledge of true jacobian," Proc. IROS 94, Sept. 1994, pp. 186-191.
80. P. Corke and M. Good, "Dynamic effects in visual closed-loop systems," IEEE Trans. Robot. Automat., this issue, pp. 671-683.