Herb 2.0: Lessons learned from developing a mobile manipulator for the home

Proceedings of the IEEE

Volumn 100, Issue 8, 2012, Pages 2410-2428

  Srinivasa, Siddhartha S ^a   Berenson, Dmitry ^b   Cakmak, Maya ^c   Collet, Alvaro ^a   Dogar, Mehmet R ^a   Dragan, Anca D ^a   Knepper, Ross A ^a   Niemueller, Tim ^d   Strabala, Kyle ^a   Vande Weghe, Mike ^a   Ziegler, Julius ^e  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c Georgia Institute of Technology   (United States)

^d RWTH AACHEN UNIVERSITY   (Germany)

^e KARLSRUHE INSTITUTE OF TECHNOLOGY   (Germany)

Author keywords

Human robot interaction; mobile manipulation; motion planning; perception; personal robotics; trajectory optimization

Indexed keywords

HUMAN ROBOT INTERACTION; MANIPULATORS; MOTION PLANNING; ROBOTICS; SENSORY PERCEPTION;

CARNEGIE MELLON UNIVERSITY , PITTSBURGH; HUMAN ENVIRONMENT; INTEGRATED PLATFORM; MOBILE MANIPULATION; MOBILE MANIPULATOR; OFFICE ENVIRONMENTS; PERSONAL ROBOTICS; TRAJECTORY OPTIMIZATION;

ROBOT PROGRAMMING;

EID: 84864117734     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2012.2200561     Document Type: Conference Paper

References (80)

1. S. Srinivasa, D. Ferguson, J. Vande Weghe, D. Berenson, R. Diankov, C. Helfrich, and H. Strasdat, "The robotic busboy: Steps towards developing a mobile robotic home assistant," in Proc. IEEE Int. Conf. Intell. Autonom. Syst., Jul. 2008.
2. S. Srinivasa, D. Ferguson, C. Helfrich, D. Berenson, A. Collet, R. Diankov, G. Gallagher, G. Hollinger, J. Kuffner, and J. Vande Weghe, "HERB: A home exploring robotic butler," Autonom. Robots, vol. 28, no. 1, pp. 5-20, 2009.
3. N. Nilsson, "A mobile automation: An application of artificial intelligence techniques," in Proc. 1st Int. Joint Conf. Artif. Intell., 1969, pp. 509-520.
4. A. P. Ambler, H. G. Barrow, C. M. Brown, R. M. Burstall, and R. J. Popplestone, "A versatile computer-controlled assembly system," in Proc. 3rd Int. Joint Conf. Artif. Intell., 1973, pp. 298-307.
5. T. Lozano-Perez, J. Jones, E. Mazer, P. O'Donnell, W. Grimson, P. Tournassoud, and A. Lanusse, "Handey: A robot system that recognizes, plans, and manipulates," in Proc. IEEE Int. Conf. Robot. Autom., 1987, vol. 4, pp. 843-849. (Pubitemid 17595695)
6. T. Lozano-Perez, J. Jones, E. Mazer, and P. O'Donnell, Handey: A robot task planner, 1992.
7. A. M. Thompson, "The navigation system of the JPL robot," in Proc. 5th Int. Joint Conf. Artif. Intell., 1977, pp. 749-757.
8. O. Khatib, K. Yokoi, K. Chang, D. Ruspini, R. Holmberg, and A. Casal, "Coordination and decentralized cooperation of multiple mobile manipulators," J. Robot. Syst., vol. 13, no. 11, pp. 755-764, 1996. (Pubitemid 126591413)
9. R. Alami, L. Aguilar, H. Bullata, S. Fleury, M. Herrb, F. Ingrand, M. Khatib, and F. Robert, "A general framework for multi-robot cooperation and its implementation on a set of three hilare robots," Exper. Robot. IV, pp. 26-39, 1997. (Pubitemid 126155688)
10. Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki, and K. Fujimura, "The intelligent asimo: System overview and integration," in Proc. IEEE Int. Conf. Intell. Robots Syst., 2002, vol. 3, pp. 2478-2483. (Pubitemid 35438181)
11. K. Nishiwaki, T. Sugihara, S. Kagami, F. Kanehiro, M. Inaba, and H. Inoue, "Design and development of research platform for perception-action integration in humanoid robot: H6," in Proc. Int. Conf. Intell. Robots Syst., 2000, vol. 3, pp. 1559-1564. (Pubitemid 32379937)
12. S. Kagami, K. Nishiwaki, J. Kuffner, Jr., Y. Kuniyoshi, M. Inaba, and H. Inoue, "Online 3D vision, motion planning and bipedal locomotion control coupling system of humanoid robot: H7," in Proc. IEEE Int. Conf. Intell. Robots Syst., 2002, vol. 3, pp. 2557-2562. (Pubitemid 35438194)
13. K. Kaneko, F. Kanehiro, S. Kajita, H. Hirukawa, T. Kawasaki, M. Hirata, K. Akachi, and T. Isozumi, "Humanoid robot hrp-2," in Proc. IEEE Int. Conf. Robot. Automat., 2004, vol. 2, pp. 1083-1090.
14. K. Akachi, K. Kaneko, N. Kanehira, S. Ota, G. Miyamori, M. Hirata, S. Kajita, and F. Kanehiro, "Development of humanoid robot hrp-3p," in Proc. 5th IEEE-RAS Int. Conf. Humanoid Robots, 2005, pp. 50-55, IEEE. (Pubitemid 46170791)
15. K. Kaneko, F. Kanehiro, M. Morisawa, K. Miura, S. Nakaoka, and S. Kajita, "Cybernetic human hrp-4c," in Proc. IEEE 9th IEEE-RAS Int. Conf. Humanoid Robots, 2009, pp. 7-14.
16. M. Ciocarlie, K. Hsiao, E. Jones, S. Chitta, R. Rusu, and I. Sucan, "Towards reliable grasping and manipulation in household environments," in Proc. RSS Workshop Strategies Eval. Mobile Manipulation in Household Environ., 2010.
17. A. Albu-Schäffer, S. Haddadin, C. Ott, A. Stemmer, T. Wimböck, and G. Hirzinger, "The DLR lightweight robot: Design and control concepts for robots in human environments," Ind. Robot: An Int. J., vol. 34, no. 5, pp. 376-385, 2007. (Pubitemid 47283169)
18. M. Beetz, L. Mosenlechner, and M. Tenorth, "Crama cognitive robot abstract machine for everyday manipulation in human environments," in Proc. IEEE/RSJ Int. Conf. Intell. Robots. Syst., 2010, pp. 1012-1017.
19. A. Jain and C. Kemp, "El-e: An assistive mobile manipulator that autonomously fetches objects from flat surfaces," Autonom. Robots, vol. 28, no. 1, pp. 45-64, 2010.
20. M. Quigley, E. Berger, and A. Ng, "Stair: Hardware and software architecture," in Proc. AAAI Robot. Workshop, Vancouver, "C, Canada, 2007.
21. M. Quigley, K. Conley, B. P. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng, BROS: An open-source robot operating system," in Proc. ICRA Workshop Open Source Softw., 2009.
22. K. Konolige, "COLBERT: A language for reactive control in sapphira," in Proc. 21st Annu. German Conf. Artif. Intell., KI-97: Adv. Artif. Intell., Freiburg, Germany, 1997, pp. 31-52. (Pubitemid 127120113)
23. M. Loetzsch, M. Risler, and M. Jungel, "XABSLVA pragmatic approach to behavior engineering," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2006, pp. 5124-5129. (Pubitemid 46928743)
24. T. Niemueller, A. Ferrein, and G. Lakemeyer, "A Lua-based behavior engine for controlling the humanoid robot nao," in Proc. RoboCup XIII Symp., Graz, Austria, 2009.
25. T. Niemueller, A. Ferrein, D. Beck, and G. Lakemeyer, "Design principles of the component-based robot software framework fawkes," in Proc. 2nd Int. Conf. Simul., Model., Program. Auton. Robots, 2010.
26. R. Ierusalimschy, Programming in Lua, 2nd ed. New York: Lua.org, 2006.
27. J. Bohren and S. Cousins, "The SMACH high-level executive," IEEE Robot. Autom. Mag., vol. 17, no. 4, pp. 18-20, 2010.
28. D. Berenson, S. Srinivasa, and J. Kuffner, "Task space regions: A framework for pose-constrained manipulation planning," Int. J. Robot. Res., Mar. 2011.
29. D. Berenson, T. Simeon, and S. Srinivasa, "Addressing cost-space chasms in manipulation planning," in Proc. IEEE Int. Conf. Robot. Autom., May 2011.
30. M. Stilman, "Task constrained motion planning in robot joint space," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2007.
31. Y. Koga, K. Kondo, J. Kuffner, and J. claude Latombe, "Planning motions with intentions," in Proc. SIGGRAPH, 1994.
32. K. Yamane, J. Kuffner, and J. Hodgins, "Synthesizing animations of human manipulation tasks," in Proc. SIGGRAPH, 2004.
33. Z. Yao and K. Gupta, "Path planning with general end-effector constraints: Using task space to guide configuration space search," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2005.
34. E. Drumwright and V. Ng-Thow-Hing, "Toward interactive reaching in static environments for humanoid robots," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2006.
35. D. Bertram, J. Kuffner, R. Dillmann, and T. Asfour, "An integrated approach to inverse kinematics and path planning for redundant manipulators," in Proc. IEEE Int. Conf. Robot. Autom., 2006.
36. D. Berenson, J. Chestnutt, S. S. Srinivasa, J. J. Kuffner, and S. Kagami, "Pose-constrained whole-body planning using task space region chains," in Proc. IEEE-RAS Int. Conf. Humanoid Robots, 2009.
37. D. Berenson, S. Srinivasa, and J. Kuffner, "Addressing pose uncertainty in manipulation planning using task space regions," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2009.
38. D. Berenson, S. S. Srinivasa, D. Ferguson, and J. Kuffner, "Manipulation planning on constraint manifolds," in Proc. IEEE Int. Conf. Robot. Autom., 2009.
39. J. H. Yakey, S. M. LaValle, and L. E. Kavraki, "Randomized path planning for linkages with closed kinematic chains," IEEE Trans. Robot. Autom., vol. 17, no. 6, pp. 951-958, 2001. (Pubitemid 34125094)
40. J. Cortes and T. Simeon, "Sampling-based motion planning under kinematic loop-closure constraints," in Proc. Workshop Algorithmic Found. Robot., 2004.
41. D. Berenson and S. Srinivasa, "Probabilistically complete planning with end-effector pose constraints," in Proc. IEEE Int. Conf. Robot. Automat., May 2010.
42. S. Dalibard, A. Nakhaei, F. Lamiraux, and J.-P. Laumond, "Whole-body task planning for a humanoid robot: A way to integrate collision avoidance," in Proc. Humanoids, 2009.
43. D. Berenson, "Constrained manipulation planning," Ph.D. dissertation, Robotics Inst., Carnegie Mellon Univ., Pittsburgh, PA, May 2011.
44. M. Dogar and S. Srinivasa, "Push-grasping with dexterous hands," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Oct. 2010.
45. M. Dogar and S. Srinivasa, "A framework for push-grasping in clutter," in Proc. Robot: Sci. Syst. VII, 2011.
46. T. Lozano-Perez, M. T. Mason, and R. H. Taylor, "Automatic synthesis of fine-motion strategies for robots," IJRR, vol. 3, no. 1, 1984.
47. S. Goyal, A. Ruina, and J. Papadopoulos, "Planar sliding with dry friction. Part 1. Limit surface and moment function," Wear, no. 143, pp. 307-330, 1991.
48. N. D. Ratliff, M. Zucker, J. A. Bagnell, and S. S. Srinivasa, "Chomp: Gradient optimization techniques for efficient motion planning," in Proc. IEEE Int. Conf. Robot. Autom., 2009, pp. 489-494.
49. A. D. Dragan, N. D. Ratliff, and S. S. Srinivasa, "Manipulation planning with goal sets using constrained trajectory optimization," in Proc. IEEE Int. Conf. Robot. Autom., 2011.
50. A. D. Dragan, G. J. Gordon, and S. S. Srinivasa, "Learning from experience in manipulation planning: Setting the right goals," in Proc. Int. Symp. Robot. Res., 2011.
51. N. D. Ratliff, J. A. Bagnell, and M. A. Zinkevich, "Maximum margin planning," in Proc. 23rd Int. Conf. Mach. Learn., 2006.
52. M. Cakmak, S. Srinivasa, M. Lee, S. Kiesler, and J. Forlizzi, "Using spatial and temporal contrast for fluent robot-human hand-overs," in Proc. 6th Int. Conf. Human-Robot Interact., 2011, pp. 489-497.
53. M. Cakmak, S. Srinivasa, M. Lee, J. Forlizzi, and S. Kiesler, "Human preferences for robot-human hand-over configurations," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2011.
54. A. Agah and K. Tanie, "Human interaction with a service robot: Mobile-manipulator handing over an object to a human," in Proc. IEEE Int. Conf. Robot. Autom., 1997, pp. 575-580.
55. M. Huber, M. Rickert, A. Knoll, T. Brandt, and S. Glasauer, "Human-robot interaction in handing-over tasks," in Proc. IEEE Int. Symp. Robot Human Interactive Commun., 2008, pp. 107-112.
56. E. Sisbot, L. Marin, and R. Alami, "Spatial reasoning for human robot interaction," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2007.
57. S. Kajikawa, T. Okino, K. Ohba, and H. Inooka, "Motion planning for hand-over between human and robot," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 1995, pp. 193-199.
58. E. Lopez-Damian, D. Sidobre, S. DeLaTour, and R. Alami, "Grasp planning for interactive object manipulation," in Proc. Int. Symp. Robot. Autom., 2006.
59. A. Edsinger and C. Kemp, "Human-robot interaction for cooperative manipulation: Handing objects to one another," in Proc. IEEE Int. Symp. Robot Human Interactive Commun., 2007.
60. K. Koay, E. Sisbot, D. Syrdal, M. Walters, K. Dautenhahn, and R. Alami, "Exploratory study of a robot approaching a person in the context of handing over an object," in Proc. AAAI Spring Symp. Multidiscipl. Collab. Socially Assistive Robot., 2007, pp. 18-24. (Pubitemid 350293250)
61. Y. Choi, T. Chen, A. Jain, C. Anderson, J. Glass, and C. Kemp, "Hand it over or set it down: A user study of object delivery with an assistive mobile manipulator," in Proc. IEEE Int. Symp. Robot Human Interactive Commun., 2009.
62. K. Nagata, Y. Oosaki, M. Kakikura, and H. Tsukune, "Delivery by hand between human and robot based on fingertip force-torque information," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 1998, pp. 750-757.
63. I. Kim and H. Inooka, "Hand-over of an object between human and robot," in Proc. IEEE Int. Symp. Robot Human Interactive Commun., 1992.
64. A. Collet, S. S. Srinivasa, and M. Hebert, "Structure discovery in multi-modal data: A region-based approach," in Proc. IEEE Int. Conf. Robot. Autom., May 2011.
65. A. Collet, M. Martinez, and S. S. Srinivasa, "The moped framework: Object recognition and pose estimation for manipulation," Int. J. Robot. Res., vol. 30, no. 10, pp. 1284-1306, 2011.
66. D. Lowe. (2004). Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. Online. 60(2), pp. 91-110. Available: http://portal.acm.org/citation.cfm?id= 996342
67. Y. Cheng, "Mean shift, mode seeking, and clustering," IEEE Trans. Pattern Anal. Mach. Intell., vol. 17, no. 8, p. 790, 1995.
68. C. F. Olson, "Efficient pose clustering using a randomized algorithm," Int. J. Comput. Vis., vol. 23, no. 2, p. 131, 1997.
69. S. Kammel, J. Ziegler, ". Pitzer, M. Werling, T. Gindele, D. Jagzent, J. Schröder, M. Thuy, M. Goebl, F. v. Hundelshausen, O. Pink, C. Frese, and C. Stiller, BTeam AnnieWAY's autonomous system for the 2007 DARPA urban challenge," Int. J. Field Robot. Res., vol. 25, pp. 615-639, 2008.
70. J. Ziegler, M. Werling, and J. Schröder, "Navigating car-like vehicles in unstructured environment," in Proc. IEEE Intell. Veh. Symp., 2008, pp. 787-791.
71. M. Werling, J. Ziegler, S. Kammel, and S. Thrun, "Optimal trajectory generation for dynamic street scenarios in a frenet frame," in Proc. IEEE Int. Conf. Robot. Autom., 2010, pp. 987-993.
72. T. Gindele, D. Jagzent, ". Pitzer, and R. Dillmann, BDesign of the planner of Team AnnieWAY's autonomous vehicle used in the DARPA urban challenge 2007," in Proc. IEEE Intell. Veh. Symp., Eindhoven, The Netherlands, 2008.
73. A. Kelly, A. Stentz, O. Amidi, M. W. Bode, D. Bradley, A. Diaz-Calderon, M. Happold, H. Herman, R. Mandelbaum, T. Pilarski, P. Rander, S. Thayer, N. M. Vallidis, and R. Warner, "Toward reliable off road autonomous vehicles operating in challenging environments," Int. J. Robot. Res., vol. 25, no. 1, pp. 449-483, May 2006. (Pubitemid 43735584)
74. T. Allen, J. Underwood, and S. Scheding, "A path planning system for autonomous ground vehicles operating in unstructured dynamic environments," in Proc. Australasian Conf. Robot. Autom., 2007.
75. C. Green and A. Kelly, "Toward optimal sampling in the space of paths," in Proc. Int. Symp. Robot. Res., Hiroshima, Japan, Nov. 2007.
76. L. Erickson and S. LaValle, "Survivability: Measuring and ensuring path diversity," in Proc. IEEE Int. Conf. Robot. Autom., Kobe, Japan, May 2009.
77. R. A. Knepper and M. T. Mason, "Path diversity is only part of the problem," in Proc. IEEE Int. Conf. Robot. Autom., May 2009.
78. M. Mason, S. S. Srinivasa, and A. Vazquez, "Generality and simple hands," in Proc. Int. Symp. Robot. Res., Jul. 2009.
79. A. R. Garcia, M. Mason, and S. S. Srinivasa, "Manipulation capabilities with simple hands," in Proc. Int. Symp. Exper. Robot., Dec. 2010.
80. M. Mason, A. R. Garcia, S. S. Srinivasa, and A. Vazquez, "Autonomous manipulation with a general-purpose simple hand," Int. J. Robot. Res., 2011.