Towards grasp-oriented visual perception for humanoid robots

International Journal of Humanoid Robotics

Volumn 6, Issue 3, 2009, Pages 387-434

  Bohg, Jeanette ^a   Barck Holst, Carl ^a   Hubner, Kai ^a   Ralph, Maria ^a   Rasolzadeh, Babak ^a   Song, Dan ^a   Kragic, Danica ^a  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

Author keywords

Affordances; Attention; Grasping; Neuroscience; Perception; Reasoning

Indexed keywords

AFFORDANCES; ATTENTION; GRASPING; NEUROSCIENCE; PERCEPTION; REASONING;

COMPUTER VISION; TERMINOLOGY; VISUAL COMMUNICATION;

ANTHROPOMORPHIC ROBOTS;

EID: 70349576912     PISSN: 02198436     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0219843609001796     Document Type: Article

References (105)

1. J. Gray, C. Breazeal, M. Berlin, A. Brooks and J. Lieberman, Action parsing and goal inference using self as simulator, IEEE Workshop on Robot and Human Interactive Communication (2005) 202-209.
2. T. Kyriacou, G. Bugmann and S. Lauria, Vision-based urban navigation procedures for verbally instructed robots, Robotics and Autonomous Systems 51(1) (2005) 69-80.
3. P. McGuire, J. Fritsch, J. J. Steil, F. Rothling, G. A. Fink, S. Wachsmuth, G. Sagerer and H. Ritter, Multi-modal human-machine communication for instructing robot grasping tasks, IEEE/RSJ Int. Conf. on Intell. Robots and Systems (2002) 1082-1089.
4. G. Rizzolatti, L. Fadiga, M. Matelli, V. Bettinardi, E. Paulesu, D. Perani and F. Fazio, Localization of grasp representations in humans by PET:(1) Observation versus execution, Experimental Brain Research 111(2) (1996) 246-252.
5. G. Rizzolatti, G. Luppino and M. Matelli, The organization of the cortical motor system: New concepts, Electroencephalography and Clinical Neurophysiology 106(4) (1998) 283-296.
6. A. H. Fagg and M. A. Arbib, Modeling parietal-premotor interactions in primate control of grasping, Neural Networks 11(7-8) (1998) 1277-1303.
7. V. Raos, M. Umilta, A. Murata, L. Fogassi and V. Gallese, Functional properties of grasping-related neurons in the ventral premotor area F5 of the macaque monkey, J. of Neurophysiology 95(2) (2006) 709-729.
8. Z. Li, A saliency map in primary visual cortex, Trends in Cognitive Sciences 6(1) (2002) 9-16.
9. M. Goodale, Separate visual pathways for perception and action, Trends in Neurosciences 15(1) (1992) 20-25.
10. M. Lu, J. Preston and P. Strick, Interconnections between the prefrontal cortex and the premotor areas in the frontal lobe, J. of Comparative Neurology 341(3) (1994) 375-392.
11. G. Luppino, M. Matelli and G. Rizzolatti, Cortico-cortical connections of two electrophysiologically identified arm representations in the mesial agranular frontal cortex, Experimental Brain Research 82 1) (1990) 214-218.
12. D. Ingle, G. Schneider, C. Trevarthen and R. Held, Locating and identifying: Two modes of visual processing, Psychological Research 31(1) (1967) 42-43.
13. E. Chinellato and A. del Pobil, Neural Coding in the Dorsal Visual Stream (Springer-Verlag Berlin/Heidelberg, 2008), pp. 230-239.
14. A. Murata, V. Gallese, G. Luppino, M. Kaseda and H. Sakata, Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP, J. of Neurophysiology 83(5) (2000) 2580-2601.
15. T. James, G. Humphrey, J. Gati, R. Menon and M. Goodale, Differential effects of viewpoint on object-driven activation in dorsal and ventral streams, Neuron 35(4) (2002) 793-801.
16. E. Chinellato, Y. Demiris and A. P. del Pobil, Studying the human visual cortex for achieving action-perception coordination with robots, IASTED Int. Conf. Artif. Intell. Soft Comput. (2006) 184-189.
17. U. Castiello and M. Jeannerod, Measuring time to awareness, Neuroreport 2(12) (1991) 797-800.
18. M. J. Webster, J. Bachevalier and L. G. Ungerleider, Connections of inferior temporal areas TEO and TE with parietal and frontal cortex in macaque monkeys, Cerebral Cortex 4(5) (1994) 470-483.
19. G. Rizzolatti and G. Luppino, The cortical motor system, Neuron 31(6) (2001) 889-901.
20. M. Petrides and D. N. Pandya, Projections to the frontal cortex from the posterior parietal region in the rhesus monkey, J. of Comparative Neurology 228(1) (1984) 105-116.
21. M. A. Arbib, A. R. Iberall and D. Lyons, Coordinated control programs for control of the hands, eds. A.W. Goodwin and I. Darian-Smith, Hand Function and the Neocortex, Experimental Brain Research Supplemental 10, (Springer-Verlag, Berlin, 1985), pp. 111-129.
22. T. M. Preuss and P. S. Goldman-Rakic, Connections of the ventral granular frontal cortex of macaques with perisylvian premotor and somatosensory areas: Anatomical evidence for somatic representation in primate frontal association cortex, J. of Comparative Neurology 282(2) (1989) 293-316.
23. G. Recatalá, E. Chinellato, Á. del Pobil, Y. Mezouar and P. Martinet, Biologically-inspired 3D grasp synthesis based on visual exploration, Autonomous Robots 25(1) (2008) 59-70.
24. R. S. Johansson and G. Westling, Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects, Experimental Brain Research 56(3) (1984) 550-564.
25. A. Ude, C. Gaskett and G. Cheng, Foveated vision systems with two cameras per eye, IEEE Int. Conf. on Robotics and Automation (2006) 3457-3462.
26. M. Björkman and J.-O. Eklundh, Attending, foveating and recognizing objects in real world scenes, British Machine Vision Conference, BMVC'04 (2004).
27. J. K. Tsotsos, Analyzing vision at the complexity level: Constraints on an architecture, an explanation for visual search performance, and computational justification for attentive processes, Technical Report, University of Toronto Vision Laboratory (1987).
28. J. K. Tsotsos, A 'complexity level' analysis of immediate vision, Int. J. of Computer Vision 1(4) (1988) 303-320.
29. J. K. Tsotsos, The complexity of perceptual search tasks, Technical Report, University of Toronto Vision Laboratory (1989).
30. S. Ekvall and D. Kragic, Learning and evaluation of the approach vector for automatic grasp generation and planning, IEEE Int. Conf. on Robotics and Automation (2007) 4715-4720.
31. A. Morales, P. Azad, T. Asfour, D. Kraft, S. Knoop, R. Dillmann, A. Kargov, C. Pylatiuk and S. Schulz, An anthropomorphic grasping approach for an assistant humanoid robot, 37th International Symposium on Robotics (2006) 149-152.
32. L. Itti, Models of bottom-up and top-down visual attention, Ph.D. thesis, California Institute of Technology (2000).
33. B. Olshausen, C. Anderson and D. van Essen, A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information, J. of Neurosci. 13 (1993) 4700-4719.
34. A. M. Treisman and G. Gelade, A feature integration theory of attention, Cognitive Psychology 12 (1980) 97-136.
35. C. Koch and S. Ullman, Shifts in selective visual attention: Towards the underlying neural circuitry, Human Neurobiology 4 (1985) 219-227.
36. T. Koike and J. Saiki, Stochastic guided search model for search asymmetries in visual search tasks, Biologically Motivated Computer Vision (2002) 408-417.
37. O. Ramström and H. I. Christensen, Object detection using background context, in Int. Conf. Patt. Recogn. (2004) 45-48.
38. S. B. Choi, S. W. Ban and M. Lee, Biologically motivated visual attention system using bottom-up saliency map and top-down inhibition, Neural Information Processing - Letters and Review 2 (2004) 19-25.
39. L. Itti, C. Koch and E. Niebur, A model of saliency-based visual attention for rapid scene analysis, IEEE Trans. Patt. Anal. Mach. Intell. 20(11) (1998) 1254-1259.
40. V. Navalpakkam and L. Itti, Sharing resources: Buy attention, get recognition, International Workshop Attention and Performance in Computer Vision (2003).
41. K. Lee, H. Buxton and J. Feng, Selective attention for cueguided search using a spiking neural network, International Workshop on Attention and Performance in Computer Vision (2003) 55-62.
42. A. Oliva, A. Torralba, M. S. Castelhano and J. M. Henderson, Top-down control of visual attention in object detection, Int. Conf. Image Process. (2003) 253-256.
43. S. Frintrop, VOCUS: A Visual Attention System for Object Detection and Goal-Directed Search 3899 (Springer, 2006) ISBN 978-3-540-32759-2.
44. J. Moren, A. Ude, A. Koene and G. Cheng, Biologically-based top-down attention modulation for humanoid interactions, Int. J. of Humanoid Robotics, (2008) 3-24.
45. A. M. Okamura, N. Smaby and M. R. Cutkosky, An overview of dexterous manipulation, IEEE Int. Conf. on Robotics and Automation, (2000) 255-262.
46. K. B. Shimoga, Robot grasp synthesis algorithms: A survey, Int. J. Robotic Research 15(3) (1996) 230-266.
47. M. Ciorcarlie, C. Goldfeder and P. Allen, Dexterous grasping via eigengrasps: A low-dimensional approach to a high-complexity problem, Robotics: Science and Systems Manipulation Workshop (2007).
48. C. Borst, M. Fischer and G. Hirzinger, Grasping the dice by dicing the grasp, IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (2003) 3692-3697.
49. T. Asfour, K. Regenstein, P. Azad, J. Schröder, A. Bierbaum, N. Vahrenkamp and R. Dillmann, ARMAR-III: An integrated humanoid platform for sensory-motor control, 6th IEEE-RAS Int. Conf. on Humanoid Robots (2006) 169-175.
50. J. Tegin, S. Ekvall, D. Kragic, B. Iliev and J. Wikander, Demonstration based learning and control for automatic grasping, Journal of Intelligent Service Robotics (2008).
51. A. Saxena, J. Driemeyer, J. Kearns and A. Y. Ng, Robotic grasping of novel objects, Neural Inform. Process. Syst. 19 (2007) 1209-1216.
52. A. Morales, E. Chinellato, A. Fagg and A. del Pobil, Using experience for assessing grasp reliability, Int. J. of Humanoid Robotics 1 4) (2004) 671-691.
53. M. Stark, P. Lies, M. Zillich, J. Wyatt and B. Schiele, Functional object class detection based on learned affordance cues, 6th Int. Conf. on Computer Vision Systems, 5008 LNAI (Springer-Verlag, 2008) 435-444.
54. J. Grezes and J. Decety, Does visual perception of object afford action? Evidence from a neuroimaging study, Neuropsychologia 40(2) (2002) 212-222.
55. M. Tucker and R. Ellis, On the relations between seen objects and components of potential actions, Journal of Experimental Psychology: Human Perception and Performance 24(3) (1998) 830-846.
56. A. M. Borghi, Grounding Cognition: The Role of Perception and Action in Memory, Language, and Thinking, Chapter Object Concepts and Action, Cambridge University Press (2005).
57. S. H. Creem and D. R. Proffitt, Grasping objects by their handles: A necessary interaction between cognition and action, Journal of Experimental Psychology: Human Perception and Performance 27(1) (2001) 218-228.
58. J. J. Gibson, The Ecological Approach to Visual Perception, Lawrence Erlbaum Associates (1979).
59. M. A. Goodale, J. P. Meenan, H. H. Bülthoff, D. A. Nicolle, K. J. Murphy and C. I. Racicot, Separate neural pathways for the visual analysis of object shape in perception and prehension, Current Biology 4(7) (1994) 604-610.
60. R. H. Cuijpers, J. B. J. Smeets and E. Brenner, On the relation between object shape and grasping kinematics, Journal of Neurophysiology 91 (2004) 2598-2606.
61. M. Gentilucci, Object motor representation and reaching-grasping control, Neuropsychologia 40(8) (2002) 1139-1153.
62. A. T. Miller, S. Knoop, H. I. Christensen and P. K. Allen, Automatic grasp planning using shape primitives, IEEE Int. Conf. on Robotics and Automation (2003) 1824-1829.
63. C. Goldfeder, P. K. Allen, C. Lackner and R. Pelossof, Grasp planning via decomposition trees, IEEE Int. Conf. on Robotics and Automation (2007) 4679-4684.
64. G. Biegelbauer and M. Vincze, Efficient 3D object detection by fitting superquadrics to range image data for robot's object manipulation, IEEE Int. Conf. on Robotics and Automation (2007) 1086-1091.
65. L. Chevalier, F. Jaillet and A. Baskurt, Segmentation and superquadric modeling of 3D objects, Journal of Winter School of Computer Graphics, WSCG'03 (2003).
66. D. Katsoulas, Reliable recovery of piled box-like objects via parabolically deformable superquadrics, 9th IEEE Int. Conf. on Computer Vision 2 (2003) 931-938.
67. E. Lopez-Damian, Grasp planning for object manipulation by an autonomous robot, Ph.D. thesis, Laboratoire d'Analyse et d'Architecture des Systmes du CNRS (2006).
68. E. Lopez-Damian, D. Sidobre and R. Alami, Grasp planning for non-convex objects, 36th Int. Symp. Robotics (2005).
69. S. El-Khoury and Anis Sahbani, Handling objects by their handles, IROS-2008 Workshop on Grasp and Task Learning by Imitation (2008).
70. N. Amenta, S. Choi and R. Kolluri, The power crust, 6th ACM Symposium on Solid Modeling and Applications (2001) 249-260.
71. G. Barequet and S. Har-Peled, Efficiently approximating the minimum-volume bounding box of a point set in three dimensions, Journal of Algorithms 38 (2001) 91-109.
72. B. Draper and A. Lionelle, Evaluation of selective attention under similarity transforms, International Workshop on Attention and Performance in Computer Vision (2003) 31-38.
73. A. Tavakoli Targhi, E. Hayman, J. O. Eklundh and M. Shahshahani, The eigen-transform and applications, Asian Conference on Computer Vision (2006) 70-79.
74. B. Rasolzadeh, A. Tavakoli Targhi and J.-O. Eklundh, An attentional system combining top-down and bottom-up influences, Workshop on Attention and Performance in Computational Vision (2007) 123-140.
75. B. Rasolzadeh, Interaction of bottom-up and top-down influences for attention in an active vision system, Master's thesis, Royal Institute of Technology, Stockholm, Sweden (2006).
76. A. Rabinovich, A. Vedaldi, C. Galleguillos, E. Wiewiora and S. Belongie, Objects in context, Int. Conf. on Computer Vision (2007) 1-8.
77. T. M. Strat and M. A. Fischler, Context-based vision: Recognition of natural scenes, Asilomar Conference on Signals, Systems and Computers (1989) 532-536.
78. T. M. Strat and M. A. Fischler, The use of context in vision, Workshop on Context-Based Vision (1995).
79. Y. Hu, X. Xie, W.-Y. Ma, L.-T. Chia and D. Rajan, Salient region detection using weighted feature maps based on the human visual attention model, IEEE Pacific-Rim Conference on Multimedia (2004) 993-1000.
80. A. K. C. Wong and P. K Sahoo, A gray-level threshold selection method based on maximum entropy principle, IEEE Trans. Systems Man and Cybernetics 19 (1989) 866-871.
81. J. Bohg and D. Kragic, Grasping familiar objects using shape context, Int. Conf. on Advanced Robotics (2009).
82. S. Belongie, J. Malik and J. Puzicha, Shape matching and object recognition using shape contexts, IEEE Trans. Patt. Anal. Mach. Intell. 24(4) (2002) 509-522.
83. M. Craven and J. Shavlik, Extracting tree-structured representations of trained networks, Advances in Neural Information Processing Systems (NIPS-8) (MIT Press, 1995), pp. 24-30.
84. D. Martens, B. Baesens, T. Van Gestel and J. Vanthienen, Comprehensible credit scoring models using rule extraction from support vector machines, Eur. J. Operational Research 183(3) (December 2007) 1466-1476.
85. W. T. Townsend, The Barrett hand grasper - Programmably flexible part handling and assembly, Industrial Robot: An International Journal 27(3) (2000) 181-188.
86. K. Huebner, S. Ruthotto and D. Kragic, Minimum volume bounding box decomposition for shape approximation in robot grasping, IEEE Int. Conf. on Robotics and Automation (2008) 1628-1633.
87. R. B. Rusu, Z. C. Marton, N. Blodow, M. E. Dolha and M. Beetz, Functional object mapping of kitchen environments, IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (2008) 3525-3532.
88. R. Triebel, R. Schmidt, Ó. Martínez Mozos and W. Burgard, Instance-based AMN classification for improved object recognition in 2D and 3D laser range data, Int. Joint Conf. Artif. Intell. (2007) 2225-2230.
89. K. Huebner, M. Björkman, B. Rasolzadeh, M. Schmidt and D. Kragic, Integration of visual and shape attributes for object action complexes, 6th Int. Conf. on Computer Vision Systems, 5008 of LNAI (Springer-Verlag, 2008) 13-22.
90. A. Saxena, L. Wong and A. Y. Ng, Learning grasp strategies with partial shape information, 23rd AAAI Conf. Artif. Intell. (2008) 1491-1494.
91. D. Aarno, J. Sommerfeld, D. Kragic, N. Pugeault, S. Kalkan, F. Wörgötter, D. Kraft and N. Krüger, Early reactive grasping with second order 3D feature relations, ICRA Workshop: From Features to Actions (2007) 319-325.
92. C. Borst, M. Fischer and G. Hirzinger, Grasp planning: How to choose a suitable task wrench space, IEEE Int. Conf. on Robotics and Automation (2004) 319-325.
93. K. Huebner and D. Kragic, Selection of robot pre-grasps using box-based shape approximation, IEEE Int. Conf. on Intelligent Robots and Systems (2008) 1765-1770.
94. A. Nealen, T. Igarashi, O. Sorkine and M. Alexa, FiberMesh: Designing freeform surfaces with 3D curve, ACM Transactions on Computer Graphics, SIGGRAPH 2007 23(3) (2007).
95. A. T. Miller and P. K. Allen, Graspit! A versatile simulator for robotic grasping, IEEE Robotics & Automation Magazine 11(4) (2004) 110-122.
96. A. Morales, Learning to predict grasp reliability with a multifinger robot hand by using visual features, Ph.D. thesis, Dep. of Computer and Engineering Science, Univ. Jaume I (2004).
97. J. Speth, A. Morales and P. J. Sanz, Vision-based grasp planning of 3D objects by extending 2D contour based algorithms, IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (2008) 2240-2245.
98. S. Geidenstam, K. Huebner, D. Banksell and D. Kragic, Learning of grasping strategies from box-based 3D object approximations, Proceedings of Robotics: Science and Systems (2009).
99. C. Geib, K. Mourao, R. Petrick, N. Pugeault, M. Steedman, N. Krüger and F. Wörgötter, Object action complexes as an interface for planning and robot control, IEEE-RAS Humanoids-06 Workshop: Toward Cognitive Humanoid Robots (2006).
100. J. Gibson, The theory of affordances, eds. R. E. Shaw and J. Bransford, Perceiving, Acting, and Knowing: Toward an Ecological Psychology (Erlbaum, NJ, 1977) 67-82.
101. M. Bar, Visual objects in context, Nature Reviews Neuroscience 5(8) (2004) 617-629.
102. M. Bar and S. Ullman, Spatial context in recognition, Technical Report, Mathematics & Computer Science, Weizmann Institute of Science (1993).
103. A. Torralba and P. Sinha, Statistical context priming for object detection, Int. Conf. on Computer Vision (2001) 763-770.
104. L. Montesano, M. Lopes, A. Bernardino and J. Santos-Victor, Learning object affordances: From sensory-motor coordination to imitation, IEEE Trans. on Robotics 24(1) (2008) 15-26.
105. M. Lopes and J. Santos-Victor, A developmental roadmap for learning by imitation in robots, IEEE Transactions on Systems, Man, and Cybernetics, Part B 37(2) (2007) 308-321.