A simple ontology of manipulation actions based on hand-object relations

IEEE Transactions on Autonomous Mental Development

Volumn 5, Issue 2, 2013, Pages 117-134

  Wörgötter, Florentin ^a   Aksoy, Eren Erdal ^a   Krüger, Norbert ^b   Piater, Justus ^c   Ude, Ales ^d   Tamosiunaite, Minija ^a  

^a UNIVERSITY OF GÖTTINGEN   (Germany)

^b UNIVERSITY OF SOUTHERN DENMARK   (Denmark)

^c UNIVERSITY OF INNSBRUCK   (Austria)

^d JO EF STEFAN INSTITUTE   (Slovenia)

Author keywords

Manipulation action; manipulation ontology; scene graph; semantic event chain

Indexed keywords

CONDENSED REPRESENTATIONS; MANIPULATION ACTION; RELATIVE POSE; SCENE GRAPH; SEMANTIC EVENT; SPACE AND TIME; TRAJECTORY SHAPES; UNDERLYING PRINCIPLES;

ARTIFICIAL INTELLIGENCE; SOFTWARE ENGINEERING;

SEMANTICS;

EID: 84879400409     PISSN: 19430604     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAMD.2012.2232291     Document Type: Article

References (60)

1. F. Guerin, N. Krüger, and D. Kraft, "A survey of the ontogeny of tool use: from sensorimotor experience to planning," IEEE Trans. Autonom. Mental Develop., vol. 5, no. 1, pp. 18-45, Mar. 2013.
2. E. E. Aksoy, A. Abramov, F. Wörgötter, and B. Dellen, "Categorizing object-action relations from semantic scene graphs," presented at the IEEE Int. Conf. Robot. Autom. (ICRA), 2010.
3. E. E. Aksoy, A. Abramov, J. Dörr, N. Kejun, B. Dellen, and W. Flo-rentin, "Learning the semantics of object-action relations by observation," Int. J. Robot. Res. (IJRR), vol. 30, no. 10, pp. 1229-1249, 2011.
4. B. Aune, Reason and Action. Boston, MA, USA: D. Reidel Publishing Comp, 1978.
5. B. Aune, "Action and Ontology," Philosoph. Studies, vol. 54, pp. 195-213, 1988.
6. V. F. Turchin, "The cybernetic ontology of action," Kybernetes, vol. 22, pp. 10-30, 1993.
7. E. Pacherie, "The phenomenology of action: A conceptual framework," Cognition, vol. 107, pp. 179-217, 2008. (Pubitemid 351384722)
8. T. Schack and H. Ritter, "The cognitive nature of action-Functional links between cognitive psychology, movement science, and robotics," Progress. Brain Res., vol. 174, pp. 231-251, 2009.
9. A. Sloman, Progress Report on the Cognition and Affect Project: Architectures, Architecture-Schemas, and the New Science of Mind 2008.
10. D. Vernon, G. Metta, and G. Sandini, "The iCub cognitive architecture: interactive development in a humanoid robot," in IEEE Int. Conf. Develop. Learn. Imperial College, London, U.K., 2007.
11. D. Vernon, C. Hofsten, and L. Fadiga, "A roadmap for cognitive devel opment in humanoid robots," in Cognitive Syst. Monographs. Berlin, Germany: Springer-Verlag, 2010, vol. 11, pp. 121-153.
12. D. Kraft, E. Baseski, M. Popovic, A. M. Batog, A. Kjær-Nielsen, N. Krüger, R. Petrick, C. Geib, N. Pugeault, M. Steedman, T. Asfour, R. Dillmann, S. Kalkan, F. Wörgötter, B. Hommel, R. Detry, and J. Piater, "Exploration and planning in a three-level cognitive architecture," presented at the Int. Conf. Cognitive Syst. (CogSys), 2008.
13. T. Metzinger and V. Gallese, "The emergence of a shared action ontology: Building blocks for a theory," Consciousness Cogn., vol. 12, pp. 549-571, 2003. (Pubitemid 37491346)
14. J. J. Gibson, The Theory of Affordances. In Perceiving, Acting, and Knowing, R. Shaw and J. Bransford, Eds. Hillsdale, NJ, USA: Lawrence Erlbaum Associates, 1977.
15. J. J. Gibson, The Ecological Approach to Visual Perception. Boston, MA, USA: Houghton Mifflin, 1979.
16. B. Hommel, J. Müsseler, G. Aschersleben, and W. Prinz, "The theory of event coding (TEC): A framework for perception and action planning," Behav. Brain Sci., vol. 24, pp. 849-87, 2001.
17. J. Piaget, The Child's Construction of Reality. London, U.K.: Rout-ledge & Kegan Paul, 1955.
18. F. Wörgötter, A. Agostini, N. Krüger, N. Shylo, and B. Porr, "Cognitive agents-a procedural perspective relying on predictability of object-action complexes (OACs)," Robot. Auton. Syst., vol. 57, pp. 420-432, 2009.
19. N. Krüger, J. Piater, C. Geib, R. Petrick, M. Steedman, F. Wörgötter, A. Ude, T. Asfour, D. Kraft, D. Omrcen, A. Agostini, and R. Dillmann, "Object-action complexes: Grounded abstractions of sensorimotor processes," Robot. Autonom. Syst. (RAS), vol. 59, pp. 740-757, 2011.
20. H. Kjellström, J. Romero, and D. Kragic, "Visual object-action recognition: Inferring object affordances from human demonstration," Comput. Vis. Image Understand., vol. 115, pp. 81-90, 2011.
21. H. Kjellström, J. Romero, D. Martínez, and D. Kragic, "Simultaneous visual recognition of manipulation actions and manipulated objects," in Eur. Conf. Comput. Vision, 2008.
22. S. Griffith, V. Sukhoy, and A. Stoytchev, "Using sequences of movement dependency graphs to form object categories," presented at the IEEE Conf. Humanoids, Bled, Slovenia, 2011.
23. M. Sridhar, A. Cohn, and D. Hogg, "Learning functional object-categories from a relational spatio-temporal representation," presented at the 18th Eur. Conf. Artif. Intell., 2008.
24. M. Sridhar, A. Cohn, and D. Hogg, "Unsupervised learning of event classes from video," presented at the 24th AAAI Conf. Artif. Intell., 2010.
25. A. Gangemi, R. Navigli, and P. Velardi, "The ontowordnet project: Extension and axiomatization of conceptual relations in wordnet," Proc. CoopIS/DOA/ODBASE'03, 2003.
26. H. Liu and P. Singh, "Conceptnet: A practical commonsense reasoning toolkit," BT Technol. J., vol. 22, pp. 221-226, 2004.
27. C. Kemke, "An action ontology framework for natural language interfaces to agent systems," in Artif. Intell. Rev.: Springer, 2007.
28. R. Trypuz, "Formal Ontology of Action: A Unifying Approach," Ph.D. dissertation, Univ. Trento, Trento, Italy, 2007.
29. M. Gutierrez, F. Vexo, and D. Thalman, "Semantic-based representation of virtual environments," Int. J. Comput. Appl. Technol., pp. 229-238, 2005.
30. S. Irawati, D. Calderón, and H. Ko, "Spatial ontology for semantic integration in 3D multimodal interaction framework," presented at the ACM Int. Conf. Virtual Reality Contin. Appl., Hongkong, 2006.
31. J. Lugrin and M. Cavazza, "Making sense of virtual environments: Action representation, grounding and common sense," presented at the 2007 Int. Conf. Intell. User Interfaces, IUI, Honolulu, Hawaii, 2007.
32. M. Tenorth, D. Nyga, and M. Beetz, "Understanding and executing instructions for everyday manipulation tasks from the world wide web," presented at the IEEE Int. Conf. Robot. Autom., 2010.
33. K. Kunze, M. E. Dolha, E. Guzman, and M. Beetz, "Simulation-based temporal projection of everyday robot object manipulation," presented at the 10th Int. Conf. on Auton. Agents Multiagent Syst., 2011.
34. M. Soury, P. Hède, P. Morignot, C. Leroux, J. Savary, and J. Canou, "Cognitive and physical stimulation services with a robotic agent in a domestic setting," presented at the 11th Conf. Eur. Assoc. Adv. Assis-tive Technol. Eur., 2011.
35. H. S. Chung, J. M. Kim, Y. C. Byun, and S. Y. Byun, "Retrieving and exploring ontology-based human motion sequences," presented at the Comput. Sci. Appl.-ICCSA,.
36. S. Kobayashi, S. Tamagawa, T. Morita, and T. Yamaguchi, "Intelligent humanoid robot with japanese Wikipedia ontology and robot action ontology," presented at the HRI'2011,.
37. D. Kulic, W. Takano, and Y. Nakamura, "On-line segmentation and clustering from continuous observation of whole body motions," IEEE Trans. Robot., vol. 25, pp. 1158-1166, 2009.
38. M. Yamamoto, H. Mitomi, F. Fujiwara, and T. Sato, "Bayesian classification of task-oriented actions based on stochastic context-free grammar," presented at the Int. Conf. Automat. Face Gesture Recognit., Southampton, 2006.
39. V. Krüger, D. Herzog, S. Baby, A. Ude, and D. Kragic, "Learning actions from observations: Primitive-based modeling and grammar," IEEE Robot. Autom. Mag., vol. 17, no. 2, pp. 30-43, Apr. 2010.
40. N. Dantam and M. Stilman, "The motion grammar: Linguistic perception, planning, and control," Robotics: Sci. and Syst., RSS, 2011.
41. M. R. Cutkosky and R. D. Howe, "Dextrous robot hands," in Human Grasp Choice and Robotic Grasp Anal. Berlin, Germany: Springer-Verlag, 1990, pp. 5-31.
42. S. Kang and K. Ikeuchi, "A grasp abstraction hierarchy for recognition of grasping tasks from observation," in Proc. IEEE/RSJ Int'l Conf. In-tell. Robots Syst., Yokohama, Japan, 1993.
43. T. Feix, R. Pawlik, H. Schmiedmayer, J. Romero, and D. Kragic, "A comprehensive grasp taxonomy," presented at the Robotics, Sci. Syst.: Workshop Understanding the Human Hand for Adv. Robotic Manipulation, 2009.
44. M. Do, J. Romero, H. Kjellstrom, P. Azad, T. Asfour, D. Kragic, and R. Dillmann, "Grasp recognition and mapping on humanoid robots," presented at the IEEE-RAS Int. Conf. Humanoid Robots, 2009.
45. K. Pastra and Y. Aloimonos, "The minimalist grammar of action," Philosophical Trans. Roy. Soc. B, vol. 367, pp. 103-117, 2012.
46. M. Beetz, D. Jain, L. Mösenlechner, and M. Tenorth, "Towards performing everyday manipulation activities," Robot. Autonom. Syst., vol. 58, no. 9, pp. 1085-1095, 2010.
47. M. Beetz, M. Tenorth, D. Jain, and J. Bandouch, "Towards automated models of activities of daily life," Technol. Disability, vol. 22, pp. 27-40, 2010.
48. A. Revonsuo, "Binding and the phenomenal unity of consciousness," Conscious Cogn., vol. 8, pp. 173-85, 1999.
49. H. Ritter, R. Haschke, and J. J. Steil, "A dual interaction perspective for robot cognition: Grasping as a "Rosetta stone"," Perspectives of Neural-Symbolic Integr., pp. 159-178, 2007.
50. C. Breazeal and B. Scassellati, "Robots that imitate humans," Trends Cogn. Sci., vol. 6, pp. 481-487, 2002. (Pubitemid 35341852)
51. R. Dillmann, "Robot learning from demonstration," Robot. Autonom. Syst. (RAS), vol. 47, pp. 109-116, 2004.
52. R. Zöllner, T. Asfour, and R. Dillmann, "Programming by demonstration: Dual-arm manipulation tasks for humanoid," presented at the IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Sendai, Japan, 2004.
53. E. Billing and T. Hellström, "A formalism for learning from demonstration," Paladyn J. Behav. Robot., vol. 1, pp. 1-13, 2010.
54. A. Billard and D. Grollman, Robot Learning by Demonstration (Under Review). Scholarpedia, 2011 [Online]. Available: http://www.schol-arpedia.org/ article/Robot-learning-by-demonstration
55. E. E. Aksoy, M. Tamosiunaite, R. Vuga, A. Ude, C. Geib, M. Steedman, and F. Wörgötter, Structural bootstrapping at the sensorimotor level for the fast acquisition of action knowledge for cognitive robots, 2013, submitted for publication.
56. M. J. Aein, E. E. Aksoy, M. Tamosiunaite, J. Papon, A. Ude, and F. Wörgötter, Toward a library of actions based on semantic object-action relations, 2013, submitted for publication.
57. A. Abramov, E. E. Aksoy, J. Dörr, F. Wörgötter, and B. Dellen, "3D semantic representation of actions from efficient stereo-image-sequence segmentation on GPUs," presented at the 5th Int. Symp. 3D Data Process., Visualiz., Trans., 2010.
58. E. E. Aksoy, B. Dellen, M. Tamosiunaite, and F. Wörgötter, "Execution of a dual-object (pushing) action with semantic event chains," presented at the IEEE-RAS Int. Conf. Humanoid Robots, 2011.
59. I. Biederman, "Recognition-by-components: A theory of human image understanding," Psychol. Rev., pp. 115-47, 1987.
60. U. Brandes, M. Eiglsperger, J. Lerner, and C. Pich, "Graph markup language (GraphML)," Handbook of Graph Drawing Visualiz., 2010. Communication Technologies Communication Technologies