Learning the semantics of object-action relations by observation

International Journal of Robotics Research

Volumn 30, Issue 10, 2011, Pages 1229-1249

  Aksoy, Eren Erdal ^a   Abramov, Alexey ^a   Dörr, Johannes ^a   Ning, Kejun ^a   Dellen, Babette ^a,b   Wörgötter, Florentin ^a  

^a UNIVERSITY OF GÖTTINGEN   (Germany)

^b INSTITUT DE ROBÒTICA I INFORMÀTICA INDUSTRIAL CSIC UPC   (Spain)

Author keywords

Action recognition; affordances; object categorization; object action complexes (OACs); semantic scene graphs; unsupervised learning

Indexed keywords

ACTION RECOGNITION; AFFORDANCES; DYNAMIC GRAPH; HIGH CONFIDENCE; HUMAN MANIPULATION; IMAGE SEGMENTS; OBJECT CATEGORIZATION; OBJECT-ACTION COMPLEXES (OACS); SEARCH ALGORITHMS; SEMANTIC EVENT; SPATIAL RELATIONS; STATISTICAL LEARNING; TIME POINTS; TOPOLOGICAL TRANSITIONS; TRANSITION MATRICES;

ALGORITHMS; ROBOTS; SEMANTICS;

TOPOLOGY;

EID: 80052577204     PISSN: 02783649     EISSN: 17413176     Source Type: Journal    
DOI: 10.1177/0278364911410459     Document Type: Article

References (49)

1. AbramovAAksoyEEDörrJPauwelsKWörgötterFDellenB (2010) 3D semantic representation of actions from efficient stereo-image-sequence segmentation on GPUs. In 3DPVT.
2. Aksoy EE, Abramov A, Wörgötter F, Dellen B. Categorizing object-action relations from semantic scene graphs. IEEE International Conference on Robotics and Automation, ICRA2010 ; 2010 Alaska ; 2010..
3. Belhumeur PN, Kriegmant DJ. What is the set of images of an object under all possible lighting conditions. IEEE CVPR. 1996 ;: 270-277
4. Breazeal C, Scassellati B. Robots that imitate humans. Trends Cogn Sci (Regul Ed). 2002 ; 6: 481-487 (Pubitemid 35341852)
5. Calinon S, Billard A. Stochastic gesture production and recognition model for a humanoid robot. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) ; 2004 ; 2004. 2769.
6. Calinon S, Billard A. Recognition and Reproduction of Gestures using a Probabilistic Framework combining PCA, ICA and HMM. Proceedings of the International Conference on Machine Learning (ICML) ; 2005 ; 2005. 105.
7. Calinon S, Billard A. Incremental learning of gestures by imitation in a humanoid robot. HRI '07: Proceedings of the ACM/IEEE International conference on Human-robot interaction ; 2007 New York: ACM Press ; 2007. 255.
8. Dan Pelleg AM. X-means: Extending k-means with efficient estimation of the number of clusters. Proceedings of the Seventeenth International Conference on Machine Learning ; 2000 San Francisco, CA: Morgan Kaufmann ; 2000. 727.
9. Dee H, Hogg D, Cohn A. Scene modelling and classification using learned spatial relations. Proceedings of Spatial Information Theory ; 2009 ; 2009. 295.
10. Dellen B, Aksoy EE, Wörgötter F. Segment tracking via a spatiotemporal linking process in an n-d lattice model. Sensors. 2009 ; 9: 9355-9379
11. Dellen B, Wörgötter F. Disparity from stereo-segment silhouettes of weakly textured images. Proceedings of the British Machine Vision Conference ; 2009 ; 2009..
12. Fergus R, Perona P, Zisserman A. Object class recognition by unsupervised scale-invariant learning. Proceedings of CVPR ; 2003 ; 2003. 264.
13. Gibson JJ Perceiving, acting, and knowing. Shaw R Bransford J, ed. Hoboken: John Wiley & Sons ; 1977: 127-143.
14. Gilbert A, Illingworth J, Bowden R. Action recognition using mined hierarchical compound features. IEEE Trans Pattern Anal Machine Intell. 2011 ; 33 (5). 883-897
15. Hakeem A, Shah M. Multiple agent event detection and representation in videos. Proceedings of AAAI ; 2005 ; 2005..
16. Harnad S. The symbol grounding problem. Physica D. 1990 ; 42: 335-346
17. Helbig HB, Steinwender J, Graf M, Kiefer M. Action observation can prime visual object recognition. Exp Brain Res. 2010 ; 200 (3-4). 251-258
18. Hoiem D, Efros AA, Hebert M. Putting objects in perspective. Int J Comput Vision. 2008 ; 80: 3-15
19. Hongeng S. Unsupervised learning of multi-object event classes. Proceedings of the 15th British Machine Vision Conference ; 2004 ; 2004. 487.
20. Ijspeert AJ, Nakanishi J, Schaal S. Movement imitation with nonlinear dynamical systems in humanoid robots. Proceedings IEEE International Conference on Robotics and Automation ; 2002 ; 2002. 1398.
21. Kjellstrom H, Romero J, Kragic D. Simultaneous visual recognition of manipulation actions and manipulated objects. European Conference on Computer Vision ; 2008 ; 2008..
22. Krüger N, Piater J, Geib C, Petrick R, Steedman M, Wörgötter F, et al. Object-action complexes: Grounded abstractions of sensorimotor processes. Robotics and Autonomous Systems. 2010 ;:
23. Laptev I, Perez P. Retrieving actions in movies. Proceedings of ICCV ; 2007 ; 2007..
24. Liao L, Fox D, Kautz H. Location-based activity recognition using relational markov networks. Proceedings of the 19th International Joint Conference on Artificial Intelligence ; 2005 ; 2005. 773.
25. Lowe DG. Distinctive image features from scale-invariant keypoints. Int J Comput Vision. 2004 ; 60: 91-110
26. Maurer A, Hersch M, Billard A. Extended Hopfield network for sequence learning: application to gesture recognition. Proceedings of ICANN'05 ; 2005 ; 2005..
27. McCarthy J, Hayes P Machine Intelligence. Meltzer B. Michie D., ed. UK: Edinburgh University Press ; 1969: 195-204.
28. Modayil J, Bai T, Kautz H. Improving the recognition of interleaved activities. Proceedings of the 10th International Conference on Ubiquitous Computing ; 2008 ; 2008. 40.
29. Mundy J, Zisserman A Geometric Invariance in Computer Vision. Cambridge, MA: The MIT Press ; 1992 :
30. Mundy JL Toward Category Level Object Recognition. New York: Springer ; 2006: 3-29.
31. Murase H, Nayar SK. Visual learning and recognition of 3-d objects from appearance. Int J Comput Vision. 1995 ; 14: 5-24
32. Niebles J, Wang H, Fei-Fei L. Unsupervised learning of human action categories using spatial-temporal words. Int J Comput Vision. 2008 ; 79: 299-318
33. Ning K, Kulvicius T, Tamosiunaite M, Wörgötter F. A novel trajectory generator based on dynamic motor primitives. IEEE Trans Robotics. 2010 ;:
34. Nister D, Stewenius H. Scalable recognition with a vocabulary tree. Proceedings of CVPR ; 2006 ; 2006. 2161.
35. Ogawara K, Takamatsu J, Kimura H, Katsushi I. Modeling manipulation interactions by hidden Markov models. IEEE/RSJ International Conference on Intelligent Robots and Systems ; 2002 ; 2002..
36. Oliva A, Torralba A. The role of context in object recognition. Trends Cognitive Sci. 2009 ; 11: 520-526
37. Pauwels K, Van Hulle M. Realtime phase-based optical flow on the GPU. IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Computer Vision on the GPU ; 2008 Anchorage, AK ; 2008..
38. Raamana PR, Grest D, Krueger V. Human action recognition in table-top scenarios: an HMM-based analysis to optimize the performance. CAIP'07: Proceedings of the 12th International Conference on Computer Analysis of Images and Patterns ; 2007 ; 2007. 101.
39. Rizzolatti G, Craighero L. The mirror-neuron system. Ann Rev Neurosci. 2004 ; 27: 169-192 (Pubitemid 39050401)
40. Sabatini S, Gastaldi G, Solari F, Diaz J, Ros E, Pauwels K, et al. Compact and accurate early vision processing in the harmonic space. International Conference on Computer Vision Theory and Applications ; 2007 Barcelona ; 2007. 213.
41. Shylo N, Wörgötter F, Dellen B. Ascertaining relevant changes in visual data by interfacing AI reasoning and low-level visual information via temporally stable image segments. Proceedings of the International Conference on Cognitive Systems (Cogsys 2008) ; 2009 ; 2009..
42. Sivic J, Zisserman A. Video google: A text retrieval approach to object matching in videos. ICCV '03: Proceedings of the Ninth IEEE International Conference on Computer Vision ; 2003 ; 2003. 1470.
43. Sridhar M, Cohn GA, Hogg D. Learning functional object-categories from a relational spatio-temporal representation. Proceedings 18th European Conference on Artificial Intelligence ; 2008 ; 2008. 606.
44. SumsiMF (2008) Theory and Algorithms on the Median Graph. Application to Graph-based Classification and Clustering. PhD thesis, Universitat Autonoma de Barcelona.
45. Thorndike E Animal Intelligence. New York: Macmillan ; 1911 :
46. Torralba A. Modeling global scene factors in attention. J Opt Soc Am A. 2003 ; 20: 1407-1418
47. Turk M, Pentland A. Eigenfaces for recognition. J Cognitive Neurosci. 1991 ; 3: 71-86
48. Vicente I, Kyrki V, Kragic D. Action recognition and understanding through motor primitives. Advanced Robotics. 2007 ; 21: 1687-1707 (Pubitemid 350112896)
49. Wörgötter F, Agostini A, Krüger N, Shylo N, Porr B. Cognitive agents-a procedural perspective relying on predictability of object-action complexes (OACs). Robotics Autonomous Syst. 2009 ; 57: 420-432