Visual object-action recognition: Inferring object affordances from human demonstration

Computer Vision and Image Understanding

Volumn 115, Issue 1, 2011, Pages 81-90

  Kjellström, Hedvig ^a   Romero, Javier ^a   Kragić, Danica ^a  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

Author keywords

Action recognition; Contextual recognition; Learning from demonstration; Object affordances; Object recognition

Indexed keywords

ACTION RECOGNITION; AFFORDANCES; CONTEXTUAL INFORMATION; CONTEXTUAL RECOGNITION; HUMAN HANDS; HUMAN MANIPULATION; LEARNING FROM DEMONSTRATION; MANIPULATED OBJECTS; OBJECT CATEGORIZATION; VISUAL OBJECTS;

OBJECT RECOGNITION;

DEMONSTRATIONS;

EID: 78751574174     PISSN: 10773142     EISSN: 1090235X     Source Type: Journal    
DOI: 10.1016/j.cviu.2010.08.002     Document Type: Article

References (48)

1. A. Billard, S. Calinon, R. Dillman, and S. Schaal Robot programming by demonstration B. Siciliano, O. Khatib, Handbook of Robotics 2008 Springer-Verlag New York, NY, USA (Chapter 59)
2. A. Blum, T. Mitchell, Combining labeled and unlabeled data with co-training, in: Conference on Computational Learning Theory, 1998.
3. J. Bohg, D. Kragić, Grasping familiar objects using shape context, in: International Conference on Advanced Robotics, 2009.
4. P. Carbonetto, N. de Freitas, K. Barnard, A statistical model for general contextual object recognition, in: European Conference on Computer Vision, vol. 1, 2004, pp. 350-362.
5. C.-C. Chang, C.-J. Lin, LIBSVM: A Library for Support Vector Machines, 2001. .
6. N. Cristianini, and J. Shawe-Taylor An Introduction to Support Vector Machines 2000 Cambridge University Press Cambridge, UK
7. N. Dalal, B. Triggs, Histograms of oriented gradients for human detection, in: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2005, pp. 886-893.
8. L. Fei-Fei, R. Fergus, A. Torralba, Recognizing and Learning Object Categories: Short Course at ICCV, 2009. .
9. R. Filipovych, E. Ribeiro, Recognizing primitive interactions by exploring actor-object states, in: IEEE Conference on Computer Vision and Pattern Recognition, 2008.
10. P. Fitzpatrick, G. Metta, L. Natale, S. Rao, G. Sandini, Learning about objects through action - initial steps towards artificial cognition, in: IEEE International Conference on Robotics and Automation, 2003.
11. W.T. Freeman, M. Roth, Orientational histograms for hand gesture recognition, in: IEEE International Conference on Automatic Face and Gesture Recognition, 1995.
12. J. Gibson The Ecological Approach to Visual Perception 1979 Lawrence Erlbaum Associates, Hillsdale, NJ, USA
13. A. Gupta, L.S. Davis, Beyond nouns: exploiting prepositions and comparative adjectives for learning visual classifiers, in: European Conference on Computer Vision, 2008.
14. A. Gupta, A. Kembhavi, and L.S. Davis Observing human-object interactions: using spatial and functional compatibility for recognition IEEE Transactions on Pattern Analysis and Machine Intelligence 31 10 2009 1775 1789
15. K. Ikeuchi, M. Hebert, Task oriented vision, in: IEEE/RSJ International Conference on Intelligent Robots and Systems, 1992, pp. 2187-2194.
16. H. Kjellström, J. Romero, D. Martínez, D. Kragić, Simultaneous visual recognition of manipulation actions and manipulated objects, in: European Conference on Computer Vision, vol. 2, 2008, pp. 336-349.
17. J. Lafferty, A. McCallum, F. Pereira, Conditional random fields: probabilistic models for segmenting and labeling sequence data, in: International Conference on Machine Learning, 2001.
18. Y. LeCun, F.J. Huang, L. Bottou, Learning methods for generic object recognition with invariance to pose and lighting, in: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2004, pp. 97-104.
19. L.-J. Li, L. Fei-Fei, What, where and who? Classifying events by scene and object recognition, in: IEEE International Conference on Computer Vision, 2007.
20. D. Marr Vision 1982 Freeman New York, NY, USA
21. M. Marszalek, I. Laptev, C. Schmid, Actions in context, in: IEEE Conference on Computer Vision and Pattern Recognition, 2009.
22. J. Miura, K. Ikeuchi, Task-oriented generation of visual sensing strategies, in: IEEE International Conference on Computer Vision, 1995.
23. T.B. Moeslund, A. Hilton, and V. Krüger A survey of advances in computer vision-based human motion capture and analysis Computer Vision and Image Understanding 104 2-3 2006 90 126
24. L. Montesano, M. Lopes, A. Bernardino, and J. Santos-Victor Learning object affordances: from sensory motor coordination to imitation IEEE Transactions on Robotics 24 1 2008 15 26
25. D.J. Moore, I.A. Essa, M.H. Hayes, Exploiting human actions and object context for recognition tasks, in: IEEE International Conference on Computer Vision, 1999.
26. K. Murphy, A CRF implementation for general graphs, 2004. .
27. K. Murphy, A. Torralba, W.T. Freeman, Using the forest to see the trees: a graphical model relating features, objects, and scenes, in: Neural Information Processing Systems, 2003.
28. P. Peursum, G. West, S. Venkatesh, Combining image regions and human activity for indirect object recognition in indoor wide-angle views, in: IEEE International Conference on Computer Vision, vol. 1, 2005, pp. 82-89.
29. A. Quattoni, M. Collins, T. Darrell, Learning visual representations using images with captions, in: IEEE Conference on Computer Vision and Pattern Recognition, 2007.
30. L.R. Rabiner A tutorial on hidden Markov models and selected applications in speech recognition Proceedings of the IEEE 77 2 1989 257 286
31. E. Rivlin, S.J. Dickinson, and A. Rosenfeld Recognition by functional parts Computer Vision and Image Understanding 62 2 1995 164 176
32. J. Romero, H. Kjellström, D. Kragić, Hands in action: real-time 3D reconstruction of hands in interaction with objects, in: IEEE International Conference on Robotics and Automation, 2010.
33. A. Saxena, J. Driemeyer, and A.Y. Ng Robotic grasping of novel objects using vision International Journal of Robotics Research 27 2 2008 157 173 (Pubitemid 351169712)
34. S. Schaal Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences 3 1999 233 242
35. I. Serrano Vicente, V. Kyrki, and J. Kragić Action recognition and understanding through motor primitives Advanced Robotics 21 13 2007 1473 1501
36. G. Shakhnarovich, P. Viola, T. Darrell, Fast pose estimation with parameter sensitive hashing, in: IEEE International Conference on Computer Vision, vol. 2, 2003, pp. 750-757.
37. Y. Shi, A. Bobick, I. Essa, Learning temporal sequence model from partially labeled data, in: IEEE Conference on Computer Vision and Pattern Recognition, 2003, pp. 1631-1638.
38. C. Sminchisescu, A. Kanujia, and D. Metaxas Conditional models for contextual human motion recognition Computer Vision and Image Understanding 104 2-3 2006 210 220
39. L. Stark, K. Bowyer, Generic Object Recognition using Form and Function, W. Sci. Ser. Machine Perception and Artificial Intelligence, vol. 10, 1996.
40. M. Stark, P. Lies, M. Zillich, J. Wyatt, B. Schiele, Functional object class detection based on learned affordance cues, in: International Conference on Computer Vision Systems, 2008, pp. 435-444.
41. C. Sutton, K. Rohanimanesh, A. McCallum, Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data, In: International Conference on Machine Learning, 2004.
42. A. Torralba Contextual priming for object detection International Journal of Computer Vision 53 2 2003 169 191
43. A. Torralba, K. Murphy, W.T. Freeman, Contextual models for object detection using boosted random fields, in: Neural Information Processing Systems, 2004.
44. J. Triesch, D.M. Ballard, M.M. Hayhoe, and B.T. Sullivan What you see is what you need Journal of Vision 3 2003 86 94
45. M. Veloso, F. von Hundelshausen, P.E. Rybski, Learning visual object definitions by observing human activities, in: IEEE-RAS International Conference on Humanoid Robots, 2005.
46. F. Wörgötter, A. Agostini, N. Krüger, N. Shylo, and B. Porr Cognitive agents - a procedural perspective relying on the predictability of object-action-complexes (OACs) Robotics and Autonomous Systems 57 4 2009 420 432
47. J. Wu, A. Osuntogun, T. Choudhury, M. Philipose, J.M. Rehg, A scalable approach to activity recognition based on object use, in: IEEE International Conference on Computer Vision, 2007.
48. R. Zöllner, M. Pardowitz, S. Knoop, R. Dillman, Towards cognitive robots: building hierarchical task representations of manipulations from human demonstration, in: IEEE International Conference on Robotics and Automation, 2005, pp. 1535-1540.