Audio-visual classification and detection of human manipulation actions

IEEE International Conference on Intelligent Robots and Systems

Volumn , Issue , 2014, Pages 3045-3052

  Pieropan, Alessandro ^a   Salvi, Giampiero ^a   Pauwels, Karl ^b   Kjellstrom, Hedvig ^a  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

^b UNIVERSITY OF GRANADA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

AGRICULTURAL ROBOTS;

ACOUSTIC DATA; AUDIO-VISUAL DATA; COHERENT REPRESENTATIONS; HUMAN MANIPULATION; MULTIPLE SOURCE; OBJECT POSE; UNSTRUCTURED ENVIRONMENTS; VISUAL DATA;

INTELLIGENT ROBOTS;

EID: 84911478073     PISSN: 21530858     EISSN: 21530866     Source Type: Conference Proceeding    
DOI: 10.1109/IROS.2014.6942983     Document Type: Conference Paper

References (40)

1. L. Montesano, M. Lopes, A. Bernardino, and J. Santos-Victor. Learning object affordances: From sensory motor coordination to imitation. IEEE TRO, 24(1): 15-26, 2008.
2. R. F. Lyon. Machine hearing: An emerging field. Signal Processing Magazine, 27(5): 131-139, 2010.
3. S. Zmigrod and B. Hommel. Feature integration across multimodal perception and action: A review. Multisensory research, 26: 143-157, 2013.
4. C. L. Teo, Y. Yang, H. Daumé III, C. Fermüller, and Y. Aloimonos. Towards a watson that sees: Language-guided action recognition for robots. In ICRA, pages 374-381. IEEE, 2012.
5. M. R. Hasan, M. Jamil, M. G. Rabbani, and M. S. Rahman. Speaker indentification using mel frequency cepstral coefficients. In Conference on Electrical and Computer Engineering, 2004.
6. K. Pauwels, L. Rubio, J. Diaz Alonso, and E. Ros. Real-time modelbased rigid object pose estimation and tracking combining dense and sparse visual cues. In CVPR, 2013.
7. C. Desai, D. Ramanan, and C. Fowlkes. Discriminative models for static human-object interactions. In CVPR, 2010.
8. A. Gupta, A. Kembhavi, and L. S. Davis. Observing human-object interactions: Using spatial and functional compatibility for recognition. IEEE PAMI, 31(10): 1775-1789, 2009.
9. H. Kjellström. Contextual action recognition. In T. B. Moeslund, A. Hilton, V. Krüger, and L. Sigal, editors, Guide to Visual Analysis of Humans: Looking at People, chapter 18. Springer, 2011.
10. H. Kjellström, J. Romero, and D. Kragic. Visual object-action recognition: Inferring object affordances from human demonstration. CVIU, 115(1): 81-90, 2011.
11. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld. Learning realistic human actions from movies. In CVPR, 2008.
12. D. J. Moore, I. A. Essa, and M. H. Hayes. Exploiting human actions and object context for recognition tasks. In ICCV, 1999.
13. P. Peursum, G. West, and S. Venkatesh. Combining image regions and human activity for indirect object recognition in indoor wide-angle views. In ICCV, 2005.
14. M. Veloso, F. von Hundelshausen, and P. E. Rybski. Learning visual object definitions by observing human activities. In IEEE-RAS International Conference on Humanoid Robots, 2005.
15. J. Wu, A. Osuntogun, T. Choudhury, M. Philipose, and J. M. Rehg. A scalable approach to activity recognition based on object use. In ICCV, 2007.
16. Benjamin Sapp, Alexander Toshev, and Ben Taskar. Cascaded models for articulated pose estimation, 2010.
17. Salman Aslam, Christopher F. Barnes, and Aaron F. Bobick. Video action recognition using residual vector quantization and hidden markov models. In IPCV, pages 659-666, 2010.
18. Aaron F. Bobick, James W. Davis, Ieee Computer Society, and Ieee Computer Society. The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23: 257-267, 2001.
19. Ivan Laptev and Tony Lindeberg. Space-time interest points. In IN ICCV, pages 432-439, 2003.
20. A. Billard, S. Calinon, R. Dillman, and S. Schaal. Robot programming by demonstration. In B. Siciliano and O. Khatib, editors, Handbook of Robotics, chapter 59. Springer, 2008.
21. T. Lang and M. Toussaint. Planning with noisy probabilistic relational rules. Journal of Artificial Intelligence Research, 39: 1-49, 2010.
22. J. Gall, A. Fossati, and L. van Gool. Functional categorization of objects using real-time markerless motion capture. In CVPR, 2011.
23. H. Grabner, J. Gall, and L. van Gool. What makes a chair a chair? In CVPR, 2011.
24. E. Rivlin, S. J. Dickinson, and A. Rosenfeld. Recognition by functional parts. CVIU, 62(2): 164-176, 1995.
25. L. Stark and K. Bowyer. Generic Object Recognition using Form and Function. World Scientific Series in Machine Perception and Artificial Intelligence-Vol. 10, 1996.
26. M. A. Sutton and L. Stark. Function-based reasoning for goal-oriented image segmentation. In B. Siciliano and O. Khatib, editors, Handbook of Robotics, chapter 59. Springer, 2008.
27. M. W. Turek, A. Hoggs, and R. Collins. Unsupervised learning of functional categories in video scenes. In ECCV, 2010.
28. J. J. Gibson. The Ecological Approach to Visual Perception. Lawrence Erlbaum Associates, 1979.
29. E. E. Aksoy, A. Abramov, F. Wörgötter, and B. Dellen. Categorizing object-action relations from semantic scene graphs. In ICRA, 2010.
30. G. Luo, N. Bergström, C. H. Ek, and D. Kragic. Representing actions with kernels. In IROS, 2011.
31. A. Pieropan, C. H. Ek, and H. Kjellström. Functional object descriptors for human activity modeling. In ICRA, 2013.
32. A. Gupta, S. Satkin, A. A. Efros, and M. Hebert. From 3D scene geometry to human workspace. In CVPR, 2011.
33. B. Moldovan, P. Moreno, M. van Otterlo, J. Santos-Victor, and L. De Raedt. Learning relational affordance models for robots in multi-object manipulation tasks. In ICRA, 2012.
34. G. Salvi, L. Montesano, A. Bernardino, and J. Santos-Victor. Language bootstrapping: Learning word meanings from perception-action association. IEEE SMC-B, 42(3): 660-671, 2012.
35. J. Stork, L. Spinello, J. Silva, and K. O. Arras. Audio-based human activity recognition using non-markovian ensemble voting. In ROMAN, pages 509-514, 2012.
36. A. Treisman. The binding problem. Current Opinion in Neurobiology, 6: 171-8, 1996.
37. G. Saponaro, G. Salvi, and A. Bernardino. Robot anticipation of human intentions through continuous gesture recognition. In International Workshop on Collaborative Robots and Human Robot Interaction, 2013.
38. G. Englebienne, T. F. Cootes, and M. Rattray. A probabilistic model for generating realistic lip movements from speech. In NIPS, 2007.
39. V. Mitra, H. Nam, C. Y. Espy-Wilson, E. Saltzman, and L. Goldstein. Gesture-based dynamic bayesian network for noise robust speech recognition. In ICASSP, pages 5172-5175. IEEE, 2011.
40. M. Mckinney and J. Breebaart. Features for audio and music classification. In International Symposium on Music Information Retrieval, pages 151-158, 2003