Towards a Watson that sees: Language-guided action recognition for robots

Proceedings - IEEE International Conference on Robotics and Automation

Volumn , Issue , 2012, Pages 374-381

  Teo, Ching L ^a   Yang, Yezhou ^a   Daumé III, Hal ^a   Fermuller, Cornelia ^a   Aloimonos, Yiannis ^a  

^a University of Maryland ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HIGH LEVEL LANGUAGES; ROBOTICS; ROBOTS;

ACTION RECOGNITION; HIGH LEVEL KNOWLEDGE; LANGUAGE FEATURES; LANGUAGE MODEL; LARGE CORPORA; ROBOTIC APPLICATIONS; STATE OF THE ART; SUPERVISED CLASSIFICATION;

HAND TOOLS;

EID: 84864473231     PISSN: 10504729     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICRA.2012.6224589     Document Type: Conference Paper

References (31)

1. B. D. Argall, S. Chernova, M. Veloso, and B. Browning. A survey of robot learning from demonstration. Robot. Auton. Syst., 57:469-483, May 2009.
2. T. L. Berg, A. C. Berg, J. Edwards, and D. A. Forsyth. Who's in the picture? In NIPS, 2004.
3. A. Bissacco, A. Chiuso, and S. Soatto. Classification and recognition of dynamical models: The role of phase, independent components, kernels and optimal transport. IEEE Trans. Pattern Anal. Mach. Intell., 29:1958-1972, November 2007.
4. T. Brox, C. Bregler, and J. Malik. Large displacement optical flow. In CVPR, pages 41-48. IEEE, 2009.
5. R. Chaudhry, A. Ravichandran, G. Hager, and R. Vidal. Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In CVPR, 2009.
6. T. Cour, C. Jordan, E. Miltsakaki, and B. Taskar. Movie/script: Alignment and parsing of video and text transcription. In ECCV. 2008.
7. P. Dollár, V. Rabaud, G. Cottrell, and S. Belongie. Behavior recognition via sparse spatio-temporal features. In VS-PETS, October 2005.
8. P. Duygulu, K. Barnard, J. F. G. de Freitas, and D. A. Forsyth. Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In A. Heyden, G. Sparr, M. Nielsen, and P. Johansen, editors, ECCV (4), volume 2353 of Lecture Notes in Computer Science, pages 97-112. Springer, 2002.
9. A. Efros, A. Berg, G. Mori, and J. Malik. Recognizing action at a distance. In ICCV, pages 726-733, 2003.
10. P. F. Felzenszwalb, R. B. Girshick, D. A. McAllester, and D. Ramanan. Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell., 32(9):1627-1645, 2010.
11. L. Gorelick, M. Blank, E. Shechtman, R. Basri, and M. Irani. Actions as space-time shapes. PAMI, 29(12):2247-2253, 2007.
12. D. Graff. English gigaword. In Linguistic Data Consortium, Philadel- phia, PA, 2003.
13. A. Gupta and L. S. Davis. Objects in action: An approach for combining action understanding and object perception. In CVPR. IEEE Computer Society, 2007.
14. A. Gupta and L. S. Davis. Beyond nouns: Exploiting prepositions and comparative adjectives for learning visual classifiers. In D. A. Forsyth, P. H. S. Torr, and A. Zisserman, editors, ECCV (1), volume 5302 of Lecture Notes in Computer Science, pages 16-29. Springer, 2008.
15. A. Gupta, A. Kembhavi, and L. S. Davis. Observing human-object interactions: Using spatial and functional compatibility for recognition. IEEE Trans on PAMI, 31(10):1775-1789, 2009.
16. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. The weka data mining software: An update. SIGKDD Explorations, Volume 11, Issue 1, 2009.
17. L. Jie, B. Caputo, and V. Ferrari. Who's doing what: Joint modeling of names and verbs for simultaneous face and pose annotation. In NIPS, December 2009.
18. F. D. la Torre, J. Hodgins, J. Montano, S. Valcarcel, R. Forcada, and J. Macey. Guide to the carnegie mellon university multimodal activity (cmu-mmac) database. Technical report, CMU-RI-TR-08-22, Robotics Institute, Carnegie Mellon University, July 2009.
19. I. Laptev and T. Lindeberg. Space-time interest points. In ICCV, pages 432-439. IEEE Computer Society, 2003.
20. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld. Learning realistic human actions from movies. In CVPR, 2008.
21. A. P. B. Lopes, E. A. d. Valle, J. M. de Almeida, and A. A. de Arajo. Action recognition in videos: from motion capture labs to the web. (arXiv:1006.3506), Jun 2010. Preprint submitted to CVIU.
22. R. Messing, C. Pal, and H. Kautz. Activity recognition using the velocity histories of tracked keypoints. In ICCV '09: Proceedings of the Twelfth IEEE International Conference on Computer Vision, Washington, DC, USA, 2009. IEEE Computer Society.
23. C. Rother, V. Kolmogorov, and A. Blake. 'grabcut': interactive foreground extraction using iterated graph cuts. ACM Trans. Graph., 23(3):309-314, 2004.
24. B. Sapp, A. Toshev, and B. Taskar. Cascaded models for articulated pose estimation. In ECCV, 2010.
25. C. Schuldt, I. Laptev, and B. Caputo. Recognizing human actions: A local svm approach. In ICPR, 2004.
26. W. Schwartz, A. Kembhavi, D. Harwood, and L. Davis. Human detection using partial least squares analysis. In International Conference on Computer Vision, 2009.
27. L. Sigal, A. O. Balan, and M. J. Black. Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. International Journal of Computer Vision, 87(1-2):4-27, 2010.
28. P. K. Turaga, R. Chellappa, V. S. Subrahmanian, and O. Udrea. Machine recognition of human activities: A survey. IEEE Trans. Circuits Syst. Video Techn., 18(11):1473-1488, 2008.
29. P. Viola and M. Jones. Robust real-time face detection. International Journal of Computer Vision, 57(2):137-154, 2004.
30. D. Weinland, R. Ronfard, and E. Boyer. A survey of visionbased methods for action representation, segmentation and recognition. Computer Vision and Image Understanding, 115(2):224-241, 2010.
31. A. Yilmaz and M. Shah. Actions sketch: A novel action representation. In CVPR, pages 984-989, 2005.