Robot learning manipulation action plans by "watching" unconstrained videos from the World Wide Web

Proceedings of the National Conference on Artificial Intelligence

Volumn 5, Issue , 2015, Pages 3686-3692

  Yang, Yezhou ^a   Li, Yi ^b   Fermüller, Cornelia ^a   Aloimonos, Yiannis ^a  

^a UNIVERSITY OF MARYLAND   (United States)

^b CSIRO   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; FORMAL LANGUAGES; NEURAL NETWORKS; OBJECT RECOGNITION; ROBOTS; SYNTACTICS; WORLD WIDE WEB;

ATOMIC ACTIONS; COMPUTATIONAL TOOLS; CONVOLUTIONAL NEURAL NETWORK; HAND GRASPS; HIGH-ACCURACY; HUMAN ACTIONS; ROBOT MANIPULATION; VIDEO DATASET;

VIDEO SIGNAL PROCESSING;

EID: 84961223498     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. Aksoy, E.; Abramov, A.; Dörr, J.; Ning, K.; Dellen, B.; and Wörgötter, F. 2011. Learning the semantics of object-action relations by observation. The International Journal of Robotics Research 30(10):1229-1249.
2. Argall, B. D.; Chernova, S.; Veloso, M.; and Browning, B. 2009. A survey of robot learning from demonstration. Robotics and Autonomous Systems 57(5):469-483.
3. Bengio, Y.; Courville, A.; and Vincent, P. 2013. Representation learning: A review and new perspectives. Pattern Analysis and Machine Intelligence, IEEE Transactions on 35(8):1798-1828.
4. Bird, S.; Klein, E.; and Loper, E. 2009. Natural language processing with Python. " O'Reilly Media, Inc.".
5. Cheng, M.-M.; Zhang, Z.; Lin, W.-Y.; and Torr, P. H. S. 2014. BING: Binarized normed gradients for objectness estimation at 300fps. In IEEE CVPR.
6. Chomsky, N. 1993. Lectures on government and binding: The Pisa lectures. Berlin: Walter de Gruyter.
7. Church, K. W. 1988. A stochastic parts program and noun phrase parser for unrestricted text. In Proceedings of the second conference on Applied natural language processing, 136-143. Association for Computational Linguistics.
8. Ciresan, D. C.; Meier, U.; and Schmidhuber, J. 2012. Multi-column deep neural networks for image classification. In CVPR 2012.
9. Dalai, N., and Triggs, B. 2005. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 1, 886-893. IEEE.
10. Das, P.; Xu, C.; Doell, R. F.; and Corso, J. J. 2013. A thousand frames in just a few words: Lingual description of videos through latent topics and sparse object stitching. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.
11. Dunning, T. 1993. Accurate methods for the statistics of surprise and coincidence. Computational Linguistics 19(1):61-74.
12. Feix, T.; Romero, J.; Ek, C. H.; Schmiedmayer, H.; and Kragic, D. 2013. A Metric for Comparing the Anthropomorphic Motion Capability of Artificial Hands. Robotics, IEEE Transactions on 29(1):82-93.
13. Graff, D. 2003. English gigaword. In Linguistic Data Consortium, Philadelphia, PA.
14. Guerra-Filho, G.; Fermiiller, C.; and Aloimonos, Y. 2005. Discovering a language for human activity. In Proceedings of the AAAI 2005 Fall Symposium on Anticipatory Cognitive Embodied Systems. Washington, DC: AAAI.
15. Guha, A.; Yang, Y.; Fermiiller, C.; and Aloimonos, Y. 2013. Minimalist plans for interpreting manipulation actions. In Proceedings of the 2013 International Conference on Intelligent Robots and Systems, 5908-5914. Tokyo: IEEE.
16. Jeannerod, M. 1984. The timing of natural prehension movements. Journal of motor behavior 16(3):235-254.
17. Jia, Y. 2013. Caffe: An open source convolutional architecture for fast feature embedding, http://caffe.berkeleyvision.org/.
18. Krizhevsky, A.; Sutskever, I.; and Hinton, G. 2013. Imagenet classification with deep convolutional neural networks. In NIPS 2012.
19. LeCun, Y., and Bengio, Y. 1998. The handbook of brain theory and neural networks. Cambridge, MA, USA: MIT Press, chapter Convolutional networks for images, speech, and time series, 255-258.
20. Lee, K.; Su, Y.; Kim, T.-K.; and Demiris, Y. 2013. A syntactic approach to robot imitation learning using probabilistic activity grammars. Robotics and Autonomous Systems 61(12):1323-1334.
21. Lenz, I.; Lee, H.; and Saxena, A. 2014. Deep learning for detecting robotic grasps. International Journal of Robotics Research to appear.
22. Li, Y.; Fermüller, C.; Aloimonos, Y.; and Ji, H. 2010. Learning shift-invariant sparse representation of actions. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2630-2637. San Francisco, CA: IEEE.
23. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. International journal of computer vision 60(2):91-110.
24. Mittal, A.; Zisserman, A.; and Torr, P. H. 2011. Hand detection using multiple proposals. In BMVC, l-11. Citeseer.
25. Oikonomidis, I.; Kyriazis, N.; and Argyros, A. 2011. Efficient model-based 3D tracking of hand articulations using Kinect. In Proceedings of the 2011 British Machine Vision Conference, 1-11. Dundee, UK: BMVA.
26. Pastra, K., and Aloimonos, Y. 2012. The minimalist grammar of action. Philosophical Transactions of the Royal Society: Biological Sciences 367(1585):103-117.
27. Saxena, A.; Driemeyer, J.; and Ng, A. Y. 2008. Robotic grasping of novel objects using vision. The International Journal of Robotics Research 27(2):157-173.
28. Shimoga, K. B. 1996. Robot grasp synthesis algorithms: A survey. The International Journal of Robotics Research l 5(3):230-266.
29. Steedman, M. 2002. Plans, affordances, and combinatory grammar. Linguistics and Philosophy 25(5-6):723-753.
30. Summers-Stay, D.; Teo, C.; Yang, Y.; Fermüller, C.; and Aloimonos, Y. 2013. Using a minimal action grammar for activity understanding in the real world. In Proceedings of the 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 4104-4111. Vilamoura, Portugal: IEEE.
31. Tenorth, M.; Ziegltrum, J.; and Beetz, M. 2013. Automated alignment of specifications of everyday manipulation tasks. In IROS. IEEE.
32. Yang, Y.: Teo, C. L.; Daumé III, H.; and Aloimonos, Y. 2011. Corpus-guided sentence generation of natural images. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 444-454. Association for Computational Linguistics.
33. Yang, Y.; Guha, A.; Fermuller, C.; and Aloimonos, Y. 2014. A cognitive system for understanding human manipulation actions. Advances in Cognitive Sysytems 3:67-86.