Grasp2Vec: Learning Object Representations from Self-Supervised Grasping

Proceedings of Machine Learning Research

Volumn 87, Issue , 2018, Pages 99-112

  Jang, Eric ^a   Devin, Coline ^b   Vanhoucke, Vincent ^a   Levine, Sergey ^a,b  

^a GOOGLE INC   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

instance grasping; reinforcement learning; unsupervised learning

Indexed keywords

HUMAN ROBOT INTERACTION;

GENERALISATION; HUMAN LABELLING; INSTANCE GRASPING; LEARNING METHODS; LEARNING OBJECTS; MANIPULATION TASK; OBJECT REPRESENTATIONS; REINFORCEMENT LEARNINGS; ROBOTIC MANIPULATION; VISUAL REPRESENTATIONS;

REINFORCEMENT LEARNING;

EID: 85113276548     PISSN: None     EISSN: 26403498     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. S. Ross, N. Melik-Barkhudarov, K. S. Shankar, A. Wendel, D. Dey, J. A. Bagnell, and M. Hebert. Learning monocular reactive uav control in cluttered natural environments. In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages 1765-1772. IEEE, 2013.
2. M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang, et al. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316, 2016.
3. A. Ghadirzadeh, A. Maki, D. Kragic, and M. Björkman. Deep predictive policy training using reinforcement learning. In Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ International Conference on, pages 2351-2358. IEEE, 2017.
4. S. Levine, P. Pastor, A. Krizhevsky, and D. Quillen. Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. In International Symposium on Experimental Robotics (ISER 2016), 2016.
5. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740-755. Springer, 2014.
6. P. Agrawal, A. V. Nair, P. Abbeel, J. Malik, and S. Levine. Learning to poke by poking: Experiential learning of intuitive physics. In Advances in Neural Information Processing Systems, pages 5074-5082, 2016.
7. F. Ebert, C. Finn, A. X. Lee, and S. Levine. Self-supervised visual planning with temporal skip connections. arXiv preprint arXiv:1710.05268, 2017.
8. L. Pinto, D. Gandhi, Y. Han, Y.-L. Park, and A. Gupta. The curious robot: Learning visual representations via physical interactions. In European Conference on Computer Vision, pages 3-18. Springer, 2016.
9. D. Jayaraman and K. Grauman. Learning image representations tied to ego-motion. In Proceedings of the IEEE International Conference on Computer Vision, pages 1413-1421, 2015.
10. R. Jonschkowski and O. Brock. Learning state representations with robotic priors. Autonomous Robots, 39(3):407-428, 2015.
11. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111-3119, 2013.
12. A. Radford, L. Metz, and S. Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.
13. X. Wang, A. Farhadi, and A. Gupta. Actions transformations. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2658-2667, 2016.
14. L. Pinto and A. Gupta. Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours. In Robotics and Automation (ICRA), 2016 IEEE International Conference on, pages 3406-3413. IEEE, 2016.
15. I. Lenz, H. Lee, and A. Saxena. Deep learning for detecting robotic grasps. The International Journal of Robotics Research, 34(4-5):705-724, 2015.
16. A. Zeng, K.-T. Yu, S. Song, D. Suo, E. Walker, A. Rodriguez, and J. Xiao. Multi-view selfsupervised deep learning for 6d pose estimation in the amazon picking challenge. In Robotics and Automation (ICRA), 2017 IEEE International Conference on, pages 1386-1383. IEEE, 2017.
17. A. ten Pas, M. Gualtieri, K. Saenko, and R. Platt. Grasp pose detection in point clouds. The International Journal of Robotics Research, page 0278364917735594, 2017.
18. E. Jang, S. Vijaynarasimhan, P. Pastor, J. Ibarz, and S. Levine. End-to-end learning of semantic grasping. arXiv preprint arXiv:1707.01932, 2017.
19. K. Fang, Y. Bai, S. Hinterstoisser, and M. Kalakrishnan. Multi-task domain adaptation for deep learning of instance grasping from simulation. arXiv preprint arXiv:1710.06422, 2017.
20. M. Andrychowicz, F.Wolski, A. Ray, J. Schneider, R. Fong, P.Welinder, B. McGrew, J. Tobin, O. Pieter Abbeel, and W. Zaremba. Hindsight experience replay. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5048-5058. Curran Associates, Inc., 2017.
21. S. Cabi, S. G. Colmenarejo, M.W. Hoffman, M. Denil, Z.Wang, and N. De Freitas. The intentional unintentional agent: Learning to solve many continuous control tasks simultaneously. arXiv preprint arXiv:1707.03300, 2017.
22. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.
23. K. Sohn. Improved deep metric learning with multi-class n-pair loss objective. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 1857-1865. Curran Associates, Inc., 2016.
24. D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke, et al. Qt-opt: Scalable deep reinforcement learning for visionbased robotic manipulation. arXiv preprint arXiv:1806.10293, 2018.
25. E. Coumans and Y. Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016-2018.
26. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211-252, 2015.
27. J. Johnson, B. Hariharan, L. van der Maaten, L. Fei-Fei, C. L. Zitnick, and R. Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, pages 1988-1997. IEEE, 2017.