Dex-Net 2.0: Deep learning to plan Robust grasps with synthetic point clouds and analytic grasp metrics

Robotics: Science and Systems

Volumn 13, Issue , 2017, Pages

  Mahler, Jeffrey ^a   Liang, Jacky ^a   Niyaz, Sherdil ^a   Laskey, Michael ^a   Doan, Richard ^a   Liu, Xinyu ^a   Ojea, Juan Aparicio ^b   Goldberg, Ken ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b SIEMENS CORPORATE RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

NEURAL NETWORKS; ROBOTICS;

CONVOLUTIONAL NEURAL NETWORK; DATA COLLECTION; FALSE POSITIVE; GRASP PLANNING; GRASP QUALITIES; PROBABILITY OF SUCCESS; RGB-D SENSORS; SYNTHETIC DATA;

DEEP LEARNING;

EID: 85038414017     PISSN: None     EISSN: 2330765X     Source Type: Conference Proceeding    
DOI: 10.15607/rss.2017.xiii.058     Document Type: Conference Paper

References (60)

1. Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467, 2016.
2. Ravi Balasubramanian, Ling Xu, Peter D Brook, Joshua R Smith, and Yoky Matsuoka. Physical human interactive guidance: Identifying grasping principles from human-planned grasps. IEEE Trans. Robotics, 28(4):899-910, 2012.
3. Jeannette Bohg and Danica Kragic. Learning grasping points with shape context. Robotics and Autonomous Systems, 58(4):362-377, 2010.
4. Jeannette Bohg, Antonio Morales, Tamim Asfour, and Danica Kragic. Data-driven grasp synthesisa survey. IEEE Trans. Robotics, 30(2):289-309, 2014.
5. Peter Brook, Matei Ciocarlie, and Kaijen Hsiao. Collaborative grasp planning with multiple object representations. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), pages 2851-2858. IEEE, 2011.
6. Lillian Chang, Joshua R Smith, and Dieter Fox. Interactive singulation of objects from a pile. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), pages 3875-3882. IEEE, 2012.
7. I-Ming Chen and Joel W Burdick. Finding antipodal point grasps on irregularly shaped objects. IEEE Trans. Robotics and Automation, 9(4):507-512, 1993.
8. Matei Ciocarlie, Kaijen Hsiao, Edward Gil Jones, Sachin Chitta, Radu Bogdan Rusu, and Ioan A Şucan. Towards reliable grasping and manipulation in household environments. In Experimental Robotics, pages 241-252. Springer, 2014.
9. Navneet Dalal and Bill Triggs. Histograms of oriented gradients for human detection. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), volume 1, pages 886-893. IEEE, 2005.
10. Renaud Detry, Carl Henrik Ek, Marianna Madry, and Danica Kragic. Learning a dictionary of prototypical grasp-predicting parts from grasping experience. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), pages 601-608. IEEE, 2013.
11. Andreas Eitel, Jost Tobias Springenberg, Luciano Spinello, Martin Riedmiller, and Wolfram Burgard. Multimodal deep learning for robust rgb-d object recognition. In Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pages 681-687. IEEE, 2015.
12. Jerome Friedman, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning, volume 1. Springer series in statistics Springer, Berlin, 2001.
13. Ken Goldberg, Brian V Mirtich, Yan Zhuang, John Craig, Brian R Carlisle, and John Canny. Part pose statistics: Estimators and experiments. IEEE Trans. Robotics and Automation, 15(5):849-857, 1999.
14. Corey Goldfeder and Peter K Allen. Data-driven grasping. Autonomous Robots, 31(1):1-20, 2011.
15. Corey Goldfeder, Matei Ciocarlie, Hao Dang, and Peter K Allen. The columbia grasp database. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), pages 1710-1716. IEEE, 2009.
16. Marcus Gualtieri, Andreas ten Pas, Kate Saenko, and Robert Platt. High precision grasp pose detection in dense clutter. arXiv preprint arXiv:1603.01564, 2016.
17. Menglong Guo, David V Gealy, Jacky Liang, Jeffrey Mahler, Aimee Goncalves, Stephen McKinley, and Ken Goldberg. Design of parallel-jaw gripper tip surfaces for robust grasping. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), 2017.
18. Saurabh Gupta, Pablo Arbeláez, Ross Girshick, and Jitendra Malik. Aligning 3d models to rgb-d images of cluttered scenes. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 4731-4740, 2015.
19. Richard Hartley and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge university press, 2003.
20. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proc. IEEE Int. Conf. on Computer Vision (ICCV), pages 1026-1034, 2015.
21. Carlos Hernandez, Mukunda Bharatheesha, Wilson Ko, Hans Gaiser, Jethro Tan, Kanter van Deurzen, Maarten de Vries, Bas Van Mil, Jeff van Egmond, Ruben Burger, et al. Team delft's robot winner of the amazon picking challenge 2016. arXiv preprint arXiv:1610.05514, 2016.
22. Alexander Herzog, Peter Pastor, Mrinal Kalakrishnan, Ludovic Righetti, Jeannette Bohg, Tamim Asfour, and Stefan Schaal. Learning of grasp selection based on shape-templates. Autonomous Robots, 36(1-2):51-65, 2014.
23. Stefan Hinterstoisser, Stefan Holzer, Cedric Cagniart, Slobodan Ilic, Kurt Konolige, Nassir Navab, and Vincent Lepetit. Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes. In Proc. IEEE Int. Conf. on Computer Vision (ICCV), pages 858-865. IEEE, 2011.
24. Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. Spatial transformer networks. In Proc. Advances in Neural Information Processing Systems, pages 2017-2025, 2015.
25. Edward Johns, Stefan Leutenegger, and Andrew J Davison. Deep learning a grasp function for grasping under gripper pose uncertainty. In Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pages 4461-4468. IEEE, 2016.
26. Daniel Kappler, Jeannette Bohg, and Stefan Schaal. Leveraging big data for grasp planning. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), 2015.
27. Alexander Kasper, Zhixing Xue, and Rüdiger Dillmann. The kit object models database: An object model database for object recognition, localization and manipulation in service robotics. Int. Journal of Robotics Research (IJRR), 31(8):927-934, 2012.
28. Ben Kehoe, Akihiro Matsukawa, Sal Candido, James Kuffner, and Ken Goldberg. Cloud-based robot grasping with the google object recognition engine. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), pages 4263-4270. IEEE, 2013.
29. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Proc. Advances in Neural Information Processing Systems, pages 1097-1105, 2012.
30. Michael Laskey, Jeffrey Mahler, Zoe McCarthy, Florian T Pokorny, Sachin Patil, Jur van den Berg, Danica Kragic, Pieter Abbeel, and Ken Goldberg. Multi-armed bandit models for 2d grasp planning with uncertainty. In Proc. IEEE Conf. on Automation Science and Engineering (CASE). IEEE, 2015.
31. Michael Laskey, Caleb Chuck, Jonathan Lee, Jeffrey Mahler, Sanjay Krishnan, Kevin Jamieson, Anca Dragan, and Ken Goldberg. Comparing human-centric and robot-centric sampling for robot deep learning from demonstrations. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA). IEEE, 2017.
32. Ian Lenz, Honglak Lee, and Ashutosh Saxena. Deep learning for detecting robotic grasps. Int. Journal of Robotics Research (IJRR), 34(4-5):705-724, 2015.
33. Sergey Levine, Peter Pastor, Alex Krizhevsky, and Deirdre Quillen. Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. arXiv preprint arXiv:1603.02199, 2016.
34. Jeffrey Mahler, Brian Hou, Sherdil Niyaz, Florian T Pokorny, Ramu Chandra, and Ken Goldberg. Privacy-preserving grasp planning in the cloud. In Proc. IEEE Conf. on Automation Science and Engineering (CASE), pages 468-475. IEEE, 2016.
35. Jeffrey Mahler, Florian T Pokorny, Brian Hou, Melrose Roderick, Michael Laskey, Mathieu Aubry, Kai Kohlhoff, Torsten Kröger, James Kuffner, and Ken Goldberg. Dex-net 1.0: A cloud-based network of 3d objects for robust grasp planning using a multiarmed bandit model with correlated rewards. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA). IEEE, 2016.
36. Tanwi Mallick, Partha Pratim Das, and Arun Kumar Majumdar. Characterizations of noise in kinect depth images: A review. IEEE Sensors Journal, 14(6):1731-1740, 2014.
37. John W Miller, Rod Goodman, and Padhraic Smyth. On loss functions which minimize to conditional expected values and posterior probabilities. IEEE Transactions on Information Theory, 39(4):1404-1408, 1993.
38. Luis Montesano and Manuel Lopes. Active learning of visual descriptors for grasping using non-parametric smoothed beta distributions. Robotics and Autonomous Systems, 60(3):452-462, 2012.
39. Anh Nguyen, Dimitrios Kanoulas, Darwin G Caldwell, and Nikos G Tsagarakis. Detecting object affordances with convolutional neural networks. In Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pages 2765-2770. IEEE, 2016.
40. John Oberlin and Stefanie Tellex. Autonomously acquiring instance-based object models from experience. In Int. S. Robotics Research (ISRR), 2015.
41. Lerrel Pinto and Abhinav Gupta. Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), 2016.
42. Lerrel Pinto, James Davidson, and Abhinav Gupta. Supervision via competition: Robot adversaries for learning tasks. arXiv preprint arXiv:1610.01685, 2016.
43. Florian T Pokorny and Danica Kragic. Classical grasp quality evaluation: New algorithms and theory. In Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pages 3493-3500. IEEE, 2013.
44. Lorenzo Porzi, Samuel Rota Bulo, Adrian Penate-Sanchez, Elisa Ricci, and Francesc Moreno-Noguer. Learning depth-aware deep representations for robotic perception. IEEE Robotics & Automation Letters, 2016.
45. Domenico Prattichizzo and Jeffrey C Trinkle. Grasping. In Springer handbook of robotics, pages 671-700. Springer, 2008.
46. Joseph Redmon and Anelia Angelova. Real-time grasp detection using convolutional neural networks. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), pages 1316-1322. IEEE, 2015.
47. Alberto Rodriguez, Matthew T Mason, and Steve Ferry. From caging to grasping. Int. Journal of Robotics Research (IJRR), page 0278364912442972, 2012.
48. Fereshteh Sadeghi and Sergey Levine. (cad)2 rl: Real singleimage flight without a single real image. arXiv preprint arXiv:1611.04201, 2016.
49. Renato F Salas-Moreno, Richard A Newcombe, Hauke Strasdat, Paul HJ Kelly, and Andrew J Davison. Slam++: Simultaneous localisation and mapping at the level of objects. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 1352-1359, 2013.
50. Ashutosh Saxena, Justin Driemeyer, and Andrew Y Ng. Robotic grasping of novel objects using vision. The International Journal of Robotics Research, 27(2):157-173, 2008.
51. Daniel Seita, Florian T Pokorny, Jeffrey Mahler, Danica Kragic, Michael Franklin, John Canny, and Ken Goldberg. Large-scale supervised learning of the grasp robustness of surface patch pairs. In Proc. IEEE Int. Conf. on Simulation, Modeling, and Programming of Autonomous Robots (SIMPAR). IEEE, 2016.
52. Andreas ten Pas and Robert Platt. Using geometry to detect grasp poses in 3d point clouds. In Intl Symp. on Robotics Research, 2015.
53. Eric Tzeng, Coline Devin, Judy Hoffman, Chelsea Finn, Pieter Abbeel, Sergey Levine, Kate Saenko, and Trevor Darrell. Adapting deep visuomotor representations with weak pairwise constraints. In Workshop on the Algorithmic Foundation of Robotics (WAFR), 2016.
54. Jacob Varley, Jonathan Weisz, Jared Weiss, and Peter Allen. Generating multi-fingered robotic grasps via deep learning. In Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pages 4415-4420. IEEE, 2015.
55. Matthew Veres, Medhat Moussa, and Graham W Taylor. Modeling grasp motor imagery through deep conditional generative models. arXiv preprint arXiv:1701.03041, 2017.
56. Jonathan Weisz and Peter K Allen. Pose error robust grasping from contact wrench space metrics. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), pages 557-562. IEEE, 2012.
57. Walter Wohlkinger, Aitor Aldoma, Radu B Rusu, and Markus Vincze. 3dnet: Large-scale object class recognition from cad models. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), pages 5384-5391. IEEE, 2012.
58. Ziang Xie, Arjun Singh, Justin Uang, Karthik S Narayan, and Pieter Abbeel. Multimodal blending for high-accuracy instance recognition. In Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pages 2214-2221. IEEE, 2013.
59. Andy Zeng, Kuan-Ting Yu, Shuran Song, Daniel Suo, Ed Walker Jr, Alberto Rodriguez, and Jianxiong Xiao. Multi-view self-supervised deep learning for 6d pose estimation in the amazon picking challenge. arXiv preprint arXiv:1609.09475, 2016.
60. Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi. Target-driven visual navigation in indoor scenes using deep reinforcement learning. arXiv preprint arXiv:1609.05143, 2016.