Deep learning for tactile understanding from visual and haptic data

Proceedings - IEEE International Conference on Robotics and Automation

Volumn 2016-June, Issue , 2016, Pages 536-543

  Gao, Yang ^a   Hendricks, Lisa Anne ^a   Kuchenbecker, Katherine J ^b   Darrell, Trevor ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

NATURAL LANGUAGE PROCESSING SYSTEMS; ROBOTS;

COGNITIVE PATTERNS; DEEP NEURAL NETWORKS; DOMAIN-SPECIFIC KNOWLEDGE; PHYSICAL INTERACTIONS; PHYSICAL WORLD; PREDICTION MODEL; UNIFIED APPROACH; VISUAL OBSERVATIONS;

ROBOTICS;

EID: 84977528092     PISSN: 10504729     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICRA.2016.7487176     Document Type: Conference Paper

References (37)

1. O. Abdel-Hamid, A.-r. Mohamed, H. Jiang, and G. Penn, "Applying convolutional neural networks concepts to hybrid nn-hmm model for speech recognition, " in Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on. IEEE, 2012, pp. 4277-4280.
2. P. Agrawal, R. Girshick, and J. Malik, "Analyzing the performance of multilayer neural networks for object recognition, " in Computer Vision-ECCV 2014. Springer, 2014, pp. 329-344.
3. A. Amedi, K. von Kriegstein, N. M. van Atteveldt, M. Beauchamp, and M. J. Naumer, "Functional imaging of human crossmodal identification and object recognition, " Experimental Brain Research, vol. 166, no. 3-4, pp. 559-571, 2005.
4. D. Bahdanau, K. Cho, and Y. Bengio, "Neural machine translation by jointly learning to align and translate, " arXiv preprint arXiv:1409. 0473, 2014.
5. S. Bell, P. Upchurch, N. Snavely, and K. Bala, "Material recognition in the wild with the materials in context database, " arXiv preprint arXiv:1412. 0623, 2014.
6. A. Burka, S. Hu, S. Krishnan, K. J. Kuchenbecker, L. A. Hendricks, Y. Gao, and T. Darrell, "Toward a large-scale visuo-haptic dataset for robotic learning, " 2015, proceedings of the Workshop on the Future of Datasets in Vision at Computer Vision and Pattern Recognition (CVPR).
7. V. Chu, I. McMahon, L. Riano, C. G. McDonald, Q. He, J. M. Perez-Tejada, M. Arrigo, T. Darrell, and K. J. Kuchenbecker, "Robotic learning of haptic adjectives through physical interaction, " Robotics and Autonomous Systems, vol. 63, no. 3, pp. 279-292, January 2015.
8. V. Chu, I. McMahon, L. Riano, C. G. McDonald, Q. He, J. M. Perez-Tejada, M. Arrigo, N. Fitter, J. Nappo, T. Darrell, and K. J. Kuchenbecker, "Using robotic exploratory procedures to learn the meaning of haptic adjectives, " in Proc. IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, May 2013, pp. 3048-3055.
9. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database, " in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009, pp. 248-255.
10. J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell, "Decaf: A deep convolutional activation feature for generic visual recognition, " arXiv preprint arXiv:1310. 1531, 2013.
11. Z. Du, W. Yu, Y. Yu, L. Chen, Y. Ni, J. Luo, and X. Yu, "Fabric handle clusters based on fuzzy clustering algorithm, " in Fuzzy Systems and Knowledge Discovery, 2008. FSKD'08. Fifth International Conference on, vol. 3. IEEE, 2008, pp. 546-550.
12. J. A. Fishel and G. E. Loeb, "Bayesian exploration for intelligent identification of textures, " Frontiers in neurorobotics, vol. 6, 2012.
13. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation, " in Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on. IEEE, 2014, pp. 580-587.
14. X. Glorot and Y. Bengio, "Understanding the difficulty of training deep feedforward neural networks, " in International conference on artificial intelligence and statistics, 2010, pp. 249-256.
15. A. Graves and N. Jaitly, "Towards end-to-end speech recognition with recurrent neural networks, " in Proceedings of the 31st International Conference on Machine Learning (ICML-14), 2014, pp. 1764-1772.
16. A. Graves, M. Liwicki, H. Bunke, J. Schmidhuber, and S. Fernández, "Unconstrained on-line handwriting recognition with recurrent neural networks, " in Advances in Neural Information Processing Systems, 2008, pp. 577-584.
17. S. Griffith, J. Sinapov, V. Sukhoy, and A. Stoytchev, "A behaviorgrounded approach to forming object categories: Separating containers from noncontainers, " Autonomous Mental Development, IEEE Transactions on, vol. 4, no. 1, pp. 54-69, 2012.
18. S. Gupta, R. Girshick, P. Arbeláez, and J. Malik, "Learning rich features from rgb-d images for object detection and segmentation, " in Computer Vision-ECCV 2014. Springer, 2014, pp. 345-360.
19. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al., "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, " Signal Processing Magazine, IEEE, vol. 29, no. 6, pp. 82-97, 2012.
20. S. Hochreiter and J. Schmidhuber, "Long short-term memory, " Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
21. R. S. Johansson and J. R. Flanagan, "Coding and use of tactile signals from the fingertips in object manipulation tasks, " Nature Reviews Neuroscience, vol. 10, no. 5, pp. 345-359, 2009.
22. S. E. Kahou, X. Bouthillier, P. Lamblin, C. Gulcehre, V. Michalski, K. Konda, S. Jean, P. Froumenty, A. Courville, P. Vincent, et al., "Emonets: Multimodal deep learning approaches for emotion recognition in video, " arXiv preprint arXiv:1503. 01800, 2015.
23. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks, " in Advances in neural information processing systems, 2012, pp. 1097-1105.
24. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
25. M. Lodén, H. Olsson, L. Skare, and T. Axéll, "Instrumental and sensory evaluation of the frictional response of the skin following a single application of five moisturizing creams, " J Soc Cosmet Chem, vol. 43, pp. 13-20, 1992.
26. J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation, " arXiv preprint arXiv:1411. 4038, 2014.
27. H. McGurk and J. MacDonald, "Hearing lips and seeing voices, " 1976.
28. T. Mikolov, M. Karafiát, L. Burget, J. Cernockỳ, and S. Khudanpur, "Recurrent neural network based language model. " in INTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, September 26-30, 2010, 2010, pp. 1045-1048.
29. Y. Mukaibo, H. Shirado, M. Konyo, and T. Maeno, "Development of a texture sensor emulating the tissue structure and perceptual mechanism of human fingers, " in Robotics and Automation, 2005. ICRA 2005. Proceedings of the 2005 IEEE International Conference on. IEEE, 2005, pp. 2565-2570.
30. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng, "Multimodal deep learning, " in Proceedings of the 28th international conference on machine learning (ICML-11), 2011, pp. 689-696.
31. N. Pan, "Quantification and evaluation of human tactile sense towards fabrics, " International Journal of Design and Nature, vol. 1, no. 1, pp. 48-60, 2007.
32. D. M. Powers, "Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlation, " 2011.
33. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions, " arXiv preprint arXiv:1409. 4842, 2014.
34. O. Vinyals, L. Kaiser, T. Koo, S. Petrov, I. Sutskever, and G. Hinton, "Grammar as a foreign language, " arXiv preprint arXiv:1412. 7449, 2014.
35. N. Wettels, V. J. Santos, R. S. Johansson, and G. E. Loeb, "Biomimetic tactile sensor array, " Advanced Robotics, vol. 22, no. 8, pp. 829-849, 2008.
36. A. T. Woods and F. N. Newell, "Visual, haptic and cross-modal recognition of objects and scenes, " Journal of physiology-Paris, vol. 98, no. 1, pp. 147-159, 2004.
37. B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva, "Learning deep features for scene recognition using places database, " in Advances in Neural Information Processing Systems, 2014, pp. 487-495.