Multimodal integration learning of robot behavior using deep neural networks

Robotics and Autonomous Systems

Volumn 62, Issue 6, 2014, Pages 721-736

  Noda, Kuniaki ^a   Arie, Hiroaki ^a   Suga, Yuki ^a   Ogata, Tetsuya ^a  

^a WASEDA UNIVERSITY   (Japan)

Author keywords

Cross modal memory retrieval; Deep learning; Multimodal integration; Object manipulation

Indexed keywords

ANTHROPOMORPHIC ROBOTS; DATA PROCESSING; EXPERIMENTS; INTEGRATION; LEARNING ALGORITHMS; LEARNING SYSTEMS;

COMPUTATIONAL FRAMEWORK; CONVENTIONAL MACHINES; DEEP LEARNING; GENERALIZATION CAPABILITY; MEMORY RETRIEVAL; MULTIMODAL INTEGRATION; OBJECT MANIPULATION; ROBOTIC APPLICATIONS;

BEHAVIORAL RESEARCH;

EID: 84899525901     PISSN: 09218890     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.robot.2014.03.003     Document Type: Article

References (40)

1. M.O. Ernst, and H.H. Bülthoff Merging the senses into a robust percept Trends Cogn. Sci. 8 4 2004 162 169
2. B.E. Stein, and M.A. Meredith The Merging of the Senses 1993 The MIT Press
3. I. Gallagher Philosophical conceptions of the self: implications for cognitive science Trends Cogn. Sci. 4 1 2000 14 21
4. T. Kawabe, W. Roseboom, and S. Nishida The sense of agency is action-effect causality perception based on cross-modal grouping Proc. R. Soc. B: Biological Sciences 280 1763 2013 20130991
5. R.A. Brooks, C.B. (Ferrell), R. Irie, C.C. Kemp, M. Marjanovic, B. Scassellati, M.M. Williamson, Alternative essences of intelligence, in: Proceedings of the 15th National Conference on Artificial Intelligence, Madison, Wisconsin, USA, 1998, pp. 961-968.
6. M. Coen, Multimodal integration-a biological view, in: Proceedings of the 17th International Joint Conference on Artificial Intelligence, vol. 2, Seattle, Washington, USA, 2001, pp. 1417-1424.
7. A. Pitti, A. Blanchard, M. Cardinaux, P. Gaussier, Distinct mechanisms for multimodal integration and unimodal representation in spatial development, in: Proceedings of the IEEE International Conference on Development and Learning and Epigenetic Robotics, San Diego, California, USA, 2012, pp. 1-6.
8. A. Jauffret, N. Cuperlier, P. Gaussier, P. Tarroux, Multimodal integration of visual place cells and grid cells for navigation tasks of a real robot, in: Proceedings of the 12th International Conference on Simulation of Adaptive Behavior, vol. 7426, Odense, Denmark, 2012, pp. 136-145.
9. E. Sauser, A. Billard, Biologically inspired multimodal integration: interferences in a human-robot interaction game, in: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China, 2006, pp. 5619-5624.
10. R.R. Murphy Introduction to AI Robotics 2000 The MIT Press
11. M. Ogino, H. Toichi, Y. Yoshikawa, and M. Asada Interaction rule learning with a human partner based on an imitation faculty with a simple visuo-motor mapping Robot. Auton. Syst. 54 5 2006 414 418
12. T. Kuriyama, T. Shibuya, T. Harada, Y. Kuniyoshi, Learning interaction rules through compression of sensori-motor causality space, in: Proceedings of the 10th International Conference on Epigenetic Robotics, Örenäs Slott, Sweden, 2010, pp. 57-64.
13. Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki, K. Fujimura, The intelligent ASIMO: system overview and integration, in: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and System, vol. 3, Lausanne, Switzerland, 2002, pp. 2478-2483.
14. K. Kaneko, F. Kanehiro, S. Kajita, H. Hirukawa, T. Kawasaki, M. Hirata, K. Akachi, T. Isozumi, Humanoid robot HRP-2, in: Proceedings of the IEEE International Conference on Robotics and Automation, vol. 2, Barcelona, Spain, 2004, pp. 1083-1090.
15. Willow Grarage, Personal Robot 2 (PR2), http://www.willowgarage.com/ pages/pr2/overview.
16. R. Bekkerman, M. Bilenko, and J. Langford Scaling up Machine Learning: Parallel and Distributed Approaches 2011 Cambridge University Press
17. Y. Bengio Learning deep architectures for AI Found. Trends Mach. Learn. 2 1 2009 1 127
18. I. Sutskever, J. Martens, G. Hinton, Generating text with recurrent neural networks, in: Proceedings of the 28th International Conference on Machine Learning, Bellevue, Washington, USA, 2011, pp. 1017-1024.
19. Q.V. Le, M. Ranzato, R. Monga, M. Devin, K. Chen, G.S. Corrado, J. Dean, A.Y. Ng, Building high-level features using large scale unsupervised learning, in: Proceedings of the 29th International Conference on Machine Learning, Edinburgh, Scotland, 2012, pp. 81-88.
20. G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. Sainath, and B. Kingsbury Deep neural networks for acoustic modeling in speech recognition IEEE Signal Process. Mag. 29 2012 82 97
21. A. Krizhevsky, I. Sutskever, G. Hinton, Imagenet classification with deep convolutional neural networks, in: Proceedings of the Advances in Neural Information Processing Systems 25, Lake Tahoe, Nevada, USA, 2012, pp. 1106-1114.
22. Rosenberg Chuck, Improving photo search: a step across the semantic gap, http://googleresearch.blogspot.jp/2013/06/improving-photo-search-step-across. html (Jun. 2013).
23. Hof Robert, Meet the guy who helped google beat Apple's Siri, http://www.forbes.com/sites/roberthof/2013/05/01/meet-the-guy-who-helped-google- beat-apples-siri/ (May 2013).
24. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, A.Y. Ng, Multimodal deep learning, in: Proceedings of the 28th International Conference on Machine Learning, Bellevue, Washington, USA, 2011, pp. 689-696.
25. N. Srivastava, R. Salakhutdinov, Multimodal learning with deep Boltzmann machines, in: Proceedings of the Advances in Neural Information Processing Systems 25, Lake Tahoe, Nevada, USA, 2012, pp. 2231-2239.
26. J. Dewey The reflex arc concept in psychology Psychol. Rev. 3 1896 357 370
27. G.E. Hinton, and R.R. Salakhutdinov Reducing the dimensionality of data with neural networks Science 313 5786 2006 504 507
28. J. Martens, Deep learning via Hessian-free optimization, in: Proceedings of the 27th International Conference on Machine Learning, Haifa, Israel, 2010, pp. 735-742.
29. K. Lang, A. Waibel, and G. Hinton A time-delay neural network architecture for isolated word recognition Neural Netw. 3 1990 23 43
30. B. Pearlmutter Fast exact multiplication by the Hessian Neural Comput. 6 1 1994 147 160
31. N.N. Schraudolph Fast curvature matrix-vector products for second-order gradient descent Neural Comput. 14 7 2002 1723 1738
32. Aldebaran Robotics, NAO Humanoid, http://www.aldebaran-robotics.com/ Downloads/Download-document/192-Datasheet-NAO-Humanoid.html (Nov. 2012).
33. A. Krizhevsky, G.E. Hinton, Using very deep autoencoders for content-based image retrieval, in: Proceedings of the 19th European Symposium on Artificial Neural Networks, Bruges, Belgium, 2011.
34. V. Franc, V. Hlavac, Statistical pattern recognition toolbox for matlab, http://cmp.felk.cvut.cz/cmp/software/stprtool/ (Aug. 2008).
35. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner Gradient-based learning applied to document recognition Proc. IEEE 86 11 1998 2278 2324
36. M.D. Grilli, and E.L. Glisky The self-imagination effect: benefits of a self-referential encoding strategy on cued recall in memory-impaired individuals with neurological damage JINS 17 5 2011 929 933
37. M.D. Grilli, and E.L. Glisky Self-imagining enhances recognition memory in memory-impaired individuals with neurological damage Neuropsychology 24 6 2010 698 710
38. A. Pouget, S. Deneve, and J. Duhamel A computational perspective on the neural basis of multisensory spatial representations Nature Rev. Neurosci. 3 2002 741 747
39. S. Deneve, and A. Pouget Bayesian multisensory integration and cross-modal spatial links J. Physiol. Paris 98 1-3 2004 249 258
40. J. Martens, I. Sutskever, Learning recurrent neural networks with Hessian-free optimization, in: Proceedings of the 28th International Conference on Machine Learning, Bellevue, Washington, USA, 2011, pp. 1033-1040.