Multimodal communication in animals, humans and robots: An introduction to perspectives in brain-inspired informatics

Neural Networks

Volumn 22, Issue 2, 2009, Pages 111-115

  Wermter, S ^a   Page, M ^a   Knowles, M ^a   Gallese, V ^b   Pulvermuller F ^c   Taylor, J ^d  

^a UNIVERSITY OF SUNDERLAND   (United Kingdom)

^b UNIVERSITY OF PARMA   (Italy)

^c MRC COGNITION AND BRAIN SCIENCES UNIT   (United Kingdom)

^d KING'S COLLEGE LONDON   (United Kingdom)

Author keywords

Brain inspired computing; Multimodal communication; Neural networks; Robotics

Indexed keywords

ANIMAL COMMUNICATION; BRAIN MAPPING; COGNITION; EDITORIAL; HUMAN; INFORMATION SCIENCE; INTERPERSONAL COMMUNICATION; LANGUAGE DEVELOPMENT; LANGUAGE PROCESSING; LEARNING THEORY; NERVE CONDUCTION; NONVERBAL COMMUNICATION; PRIORITY JOURNAL; ROBOTICS; SPATIAL MEMORY; SUPERIOR COLLICULUS; VERBAL COMMUNICATION; VISUAL INFORMATION;

ANIMAL COMMUNICATION; ANIMALS; BRAIN; COMMUNICATION; HUMANS; INFORMATICS; LANGUAGE; NEURAL NETWORKS (COMPUTER); ROBOTICS; VISION, OCULAR;

EID: 61749087943     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2009.01.004     Document Type: Editorial

References (59)

1. Belin P., Zatorre R.J., Lafaille P., Ahad P., and Pike B. Voice-selective areas in human auditory cortex. Nature 403 6767 (2000) 309-312
2. Buccino G., Riggio T.L., Melli G., Binkofski F., Gallese V., and Rizzolatti G. Listening to action-related sentences modulates the activity of the motor system: A combined TMS and behavioral study. Cognitive Brain Research 24 3 (2005) 355-363
3. Cohen L., Lehericy S., Chochon F., Rivaud S., and Dehaene S. The visual word form area: Spatial and temporal characterisation of an initial stage of reading in normal subjects and posterior split-brain patients. Brain 123 (2002) 291-307
4. CHLaSC. Online Available at: http://www.zas.gwz-berlin.de/chlasc/index.html Accessed 20.10.08
5. Dehaene S., Jobert A., Naccache L., Ciuciu P., Poline JB., Le Bihan D., et al. Letter binding and invariant recognition of masked words: Behavioural and neuroimaging evidence. Psychological Science 15 5 (2004) 307-313
6. Edici Project. Online Available at: http://www.univie.ac.at/edici/ Accessed 20.10.08
7. Elshaw, M., Weber, C., Zochios, A., & Wermter, S. (2004). An associator network approach to robot learning by imitation through vision, motor control and language. In Proceedings of the international joint conference on neural networks, (pp. 591-596)
8. Far Project. Online Available at: http://www.cbcd.bbk.ac.uk/research/projects/far/FAR Accessed 20.10.08
9. Fay R., Kaufmann U., Knoblauch A., Markert H., and Palm G. Combining visual attention, object recognition and associative information processing in a neurobiotic system. In: Wermter S., Palm G., and Elshaw M. (Eds). Biomimetic neural learning for intelligent robots (2005), Springer Verlag, Heidelberg, Germany 118-143
10. Friederici A.D. Towards a neural basis of auditory sentence processing. Trends in Cognitive Sciences 6 2 (2002) 78-84
11. Friederici A.D., Friedrich M., and Christophe A. Brain responses in 4-month-old infants are already language specific. Current Biology 17 14 (2007) 1208-1211
12. Gallese V., Keysers C., and Rizzollati G. A unifying view of the basis of social cognition. Trends in Cognitive Science 8 9 (2004) 396-403
13. Gallese V., and Lakoff G. The brain's concepts: The role of the sensory-motor system in reason and language. Cognitive Neuropsychology 22 (2005) 455-479
14. Gallese V. The intentional attunement hypothesis-the mirror neuron system and its role in interpersonal relations. In: Wermter S., Palm G., and Elshaw M. (Eds). Biomimetic neural learning for intelligent robots (2005), Springer, Berlin 19-30
15. Garagnani M., Wennekers T., and Pulvermüller F. A neuroanatomically-grounded Hebbian learning model of attention-language interactions in the human brain. European Journal of Neuroscience 27 2 (2008) 492-513
16. Gervain J., Macagno F., Cogoi S., Peña M., and Mehler J. The neonate brain detects speech structure. Proceedings of the National Academy of Sciences 105 37 (2008) 14222-14227
17. Gillmeister, H., Catmur, C., Bird, G., Liepelt, R., Brass, M., & Heyes, C. (2008). Body part priming in action imitation and the mirror neuron system depends on correlated sensorimotor experience. In Poster presented at the Evolution, Development and Intentional Control of Imitation Workshop
18. Gillmeister H., Catmur C., Liepelt R., Brass M., and Heyes C.M. Experience-based priming of body parts: A study of action imitation. Brain Research 1217C (2008) 157-170
19. Gómez J.C. Pointing behaviors in apes and human infants: A balanced interpretation. Child Development 78 3 (2007) 729-734
20. Hauk O., Johnsrude I., and Pulvermüller F. Somatotopic representation of action words in the motor and premotor cortex. Neuron 41 (2004) 301-307
21. Hauk O., Shtyrov Y., and Pulvermüller F. The sounds of actions as reflected by mismatch negativity: Rapid activation of sensory-motor networks by sounds associated with finger and tongue movements. European Journal of Neuroscience 23 (2006) 811-821
22. Iba S., Paredis C.J.J., and Khosla P.K. Interactive multi-modal robot programming. In: Ang M.H., and Khatib O. (Eds). : Experimental Robotics IX. STAR 21 (2006), Springer, Berlin, Heidelberg 503-512
23. Knoblauch A., and Palm G. Scene segmentation by spike synchronization in reciprocally connected visual areas. II. Global assemblies and synchronization on larger space and time scales. Biological Cybernetics (Berlin) 87 3 (2002) 168-184
24. Kohler E., Keysers C., Umiltà M.A., Fogassi L., Gallese V., and Rizzolatti G. Hearing sounds, understanding actions: Action representation in mirror neurons. Science 297 5582 (2002) 846-848
25. Kriegstein K.V., and Giraud A.L. Distinct functional substrates along the right superior temporal sulcus for the processing of voices. Neuroimage 22 2 (2004) 948-955
26. Markert, H., Kaufmann, U., Kara Kayikci, Z., & Palm, G. (2009). Neural associative memories for the integration of language, vision and action in an autonomous agent. Neural Networks, in this issue (doi:10.1016/j.neunet.2009.01.011)
27. Mirrorbot Project. Online Available at: http://www.his.sunderland.ac.uk/mirrorbot/ Accessed 20.10.08
28. Moscoso Del Prado Martin F., Hauk O., and Pulvermüller F. Category specificity in the processing of color-related and form-related words: An ERP study. Neuroimage 29 1 (2006) 29-37
29. Murray, J. C, Erwin, H. R, & Wermter, S. (2008). A hybrid architecture using cross-correlation and recurrent neural networks for robotic sound-source localisation, Elsevier (in this issue)
30. Noordzij, M. L. (2004). Communicating spatial information from verbal descriptions. In Thesis submitted to Helmholtz Institute. Utrecht University. http://wayfinding.fss.uu.nl/
31. Opitz B., and Friederici A.D. Brain correlates of language learning: The neuronal dissociation of rule-based versus similarity-based learning. Journal of Neuroscience (New York, NY) 24 39 (2004) 8436-8440
32. Preissl H., Pulvermüller F., Lutzenberger W., and Birbaumer N. Evoked potentials distinguish nouns from verbs. Neuroscience Letters 197 (1995) 81-83
33. Price C.J., and Devlin J.T. The myth of the visual word form area. Neuroimage 19 3 (2003) 473-481
34. Press, C., Cook, R., Dickinson, A., & Heyes, C. M. (2008). Does learning to imitate depend on contingency. In International Workshop on the Evolution, Development and Intentional Control of Imitation
35. Pulvermüller F. Words in the brain's language. Behavioral and Brain Sciences 22 (1999) 253-336
36. Pulvermüller F. Brain reflections of words and their meaning. Trends in Cognitive Sciences 5 12 (2001) 517-524
37. Pulvermüller F. Brain mechanisms linking language and action. Nature Reviews Neuroscience 6 7 (2005) 576-582
38. Pulvermüller F., and Hauk O. Category-specific processing of color and form words in left fronto-temporal cortex. Cerebral Cortex 16 8 (2006) 1193-1201
39. Pulvermüller, F., & Knoblauch, A. (2009). Discrete rule formation in brain-inspired neural networks. Neural Networks (in this issue)
40. Pulvermüller F., Lutzenberger W., and Preissl H. Nouns and verbs in the intact brain: Evidence from event-related potentials and high-frequency cortical responses. Cerebral Cortex 9 (1999) 498-508
41. Pulvermüller F., Preissl H., Lutzenberger W., and Birbaumer N. Brain rhythms of language: Nouns versus verbs. European Journal of Neuroscience 8 (1996) 937-941
42. Refcom Project. Online Available at: http://psy.st-andrews.ac.uk/research/refcom/ Accessed 20.10.08
43. Revel, A., & Andry, (2008). Emergence of structured interactions: From a theoretical model to pragmatic robotics. Neural Networks, in this issue (doi:10.1016/j.neunet.2009.01.005)
44. Rizzolatti G., and Craighero L. The mirror-neuron system. Annual Review of Neuroscience 27 (2004) 169-192
45. Rybski, P. E., Yoon, K., Stolarz, J., & Veloso, M. M. (2007). Interactive robot task training through dialog and demonstration. In HRI'07
46. Salvesen C. The landscape of memory (1965), Arnold, London
47. Shtyrov Y., Hauk O., and Pulvermüller F. Distributed neuronal networks for encoding category-specific semantic information: The mismatch negativity to action words. European Journal of Neuroscience 19 4 (2004) 1083-1092
48. Spiliotopoulos, D., Andreoutsopoulos, I., & Spyropoulos, C. D. (2001). Human-robot interaction based on spoken natural language dialogue. In Proc. Euro. Workshop on Service and Humanoid Robotics
49. Stein B.E., and Meredith M.A. The merging of the senses (1993), MIT Press
50. Sullivan W.E., and Konishi M. Neural map of interaural phase difference in the owl's brainstem. Proceedings of the National Academy of Sciences of the United States of America 83 21 (1986) 8400-8404
51. Voelkl, B., & Huber, L. (2007). Imitation as faithful copying of a novel technique in marmoset monkeys. PLoS ONE 2(7) Available at http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1905941 A ccessed 14.01.08
52. Wayfinding Project. Online Available at: http://wayfinding.fss.uu.nl/ Accessed 20.10.08
53. Wennekers T., Garagnani M., and Pulvermüller F. Language models based on Hebbian cell assemblies. Journal de Physiologie (Paris) 100 (2006) 16-30
54. Wermter S., and Elshaw M. Learning robot actions based on self-organising language memory. Neural Networks 16 5 (2003) 661-669
55. Wermter S., Weber C., Elshaw M., Panchev C., Erwin H., and Pulvermüller F. Towards multimodal neural robot learning. Robotics and Autonomous Systems Journal 47 2-3 (2004) 171-175
56. In: Wermter S., Palm G., and Elshaw M. (Eds). Biomimetic neural learning for intelligent robots (2005), Springer, Heidelberg, Germany
57. Wermter S., Weber C., Elshaw M., Gallese V., and Pulvermüller F. Grounding neural robot language in action. Biomimetic Neural Learning for Intelligent Robots: Intelligent Systems, Cognitive Robotics, and Neuroscience 3575 (2005) 162-181
58. Wermter S., Weber C., Elshaw M., Gallese V., and Pulvermüller F. Neural grounding of robot language in action. In: Wermter S., Palm G., and Elshaw M. (Eds). Biomimetic neural learning for intelligent robots (2005), Springer, Berlin 162-181
59. Van Asselen, M. (2005). The neurocognitive basis of spatial memory. In Thesis submitted to Helmholz Institute. Utrecht University. http://wayfinding.fss.uu.nl/ Accessed 5.01.09