Describing visual scenes: Towards a neurolinguistics based on construction grammar

Brain Research

Volumn 1225, Issue , 2008, Pages 146-162

  Arbib, Michael A ^a,b   Lee, JinYong ^a  

^a University of Southern California ^*  (United States)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Action; Action recognition; Brain mechanism; Competition and cooperation; Construction grammar; Dynamic visual scene; Language perception; Language production; Mirror neuron; Scene description; Schema theory; SemRep; Vision; Visual perception

Indexed keywords

ARTICLE; BRAIN FUNCTION; CENTRAL NERVOUS SYSTEM; COMPREHENSION; COMPUTER ANALYSIS; COMPUTER INTERFACE; DATA BASE; GRAMMAR; HUMAN; LANGUAGE; LEARNING; NERVE CELL; PRIORITY JOURNAL; RECOGNITION; SEMANTICS; TEMPLATE CONSTRUCTION GRAMMAR; VISION; VISUAL SYSTEM; WORKING MEMORY;

BRAIN; COGNITION; COMPUTER SIMULATION; HUMANS; LANGUAGE; LINGUISTICS; NEURAL NETWORKS (COMPUTER); NEUROPSYCHOLOGY; SEMANTICS; SPEECH; SPEECH PERCEPTION; VERBAL BEHAVIOR; VISUAL PERCEPTION;

EID: 48349112978     PISSN: 00068993     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.brainres.2008.04.075     Document Type: Article

References (65)

1. Aboitiz F., and Garcia V.R. The evolutionary origin of the language areas in the human brain. A neuroanatomical perspective. Brain Res. Brain Res. Rev. 25 (1997) 381-396
2. Aboitiz F., García R., Brunetti E., and Bosman C. The origin of Broca's area and its connections from an ancestral working memory network. In: Grodzinsky Y., and Amunts K. (Eds). Broca's region (2006), Oxford University Press, Oxford 3-16 Vol.
3. Aboitiz F., Garcia R.R., Bosman C., and Brunetti E. Cortical memory mechanisms and language origins. Brain Lang. 98 (2006) 40-56
4. Arbib M.A. Perceptual structures and distributed motor control. In: Brooks V.B. (Ed). Handbook of Physiology - The Nervous System II. Motor Control (1981), American Physiological Society, Bethesda, MD 1449-1480 Vol.
5. Arbib M.A. Rana computatrix to human language: towards a computational neuroethology of language evolution. Philos. Transact. A Math. Phys. Eng. Sci. 361 (2003) 2345-2379
6. Arbib M.A. From monkey-like action recognition to human language: an evolutionary framework for neurolinguistics (with commentaries and author's response). Behav. brain sci. 28 (2005) 105-167
7. Arbib M.A. Aphasia, apraxia and the evolution of the language-ready brain. Aphasiology 20 (2006) 1-30
8. Arbib M.A., and Bota M. Language evolution: neural homologies and neuroinformatics. Neural Netw 16 (2003) 1237-1260
9. Arbib M.A., and Bota M. Neural homologies and the grounding of neurolinguistics. In: Arbib M.A. (Ed). Action to language via the mirror neuron system (2006), Cambridge University Press, Cambridge 136-173 Vol.
10. Arbib M.A., and Caplan D. Neurolinguistics must be computational. Behav. brain sci. 2 (1979) 449-483
11. Arbib M.A., and Didday R.L. The organization of action-oriented memory for a perceiving system. I. The basic model. J. cybern. 1 (1971) 3-18
12. Arbib M.A., and Lee J. Vision and action in the language-ready brain: from mirror neurons to SemRep. In: Mele F. (Ed). BVAI 2007 (Brain Vision & Artificial Intelligence, 2007), LNCS 4729 (2007), Springer-Verlag, Berlin 104-123 Vol.
13. Arbib M.A., and Liaw J.-S. Sensorimotor transformations in the worlds of frogs and robots. Artif. Intell. 72 (1995) 53-79
14. Arbib M.A., Conklin E.J., and Hill J.C. From Schema Theory to Language (1987), Oxford University Press, New York Vol.
15. Arbib M.A., Érdi P., and Szentágothai J. Neural Organization: Structure, Function, and Dynamics (1998), The MIT Press, Cambridge, MA Vol.
16. Barsalou L.W. Perceptual symbol systems. Behav. Brain Sci. 22 (1999) 577-609 discussion 610-60
17. Barsalou L.W., Kyle Simmons W., Barbey A.K., and Wilson C.D. Grounding conceptual knowledge in modality-specific systems. Trends Cogn. Sci. 7 (2003) 84-91
18. Bartlett F.C. Remembering (1932), Cambridge University Press, Cambridge Vol.
19. Bergen B., and Chang N. Embodied construction grammar in simulation-based language understanding. In: Östman J.-O., and Fried M. (Eds). Construction grammar(s): Cognitive and cross-language dimensions (2005), John Benjamins, Amsterdam 147-190 Vol.
20. Bonaiuto J., Rosta E., and Arbib M. Extending the mirror neuron system model, I : audible actions and invisible grasps. Biol. Cybern. 96 (2007) 9-38
21. Broca P.P. Perte de la parole. Bull. Soc. anthropol. Paris 2 (1861) 235-238
22. Buffalo E.A., Reber P.J., and Squire L.R. The human perirhinal cortex and recognition memory. Hippocampus 8 (1998) 330-339
23. Chao L.L., and Martin A. Representation of manipulable man-made objects in the dorsal stream. Neuroimage 12 (2000) 478-484
24. Corina D.P., McBurney S.L., Dodrill C., Hinshaw K., Brinkley J., and Ojemann G. Functional roles of Broca's area and SMG: evidence from cortical stimulation mapping in a deaf signer. Neuroimage 10 (1999) 570-581
25. Coslett H.B., and Saffran E. Simultanagnosia. To see but not two see. Brain 114 Pt 4 (1991) 1523-1545
26. Croft W., and Cruse D.A. Cognitive Linguistics (2005), Cambridge University Press, Cambridge Vol.
27. Culicover P., and Jackendoff R. Simpler Syntax (2005), Oxford University Press, Oxford Vol.
28. Damasio H., Grabowski T.J., Tranel D., Hichwa R.D., and Damasio A.R. A neural basis for lexical retrieval. Nature 380 (1996) 499-505
29. De Beule J., and Steels L. Hierarchy in fluid construction grammar. In: Furbach U. (Ed). Proceedings of the 28th Annual German Conference on AI, KI 2005, Lecture Notes in Artificial Intelligence vol. 3698 (2005), Springer-Verlag, Berlin 1-15 Vol.
30. Dominey P.F., and Hoen M. Structure mapping and semantic integration in a construction-based neurolinguistic model of sentence processing. Cortex 42 (2006) 476-479
31. Draper B.A., Collins R.T., Brolio J., Hanson A.R., and Riseman E.M. The schema system. Int. j. comput. vis. 2 (1989) 209-250
32. Erman L.D., Hayes-Roth F., Lesser V.R., and Reddy D.R. The HEARSAY-II speech understanding system: integrating knowledge to resolve uncertainty. Comput. Surv. 12 (1980) 213-253
33. Fagg A.H., and Arbib M.A. Modeling parietal-premotor interactions in primate control of grasping. Neural Netw. 11 (1998) 1277-1303
34. Fillmore C.J., Kay P., and O'Connor M.K. Regularity and idiomaticity in grammatical constructions: the case of let alone. Lang. Cogn. Processes 64 (1988) 501-538
35. Gallese V., and Goldman A. Mirror neurons and the simulation theory of mind-reading. Trends Cogn. Sci. 2 (1998) 493-501
36. Goodale M.A., and Milner A.D. Separate visual pathways for perception and action. Trends Neurosci. 15 (1992) 20-25
37. Grice H.P. Utterer's meaning and intention. Philos. Rev 78 (1969) 147-177
38. Head H., and Holmes G. Sensory disturbances from cerebral lesions. Brain 34 (1911) 102-254
39. Hickok G., and Poeppel D. Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language. Cognition 92 (2004) 67-99
40. Hickok G., and Poeppel D. The cortical organization of speech processing. Nat. Rev. Neurosci. 8 (2007) 393-402
41. Horwitz B., Amunts K., Bhattacharyya R., Patkin D., Jeffries K., Zilles K., and Braun A.R. Activation of Broca's area during the production of spoken and signed language: a combined cytoarchitectonic mapping and PET analysis. Neuropsychologia 41 (2003) 1868-1876
42. Itti L., and Arbib M.A. Attention and the minimal subscene. In: Arbib M.A. (Ed). Action to language via the mirror neuron system (2006), Cambridge University Press, Cambridge 289-346 Vol.
43. Kaan E., and Swaab T.Y. The brain circuitry of syntactic comprehension. Trends Cogn. Sci. 6 (2002) 350-356
44. Kemmerer D. Action verbs, argument structure constructions, and the mirror neuron system. In: Arbib M.A. (Ed). Action to language via the mirror neuron system (2006), Cambridge University Press, Cambridge 347-373 Vol.
45. Kuperberg G.R., McGuire P.K., Bullmore E.T., Brammer M.J., Rabe-Hesketh S., Wright I.C., Lythgoe D.J., Williams S.C., and David A.S. Common and distinct neural substrates for pragmatic, semantic, and syntactic processing of spoken sentences: an fMRI study. J. Cogn. Neurosci. 12 (2000) 321-341
46. Lyons D.M., and Arbib M.A. A formal model of computation for sensory-based robotics. IEEE trans. robot. autom. 5 (1989) 280-293
47. Marr D. In: Freeman W.H. (Ed). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (1982) Vol., New York
48. Martin A., Wiggs C.L., Ungerleider L.G., and Haxby J.V. Neural correlates of category-specific knowledge. Nature 379 (1996) 649-652
49. Michel F., and Henaff M.A. Seeing without the occipito-parietal cortex: simultagnosia as a shrinkage of the attentional visual field. Behav. Neurol. 15 (2004) 3-13
50. Murray E.A., and Richmond B.J. Role of perirhinal cortex in object perception, memory, and associations. Curr. Opin. Neurobiol. 11 (2001) 188-193
51. Narayanan S.S. Knowledge-based Action Representations for Metaphor and Aspect (KARMA). Engineering: Computer Science (1997), University of California, Berkeley Vol.
52. Oztop E., and Arbib M.A. Schema design and implementation of the grasp-related mirror neuron system. Biol. Cybern. 87 (2002) 116-140
53. Piaget J. Biology and knowledge: An essay on the relations between organic regulations and cognitive processes [Translation of (1967) Biologie et connaissance: Essai sur les relations entre les régulations organiques et les processus cognitifs. Paris: Gallimard.] Vol. (1971), Edinburgh University Press, Edinburgh
54. Piñango M.M. Understanding the architecture of language: the possible role of neurology. Trends Cogn. Sci. 10 (2006) 49-51
55. Pulvermüller F. Brain reflections of words and their meaning. Trends Cogn. Sci. 5 (2001) 517-524
56. Pulvermüller F., Hauk O., Nikulin V.V., and Ilmoniemi R.J. Functional links between motor and language systems. Eur. J. Neurosci. 21 (2005) 793-797
57. Rainer G., Asaad W.F., and Miller E.K. Memory fields of neurons in the primate prefrontal cortex. Proc. Natl. Acad. Sci. U. S. A. 95 (1998) 15008-15013
58. Rizzolatti G., and Arbib M.A. Language within our grasp. Trends Neurosci. 21 (1998) 188-194
59. Schmidt R.A. A schema theory of discrete motor skill learning. Psychol. Rev. 82 (1975) 225-260
60. Smith E.E., Jonides J., Marshuetz C., and Koeppe R.A. Components of verbal working memory: evidence from neuroimaging. Proc. Natl. Acad. Sci. U. S. A. 95 (1998) 876-882
61. Stromswold K., Caplan D., Alpert N., and Rauch S. Localization of syntactic comprehension by positron emission tomography. Brain Lang. 52 (1996) 452-473
62. Tranel D., Kemmerer D., Adolphs R., Damasio H., and Damasio A.R. Neural correlates of conceptual knowledge for actions. Cogn. Neuropsychol. 20 (2003) 409-432
63. Ullman M.T. Contributions of memory circuits to language: the declarative/procedural model. Cognition 92 (2004) 231-270
64. Wernicke C. Der aphasische symptomencomplex (1874), Cohn and Weigert, Breslau Vol.
65. Wilson F.A., Scalaidhe S.P., and Goldman-Rakic P.S. Dissociation of object and spatial processing domains in primate prefrontal cortex. Science 260 (1993) 1955-1958