The Perception-for-Action-Control Theory (PACT): A perceptuo-motor theory of speech perception

Journal of Neurolinguistics

Volumn 25, Issue 5, 2012, Pages 336-354

  Schwartz, Jean Luc ^a   Basirat, Anahita ^a   Ménard, Lucie ^b   Sato, Marc ^a  

^a UNIV GRENOBLE ALPES   (France)

^b UNIVERSITÉ DU QUÉBEC À MONTRÉAL   (Canada)

Author keywords

Dorsal route; Multisensory interactions; Neurocomputational model; Perceptual organisation; Perceptuo motor interaction; Speech perception; Vowel perception

Indexed keywords

ADAPTATION; ARTICLE; AUDITORY STIMULATION; BEHAVIORAL SCIENCE; CONCEPTUAL FRAMEWORK; EVOKED MUSCLE RESPONSE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; GESTURE; GRAMMAR; HEARING; HUMAN; INFERIOR FRONTAL GYRUS; LANGUAGE; LINGUISTICS; MIRROR NEURON; MOTOR SYSTEM; MOVEMENT PERCEPTION; NERVE STIMULATION; NEUROPHYSIOLOGY; NONHUMAN; PERCEPTION FOR ACTION CONTROL THEORY; PHONETICS; PREMOTOR CORTEX; PRIMARY MOTOR CORTEX; PRIORITY JOURNAL; SOUND INTENSITY; SPEECH; SPEECH PERCEPTION; STIMULUS RESPONSE; VOCALIZATION; VOWEL;

EID: 84860916053     PISSN: 09116044     EISSN: 18738052     Source Type: Journal    
DOI: 10.1016/j.jneuroling.2009.12.004     Document Type: Article

References (121)

1. Abry C. [b]-[d]-[g] as a universal triangle as acoustically optimal as [i]-[a]-[u]. Proceedings of the 15th international congress of phonetic sciences 2003, 727-730. Barcelona.
2. Abry C., Badin P. Speech mapping as a framework for an integrated approach to the sensori-motor foundations of language. Proceedings of the fourth speech production seminar 1996, 175-184. Autrans.
3. Abry C., Boë L.J., Schwartz J.L. Plateaus, catastrophes and the structuring of vowel systems. Journal of Phonetics 1989, 17:47-54.
4. Arbib M.A. From monkey-like action recognition to human language: an evolutionary framework for neurolinguistics. Behavioral and Brain Sciences 2005, 28:105-167.
5. d'Ausilio A., Pulvermüller F., Salmas P., Bufalari I., Begliomini C., Fadiga L. The motor somatotopy of speech perception. Current Biology 2009, 19:1-5.
6. Basirat A., Sato M., Schwartz J.L., Kahane P., Lachaux J.P. Parieto-frontal oscillatory synchronization during the perceptual emergence and stabilization of speech forms. NeuroImage 2008, 42:404-413.
7. Bell-Berti F., Raphael L.J., Pisoni D.B., Sawusch J.R. Some relationships between speech production and perception. Phonetica 1979, 36:373-383.
8. Binder J.R., Liebenthal E., Possing E.T., Medler D.A., DouglasWard B. Neural correlates of sensory and decision processes in auditory object identification. Nature Neuroscience 2004, 7:295-301.
9. Blake R., Logothetis N.K. Visual competition. Nature Review Neuroscience 2002, 3:13-21.
10. Boë L.-J. Modeling the growth of the vocal tract vowel spaces of newly-born infants and adults. Consequences for ontogenesis and phylogenesis. Proceedings of the International Congress of phonetic Sciences 1999, Vol. 3:2501-2504. San Francisco.
11. Bregman A.S. Auditory scene analysis: The perceptual organization of sound 1990, MIT Press, Cambridge, MA.
12. Callan D.E., Jones J., Callan A., Akahane-Yamada R. Phonetic perceptual identification by native- and second-language speakers differentially activates brain regions involved with acoustic phonetic processing and those involved with articulatory-auditory/orosensory internal models. NeuroImage 2004, 22:1182-1194.
13. Chomsky N. Language and thought: some reflections on venerable themes. Powers and prospects: Reflections on human nature and the social order 1996, South End Press, Boston.
14. Cooper W.E., Billings D., Cole R.A. Articulatory effects on speech perception: a second report. Journal of Phonetics 1976, 4:219-232.
15. Cooper W.E., Blumstein S.E., Nigro G. Articulatory effects on speech perception: a preliminary report. Journal of Phonetics 1975, 3:87-98.
16. Cooper W.E., Lauritsen M.S. Feature processing in the perception and production of speech. Nature 1974, 252:121-123.
17. Delattre P., Freeman D. A dialect study of American R's by x-ray motion picture. Linguistics 1968, 44:29-68.
18. Diehl R.L., Lindblom B., Creeger C.P. Increasing realism of auditory representations yields further insights into vowel phonetics. Proceedings of the 15th international congress of phonetic sciences 2003, Vol. 2:1381-1384. Causal Publications, Adelaide.
19. Diehl R.L., Lotto A.J., Holt L.L. Speech perception. Annual Review of Psychology 2004, 74:431-461.
20. Eimas P.D., Corbit J.D. Selective adaptation of linguistic feature detectors. Cognitive Psychology 1973, 4:99-109.
21. Fadiga L., Craighero L., Buccino G., Rizzolatti G. Speech listening specifically modulates the excitability of tongue muscles: a TMS study. European Journal of Neuroscience 2002, 15:399-402.
22. Fadiga L., Fogassi L., Gallese V., Rizzolatti G. Visuomotor neurons: ambiguity of the discharge or 'motor' perception?. International journal of psychophysiology 2000, 35:165-177.
23. Fowler C. An event approach to the study of speech perception from a direct-realist perspective. Journal of Phonetics 1986, 14:3-28.
24. Fowler C.A., Rosenblum L.D. The perception of phonetic gestures. Modularity and the motor theory of speech perception 1991, 33-60. Erlbaum, Lawrence, NJ. I. Mattingly, K. Studdert-Kennedy (Eds.).
25. Galantucci B., Fowler C.A., Turvey M.T. The motor theory of speech perception reviewed. Psychonomic Bulletin & Review 2006, 13:361-377.
26. Gentilucci M., Corballis M.C. From manual gesture to speech: a gradual transition. Neuroscience & Biobehavioral Reviews 2006, 30:949-960.
27. Guenther F.H., Hampson M., Johnson D. A theoretical investigation of reference frames for the planning of speech movements. Psychological Review 1998, 105:611-633.
28. Hickok G., Poeppel D. Towards a functional neuroanatomy of speech perception. Trends in Cognitive Science 2000, 4:131-138.
29. Hickok G., Poeppel D. Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language. Cognition 2004, 92:67-99.
30. Hickok G., Poeppel D. The cortical organization of speech processing. Nature Reviews Neuroscience 2007, 8:393-402.
31. Hommel B., Müsseler J., Aschersleben G., Prinz W. The theory of event coding (TEC): a framework for perception and action planning. Behavioral and Brain Sciences 2001, 24:849-878.
32. Jones J., Callan D.E. Brain activity during audiovisual speech perception: an fMRI study of the McGurk effect. NeuroReport 2003, 14:1129-1133.
33. Kohler E., Keysers C., Umiltà M.A., Fogassi L., Gallese V., Rizzolatti G. Hearing sounds, understanding actions: action representation in mirror neurons. Science 2002, 297:846-848.
34. Kondo H.M., Kashino M. Neural mechanisms of auditory awareness underlying verbal transformations. NeuroImage 2007, 36:123-130.
35. Konishi M., Takahashi T., Wagner H., Sullivan W.E., Carr C.E. Neurophysiological and anatomical substrates of sound localization in the owl. Auditory function 1988, 721-745. Wiley, New York. G.M. Edelman, W.E. Gall, W.M. Cowan (Eds.).
36. Kuzla, C., Ernestus, M., & Mitterer, H. Compensation for assimilatory devoicing and prosodic structure in German fricative perception. Laboratory Phonology, 10, in press.
37. Leopold D.A., Logothetis N.K. Multistable phenomena: changing views in perception. Trends in Cognitive Sciences 1999, 3:254-264.
38. Liberman A.M. Some results of research on speech perception. Journal of the Acoustical Society of America 1957, 29:117-123.
39. Liberman A.M., Cooper F.S., Harris K.S., MacNeilage P.F. A motor theory of speech perception. Proceedings of the speech communication seminar 1962, Stockholm.
40. Liberman A.M., Cooper F.S., Shankweiler D.P., Studdert-Kennedy M. Perception of the speech code. Psychological Review 1967, 74:431-461.
41. Liberman A.M., Delattre P.C., Cooper F.S. The role of selected stimulus variables in the perception of the unvoiced stop consonants. American Journal of Psychology 1952, 65:497-516.
42. Liberman A.M., Delattre P.C., Cooper F.S., Gerstman L.J. The role of consonant-vowel transitions in the perception of the stop and nasal consonants. Psychological Monographs 1954, 68:1-13.
43. Liberman A.M., Mattingly I.G. The motor theory of speech perception revised. Cognition 1985, 21:1-36.
44. Liberman A.M., Whalen D.H. On the relation of speech to language. Trends in Cognitive Science 2000, 4:187-196.
45. Liljencrants J., Lindblom B. Numerical simulations of vowel quality systems: the role of perceptual contrast. Language 1972, 48:839-862.
46. Lindblom B. Can the models of evolutionary biology be applied to phonetic problems?. Proceedings of the 10th international congress of phonetic sciences 1984, 67-81. Utrecht.
47. Lindblom B. Phonetic universals in vowel systems. Experimental phonology 1986, 13-44. Academic Press, New York. J.J. Ohala, J.J. Jaeger (Eds.).
48. Lindblom B. On the notion of possible speech sound. Journal of Phonetics 1990, 18:135-152.
49. Lotto A., Hickok G., Holt L. Reflections on mirror neurons and speech perception. Trends in Cognitive Sciences 2009, 13:110-114.
50. MacNeilage P.F. The frame/content theory of evolution of speech production. Behavioral and Brain Sciences 1998, 21:499-546.
51. MacNeilage P.F., Davis B.L. On the origin of internal structure of word forms. Science 2000, 288:527-531.
52. Maddieson I. Patterns of sounds 1984, Cambridge Univ. Press, Cambridge, 1986. 2nd ed.
53. Maeda S. An articulatory model of the tongue based on a statistical analysis. Journal of Acoustical Society of America 1979, 65:S22.
54. Mattingly I.G. The global character of phonetic gestures. Journal of Phonetics 1990, 18:445-452.
55. Meister I.G., Wilson S.M., Deblieck C., Wu A.D., Iacoboni M. The essential role of premotor cortex in speech perception. Current Biology 2007, 17:1692-1696.
56. Ménard, L., & Schwartz, J. L. Normalization effects and perceptuo-motor biases in the perceptual organization of the height feature in French vowels, submitted for publication.
57. Ménard L., Schwartz J.L., Boë L.J. The role of vocal tract morphology in speech development: perceptual targets and sensori-motor maps for French synthesized vowels from birth to adulthood. Journal of Speech Language and Hearing Research 2004, 47:1059-1080.
58. Ménard L., Schwartz J.L., Boë L.J., Kandel S., Vallée N. Auditory normalization: of French vowels synthesized by an articulatory model simulating growth from birth to adulthood. Journal of Acoustical Society of America 2002, 111:1892-1905.
59. Mielke, J., Baker, A., & Archangeli, D. Covert /®/ allophony in English: variation in a socially uninhibited sound pattern. Laboratory Phonology, 10, in press.
60. Möttönen R., Järveläinen J., Sams M., Hari R. Viewing speech modulates activity in the left SI mouth cortex. NeuroImage 2004, 24:731-737.
61. Moulin-Frier, C., Schwartz, J. L., Diard, J., & Bessière, P. Emergence of phonology through deictic games within a society of sensori-motor agents in interaction. Book chapter in vocalization, communication, imitation and deixis, in press.
62. Obleser J., Boecker H., Drzezga A., Haslinger B., Hennenlotter A., Roettinger M., et al. Vowel sound extraction in anterior superior temporal cortex. Human Brain Mapping 2006, 27:562-571.
63. Obleser J., Elbert T., Lahiri A., Eulitz C. Cortical representation of vowels reflects acoustic dissimilarity determined by formant frequencies. Cognitive Brain Research 2003, 15:207-213.
64. Obleser J., Lahiri A., Eulitz C. Magnetic brain response mirrors extraction of phonological features from spoken vowels. Journal of Cognitive Neuroscience 2004, 16:31-39.
65. Ohala J.J. Moderator's introduction to symposium on phonetic universals in phonological systems and their explanation. Proceedings of the 9th International Congress of Phonetic Sciences 1979, Vol. 3:181-185.
66. Ojanen V., Möttönen R., Pekkola J., Jääskeläinen I.P., Joensuu R., Autti T., et al. Processing of audiovisual speech in Broca's area. NeuroImage 2005, 25:333-338.
67. Pekkola J., Laasonen M., Ojanen V., Autti T., Jaaskelainen L.P., Kujala T., et al. Perception of matching and conflicting audiovisual speech in dyslexic and fluent readers: an fMRI study at 3T. NeuroImage 2006, 29:797-807.
68. Perkell J.S., Klatt D.H. Invariance and variability in speech processes 1986, L. Erlbaum, Hillsdale N.J, New Jersey.
69. Perrier P. About speech motor control complexity. Speech production: Models, phonetic processes, and techniques 2006, 13-25. Psychology Press, New York. J. Harrington, M. Tabain (Eds.).
70. Pitt M., Shoaf L. The source of a lexical bias in the verbal transformation effect. Language and Cognitive Processes 2001, 16:715-721.
71. Pitt M., Shoaf L. Linking verbal transformations to their causes. Journal of Experimental Psychology: Human Perception and Performance 2002, 28:150-162.
72. Porter R.J., Castellanos F.X. Speech-production measures of speech perception: rapid shadowing of VCV syllables. Journal of Acoustical Society of America 1980, 67:1349-1356.
73. Porter R.J., Lubker J.F. Rapid reproduction of vowel-vowel sequences: evidence for a fast and direct acoustic-motoric linkage in speech. Journal of Speech and Hearing Research 1980, 23:593-602.
74. Pulvermuller F., Huss M., Kherif F., Moscoso del Prado Martin F., Hauk O., Shtyrov Y. Motor cortex maps articulatory features of speech sounds. Proceedings of the National Academy of Sciences of the United States of America 2006, Vol. 103:7865-7870.
75. Remez R.E., Rubin P.E., Berns S.M., Pardo J.S., Lang J.M. On the perceptual organization of speech. Psychological Review 1994, 101:129-156.
76. Rizzolatti G., Craighero L. The mirror-neuron system. Annual Review of Neuroscience 2004, 27:169-192.
77. Rizzolatti G., Fogassi L., Gallese V. Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews Neuroscience 2001, 2:661-670.
78. Rizzolatti G., Arbib M.A. Language within our grasp. Trends in Neurosciences 1998, 21:188-194.
79. Rochet-Capellan A., Schwartz J.L. An articulatory basis for the labial-to-coronal effect: /pata/ seems a more stable articulatory pattern than /tapa/. Journal of Acoustical Society of America 2007, 121:3740-3754.
80. Rosch-Heider E. Universals in color naming and memory. Journal of Experimental Psychology 1972, 93:10-20.
81. Roy A.C., Craighero L., Fabbri-Destro M., Fadiga L. Phonological and lexical motor facilitation during speech listening: a transcranial magnetic stimulation study. Journal of Physiology, Paris 2008, 102:101-105.
82. Sato M., Baciu M., Lœvenbruck H., Schwartz J.-L., Cathiard M.-A., Segebarth C., et al. Multistable perception of speech forms in working memory: an fMRI study of the verbal transformation effect. NeuroImage 2004, 23:1143-1151.
83. Sato M., Basirat A., Schwartz J.-L. Visual contribution to the multistable perception of speech. Perception and Psychophysics 2007, 69:1360-1372.
84. Sato M., Rousset I., Schwartz J.-L., Vallée N. A perceptual correlate of the labial-coronal effect. Journal of Speech, Language and Hearing Research 2007, 50:1466-1480.
85. Sato M., Schwartz J.L., Cathiard M.A., Abry C., Loevenbruck H. Multistable syllables as enacted percepts: a source of an asymmetric bias in the verbal transformation effect. Perception & Psychophysics. 2006, 68:458-474.
86. Sato M., Tremblay P., Gracco V. A mediating role of the premotor cortex in speech segmentation. Brain and Language 2009, 111(1):1-7.
87. Savariaux C., Perrier P., Orliaguet J.-P. Compensation strategies for the perturbation of the rounded vowel [u] using a lip-tube: a study of the control space in speech production. Journal of Acoustical Society of America 1995, 98:2428-2442.
88. Savariaux C., Perrier P., Orliaguet J.-P., Schwartz J.-L. Compensation strategies for the perturbation of French [u] using lip-tube. II. Perceptual analysis. Journal of Acoustical Society of America 1999, 106:381-393.
89. Schwartz, J. L. Filling the perceptuo-motor gap. Laboratory Phonology, 10, in press.
90. Schwartz J.L., Abry C., Boë L.J., Ménard L., Vallée N. Asymmetries in vowel perception, in the context of the dispersion-focalisation theory. Speech Communication 2005, 45:425-434.
91. Schwartz J.L., Berthommier F., Savariaux C. Seeing to hear better: evidence for early audio-visual interactions in speech identification. Cognition 2004, 93:B69-B78.
92. Schwartz J.L., Boë L.J. Grounding plosive place features in perceptuo-motor substance 2007, Workshop on "International Conference on Features" (N. Clements & R. Ridouane), Paris.
93. Schwartz J.L., Boë L.J., Abry C. Linking the dispersion-focalization theory (DFT) and the maximum utilization of the available distinctive features (MUAF) principle in a perception-for-action-control theory (PACT). Experimental approaches to phonology 2007, 104-124. Oxford University Press. M.J. Solé, P. Beddor, M. Ohala (Eds.).
94. Schwartz J.L., Boë L.J., Vallée N., Abry C. The dispersion-focalization theory of vowel systems. Journal of Phonetics 1997, 25:255-286.
95. Schwartz J.L., Escudier P. A strong evidence for the existence of a large scale integrated spectral representation in vowel perception. Speech Communication 1989, 8:235-259.
96. Schwartz J.L., Robert-Ribes J., Escudier P. Ten years after summerfield. A taxonomy of models for audiovisual fusion in speech perception. Hearing by eye, II. Perspectives and directions in research on audiovisual aspects of language processing 1998, 85-108. Psychology Press, Hove (UK). R. Campbell, B. Dodd, D. Burnham (Eds.).
97. Schwartz J.L., Vallée N., Kandel S. Hearing the tongue and lips of vowel gestures: a new differential paradigm. Journal of Acoustical Society of America 2008, 123:3179.
98. Serkhane J.E., Schwartz J.L., Boë L.J., Bessière P. Building a talking baby robot: a contribution to the study of speech acquisition and evolution. Interaction Studies 2005, 6:253-286.
99. Serkhane J.E., Schwartz J.L., Boë L.J., Davis B.L., Matyear C.L. Infants' vocalizations analyzed with an articulatory model: a preliminary report. Journal of Phonetics 2007, 35:321-340.
100. Scott S.K., Wise R.J. The functional neuroanatomy of prelexical processing in speech perception. Cognition 2004, 92:13-45.
101. Shoaf L., Pitt M. Does node stability underlie the verbal transformation effect? A test of node structure theory. Perception & Psychophysics 2002, 64:795-803.
102. Shiller D., Sato M., Gracco V., Baum S. Perceptual recalibration of speech sounds following motor learning. Journal of Acoustical Society of America 2009, 125:1103-1113.
103. Skipper J.I., Nusbaum H.C., Small S.L. Listening to talking faces: motor cortical activation during speech perception. NeuroImage 2005, 25:76-89.
104. Skipper J.I., Van Wassenhove V., Nusbaum H.C., Small S.L. Hearing lips and seeing voices: how cortical areas supporting speech production mediate audiovisual speech perception. Cerebral Cortex 2007, 17:2387-2399.
105. Solé, M. J., & Ohala, J. J. What is and what is not under the control of the speaker: intrinsic vowel duration. Laboratory Phonology, 10, in press.
106. Stevens K.N. The quantal nature of speech: evidence from articulatory-acoustic data. Human communication: A unified view 1972, 51-66. Mc Graw-Hill, New-York. E.E. Davis, P.B. Denes (Eds.).
107. Stevens K.N. On the quantal nature of speech. Journal of Phonetics 1989, 17:3-45.
108. Sundara M., Namasivayam A.K., Chen R. Observation-execution matching system for speech: a magnetic stimulation study. Neuroreport 2001, 12:1341-1344.
109. Sussman H.M., Fruchter D., Hilbert J., Sirosh J. Linear correlates in the speech signal: the orderly output constraint. Behavioral and Brain Sciences 1998, 21:241-259.
110. Tiede M.K., Boyce S.E., Holland C.K., Choe K.A. A new taxonomy of American English /®/ using MRI and ultrasound. Journal of Acoustical Society of America 2004, 115:2633-2634. (A).
111. Vallée N., Schwartz J.L., Escudier P. Phase spaces of vowel systems: a typology in the light of the dispersion-focalisation theory (DFT). Proceedings of the XIVth International Congress of Phonetic Sciences 1999, Vol. 1:333-336.
112. Villacorta V.M., Perkell J.S., Guenther F.H. Sensorimotor adaptation to feedback perturbations on vowel acoustics and its relation to perception. Journal of Acoustical Society of America 2007, 122:2306-2319.
113. Viviani P., Stucchi N. Biological movements look uniform: evidence of motor-perceptual interactions. Journal of Experimental Psychology: Human Perception and Performance 1992, 18:603-623.
114. Wagner H., Takahashi T., Konishi M. Representation of interaural t time difference in the central nucleus of the barn owl's inferior colliculus. Journal of Neuroscience 1987, 7:3105-3116.
115. Warren M.R. Illusory changes of distinct speech upon repetition - the verbal transformation effect. British Journal of Psychology 1961, 52:249-258.
116. Warren M.R., Gregory R.L. An auditory analogue of the visual reversible figure. American Journal of Psychology 1958, 71:612-613.
117. Watkins K.E., Paus T. Modulation of motor excitability during speech perception: the role of Broca's area. Journal of Cognitive Neuroscience 2004, 16:978-987.
118. Watkins K.E., Strafella A.P., Paus T. Seeing and hearing speech excites the motor system involved in speech production. Neuropsychologia 2003, 41:989-994.
119. Wilson S.M., Iacoboni M. Neural responses to non-native phonemes varying in producibility: evidence for the sensorimotor nature of speech perception. NeuroImage 2006, 33:316-325.
120. Wilson S.M., Saygin A.P., Sereno M.I., Iacoboni M. Listening to speech activates motor areas involved in speech production. Nature Neuroscience 2004, 7:701-702.
121. Zekveld A.A., Heslenfeld D.J., Festen J.M., Schoonhoven R. Top-down and bottom-up processes in speech comprehension. NeuroImage 2006, 32:1826-1836.