Dynamic, rhythmic facial expressions and the superior temporal sulcus of macaque monkeys: Implications for the evolution of audiovisual speech

European Journal of Neuroscience

Volumn 31, Issue 10, 2010, Pages 1807-1817

  Ghazanfar, Asif A ^a   Chandrasekaran, Chandramouli ^a   Morrill, Ryan J ^a  

^a PRINCETON UNIVERSITY   (United States)

Author keywords

Dynamic faces; Face processing; Facial expressions; Jaw movement; Monkey vocalizations; Multisensory integration

Indexed keywords

ANIMAL EXPERIMENT; ARTICLE; AUDITORY CORTEX; BRUXISM; DYNAMICS; ELECTRIC POTENTIAL; EVENT RELATED POTENTIAL; EVOKED VISUAL RESPONSE; FACE; FACIAL EXPRESSION; JAW MOVEMENT; MACACA; MALE; NERVE CELL; NEUROPHYSIOLOGY; NONHUMAN; PRIORITY JOURNAL; SOCIAL INTERACTION; SPEECH ANALYSIS; SUBICULUM; VOCALIZATION; ANIMAL; AUDITORY STIMULATION; ELECTROENCEPHALOGRAPHY; EVOKED RESPONSE; EVOLUTION; GESTURE; MOUTH; MOVEMENT (PHYSIOLOGY); NUCLEAR MAGNETIC RESONANCE IMAGING; PHOTOSTIMULATION; PHYSIOLOGY; RHESUS MONKEY; SPEECH; STATISTICAL ANALYSIS; TEMPORAL LOBE;

ACOUSTIC STIMULATION; ANIMALS; BIOLOGICAL EVOLUTION; DATA INTERPRETATION, STATISTICAL; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS; FACIAL EXPRESSION; GESTURES; MACACA MULATTA; MAGNETIC RESONANCE IMAGING; MALE; MOUTH; MOVEMENT; NEURONS; PHOTIC STIMULATION; SPEECH; TEMPORAL LOBE;

EID: 77952667640     PISSN: 0953816X     EISSN: 14609568     Source Type: Journal    
DOI: 10.1111/j.1460-9568.2010.07209.x     Document Type: Article

References (64)

1. Barraclough, N.E., Xiao, D., Baker, C.I., Oram, M.W. Perrett, D.I. (2005) Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J. Cogn. Neurosci., 17, 377 391.
2. Bruce, C., Desimone, R. Gross, C.G. (1981) Visual properties of neurons in a polysensory area in superior temporal sulcus of the Macaque. J. Neurophysiol., 46, 369 384.
3. Burrows, A.M., Waller, B.M. Parr, L.A. (2009) Facial musculature in the rhesus macaque (Macaca mulatta): evolutionary and functional contexts with comparisons to chimpanzees and humans. J. Anat., 215, 320 334.
4. Callan, D.E., Jones, J.A., Munhall, K., Callan, A.M., Kroos, C. Vatikiotis-Bateson, E. (2003) Neural processes underlying perceptual enhancement by visual speech gestures. Neuroreport, 14, 2213 2218.
5. Calvert, G.A. (2001) Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb. Cortex, 11, 1110 1123.
6. Calvert, G.A. Campbell, R. (2003) Reading speech from still and moving faces: the neural substrates of visible speech. J. Cogn. Neurosci., 15, 57 70.
7. Calvert, G.A., Campbell, R. Brammer, M.J. (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol., 10, 649 657.
8. Campbell, R. (2008) The processing of audio-visual speech: empirical and neural bases. Philos. Trans. R. Soc. Lond. B Biol. Sci., 363, 1001 1010.
9. Chandrasekaran, C. Ghazanfar, A.A. (2009) Different neural frequency bands integrate faces and voices differently in the superior temporal sulcus. J. Neurophysiol., 101, 773 788.
10. Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A. Ghazanfar, A.A. (2009) The natural statistics of audiovisual speech. PLoS Comput. Biol., 5, e1000436.
11. Dahl, C.D., Logothetis, N.K. Kayser, C. (2009) Spatial organization of multisensory responses in temporal association cortex. J. Neurosci., 29, 11924 11932.
12. De Souza, W.C., Eifuku, S., Tamura, R., Nishijo, H. Ono, T. (2005) Differential characteristics of face neuron responses within the anterior superior temporal sulcus of macaques. J. Neurophysiol., 94, 1252 1266.
13. Drullman, R., Festen, J.M. Plomp, R. (1994) Effect of reducing slow temporal modulations on speech reception. J. Acoust. Soc. Am., 95, 2670 2680. (Pubitemid 24152861)
14. Eifuku, S., De Souza, W.C., Tamura, R., Nishijo, H. Ono, T. (2004) Neuronal correlates of face identification in the monkey anterior temporal cortical areas. J. Neurophysiol., 91, 358 371.
15. Freiwald, W.A., Tsao, D.Y. Livingstone, M.S. (2009) A face feature space in the macaque temporal lobe. Nat. Neurosci., 12, 1187 1196.
16. Ghazanfar, A.A., Maier, J.X., Hoffman, K.L. Logothetis, N.K. (2005) Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J. Neurosci., 25, 5004 5012.
17. Ghazanfar, A.A., Chandrasekaran, C. Logothetis, N.K. (2008) Interactions between the Superior Temporal Sulcus and Auditory Cortex Mediate Dynamic Face/Voice Integration in Rhesus Monkeys. J. Neurosci., 28, 4457 4469.
18. Giraud, A.L., Kleinschmidt, A., Poeppel, D., Lund, T.E., Frackowiak, R.S.J. Laufs, H. (2007) Endogenous cortical rhythms determine cerebral specialization for speech perception and production. Neuron, 56, 1127 1134.
19. Greenberg, S., Carvey, H., Hitchcock, L. Chang, S. (2003) Temporal properties of spontaneous speech - a syllable-centric perspective. J. Phon., 31, 465 485.
20. Hackett, T.A., Stepniewska, I. Kaas, J.H. (1998) Subdivisions of auditory cortex and ipsilateral cortical connections of the parabelt auditory cortex in macaque monkeys. J. Comp. Neurol., 394, 475 495.
21. Harries, M.H. Perrett, D.I. (1991) Visual processing of faces in temporal cortex - physiological evidence for a modular organization and possible anatomical correlates. J. Cogn. Neurosci., 3, 9 24.
22. Hasselmo, M.E., Rolls, E.T. Baylis, G.C. (1989) The role of expression and identity in the face-selective responses of neurons in the temporal visual-cortex of the monkey. Behav. Brain Res., 32, 203 218.
23. Hinde, R.A. Rowell, T.E. (1962) Communication by posture and facial expressions in the rhesus monkey (Macaca mulatta). Proc. Zool. Soc. Lond., 138, 1 21.
24. Kayser, C. Logothetis, N.K. (2009) Directed interactions between auditory and superior temporal cortices and their role in sensory integration. Front. Integr. Neurosci., 3, 7.
25. Kim, J. Davis, C. (2004) Investigating the audio-visual speech detection advantage. Speech. Commun., 44, 19 30.
26. Lee, D. (2002) Analysis of phase-locked oscillations in multi-channel single-unit spike activity with wavelet cross-spectrum. J. Neurosci. Methods, 115, 67 75.
27. Lieberman, P. Blumstein, S.E. (1988) Speech Physiology, Speech Perception, and Acoustic Phonetics. Cambridge University Press
28. Logothetis, N.K., Merkle, H., Augath, M., Trinath, T. Ugurbil, K. (2002) Ultra high-resolution fMRI in monkeys with implanted RF coils. Neuron, 35, 227 242.
29. Luo, H. Poeppel, D. (2007) Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex. Neuron, 54, 1001 1010.
30. MacNeilage, P.F. (1998) The frame/content theory of evolution of speech production. Behav. Brain. Sci., 21, 499 511.
31. MacNeilage, P.F. (2008) The Origin of Speech. Oxford University Press
32. Miki, K., Watanabe, S., Kakigi, R. Puce, A. (2004) Magnetoencephalographic study of occipitotemporal activity elicited by viewing mouth movements. Clin. Neurophysiol., 115, 1559 1574.
33. Munhall, K. Vatikiotis-Bateson, E. (1998) The moving face during speech communication. In Campbell, R., Dodd, B. Burnham, D. (Eds Hearing by Eye II. Taylor and Francis
34. Ohala, J. (1975) Temporal Regulation of Speech. In Fant, G. Tatham, M.A.A. (Eds Auditory Analysis and Perception of Speech. Academic Press
35. Perrett, D.I., Rolls, E.T. Caan, W. (1982) Visual neurones responsive to faces in the monkey temporal cortex. Exp. Brain Res., 47, 329 342.
36. Perrett, D.I., Smith, P.A.J., Potter, D.D., Mistlin, A.J., Head, A.S., Milner, A.D. Jeeves, M.A. (1985) Visual cells in the temporal cortex sensitive to face view and gaze direction. Proc. R. Soc. Lond., B, Biol., 223, 293 317.
37. Pfingst, B.E. O'Connor, T.A. (1980) Vertical Stereotaxic Approach to Auditory-Cortex in the Unanesthetized Monkey. J. Neurosci. Methods, 2, 33 45.
38. Pinker, S. Bloom, P. (1990) Natural language and natural selection. Behav. Brain. Sci., 13, 707 784.
39. Pinsk, M.A., DeSimone, K., Moore, T., Gross, C.G. Kastner, S. (2005) Representations of faces and body parts in macaque temporal cortex: a functional MRI study. Proc. Natl Acad. Sci. USA, 102, 6996 7001.
40. Pinsk, M.A., Arcaro, M., Weiner, K.S., Kalkus, J.F., Inati, S.J., Gross, C.G. Kastner, S. (2009) Neural representations of faces and body parts in macaque and human cortex: a comparative FMRI study. J. Neurophysiol., 101, 2581 2600.
41. Poeppel, D. (2003) The analysis of speech in different temporal integration windows: cerebral lateralization as 'asymmetric sampling in time'. Speech. Commun., 41, 245 255.
42. Puce, A., Allison, T., Bentin, S., Gore, J.C. McCarthy, G. (1998) Temporal cortex activation in humans viewing eye and mouth movements. J. Neurosci., 18, 2188 2199.
43. Puce, A., Smith, A. Allison, T. (2000) ERPs evoked by viewing facial movements. Cognitive Neuropsychology, 17, 221 239.
44. Puce, A., Syngeniotis, A., Thompson, J.C., Abbott, D.F., Wheaton, K.J. Castiello, U. (2003) The human temporal lobe integrates facial form and motion: evidence from fMRI and ERP studies. Neuroimage, 19, 861 869.
45. Puce, A., Epling, J.A., Thompson, J.C. Carrick, O.K. (2007) Neural responses elicited to face motion and vocalization pairings. Neuropsychologia, 45, 93 106.
46. Reale, R.A., Calvert, G.A., Thesen, T., Jenison, R.L., Kawasaki, H., Oya, H., Howard, M.A. Brugge, J.F. (2007) Auditory-visual processing represented in the human superior temporal gyrus. Neuroscience, 145, 162 184.
47. Redican, W.K. (1975) Facial expressions in nonhuman primates. In Rosenblum, L.A. (ed Primate Behavior: Developments in Field and Laboratory Research. Academic Press
48. Saberi, K. Perrott, D.R. (1999) Cognitive restoration of reversed speech. Nature, 398, 760.
49. Schroeder, C.E., Molholm, S., Lakatos, P., Ritter, W. Foxe, J.J. (2004) Human-simian correspondence in the early cortical processing of multisensory cues. Cogn. Process., 5, 140 151.
50. Schroeder, C.E., Lakatos, P., Kajikawa, Y., Partan, S. Puce, A. (2008) Neuronal oscillations and visual amplification of speech. Trends Cogn Sci., 12, 106 113.
51. Shannon, R.V., Zeng, F.-G., Kamath, V., Wygonski, J. Ekelid, M. (1995) Speech recognition with primarily temporal cues. Science, 270, 303 304.
52. Sherwood, C.C. (2005) Comparative anatomy of the facial motor nucleus in mammals, with an analysis of neuron numbers in primates. Anat. Rec. A Discov. Mol. Cell. Evol. Biol., 287A, 1067 1079.
53. Sherwood, C.C., Holloway, R.L., Erwin, J.M. Hof, P.R. (2004a) Cortical orofacial motor representation in old world monkeys, great apes, and humans - II. Stereologic analysis of chemoarchitecture. Brain Behav. Evol., 63, 82 106.
54. Sherwood, C.C., Holloway, R.L., Erwin, J.M., Schleicher, A., Zilles, K. Hof, P.R. (2004b) Cortical orofacial motor representation in old world monkeys, great apes, and humans - I. Quantitative analysis of cytoarchitecture. Brain Behav. Evol., 63, 61 81.
55. Sherwood, C.C., Hof, P.R., Holloway, R.L., Semendeferi, K., Gannon, P.J., Frahm, H.D. Zilles, K. (2005) Evolution of the brainstem orofacial motor system in primates: a comparative study of trigeminal, facial, and hypoglossal nuclei. J. Human Evol., 48, 45 84.
56. Smith, Z.M., Delgutte, B. Oxenham, A.J. (2002) Chimaeric sounds reveal dichotomies in auditory perception. Nature, 416, 87 90.
57. Steckenfinger, S.A. Ghazanfar, A.A. (2009) Monkey visual behavior falls into the uncanny valley. Proc. Natl. Acad. Sci. USA, 106, 18362 18466.
58. Sugihara, T., Diltz, M.D., Averbeck, B.B. Romanski, L.M. (2006) Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex. J. Neurosci., 26, 11138 11147.
59. Szucs, A. (1998) Applications of the spike density function in analysis of neuronal firing patterns. J. Neurosci. Methods, 81, 159 167.
60. Tsao, D.Y., Freiwald, W.A., Knutsen, T.A., Mandeville, J.B. Tootell, R.B.H. (2003) Faces and objects in macaque cerebral cortex. Nat. Neurosci., 6, 989 995.
61. Vitkovitch, M. Barber, P. (1994) Effect of video frame rate on subjects' ability to shadow one of two competing verbal passages. J. Speech Hear. Res., 37, 1204 1210.
62. Vitkovitch, M. Barber, P. (1996) Visible speech as a function of image quality: effects of display parameters on lipreading ability. Appl. Cogn. Psychol., 10, 121 140.
63. Waller, B.M., Parr, L.A., Gothard, K.M., Burrows, A.M. Fuglevand, A.J. (2008) Mapping the contribution of single muscles to facial movements in the rhesus macaque. Physiol. Behav., 95, 93 100.
64. Wright, T.M., Pelphrey, K.A., Allison, T., McKeown, M.J. McCarthy, G. (2003) Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb. Cortex, 13, 1034 1043.