Speech-and-gesture integration in high functioning autism

Cognition

Volumn 115, Issue 3, 2010, Pages 380-393

  Silverman, Laura B ^a,b   Bennetto, Loisa ^a   Campana, Ellen ^c   Tanenhaus, Michael K ^a  

^a UNIVERSITY OF ROCHESTER   (United States)

^b UNIVERSITY OF ROCHESTER SCHOOL OF MEDICINE AND DENTISTRY   (United States)

^c ARIZONA STATE UNIVERSITY   (United States)

Author keywords

Autism; Cross modal integration; Eye movements; Gesture; Semantic processing

Indexed keywords

ADULT; AGE; ARTICLE; AUTISM; CLINICAL ARTICLE; COMPREHENSION; COMPUTER; CONTROLLED STUDY; EYE FIXATION; EYE MOVEMENT; FEMALE; GENDER; GESTURE; HEARING; HUMAN; INTELLIGENCE QUOTIENT; MALE; PRIORITY JOURNAL; QUANTITATIVE ANALYSIS; SOCIAL STATUS; SPEECH; TASK PERFORMANCE; VIDEORECORDING; VISION;

ADOLESCENT; AUTISTIC DISORDER; CHILD; EYE MOVEMENTS; FEMALE; FIXATION, OCULAR; GESTURES; HUMANS; MALE; NEUROPSYCHOLOGICAL TESTS; NONVERBAL COMMUNICATION; REACTION TIME; SPEECH; VOCABULARY;

EID: 77951666247     PISSN: 00100277     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cognition.2010.01.002     Document Type: Article

References (89)

1. Achenbach T.M., Rescorla L.A., McConaughey S.H., Pecora P.J., Wetherbee K.M., and Ruffle T.M. Child behavior checklist for ages 6-18 (2001), Achenbach System of Empirically Based Assessment, Burlington, VT
2. Acredolo L., and Goodwyn S. Symbolic gesturing in normal infants. Child Development 59 (1988) 450-466
3. Alcantara J.I., Weisblatt E.J.L., Moore B.C.J., and Bolton P.F. Speech-in-noise perception in high-functioning individuals with autism or Asperger's syndrome. Journal of Child Psychology and Psychiatry 45 (2004) 1107-1114
4. Alibali M.W., and Goldin-Meadow S. Gesture speech mismatch and mechanisms of learning - What the hands reveal about a child's state of mind. Cognitive Psychology 25 (1993) 468-523
5. American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-IV). 4th ed. (1994), APA, Washington, DC
6. Ashwin C., Chapman E., Colle L., and Baron-Cohen S. Impaired recognition of negative basic emotions in autism: A test of the amygdala theory. Social Neuroscience 1 (2006) 349-363
7. Baron-Cohen S. Perceptual role-taking and protodeclarative pointing in autism. British Journal of Developmental Psychology 7 (1989) 113-127
8. Baron-Cohen S., Baldwin D.A., and Crowson M. Do children with autism use the speaker's direction of gaze strategy to crack the code of language?. Child Development 68 (1997) 48-57
9. Bartak L., Rutter M., and Cox A. Comparative study of infantile autism and specific developmental receptive language disorder. 1. Children. British Journal of Psychiatry 126 (1975) 127-145
10. Bates E. Language and context: The acquisition of the performatives (1976), Academic Press, New York
11. Beattie G., and Shovelton H. Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation. Semiotica 123 (1999) 1-30
12. Bebko J.M., Weiss J.A., Demark J.L., and Gomez P. Discrimination of temporal synchrony in intermodal events by children with autism and children with developmental disabilities without autism. Journal of Child Psychology and Psychiatry 47 (2006) 88-98
13. Berger K.W., and Popelka G.R. Extra-facial gestures in relation to speech-reading. Journal of Communication Disorders 3 (1971) 302-308
14. Boucher J., Lewis V., and Collis G. Familiar face and voice matching and recognition in children with autism. Journal of Child Psychology and Psychiatry 39 (1998) 171-181
15. Brownell R. Expressive one-word vocabulary test. 2000 Edition (2000), Academic Therapy Publications, Novato, CA
16. Bryson C.Q. Systematic identification of perceptual disabilities in autistic children. Perception and Motor Skills 31 (1970) 239-246
17. Bryson C.Q. Short-term memory and cross-modal information processing in autistic children. Journal of Learning Disabilities 5 (1972) 81-91
18. Burke P. The language of gesture in early modern Italy. In: Bremmer J., and Roodenburg H. (Eds). A cultural history of gesture (1991), Cornell University Press, Ithaca, NY
19. Čeponienė R., Lepistö T., Alku P., Aro H., and Näätänen R. Event-related potential indices of auditory vowel processing in 3-year-old children. Clinical Neurophysiology 114 (2003) 652-661
20. Cook S.W., and Tanenhaus M.K. Embodied communication: Speakers' gestures affect listeners' actions. Cognition 113 (2009) 98-104
21. Cooper R.M. The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology 6 (1974) 84-107
22. Dawson G., Meltzoff A.N., Osterling J., Rinaldi J., and Brown E. Children with autism fail to orient to naturally occurring social stimuli. Journal of Autism and Developmental Disorders 28 (1998) 479-485
23. de Gelder B., Vroomen J., and Van der Heide L. Face recognition and lip-reading in autism. European Journal of Cognitive Psychology 3 (1991) 69-86
24. DeRuiter, J.-P (1998). Gesture and speech production. MPI Series in Psycholinguistics, no. 6. Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands.
25. Dunn L.M., and Dunn L.M. Peabody picture vocabulary test. 3rd ed. (1997), American Guidance Services, Circle Pines, MN
26. Dziobek I., Fleck S., Kalbe E., Rogers K., Hassenstab J., Brand M., et al. Introducing MASC: A movie for the assessment of social cognition. Journal of Autism and Developmental Disorders 36 (2006) 623-636
27. Eberhard K.M., Spivey-Knowlton M.J., Sedivy J.C., and Tanenhaus M.K. Eye-movements as a window into spoken language comprehension in natural contexts. Journal of Psycholinguistic Research 24 (1995) 409-436
28. Edelson, L. R., Fine, A., Falk, A., & Tager-Flusberg, H. (2009). Using eye-tracking measures to assess verb comprehension in toddlers with and without autism spectrum disorders. In Paper symposium session at the biennial meeting of the Society for Research in Child Development, Denver, CO, April.
29. Ejiri K., and Masataka N. Co-occurrence of preverbal vocal behavior and motor action in early infancy. Developmental Science 4 (2001) 40-48
30. Ekman P., and Friesen W.V. The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica 1 (1969) 49-98
31. Feyereisen P., Van de Wiele M., and Dubois F. The meaning of gestures: What can be understood without speech?. Chaiers de Psychologie 8 (1988) 3-25
32. Gaffrey, M. S., Kleinhans, N., Haist, F., Akshoomoff, N., Campbell, A., Courchesne, E., et al. (2004). Reduced left inferior frontal and enhanced occipital activation during semantic decision in autism. In International Meeting for Autism Research (vol. 123, pp. S3.2.1).
33. Goldin-Meadow, S., & Butcher, C. (2003). Pointing toward two-word speech in young children. In S. Kita (Ed.), Pointing; Where language, culture, and cognition meet (pp. 85-107).
34. Goldin-Meadow S., Goodrich W., Sauer E., and Iverson J. Young children use their hands to tell their mothers what to say. Developmental Science 10 (2007) 778-785
35. Goldin-Meadow S., and Sandhofer C.M. Gesture convey substantive information about a child's thoughts to ordinary listeners. Developmental Science 2 (1999) 67-74
36. Graham J.A., and Argyle M. A cross-cultural study of the communication of extra-verbal meaning by gestures. International Journal of Psychology 10 (1975) 57-67
37. Hadar U., and Pinchas-Zamir L. The semantic specificity of gesture - Implications for gesture classification and function. Journal of Language and Social Psychology 23 (2004) 204-214
38. Hall G.B.C., Szechtman H., and Hangmias C. Enhanced salience and emotion recognition in autism: A PET study. American Journal of Psychiatry 160 (2003) 1439-1441
39. Hallett P.E. Eye movements. In: Boff K., Kaufman L., and Thomas J. (Eds). Handbook of perception and human performance (1986), Wiley, New York (pp. 10.11-10.112)
40. Harris N.G.S., Bellugi U., Bates E., Jones W., and Rossen M. Contrasting profiles of language development in children with Williams and Down syndromes. Developmental Neuropsychology 13 (1997) 345-370
41. Holle H., and Gunter T.C. The role of iconic gestures in speech disambiguation: ERP evidence. Journal of Cognitive Neuroscience 19 (2007) 1175-1192
42. Hollingshead A.B. Four-factor index of social status (1975), Yale University Department of Sociology, New Haven, CT
43. Humphreys K., Minshew N., Leonard G.L., and Behrmann M. A fine-grained analysis of facial expression processing in high-functioning adults with autism. Neuropsychologia 45 (2007) 685-695
44. Iarocci G., and McDonald J. Sensory integration and the perceptual experience of persons with autism. Journal of Autism and Developmental Disorders 36 (2006) 77-90
45. Iverson J.M., and Wozniak R.H. Variation in vocal-motor development in infant siblings of children with autism. Journal of Autism and Developmental Disorders 37 (2007) 158-170
46. Joseph R., Keehn B., Connolly C., Wolfe J.M., and Horowitz T.S. Why is visual search superior in autism spectrum disorder?. Developmental Science 12 (2009) 1083-1096
47. Kana R.K., Keller T.A., Cherkassky V.L., Minshew N.J., and Just M.A. Sentence comprehension in autism: Thinking in pictures with decreased functional connectivity. Brain 129 (2006) 2484-2493
48. Kelly S.D., and Church R.B. Can children detect conceptual information conveyed through other children's nonverbal behaviors?. Cognition and Instruction 15 (1997) 107-134
49. Kelly S.D., Kravitz C., and Hopkins M. Neural correlates of bimodal speech-and-gesture comprehension. Brain and Language 89 (2004) 253-260
50. Kelly S.D., Ward S., Creigh P., and Bartolotti J. An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language 101 (2007) 222-233
51. Kemner C., van Ewijk L., van Engeland H., and Hooge I. Brief report: Eye movements during visual search tasks indicate enhanced stimulus discriminability in subjects with PDD. Journal of Autism and Developmental Disorders 38 (2008) 553-557
52. Kendon A. Gesticulation and speech: Two aspects of the process of utterance. In: Key M.R. (Ed). The Relationship of Verbal and Nonverbal Communication (1980), Mouton and Co., The Hague 207-227
53. Kendon A. The study of gesture: Some remarks on its history. Recherches Semiotiques/Semiotic Inquiry 2 (1982) 45-62
54. Kendon A. Do gestures communicate? A review. Research on Language and Social Interaction 27 (1994) 175-200
55. Klin A., Jones W., Schultz R., and Volkmar F. The enactive mind, or from actions to cognition: Lessons from autism. Philosophical Transactions of the Royal Society of London Series B - Biological Sciences 358 1430 (2003) 345-360
56. Klin A., Jones W., Schultz R., Volkmar F., and Cohen D. Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry 59 (2002) 809-816
57. Klin A., Sparrow S.S., de Bildt A., Cicchetti D.V., Cohen D.J., and Volkmar F.R. A normed study of face recognition in autism and related disorders. Journal of Autism and Developmental Disorders 29 (1999) 499-508
58. Krauss R.M., Dushay R.A., Chen Y., and Rauscher F. The communicative value of conversational hand gestures. Journal of Experimental Social Psychology 31 (1995) 533-553
59. Krauss R.M., Morrel-Samuels P., and Colasante C. Do conversational hand gestures communicate?. Journal of Personality and Social Psychology 61 (1991) 743-754
60. Kutas, M. (2006). One lesson learned: Frame language processing - literal and figurative - as a human brain function. In A. Katz (Ed.), Metaphor and symbol (Vol. 21, pp. 285-325).
61. Layton T.L. Language training with autistic-children using 4 different modes of presentation. Journal of Communication Disorders 21 (1988) 333-350
62. Leekam S., Baron-Cohen S., Perrett D., Milders M., and Brown S. Eye-direction detection: A dissociation between geometric and joint attention skills in autism. British Journal of Developmental Psychology 15 (1997) 77-95
63. Lord C., Rutter M., DiLavore P.C., and Risi S. Autism diagnostic observation schedule (1999), Western Psychological Services, Los Angeles, CA
64. Loveland K.A., Tunalikotoski B., Chen R., Brelsford K.A., Ortegon J., and Pearson D.A. Intermodal perception of affect in persons with autism or Down syndrome. Development and Psychopathology 7 (1995) 409-418
65. Magnée M.J.C.M., deGelder B., van Engeland H., and Kemner C. Audiovisual speech integration in pervasive developmental disorder: Evidence from event-related potentials. Journal of Child Psychology and Psychiatry 49 (2008) 995-1000
66. Masataka N. Why early linguistic milestones are delayed in children with Williams syndrome: late onset of hand banging as a possible rate-limiting constraint on the emergence of canonical babbling. Developmental Science 4 (2001) 158-164
67. McGurk H., and McDonald J.W. Hearing lips and seeing voices. Nature 264 (1976) 746-748
68. McNeill D. Hand and mind: What gestures reveal about thought (1992), University of Chicago Press, Chicago
69. McNeill D., Cassell J., and McCullough K.E. Communicative effects of speech-mismatched gestures. Research on Language and Social Interaction 27 (1994) 223-237
70. Morrel-Samuels P., and Krauss R.M. Word familiarity predicts temporal asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning, Memory, and Cognition 18 (1992) 615-622
71. O'Riordan M., and Plaisted K. Enhanced discrimination in autism. Journal of Experimental Psychology 54A (2001) 961-979
72. O'Riordan M.A., Plaisted K.C., Driver J., and Baron-Cohen S. Superior visual search in autism. Journal of Experimental Psychology: Human Perception and Performance 27 (2001) 719-730
73. Osterling J.A., Dawson G., and Munson J.A. Early recognition of 1-year-old infants with autism spectrum disorder versus mental retardation. Development and Psychopathology 14 (2002) 239-251
74. Özyürek A., Willems R.M., Kita S., and Hagoort P. On-line integration of semantic information from speech-and-gesture: Insights from event-related potentials. Journal of Cognitive Neuroscience 19 (2007) 605-616
75. Phillips W., Baron-Cohen S., and Rutter M. The role of eye contact in goal detection: Evidence from normal infants and children with autism or mental handicap. Development and Psychopathology 4 (1992) 375-383
76. Riseborough M.G. Physiographic gestures as decoding facilitators: Three experiments exploring a neglected facet of communication. Journal of Nonverbal Behavior 5 (1981) 172-183
77. Rutter M., LeCouteur A., and Lord C. Autism diagnostic interview-revised (2003), Western Psychological Services, Los Angeles
78. Smith E.G., and Bennetto L. Audiovisual speech integration and lipreading in autism. Journal of Child Psychology and Psychiatry 48 (2007) 813-821
79. Tager-Flusberg H., Paul R., and Lord C. Language and communication in autism. In: Volkmar F.R., Paul R., and Cohen D.J. (Eds). Handbook of autism and pervasive developmental disorders. Vol. 1, 3rd ed. (2005), Wiley, Hoboken 335-364
80. Tanenhaus M.K., Spivey-Knowlton M.J., Eberhard K.M., and Sedivy J.C. Integration of visual and linguistic information in spoken language comprehension. Science 268 (1995) 1632-1634
81. Thal D., Bates E., Goodman J., and Jahn-Samilo J. Continuity of language abilities: An exploratory study of late- and early-talking toddlers. Developmental Neuropsychology 13 (1997) 239-273
82. Thal D., and Tobias S. Relationships between language and gesture in normally developing and late-talking toddlers. Journal of Speech and Hearing Research 37 (1994) 157-170
83. Thompson L.A., and Massaro D.W. Evaluation and integration of speech and pointing gestures during referential understanding. Journal of Experimental Child Psychology 42 (1986) 144-168
84. van der Smagt M.J., van Engeland H., and Kemner C. Brief report: Can you see what is not there? Low-level auditory-visual integration in autism spectrum disorder. Journal of Autism and Developmental Disorders 37 (2007) 2014-2019
85. Wesp R., Hesse J., Keutmann D., and Wheaton K. Gestures maintain spatial imagery. American Journal of Psychology 114 (2001) 591-600
86. Williams J.H.G., Massaro D.W., Peel N.J., Bosseler A., and Suddendorf T. Visual-auditory integration during speech imitation in autism. Research in Developmental Disabilities 25 (2004) 559-575
87. Wu Y.C., and Coulson S. Meaningful gestures: Electrophysiological indices of iconic gesture comprehension. Psychophysiology 42 (2005) 654-667
88. Wu Y.C., and Coulson S. How iconic gestures enhance communication: An ERP study. Brain and Language 101 (2007) 234-245
89. Yoder P.J., and Layton T.L. Speech following sign language training in autistic-children with minimal verbal language. Journal of Autism and Developmental Disorders 18 (1988) 217-229