Facial speech gestures: the relation between visual speech processing, phonological awareness, and developmental dyslexia in 10-year-olds

Developmental Science

Volumn 19, Issue 6, 2016, Pages 1020-1034

  Schaadt, Gesa ^a,b   Männel, Claudia ^a   van der Meer, Elke ^b,c   Pannekamp, Ann ^b   Friederici, Angela D ^a  

^a MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

^b HUMBOLDT UNIVERSITY   (Germany)

^c Berlin School of Mind and Brain   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

AWARENESS; CHILD; DYSLEXIA; EVOKED RESPONSE; FACIAL EXPRESSION; GESTURE; HUMAN; PATHOPHYSIOLOGY; PHONETICS; PHYSIOLOGY; SPEECH PERCEPTION;

AWARENESS; CHILD; DYSLEXIA; EVOKED POTENTIALS; FACIAL EXPRESSION; GESTURES; HUMANS; PHONETICS; SPEECH PERCEPTION;

EID: 84944456982     PISSN: 1363755X     EISSN: 14677687     Source Type: Journal    
DOI: 10.1111/desc.12346     Document Type: Article

References (76)

1. American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders: DSM-5. Washington, DC: American Psychiatric Association.
2. Arnal, L.H., Morillon, B., Kell, C.A., & Giraud, A.L. (2009). Dual neural routing of visual facilitation in speech processing. Journal of Neuroscience, 29 (43), 13445–13453. doi: 10.1523/JNEUROSCI.3194-09.2009
3. Arnold, P., & Hill, F. (2001). Bisensory augmentation: a speechreading advantage when speech is clearly audible and intact. British Journal of Psychology, 92 (2), 339–355. doi: 10.1348/000712601162220
4. Benoît, C., Guiard-Marigny, T., Goff, B., & Adjoudani, A. (1996). Which components of the face do humans and machines best speechread? In D. Stork & M. Hennecke (Eds.), Speechreading by humans and machines (Vol. 150, pp. 315–328). Berlin and Heidelberg: Springer.
5. Bernstein, L.E., Auer, E.T., Jr., & Takayanagi, S. (2004). Auditory speech detection in noise enhanced by lipreading. Speech Communication, 44, 5–18. doi: 10.1016/j.specom.2004.10.011
6. Brambati, S.M., Termine, C., Ruffino, M., Danna, M., Lanzi, G. et al. (2006). Neuropsychological deficits and neural dysfunction in familial dyslexia. Brain Research, 1113 (1), 174–185. doi: 10.1016/j.brainres.2006.06.099
7. Campbell, R. (2008). The processing of audio-visual speech: empirical and neural bases. Philosophical Transactions of the Royal Society B, 363 (1493), 1001–1010. doi: 10.1098/rstb.2007.2155
8. Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A., & Ghazanfar, A.A. (2009). The natural statistics of audiovisual speech. PLoS Computational Biology, 5 (7), e1000436. doi: 10.1371/journal.pcbi.1000436
9. Cleary, K.M., Donkers, F.C., Evans, A.M., & Belger, A. (2013). Investigating developmental changes in sensory processing: visual mismatch response in healthy children. Frontiers in Human Neuroscience, 7, 922. doi: 10.3389/fnhum.2013.00922
10. Clery, H., Roux, S., Besle, J., Giard, M.H., Bruneau, N. et al. (2012). Electrophysiological correlates of automatic visual change detection in school-age children. Neuropsychologia, 50 (5), 979–987. doi: 10.1016/j.neuropsychologia.2012.01.035
11. Colin, C., Radeau, M., Soquet, A., Demolin, D., Colin, F. et al. (2002). Mismatch negativity evoked by the McGurk-MacDonald effect: a phonetic representation within short-term memory. Clinical Neurophysiology, 113 (4), 495–506.
12. Corbera, S., Escera, C., & Artigas, J. (2006). Impaired duration mismatch negativity in developmental dyslexia. NeuroReport, 17, 1051–1055.
13. Cornelissen, P., Richardson, A., Mason, A., Fowler, S., & Stein, J. (1995). Contrast sensitivity and coherent motion detection measured at photopic luminance levels in dyslexics and controls. Vision Research, 35 (10), 1483–1494.
14. Czigler, I. (2014). Visual mismatch negativity and categorization. Brain Topography, 27 (4), 590–598. doi: 10.1007/s10548-013-0316-8
15. Czigler, I., Weisz, J., & Winkler, I. (2006). ERPs and deviance detection: visual mismatch negativity to repeated visual stimuli. Neuroscience Letters, 401 (1–2), 178–182. doi: 10.1016/j.neulet.2006.03.018
16. de Gelder, B., & Vroomen, J. (1998). Impaired speech perception in poor readers: evidence from hearing and speech reading. Brain and Language, 64, 296–281.
17. de Haan, M., Johnson, M.H., & Halit, H. (2003). Development of face-sensitive event-related potentials during infancy: a review. International Journal of Psychophysiology, 51 (1), 45–58. doi: 10.1016/s0167-8760(03)00152-1
18. de Haan, M., & Nelson, C.A. (1999). Brain activity differentiates face and object processing in 6-month-old infants. Developmental Psychology, 35 (4), 1113–1121. doi: 10.1037/0012-1649.35.4.1113
19. Demb, J.B., Boynton, G.M., & Heeger, D.J. (1997). Brain activity in visual cortex predicts individual differences in reading performance. Proceedings of the National Academy of Sciences, USA, 94, 13363–13366.
20. Files, B.T., Auer, E.T., Jr., & Bernstein, L.E. (2013). The visual mismatch negativity elicited with visual speech stimuli. Frontiers in Human Neuroscience, 7, 371. doi: 10.3389/fnhum.2013.00371
21. Hansen, P.C., Stein, J.F., Orde, S.R., Winter, J.L., & Talcott, J.B. (2001). Are dyslexics’ visual deficits limited to measures of dorsal stream function? NeuroReport, 12 (7), 1527–1530.
22. Hayes, E.A., Tiippana, K., Nicol, T.G., Sams, M., & Kraus, N. (2003). Integration of heard and seen speech: a factor in learning disabilities in children. Neuroscience Letters, 351 (1), 46–50. doi: 10.1016/s0304-3940(03)00971-6
23. Heim, S., Grande, M., Pape-Neumann, J., van Ermingen, M., Meffert, E. et al. (2010). Interaction of phonological awareness and ‘magnocellular’ processing during normal and dyslexic reading: behavioural and fMRI investigations. Dyslexia, 16 (3), 258–282. doi: 10.1002/dys.409
24. Jansen, H., Mannhaupt, G., Marx, H., & Skowronek, H. (2002). Bielefelder Screening zur Früherkennung von Lese- Rechtschreibschwierigkeiten [Bielefeld Screening for the Early Recognition of Reading and Spelling Difficulties]. Göttingen: Hogrefe.
25. Jasper, H.H. (1958). The ten-twenty electrode system of the international federation. Electroencephalography and Clinical Neurophysiology, 10, 371–375.
26. Jiang, J., Auer, E.T., Jr., Alwan, A., Keating, P.A., & Bernstein, L.E. (2007). Similarity structure in visual speech perception and optical phonetic signals. Perception & Psychophysics, 69 (7), 1070–1083. doi: 10.3758/BF03193945
27. Kaufman, A.S., Kaufman, N.L., Melchers, P., & Preuß, U. (2009). Kaufman Assessment Battery for Children (K- ABC). Göttingen: Hogrefe.
28. Kayser, C., Petkov, C.I., & Logothetis, N.K. (2008). Visual modulation of neurons in auditory cortex. Cerebral Cortex, 18 (7), 1560–1574. doi: 10.1093/cercor/bhm187
29. Kecskes-Kovacs, K., Sulykos, I., & Czigler, I. (2013). Visual mismatch negativity is sensitive to symmetry as a perceptual category. European Journal of Neuroscience, 37 (4), 662–667. doi: 10.1111/ejn.12061
30. Kronbichler, M., Hutzler, F., Staffen, W., Mair, A., Ladurner, G. et al. (2006). Evidence for a dysfunction of left posterior reading areas in German dyslexic readers. Neuropsychologia, 44 (10), 1822–1832. doi: 10.1016/j.neuropsychologia.2006.03.010
31. Kujala, T., Myllyviita, K., Tervaniemi, M., Alho, K., Kallio, J. et al. (2000). Basic auditory dysfunction in dyslexia as demonstrated by brain activity measurements. Psychophysiology, 37 (2), 262–266. doi: 10.1111/1469-8986.3720262
32. Landerl, K., & Wimmer, H. (2008). Development of word reading fluency and spelling in a consistent orthography: an 8-year follow-up. Journal of Educational Psychology, 100 (1), 150–161. doi: 10.1037/0022-0663.100.1.150
33. Landerl, K., Wimmer, H., & Frith, U. (1997). The impact of orthographic consistency on dyslexia: a German-English comparison. Cognition, 63, 315–334.
34. Lewkowicz, D.J., & Hansen-Tift, A.M. (2012). Infants deploy selective attention to the mouth of a talking face when learning speech. Proceedings of the National Academy of Sciences, USA, 109 (5), 1431–1436. doi: 10.1073/pnas.1114783109
35. Liberman, I.Y. (1973). Segmentation of the spoken word and reading acquisition. Bulletin of the Orton Society, 23 (1), 64–77. doi: 10.1007/BF02653842
36. Lovio, R., Näätänen, R., & Kujala, T. (2010). Abnormal pattern of cortical speech feature discrimination in 6-year-old children at risk for dyslexia. Brain Research, 1335, 53–62. doi: 10.1016/j.brainres.2010.03.097
37. Lyon, G.R., Shaywitz, S.E., & Shaywitz, B.A. (2003). A definition of dyslexia. Annals of Dyslexia, 53 (1), 1–14. doi: 10.1007/s11881-003-0001-9
38. Maekawa, T., Goto, Y., Kinukawa, N., Taniwaki, T., Kanba, S. et al. (2005). Functional characterization of mismatch negativity to a visual stimulus. Clinical Neurophysiology, 116 (10), 2392–2402. doi: 10.1016/j.clinph.2005.07.006
39. Mann, V.A., & Liberman, I.Y. (1984). Phonological awareness and verbal short-term memory. Journal of Learning Disabilities, 17 (10), 592–599. doi: 10.1177/002221948401701005
40. Meng, X., Cheng-Lai, A., Zeng, B., Stein, J.F., & Zhou, X. (2011). Dynamic visual perception and reading development in Chinese school children. Annals of Dyslexia, 61 (2), 161–176. doi: 10.1007/s11881-010-0049-2
41. Mohammed, T., Campbell, R., Macsweeney, M., Barry, F., & Coleman, M. (2006). Speechreading and its association with reading among deaf, hearing and dyslexic individuals. Clinical Linguistics & Phonetics, 20 (7–8), 621–630. doi: 10.1080/02699200500266745
42. Moll, K., Ramus, F., Bartling, J., Bruder, J., Kunze, S. et al. (2014). Cognitive mechanisms underlying reading and spelling development in five European orthographies. Learning and Instruction, 29, 65–77. doi: 10.1016/j.learninstruc.2013.09.003
43. Näätänen, R., Gaillard, A.W.K., & Mäntysalo, S. (1978). Early selective-attention effect on evoked potential reinterpreted. Acta Psychologica, 42, 313–329.
44. Näätänen, R., Paavilainen, P., Rinne, T., & Alho, K. (2007). The mismatch negativity (MMN) in basic research of central auditory processing: a review. Clinical Neurophysiology, 118 (12), 2544–2590. doi: 10.1016/j.clinph.2007.04.026
45. Navarra, J., & Soto-Faraco, S. (2007). Hearing lips in a second language: visual articulatory information enables the perception of second language sounds. Psychological Research, 71 (1), 4–12. doi: 10.1007/s00426-005-0031-5
46. Neuhoff, N., Bruder, J., Bartling, J., Warnke, A., Remschmidt, H. et al. (2012). Evidence for the late MMN as a neurophysiological endophenotype for dyslexia. PLoS ONE, 7 (5), e34909. doi: 10.1371/journal.pone.0034909
47. Patterson, M.L., & Werker, J.F. (2003). Two-month-old infants match phonetic information in lips and voice. Developmental Science, 6 (2), 191–196. doi: 10.1111/1467-7687.00271
48. Pazo-Alvarez, P., Amenedo, E., Lorenzo-Lopez, L., & Cadaveira, F. (2004). Effects of stimulus location on automatic detection of changes in motion direction in the human brain. Neuroscience Letters, 371 (2–3), 111–116. doi: 10.1016/j.neulet.2004.08.073
49. Pekkola, J., Laasonen, M., Ojanen, V., Autti, T., Jaaskelainen, I.P. et al. (2006). Perception of matching and conflicting audiovisual speech in dyslexic and fluent readers: an fMRI study at 3 T. NeuroImage, 29 (3), 797–807. doi: 10.1016/j.neuroimage.2005.09.069
50. Peterson, R.L., & Pennington, B.F. (2015). Developmental dyslexia. Annual Review of Clinical Psychology, 11, 283–307. doi: 10.1146/annurev-clinpsy-032814-112842
51. Ponton, C.W., Bernstein, L.E., & Auer, E.T., Jr. (2009). Mismatch negativity with visual-only and audiovisual speech. Brain Topography, 21 (3–4), 207–215. doi: 10.1007/s10548-009-0094-5
52. Pugh, K.R., Mencl, W.E., Jenner, A.R., Katz, L., Frost, S.J. et al. (2000). Functional neuroimaging studies of reading and reading disability (developmental dyslexia). Mental Retardation & Developmental Disabilities Research Reviews, 6, 207–213.
53. Pugh, K.R., Mencl, W.E., Jenner, A.R., Katz, L., Frost, S.J. et al. (2001). Neurobiological studies of reading and reading disability. Journal of Communication Disorders, 34, 479–492.
54. Restuccia, D., Della Marca, G., Marra, C., Rubino, M., & Valeriani, M. (2005). Attentional load of the primary task influences the frontal but not the temporal generators of mismatch negativity. Cognitive Brain Research, 25 (3), 891–899. doi: 10.1016/j.cogbrainres.2005.09.023
55. Salmelin, R., Service, E., Kiesilä, P., Uutela, K., & Salonen, O. (1996). Impaired Visual word processing in dyslexia revealed with magnetoencephalography. Annals of Neurology, 40 (2), 157–162. doi: 10.1002/ana.410400206
56. Schaadt, G., Männel, C., van der Meer, E., Pannekamp, A., & Oberecker, R. et al. (under review). Present and past: can writing abilities in school children be associated with their auditory discrimination capacities in infancy?
57. Schulte-Körne, G. (2001). Lese-Rechtschreibstörung und Sprachwahrnehmung: psychometrische und neurophysiologische Untersuchungen zur Legasthenie. Münster: Waxmann.
58. Schulte-Körne, G., Deimel, W., Bartling, J., & Remschmidt, H. (1998). Auditory processing and dyslexia: evidence for a specific speech processing deficit. NeuroReport, 9, 337–340.
59. Schulte-Körne, G., Deimel, W., Bartling, J., & Remschmidt, H. (2001). Speech perception deficit in dyslexic adults as measured by mismatch negativity (MMN). International Journal of Psychophysiology, 40, 77–87.
60. Shaywitz, S.E., & Shaywitz, B.A. (2008). Paying attention to reading: the neurobiology of reading and dyslexia. Developmental Psychopathology, 20 (4), 1329–1349. doi: 10.1017/S0954579408000631
61. Snowling, M.J. (1998). Dyslexia as a phonological deficit: evidence and implications. Child Psychology & Psychiatry, 3 (1), 4–11.
62. Snowling, M.J., Gallagher, A., & Frith, U. (2003). Family risk of dyslexia is continuous: individuals differences in the precursors of reading skill. Child Development, 74 (2), 358–373. doi: 10.1111/1467-8624.7402003
63. Stefanics, G., Csukly, G., Komlosi, S., Czobor, P., & Czigler, I. (2012). Processing of unattended facial emotions: a visual mismatch negativity study. NeuroImage, 59 (3), 3042–3049. doi: 10.1016/j.neuroimage.2011.10.041
64. Stein, J. (2001). The magnocellular theory of developmental dyslexia. Dyslexia, 7 (1), 12–36. doi: 10.1002/dys.186
65. Stein, J., & Walsh, V. (1997). To see but not to read: the magnocellular theory of dyslexia. Trends in Neurosciences, 20 (4), 147–152. doi: 10.1016/S0166-2236(96)01005-3
66. Stock, C., Marx, P., & Schneider, W. (2003). Basiskompetenzen für Lese-Rechtschreibleistung (test for basic competences for reading and spelling). Göttingen: Beltz Test GmbhH.
67. Stock, C., & Schneider, W. (2008). Deutscher Rechtschreibtest (DERET) (German spelling test). Göttingen: Hogrefe.
68. Talcott, J.B., Gram, A., Van Ingelghem, M., Witton, C., Stein, J.F. et al. (2003). Impaired sensitivity to dynamic stimuli in poor readers of a regular orthography. Brain and Language, 87 (2), 259–266. doi: 10.1016/s0093-934x(03)00105-6
69. Tales, A., Newton, P., Troscianko, T., & Butler, S. (1999). Mismatch negativity in the visual modality. NeuroReport, 10 (6), 3363–3367.
70. Teinonen, T., Aslin, R.N., Alku, P., & Csibra, G. (2008). Visual speech contributes to phonetic learning in 6-month-old infants. Cognition, 108 (3), 850–855. doi: 10.1016/j.cognition.2008.05.009
71. van Wassenhove, V., Grant, K.W., & Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences, USA, 102 (4), 1181–1186. doi: 10.1073/pnas.0408949102
72. Wagner, R.K., & Trogensen, J.K. (1987). The nature of phonological processing and its causal role in the acquisition of reading skills. Psychological Bulletin, 101, 192–212. doi: 10.1037/0033-2909.101.2.192
73. Wimmer, H. (1996). The early manifestation of developmental dyslexia: evidence from German children. Reading and Writing: An Interdisciplinary Journal, 8, 171–188.
74. Winkler, I., & Czigler, I. (2012). Evidence from auditory and visual event-related potential (ERP) studies of deviance detection (MMN and vMMN) linking predictive coding theories and perceptual object representations. International Journal of Psychophysiology, 83 (2), 132–143. doi: 10.1016/j.ijpsycho.2011.10.001
75. World Medical Association (2013). World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. Journal of the American Medical Association, 310 (20), 2191–2194. doi: 10.1001/jama.2013.281053
76. Ziegler, J.C., & Goswami, U. (2006). Becoming literate in different languages: similar problems, different solutions. Developmental Science, 9 (5), 429–453. doi: 10.1111/j.1467-7687.2006.00509.x