Automatic intonation recognition for the prosodic assessment of language-impaired children

IEEE Transactions on Audio, Speech and Language Processing

Volumn 19, Issue 5, 2011, Pages 1328-1342

  Ringeval, Fabien ^a   Demouy, Julie ^b   Szaszák, György ^c   Chetouani, Mohamed ^a   Robel, Laurence ^b   Xavier, Jean ^b   Cohen, David ^a,b   Plaza, Monique ^a,b  

^a UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

^b PITIÉ SALPÊTRIÈRE HOSPITAL   (France)

^c BUDAPEST UNIVERSITY OF TECHNOLOGY AND ECONOMICS   (Hungary)

Author keywords

Automatic intonation recognition; prosodic skills assessment; social communication impairments

Indexed keywords

AUTISM DISORDERS; AUTOMATIC ASSESSMENT; AUTOMATIC INTONATION RECOGNITION; AUTOMATIC SYSTEMS; CONTROL GROUPS; LANGUAGE IMPAIRMENTS; PROSODIC SKILLS ASSESSMENT; SOCIAL COMMUNICATION IMPAIRMENTS; SPECIFIC LANGUAGE IMPAIRMENTS;

CONVOLUTION; EDUCATION; RATING;

HANDICAPPED PERSONS;

EID: 79956290068     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2010.2090147     Document Type: Article

References (73)

1. S. Ananthakrishnan and S. Narayanan, "Unsupervised adaptation of categorical prosody models for prosody labeling and speech recognition," IEEE Trans. Audio, Speech Lang. Process., vol. 17, no. 1, pp. 138-149, Jan. 2009.
2. P. K. Kuhl, "Early language acquisition: Cracking the speech code," Nature Rev. Neurosci., vol. 5, pp. 831-843, Nov. 2004. (Pubitemid 39482152)
3. R. Paul, A. Augustyn, A. Klin, and F. R. Volkmar, "Perception and production of prosody by speakers with autism spectrum disorders," J. Autism Develop. Disorders, vol. 35, no. 2, pp. 205-220, Apr. 2005. (Pubitemid 40569143)
4. P. Warren, "Parsing and prosody: An introduction," Lang. Cognitive Process., Psychol. Press, vol. 11, pp. 1-16, 1996.
5. D. Van Lancker, D. Canter, and D. Terbeek, "Disambiguation of ditropic sentences: Acoustic and phonetic cues," J. Speech Hear. Res., vol. 24, no. 3, pp. 330-335, Sep. 1981. (Pubitemid 11002155)
6. E.Winner, The Point ofWords: Children's Understanding of Metaphor and Irony. Cambridge, MA: Harvard Univ. Press, 1988.
7. D. Bolinger, Intonation and Its Uses: Melody in Grammar and Discourse. Stanford, CA: Stanford Univ. Press, Aug. 1989.
8. Diagnostic and Statistical Manual of Mental Disorders, 4th ed. Washington, DC: American Psychiatric Assoc., 1994.
9. I. Rapin and D. A. Allen, "Developmental language: Nosological consideration," in Neuropsychology of Language, Reading and Spelling, V. Kvik, Ed. New York: Academic Press, 1983.
10. L. Wing and J. Gould, "Severe impairments of social interaction and associated abnormalities in children: Epidemiology and classification," J. Autism Develop. Disorders, vol. 9, no. 1, pp. 21-29, Mar. 1979.
11. D. A. Allen and I. Rapin, "Autistic children are also dysphasic," in Neurobiology of Infantile Autism, H. Naruse and E. M. Ornitz, Eds. Amsterdam, The Netherlands: Excerpta Medica, 1992, pp. 157-168.
12. J. McCann and S. Peppé, "Prosody in autism: A critical review," Int. J. Lang. Commun. Disorders, vol. 38, no. 4, pp. 325-350, May 2003.
13. B.Wells and S. Peppé, "Intonation abilities of children with speech and language impairments," J. Speech, Lang. Hear. Res., vol. 46, pp. 5-20, Feb. 2003. (Pubitemid 44718453)
14. J. Morgan and K. Demuth, Signal to Syntax: Bootstrapping From Speech to Grammar in Early Acquisition. Mahwah, NJ: Erlbaum, 1996.
15. S. Weinert, "Sprach- und Gedächtnisprobleme dysphasischsprachgestörter Kinder: Sind rhytmisch-prosodische Defizite eine Ursache?," in [Language and Short-Term Memory Problems of Specifically Language Impaired Children: Are Rhythmic Prosodic Deficits a Cause?] Rhytmus Ein interdisziplinäres Handbuch, K. Müller and G. Aschersleben, Eds. Bern, Switzerland: Huber, 2000, pp. 255-283.
16. D. Snow, "Children's imitations of intonation contours: Are rising tones more difficult than falling tones?," J. Speech, Lang. Hear. Res., vol. 41, pp. 576-587, Jun. 1998. (Pubitemid 28299851)
17. D. Snow, "Prosodic markers of syntactic boundaries in the speech of 4-year-old children with normal and disordered language development," J. Speech, Lang. Hear. Res., vol. 41, pp. 1158-1170, Oct. 1998. (Pubitemid 28450242)
18. C. R. Marshall, S. Harcourt Brown, F. Ramus, and H. J. K Van der Lely, "The link between prosody and language skills in children with SLI and/or dyslexia," Int. J. Lang. Commun. Disorders, vol. 44, no. 4, pp. 466-488, Jul. 2009.
19. P. Hargrove and C. P. Sheran, "The use of stress by language impaired children," J. Commun. Disorders, vol. 22, no. 5, pp. 361-373, Oct. 1989. (Pubitemid 20050139)
20. C. Samuelsson, C. Scocco, and U. Nettelbladt, "Towards assessment of prosodic abilities in Swedish children with language impairment," Logopedics Phoniatrics Vocology, vol. 28, no. 4, pp. 156-166, Oct. 2003. (Pubitemid 37526900)
21. S. Van der Meulen and P. Janssen, "Prosodic abilities in children with Specific Language Impairment," J. Commun. Disorders, vol. 30, pp. 155-170, May-Jun. 1997. (Pubitemid 27220803)
22. L. Kanner, "Autistic disturbances of affective contact," Nervous Child, vol. 2, pp. 217-250, 1943.
23. R. Paul, L. Shriberg, J. Mc Sweeny, D. Ciccheti, A. Klin, and F. Volkmar, "Brief report: Relations between prosodic performance and communication and socialization ratings in high functioning speakers with autism spectrum disorders," J. Autism Develop. Disorders, vol. 35, no. 6, pp. 861-869, Dec. 2005.
24. S. Fosnot and S. Jun, "Prosodic characteristics in children with stuttering or autism during reading and imitation," in Proc. 14th Annu. Congr. Phonetic Sci., San Francisco, CA., Aug. 1-7, 1999, pp. 103-115.
25. J. McCann, S. Peppé, F. Gibbon, A. O'Hare, and M. Rutherford, "Prosody and its relationship to language in school-aged children with high functioning autism," Int. J. Lang. Commun. Disorders, vol. 47, no. 6, pp. 682-702, Nov. 2007. (Pubitemid 350034905)
26. M. T. Le Normand, S. Boushaba, and A. Lacheret-Dujour, "Prosodic disturbances in autistic children speaking French," in Proc. Speech Prosody, Campinas, Brazil, May 6-9, 2008, pp. 195-198.
27. R. Paul, N. Bianchi, A. Agustyn, A. Klin, and F. Volkmar, "Production of syllable stress in speakers with autism spectrum disorders," Research in Autism Spectrum Disorders, vol. 2, pp. 110-124, Jan.-Mar. 2008. (Pubitemid 351181803)
28. F. Volkmar, Handbook of Autism and Pervasive Develop. Disorders. Hoboken, NJ: Wiley, 2005.
29. E. Fombonne, "Epidemiological surveys of autism and other pervasive developmental disorders: An update," J. Autism Develop. Disorders, vol. 33, no. 4, Aug. 2003.
30. L. D. Schriberg, J. Kwiatkowski, and C. Rasmussen, The Prosody- Voice Screening Profile. Tuscon, AZ: Communication Skill Builders, 1990.
31. D. Crystal, Profiling Linguist. Disability. London, U.K.: Edward Arnold, 1982.
32. P. Martínez-Castilla and S. Peppé, "Developing a test of prosodic ability for speakers of Iberian-Spanish," Speech Commun., vol. 50, no. 11-12, pp. 900-915, Mar. 2008.
33. J. P. H. van Santen, E. T. Prud'hommeaux, and L. M. Black, "Automated assessment of prosody production," Speech Commun., vol. 51, no. 11, pp. 1082-1097, Nov. 2009.
34. A. Maier, T. Haderlein, U. Eysholdt, F. Rosanowski, A. Batliner, M. Schuster, and E. Nöth, "PEAKS-A system for the automatic evaluation of voice and speech disorder," Speech Commun., vol. 51, no. 5, pp. 425-437, May 2009.
35. M. Black, J. Tepperman, A. Kazemzadeh, S. Lee, and S. Narayanan, "Automatic pronunciation verification of English letter-names for early literacy assessment of preliterate children," in Proc. ICASSP, Taipei, Taiwan, Apr. 19-24, 2009, pp. 4861-4864.
36. C. Min Lee and S. Narayanan, "Toward detecting emotions in spoken dialogs," IEEE Trans. Speech Audio Process., vol. 13, no. 2, pp. 293-303, Mar. 2005. (Pubitemid 40320247)
37. G. P. M. Laan, "The contribution of intonation, segmental durations, and spectral features to the perception of a spontaneous and read speaking style," Speech Commun., vol. 22, pp. 43-65, Mar. 1997. (Pubitemid 127403439)
38. A. Potamianos and S. Narayanan, "A review of the acoustic and linguistic properties of children's speech," in Proc. IEEE 9th Workshop Multimedia Signal Process., Chania, Greece, Oct. 23, 2007, pp. 22-25.
39. R. D.Kent, "Hearing and believing: Some limits to theauditory-perceptual assessment of speech and voice disorders," Amer. J. Speech-Lang. Pathol., vol. 5, no. 3, pp. 7-23, Aug. 1996.
40. A. Tversky, "Intransitivity of preferences," Psychol. Rev., vol. 76, pp. 31-48, Jan. 1969.
41. A. Pentland, "Social signal processing," IEEE Signal Process. Mag., vol. 24, no. 4, pp. 108-111, Jul. 2007. (Pubitemid 47316171)
42. B. Schuller, A. Batliner, D. Seppi, S. Steidl, T. Vogt, J. Wagner, L. Devillers, L. Vidrascu, N. Amir, L. Kessous, and V. Aharonson, "The relevance of feature type for the automatic classification of emotional user states: Low level descriptors and functionals," in Proc. Interspeech ICSLP, Antwerp, Belgium, Aug. 27-31, 2007, pp. 2253-2256.
43. J. Nadel, "Imitation and imitation recognition: Functional use in preverbal infants and nonverbal children with autism," in The Imitative Mind: Development, Evolution and Brain Bases, A. N. Meltzoff and W. Prinz, Eds. Cambridge, MA: Cambridge Univ. Press, 2002, pp. 2-14.
44. G. Szaszák, D. Sztahó, and K. Vicsi, "Automatic intonation classification for speech training systems," in Proc. Interspeech, Brighton, U.K., Sep. 6-10, 2009, pp. 1899-1902.
45. D. Ververidis and C. Kotropoulos, "Emotional speech recognition: Resources, features and methods," Speech Commun., vol. 48, no. 9, pp. 1162-1181, Sep. 2006. (Pubitemid 44128615)
46. A. G. Adami, "Modeling prosodic differences for speaker recognition," Speech Commun., vol. 49, no. 4, pp. 1162-1181, Apr. 2007.
47. D. H. Milone and A. J. Rubio, "Prosodic and accentual information for automatic speech recognition," IEEE Trans. Speech Audio Process., vol. 11, no. 4, pp. 321-333, Jul. 2003.
48. A. Mahdhaoui, M. Chetouani, C. Zong, R. S. Cassel, C. Saint-Georges, M.-C. Laznik, S. Maestro, F. Apicella, F. Muratori, and D. Cohen, "Automatic motherese detection for face-to-face interaction analysis," Multimodal Signals: Cognitive and Algorithmic Issues, vol. LNAI 5398, pp. 248-255, Feb. 2009, Springer-Verlag.
49. V.-M. Quang, L. Besacier, and E. Castelli, "Automatic question detection: Prosodic-lexical features and crosslingual experiments," in Proc. Interspeech ICSLP, Antwerp, Belgium, Aug. 27-31, 2007, pp. 2257-2260.
50. S. Yildirim and S. Narayanan, "Automatic detection of disfluency boundaries in spontaneous speech of children using audio-visual information," IEEE Trans. Audio Speech Lang. Process., vol. 17, no. 1, pp. 2-12, Jan. 2009.
51. H. Pon-Barry and S. Shieber, "The importance of sub-utterance prosody in predicting level of certainty," in Proc. Human Lang. Tech. Conf., Poznan, Poland, May 31-Jun. 5 2009, pp. 105-108.
52. D. Elenius and M. Blomberg, "Comparing speech recognition for adults and children," in Proc. FONETIK, Stockholm, Sweden, May 26-28, 2004, pp. 105-108.
53. J.-F. Bonastre, C. Fredouille, A. Ghio, A. Giovanni, G. Pouchoulin, J. Révis, B. Teston, and P. Yu, "Complementary approaches for voice disorder assessment," in Proc. Interspeech ICSLP, Antwerp, Belgium, Aug. 27-31, 2007, pp. 1194-1197.
54. M. Chetouani, A. Mahdhaoui, and F. Ringeval, "Time-scalefeature extractions for emotional speech characterization," Cognitive Comp., vol. 1, no. 2, pp. 194-201, 2009, Springer.
55. L. I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms. Hoboken, NJ: Wiley, 2004.
56. E. Monte-Moreno, M. Chetouani, M. Faundez-Zanuy, and J. Sole-Casals, "Maximum likelihood linear programming data fusion for speaker recognition," Speech Commun., vol. 51, no. 9, pp. 820-830, Sep. 2009.
57. F. Ringeval and M. Chetouani, "A vowel based approach for acted emotion recognition," in Proc. Interspeech, Brisbane, Australia, Sep. 22-26, 2008, pp. 2763-2766.
58. A. Mahdhaoui, F. Ringeval, and M. Chetounani, "Emotional speech characterization based on multi-features fusion for face-to-face communication," in Proc. Int. Conf. SCS, Jerba, Tunisia, Nov. 6-8, 2009.
59. A. Mahdhaoui, M. Chetouani, and C. Zong, "Motherese detection based on segmental and supra-segmental features," in Proc. Int. Conf. Pattern Recogn., Tampa, FL., Dec. 8-11, 2008.
60. S. Ananthakrishnan and S. Narayanan, "Fine-grained pitch accent and boundary tones labeling with parametric f0 features," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Las Vegas, NV, Mar. 30-Apr. 4 2008, pp. 4545-4548.
61. A. Rosenberg and J. Hirschberg, "Detecting pitch accents at the word, syllable and vowel level," in Proc. Human Lang. Tech.: 2009 Annu. Conf. North Amer. Chapter Assoc. for Comput. Ling., Boulder, CO, May 31-Jun. 5 2009, pp. 81-84.
62. Snack Sound Toolkit [Online]. Available: http://www.speech.kth.se/ snack/
63. R.-O. Duda, P.-E. Hart, and D.-G. Stork, Pattern Classification, 2nd ed. New York: Wiley, 2000.
64. M. Robnik and I. Konenko, "Theoretical and empirical analysis of ReliefF and RReliefF," Mach. Learn. J., vol. 53, pp. 23-69, Oct.-Nov. 2003.
65. L. Kuncheva and C. Whitaker, "Measure of diversity in classifier ensembles," Mach. Learn., vol. 51, no. 2, pp. 181-207, May 2003.
66. C. Lord, M. Rutter, and A. Le Couteur, "Autism diagnostic interviewrevised: A revision version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders," J. Autism Develop. Disorders, vol. 24, no. 5, pp. 659-685, 1994. (Pubitemid 24309810)
67. E. Schopler, R. Reichler, R. Devellis, and K. Daly, "Toward objective classification of childhood autism: Childhood Autism Rating Scale (CARS)," J. Autism Develop. Disorders, vol. 10, no. 1, pp. 91-103, 1980. (Pubitemid 10050041)
68. R. Van der Gaag, J. Buitelaar, E. Van den Ban, M. Bezemer, L. Njio, and H. Van Engeland, "A controlled multivariate chart review of multiple complex developmental disorder," J. Amer. Acad. Child Adolesc. Psychiatry, vol. 34, pp. 1096-1106, 1995.
69. J. Buitelaar and R. Van der Gaag, "Diagnostic rules for children with PDD-NOS and multiple complex developmental disorder," J. Child Psychol. Psychiatry, vol. 39, pp. 91-919, 1998.
70. E. Rondeau, L. Klein, A. Masse, N. Bodeau, D. Cohen, and J. M. Guilé, "Is pervasive developmental disorder not otherwise specified less stable than autistic disorder?," J. Autism Develop. Disorder, 2010, to be published.
71. A. Khomsi, Evaluation du Langage Oral. Paris, France: ECPA, 2001.
72. K. Sjölander and J. Beskow, "WaveSurfer-An open source speech tool," in Proc. 6th ICSLP, Beijing, China, Oct. 2000, vol. 4, pp. 464-467 [Online]. Available: http://www.speech.kth.se/wavesurfer/
73. K. Vicsi, A Multimedia Multilingual Teaching and Training System for Speech Handicapped Children Univ. of Technol. and Economics, Dept. of Telecommunications and Telematics, Final Annual Report, Speech Corrector, SPECO-977126 [Online]. Available: http://alpha.tmit.bme.hu/speech/speco/index. html, 09.1998-08.2001