Detecting emotional state of a child in a conversational computer game

Computer Speech and Language

Volumn 25, Issue 1, 2011, Pages 29-44

  Yildirim, Serdar ^a   Narayanan, Shrikanth ^b   Potamianos, Alexandros ^c  

^a MUSTAFA KEMAL UNIVERSITY   (Turkey)

^b University of Southern California   (United States)

^c TECHNICAL UNIVERSITY OF CRETE   (Greece)

Author keywords

Child computer interaction; Children speech; Emotion recognition; Feature extraction; Natural emotions; Spoken dialog systems; Spontaneous speech

Indexed keywords

ACOUSTIC FEATURES; AGE GROUPS; AUTOMATIC RECOGNITION; CHILDREN SPEECH; CLASSIFICATION ACCURACY; CLASSIFICATION PERFORMANCE; CLASSIFICATION RESULTS; COMPUTER GAME; COMPUTER INTERACTION; CONTEXTUAL CUE; CONTEXTUAL FEATURE; CONTEXTUAL INFORMATION; DETECTION TASKS; DIALOG SYSTEMS; EMOTIONAL STATE; INFORMATION SOURCES; LEXICAL INFORMATION; NATURAL INTERACTIONS; PAIRWISE CLASSIFICATION; SPOKEN DIALOG SYSTEMS; SPONTANEOUS SPEECH; WIZARD OF OZ;

COMPUTER SOFTWARE; FEATURE EXTRACTION; SPEECH COMMUNICATION;

SPEECH RECOGNITION;

EID: 77955414344     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2009.12.004     Document Type: Article

References (40)

1. E. Andersen, M. Brizuela, B. DuPuy, and L. Gonnerman Cross-linguistic evidence for the early acquisition of discourse markers as register variable Journal of Pragmatics 31 1999 1339 1351
2. Ang, J., Dhillon, R., Krupski, A., Shriberg, E., Stolcke, A., 2002. Prosody-based automatic detection of annoyance and frustration in human-computer dialog. In: Proceedings of ICSLP, Denver, pp. 2037-2039.
3. Arunachalam, S., Gould, D., Andersen, E., Byrd, D., Narayanan, S., 2001. Politness and frustration language in child-computer interactions. In: Proceedings of Eurospeech, Aalborg, Denmark, pp. 2675-2678.
4. Batliner, A., Blomberg, M., D'Arcy, S., Elenius, D., Giuliani, D., Gerosa, M., Hacker, C., Russell, M., Steidl, S., Wong, M., 2005. Children's speech recognition with application to interactive book and tutors. In: Proceedings of InterSpeech, pp. 2761-2764.
5. Batliner, A., Steidl, S., Schuller, B., Seppi, D., Laskowski, K., Vogt, T., Devillers, L., Vidrascu, L., Amir, N., Kessous, L., Aharonson, V., 2006. Combining efforts for improving automatic classification of emotional user states. In: Erjavec, T., Gros, J.Z. (Eds.), Language Technologies, IS-LTC 2006. Infornacijska Druzba (Information Society), Ljubljana, Slovenia, pp. 240-245.
6. Bell, L., 2003. Linguistic adaptations in spoken humancomputer dialogues - empirical studies of user behavior. Ph.D. thesis, KTH, Sweden.
7. Bell, L., Boye, J., Gustafson, J., Heldner, M., Lindstrom, A., Wiren, M., 2005. The Swedish NICE Corpus Spoken dialogues between children and embodied characters in a computer game scenario. In: Proceedings of InterSpeech, Lisbon, Portugal, pp. 2765-2768.
8. Z. Callejas, and R. Lopez-Cozar Influence of contextual information in emotion annotation for spoken dialogue systems Speech Communication 50 5 2008 416 433
9. Z. Chair, and P. Varshney Optimal data fusion in multiple sensor detection systems AES-22 1 1986 98 101
10. Chang, C.-C., Lin, C.-J., 2001. LIBSVM: a library for support vector machines. .
11. N.V. Chawla, K.W. Bowyer, and P.W. Kegelmeyer Smote: synthetic minority over-sampling technique Journal of Artificial Intelligence Research 16 2002 321 357
12. J. Chu-Carroll, and B. Carpenter Vector-based natural language call routing Computational Linguistics 25 3 1999 361 388
13. T. Cover, and J. Thomas Elements of Information Theory 1991 John Wiley & Sons New York, NY
14. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. Taylor Emotion recognition in human-computer interaction IEEE Signal Processing Magazine 18 1 2001 3280
15. S.C. Deerwester, S.T. Dumais, T.K. Landauer, G.W. Furnas, and R.A. Harshman Indexing by latent semantic analysis Journal of the American Society of Information Science 41 6 1990 391 407
16. M. Eskarnazi Kids: a database of children's speech Journal of the Acoustical Society of America 100 4 1996 2759
17. Eyben, F., Wöllmer, M., Schuller, B., 2009. openEAR - introducing the Munich open-source emotion and affect recognition toolkit. In: 4th International HUMAINE Association Conference on Affective Computing and Intelligent Interaction (ACII).
18. Farantouri, V., Potamianos, A., Narayanan, S., 2008. Linguistic analysis of spontaneous children speech. In: Berkling, K., Giuliani, D., Potamianos, A. (Eds.), Proceedings of the 1st Workshop on Child, Computer and Interaction, Chania, Greece.
19. A. Gorin On automated language acquisition J. Acoust. Soc. Am. 97 6 1995 3441 3461
20. J. Gustafson, and L. Bell Speech technology on trial - experiences from the August system Natural Language Engineering 6 3-4 2000 273 286
21. Hagen, A., Pellom, B., Cole, R., 2003. Children's speech recognition with application to interactive book and tutors. In: Proceedings of the ASRU Workshop, pp. 186-191.
22. A. Kapoor, W. Burleson, and R.W. Picard Automatic prediction of frustration International Journal of Human-Computer Studies 65 2007 724 736
23. T.K. Landauer, P.W. Foltz, and D. Laham Introduction to latent semantic analysis Discourse Processes 25 1998 259 284
24. Lee, C.-C., Mower, E., Busso, C., Lee, S., Narayanan, S., 2009. Emotion recognition using a hierarchical binary decision tree approach. In: Proceedings of InterSpeech, Brighton, UK, pp. 320-323.
25. C.M. Lee, and S. Narayanan Towards detecting emotions in spoken dialogs IEEE Transactions on Speech and Audio Processing 13 2 2005 293 302
26. S. Lee, A. Potamianos, and S. Narayanan Acoustics of children's speech: developmental changes of temporal and spectral parameters Journal of the Acoustical Society of America 1999 1455 1468
27. Liscombe, J., Riccardi, G., Hakkani-Tur, D., 2005. Using context to improve emotion detection in spoken dialog systems. In: Proceedings of InterSpeech, Lisbon, Portugal, pp. 1845-1848.
28. Litman, D.J., Forbes-Riley, K., 2004. Predicting student emotions in computer-human tutoring dialogues. In: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL), Barcelona, Spain, p. 351.
29. C. Manning, P. Raghavan, and H. Schutze Introduction to Information Retrieval 2008 Cambridge University Press Cambridge
30. D. Morrison, R. Wang, and L.C.D. Silva Ensemble methods for spoken emotion recognition in call-centers Speech Communication 49 5 2007 98 112
31. S. Narayanan, and A. Potamianos Creating conversational interface for children IEEE Transactions on Speech and Audio Processing 10 2 2002 65 78
32. Neiberg, D., Elenius, K., 2008. Automatic recognition of anger in spontaneous speech. In: Proceedings of InterSpeech, ISCA, pp. 2755-2758.
33. T.L. Nwe, S.W. Foo, and L.C.D. Silva Speech emotion recognition using hidden Markov models Speech Communication 41 4 2003 603 623
34. Potamianos, A., Narayanan, S., 1998. Spoken dialogue systems for children. In: Proceedings of ICASSP, Seattle, WA, pp. 197-201.
35. Schuller, B., Batliner, A., Seppi, D., Steidl, S., 2009a. Emotion recognition from speech: putting ASR in the loop. In: ICASSP (Ed.), IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4585-4588.
36. Schuller, B., Batliner, A., Seppi, D., Steidl, S., Vogt, T., Wagner, J., Devillers, L., Vidrascu, L., Amir, N., Kessous, L., 2007a. The relevance of feature type for automatic classification of emotional user states: low level descriptors and functionals. In: Proceedings of InterSpeech, pp. 2253-2256.
37. Schuller, B., Seppi, D., Batliner, A., Maier, A., Steidl, S., 2007b. Towards more reality in the recognition of emotional speech. In: Proceedings of ICASSP, vol. 4, pp. 941-944.
38. Schuller, B., Steidl, S., Batliner, A., 2009b. The InterSpeech 2009 emotion challenge. In: ISCA (Ed.), Proceedings of InterSpeech, pp. 312-315.
39. Seppi, D., Gerosa, M., Schuller, B., Batliner, A., Steidl, S., 2008. Detecting problems in spoken child-computer-interaction. In: Berkling, K., Giuliani, D., Potamianos, A. (Eds.), Proceedings of the 1st Workshop on Child, Computer and Interaction.
40. T. Zhang, M. Hasegawa-Johnson, and S.E. Levinson Cognitive state classification in a spoken tutorial dialogue system Speech Communication 48 2006 616 632