Cognitive state classification in a spoken tutorial dialogue system

Speech Communication

Volumn 48, Issue 6, 2006, Pages 616-632

  Zhang, Tong ^a   Hasegawa Johnson, Mark ^a   Levinson, Stephen E ^a  

^a University of Illinois at Urbana Champaign   (United States)

Author keywords

Intelligent tutoring system; Spoken language processing; User affect recognition

Indexed keywords

AUTOMATIC PROGRAMMING; CLASSIFICATION (OF INFORMATION); COGNITIVE SYSTEMS; COMPUTER SIMULATION; SPEECH ANALYSIS; STATE SPACE METHODS; STUDENTS;

INTELLIGENT TUTORING SYSTEM; SPOKEN LANGUAGE PROCESSING; USER AFFECT RECOGNITION;

INFORMATION RETRIEVAL SYSTEMS;

EID: 33646257071     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2005.09.006     Document Type: Article

References (50)

1. Alpert, S.R., Singley, M.K., Carroll, J.M., 1999. Multiple instructional agents in an intelligent tutoring system. In: Internat. Workshop on Instructional Uses of Animated and Personified Agents (at the 9th Internat. Conf. on AI in Education), Le Mans, France.
2. Ang, J., Dhillon, R., Krupski, A., Shriberg, E., Stolcke, A., 2002. Prosody-based automatic detection of annoyance and frustration in human-computer dialog. In: Proc. of Internat. Conf. on Spoken Language Processing, Denver, CO.
3. Batliner A., Fischer K., Huber R., Spilker J., and Nöth E. How to find trouble in communication. Speech Commun. 40 (2003) 117-143
4. Batliner A., Huber R., Niemann H., Nöth E., Spilker J., and Fischer K. The recognition of emotion. In: Wahlster W. (Ed). Verbmobil: Foundations of Speech-to-Speech Translations (2000), Springer, New York, Berlin
5. Beck, J., Jia, P., Mostow, J., 2003. Assessing student proficiency in a reading tutor that Listens. In: Proc. 9th Internat. Conf. on User Modelling, Johnstown, PA.
6. Clark, B., Fry, J., Ginzton, M., Peters, S., Pon-Barry, H., Thomsen-Gray, Z., 2001. A multimodal intelligent tutoring system for shipboard damage control. In: Proc. 2001 Internat. Workshop on Information Presentation and Multimodal Dialogue, Verona, Italy.
7. Cole, R., van Vuuren, S., Pellom, B., Hacioglu, K., Ma, J., Movellan, J., Schwartz, S., Wade-Stein, D., Ward, W., Yan, J., 2003. Perceptive animated interfaces: first steps toward a new paradigm for human-computer interaction. In: Proc. IEEE: Special Issue on Multimodal Human Computer Interface.
8. Corbett A.T., and Anderson J.R. The Lisp intelligent tutoring system: research in skill acquisition. In: Larkin J., and Chabay R. (Eds). Computer Assisted Instruction and Intelligent Tutoring System: Shared Goals and Complementary Approaches (1992), Lawrence Eflbaum, Hillsdale, NJ
9. Crowley, R., Medvedeva, O., Jukic, D., 2003. SlideTutor: a model-tracing intelligent tutoring system for teaching microscopic diagnosis. In: Proc. 11th Internat. Conf. on Artificial Intelligence in Education, Sydney, Australia.
10. Fernandez R., and Picard R.W. Modeling driver's speech under stress. Speech Commun. 40 (2003) 145-159
11. Flammia, G., 1998. Discourse segmentation of spoken dialogue: an empirical approach. Ph.D. Thesis. MIT.
12. Forbes-Riley, K., Litman, D., 2004. Predicting emotion in spoken dialogue from multiple knowledge sources. In: Proc. Human Language Technology Conference/North American Chapter of the Association for Computational Linguistics Annual Meeting, Boston, MA.
13. Gobl C., and Chasaide A.N. The role of voice quality in communicating emotion, mood and attitude. Speech Commun. 40 (2003) 189-212
14. Godfrey, J.J., Holliman, E.C., McDaniel, J., 1991. SWITCHBOARD: telephone speech corpus for research and development. In: Proc. IEEE Internat. Conf. on Acoustics, Speech, and Signal Processing, Toronto, Canada.
15. Graesser A.C., Lehn K.V., Rose C.P., Jordan P.W., and Harter D. Intelligent tutoring system with conversational dialogue. AI Mag. 22 4 (2001) 39-52
16. Juang B.-H., and Furui S. Automatic recognition and understanding of spoken language-a first step towards natural human-machine communication. Proc. IEEE 88 8 (2000) 1142-1165
17. Jurafsky, D., Bates, R., Coccaro, N., Martin, R., Meteer, M., Ries, K., Shriberg, E., Stolcke, A., Taylor, P., Ess-Dykema, C.V., 1997. Switchboard discourse language modeling project final report. Johns Hopkins LVCSR Workshop, Baltimore, MD.
18. Kafai Y., and Resnick M. Constructionism in Practice: Designing, Thinking and Learning in a Digital World (1996), Lawrence Erlbaum Associates, Mahwah, NJ
19. Kang, B.-S., Han, C.-H., Lee, S.-T., Youn, D.-H., Lee, C., 2000. Speaker dependent emotion recognition using speech signals. In: Proc. Internat. Conf. on Spoken Language Processing, Beijing, China.
20. Katz S.M. Estimation of probabilities from sparse data for the language model component of a speech recognizer. IEEE Trans. Acoust. Speech Signal Process. 35 3 (1987) 400-401
21. Kitazawa, S., Ichikawa, H., Kobayashi, S., Nishinuma, Y., 1997. Extraction and representation rhythmic components of spontaneous speech. In: Proc. European Conf. on Speech Communication and Technology, Rhodes, Greece.
22. Lee C.M., and Narayanan S. Toward detecting emotions in spoken dialogs. IEEE Trans. Speech Audio Process. 13 2 (2005) 293-303
23. Lee L., and Rose R. A frequency warping approach to speaker normalization. IEEE Trans. Speech Audio Process. 6 1 (1998) 49-59
24. Lee S., Potamianos A., and Narayanan S. Acoustics of children's speech: developmental changes of temporal and spectral parameters. J. Acoust. Soc. Amer. 105 3 (1999) 1455-1468
25. Lesgold A., Lajoie S., Bunzo M., and Eggan G. Sherlock: a coached practice environment for an electronics troubleshooting job. In: Larkin J., Chabay R., and Sheftic C. (Eds). Computer Assisted Instruction and Intelligent Tutoring Systems Establishing Communication and Collaboration (1990), Erlbaum, Hillsdale, NJ
26. Li Q., Zheng J., Tsai A., and Zhou Q. Robust endpoint detection and energy normalization for real-time speech and speaker recognition. IEEE Trans. Speech Audio Process. 10 3 (2002) 146-157
27. Lieberman P., and Michaels S.B. Some aspects of fundamental frequency and envelope amplitude as related to the emotional content of speech. J. Acoust. Soc. Amer. 34 (1962) 922-927
28. Litman, D., Rose, C., Forbes-Riley, K., VanLehn, K., Bhembe, D., Silliman, S., 2004. Spoken versus typed human and computer dialogue tutoring. In: Proc. 7th Internat. Conf. on Intelligent Tutoring Systems, Maceió, Brazil.
29. Litman, D., Silliman, S., 2004. ITSPOKE: an intelligent tutoring spoken dialogue system. In: Proc. Human Language Technology Conference/North American Chapter of the Association for Computational Linguistics Annual Meeting, Boston, MA.
30. Martinovsky, B., Traum, D., 2003. Breakdown in human-machine interaction: the error is the cue. In: Proc. ISCA Tutorial and Research Workshop on Error Handling in Dialogue Systems, Château-d'Oex, Switzerland.
31. Morgan, N., Fosler-Lussier, E., 1998. Combining multiple estimators of speaking rate. In: Proc. IEEE Internat. Conf. on Acoustics, Speech, and Signal Processing, Seattle, WA.
32. Mostow, J., Beck, J., Winter, S.V., Wang, S., Tobin, B., 2002. Predicting oral reading miscues. In: Proc. Internat. Conf. on Spoken Language Processing, Denver, CO.
33. Mozziconacci, S., Hermes, D., 1998. Study of intonation patterns in speech expressing emotion or attitude: production and perception. IPO Annual Progress Report. IPO, Eindhoven.
34. Munoz, M., Punyakanok, V., Roth, D., Zimak, D., 1999. A learning approach to shallow parsing. In: Proc. Conf. on Empirical Methods in Natural Language Processing and Very Large Corpora, College Park, MD.
35. Narayanan S., and Potamianos A. Creating conversational interfaces for children. IEEE Trans. Speech Audio Process. 10 2 (2002) 65-78
36. Pellom, B., Ward, W., Pradhan, S., 2000. The CU communicator: an architecture for dialogue systems. In: Proc. Internat. Conf. on Spoken Language Processing, Beijing, China.
37. Petrushin V. Emotion in speech: recognition and application to call centers. In: Dagli C., Buczak A., Ghosh J., Embrechts M.J., and Ersoy O. (Eds). Intelligent Engineering Systems Through Artificial Neural Networks vol. 9 (1999) 1085-1092
38. Petrushin, V.A., 2000. Emotion recognition in speech signal: experimental study, development, and application. In: Proc. Internat. Conf. on Spoken Language Processing, Beijing, China.
39. Polzin, T.S., Waibel, A., 2000. Emotion-sensitive human-computer interfaces. In: ISCA Workshop on Speech and Emotion: a Computational Framework for Research, Newcastle, Northern Ireland.
40. Pon-Berry, H., Clark, B., Bratt, E.O., Schultz, K., Peters, S., 2004. Evaluating the effectiveness of SCoT: a spoken conversational tutor. In: Proc. ITS 2004 Workshop on Dialogue-based Intelligent Tutoring Systems, Maceio, Brazil.
41. Potamianos A., and Narayanan S. Robust recognition of children's speech. IEEE Trans. Speech Audio Process. 11 (2003) 603-616
42. Reyes, R.L., Galvey, C., Gocolay, M.C., Ordona, E., Ruiz, C., 2000. Multimedia intelligent tutoring system for context-free grammar. In: Proc. Philippine Computing Science Congress, Manila, Philippines.
43. Rulequest Research, 2004. Data mining tools. Available from: .
44. Schultz, K., Bratt, E.O., Clark, B., Peters, S., Pon-Barry, H., Treeratpituk, P., 2003. A scalable, reusable spoken conversational tutor: SCoT. In: AIED 2003 Supplementary Proceedings, Sydney, Australia.
45. Steele M.M., and Steele J.W. Discover: an intelligent tutoring system for teaching students with learning difficulties for solve word problems. J. Comput. Math. Sci. Teaching 18 4 (1999) 351-359
46. Ward, W., Pellom, B., 1999. The CU communicator system. In: IEEE Workshop on Automatic Speech Recognition and Understanding, Keystone, CO.
47. Wilensky U. Abstract meditations on the concrete. In: Harel I., and Papert S. (Eds). Constructionism (1991), Ablex, Norwood, NJ
48. Young, S., Kershaw, D., Odell, J., Ollason, D., Valtchev, V., Woodland, P., 2000. The HTK book. Available from: .
49. Zhan, P., Westphal, M., 1997. Speaker normalization based on frequency warping. In: Proc. IEEE Internat. Conf. on Acoustics, Speech, and Signal Processing, Munich, Germany.
50. Zue V., Seneff S., Glass J., Polifroni J., Pao C., Hazen T., and Hetherington L. Jupiter: a telephone-based conversational interface for weather information. IEEE Trans. Speech Audio Process. 8 (2000) 85-96