Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech?

Language and Speech

Volumn 41, Issue 3-4, 1998, Pages 443-492

  Shriberg, Elizabeth ^a   Stolcke, Andreas ^a   Jurafsky, Daniel ^b   Coccaro, Noah ^b   Meteer, Marie ^c   Bates, Rebecca ^g   Taylor, Paul ^g   Ries, Klaus ^d   Martin, Rachel ^e   Van Ess Dykema, Carol ^f  

^a SRI INTERNATIONAL   (United States)

^b UNIVERSITY OF COLORADO   (United States)

^c BBN TECHNOLOGIES   (United States)

^d CARNEGIE MELLON UNIVERSITY   (United States)

^e JOHNS HOPKINS UNIVERSITY   (United States)

^f ARMED FORCES INSTITUTE OF PATHOLOGY   (United States)

^g NONE

Author keywords

Automatic dialog act classification; Discourse modeling; Prosody; Recognition; Speech; Spontaneous speech; Understanding

Indexed keywords

ARTICLE; DECISION TREE; HUMAN; LINGUISTICS; PHONETICS; SOUND DETECTION; SPEECH PERCEPTION; TELEPHONE; VERBAL BEHAVIOR;

DECISION TREES; HUMANS; PHONETICS; PSYCHOLINGUISTICS; SOUND SPECTROGRAPHY; SPEECH PERCEPTION; TELEPHONE; VERBAL BEHAVIOR;

EID: 0342310704     PISSN: 00238309     EISSN: None     Source Type: Journal    
DOI: 10.1177/002383099804100410     Document Type: Article

References (70)

1. AYERS, G. M. (1994). Discourse functions of pitch range in spontaneous and read speech. In Working Papers in Linguistics No. 44 (pp. 1-49). Ohio State University.
2. BENNACEF, S. K., NÉEL, F., & MAYNARD, H. B. (1995). An oral dialog model based on speech acts categorization. In P. Dalsgaard, E. Larsen, E. Boves, & I. Thomsen (Eds.), ESCA Workshop on Spoken Dialog Systems - Theories and applications (pp. 237-240). Vigsø, Denmark.
3. BLAAUW, E. (1995). On the perceptual classification of spontaneous and read speech. Doctoral dissertation, Utrecht University. Published by LEd, Utrecht, The Netherlands.
4. BREIMAN, L., FRIEDMAN, J. H., OLSHEN, R. A., & STONE, C. J. (1984). Classification and regression trees. Pacific Grove, CA: Wadsworth and Brooks.
5. BRIDLE, J. (1990). Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In F. Soulie & J. Herault (Eds.), Neurocomputing: Algorithms, architectures and applications (pp. 227-236). Berlin: Springer.
6. BRUCE, G., GRANSTRÖM, B., GUSTAFSON, K., HORNE, M., HOUSE, D., & TOUATI, P. (1997). On the analysis of prosody in interaction. In Y. Sagisaka, N. Campbell, & N. Higuchi (Eds.), Computing prosody: Computational models for processing spontaneous speech (pp. 42-59). New York: Springer.
7. CARLETTA, J. (1996). Assessing agreement on classification tasks: The Kappa statistic. Computational Linguistics, 22(2), 249-254.
8. CARLETTA, J., ISARD, A., ISARD, S., KOWTKO, J., DOHERTY-SNEDDON, G., & ANDERSON, A. H. (1995). The coding of dialog structure in a corpus. In J. Andernach, S. van de Burgt, & G. van der Hoeven (Eds.), Proceedings of the Ninth Twente Workshop on Language Technology: Corpus-based approaches to dialog modeling. Universiteit Twente, Enschede.
9. CHU-CARROLL, J. (1998). A statistical model for discourse act recognition in dialog interactions. In J. Chu-Carroll & N. Green (Eds.), Applying machine learning to discourse processing. Papers from the 1998 AAAI Spring Symposium. Technical Report SS-98-01 (pp. 12-17). Menlo Park, CA: AAAI Press.
10. CLARK, H. (1996). Using language. Cambridge, U.K.: Cambridge University Press.
11. COHEN, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement, 20, 37-46.
12. CORE, M., & ALLEN, J. (1997). Coding dialogs with the DAMSL annotation scheme. In AAAI Fall Symposium on Communicative Action in Humans and Machines. Cambridge, MA. (http://www.cs.rochester.edu/u/trains/annotation/)
13. DALY, N. A., & ZUE, V. W. (1992). Statistical and linguistic analyses of F0 in read and spontaneous speech. In J. J. Ohala, T. M. Nearey, B. L. Derwing, M. M. Hodge, & G. E. Wiebe (Eds.), Proceedings of the International Conference on Spoken Language Processing (Vol. 1, pp. 763-766). Banff, Canada.
14. FINKE, M., LAPATA, M., LAVIE, A., LEVIN, L., TOMOKIYO, L. M., POLZIN, T., RIES, K., WAIBEL, A., & ZECHNER, K. (1998). Clarity: Inferring discourse structure from speech. In J. Chu-Carroll & N. Green (Eds.), Applying machine learning to discourse processing. Papers from the 1998 AAAI Spring Symposium. Technical Report SS-98-01 (pp. 25-32). Menlo Park, CA: AAAI Press.
15. GODFREY, J., HOLLIMAN, E., & McDANIEL, J. (1992). SWITCHBOARD: Telephone speech corpus for research and development. Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing (pp. 517-520). San Francisco.
16. GROSZ, B., & HIRSCHBERG, J. (1992). Some intonational characteristics of discourse structure. In J. J. Ohala, T. M. Nearey, B. L. Derwing, M. M. Hodge, & G. E. Wiebe (Eds.), Proceedings of the International Conference on Spoken Language Processing (Vol. 1, pp. 429-432). Banff, Canada.
17. HAAN, J., HEUVEN, V. J. van, PACILLY, J. J. A., & BEZOOIJEN, R. van (1997a). Intonational characteristics of declarativity and interrogativity in Dutch: A comparison. In A. Botonis, G. Kouroupetroglou, & G. Carayiannis (Eds.), ESCA, Workshop on Intonation: Theory, Models and Applications (pp. 173-176). Athens, Greece.
18. HAAN, J., van HEUVEN, V. J., PACILLY, J. J. A., & van BEZOOIJEN, R. (1997b). An anatomy of Dutch question intonation. In H. de Hoop & M. den Dikken (Eds.). Linguistics in the Netherlands. Amsterdam: John Benjamins.
19. HIRSCHBERG, J., & NAKATANI, C. (1996). A prosodic analysis of discourse segments in direction-giving monologs. Proceedings of the 34th Annual Meeting of The Association for Computational Linguistics (pp. 286-293). Santa Cruz, CA.
20. HIRSCHBERG, J., & NAKATANI, C. H. (1998). Using machine learning to identify intonational segments. In J. Chu-Carroll & N. Green (Eds.), Applying machine learning to discourse processing. Papers from the 1998 AAAI Spring Symposium. Technical Report SS-98-01 (pp. 52-59). Menlo Park, CA: AAAI Press.
21. JORDAN, M. I., & JACOBS, R. A. (1993). Hierarchical mixtures of experts and the EM algorithm. Neural Computation, 6(2), 181-214.
22. JURAFSKY, D., BATES, R., COCCARO, N., MARTIN, R., METEER, M., RIES, K., SHRIBERG, E., STOLCKE, A., TAYLOR, P., & van ESS-DYKEMA, C. (1997a). Automatic detection of discourse structure for speech recognition and understanding. IEEE Workshop on Speech Recognition and Understanding. Santa Barbara, CA.
23. JURAFSKY, D., SHRIBERG, E., & BIASCA, D. (1997b). Switchboard-DAMSL Labeling Project Coder's Manual (Tech. Rep. No. 97-02). University of Colorado Institute of Cognitive Science. (http://stripe.colorado.edu/~jurafsky/manual.augustl.html)
24. JURAFSKY, D., SHRIBERG, E., FOX, B., & CURL, T. (1998a). Lexical, prosodic, and syntactic cues for dialog acts. Proceedings of ACL-COLING Workshop on Discourse Relations and Discourse Markers (pp. 114-120). Montreal.
25. JURAFSKY, D., BATES, R., COCCARO, N., MARTIN, R., METEER, M., RIES, K., SHRIBERG, E., STOLCKE, A., TAYLOR, P., & van ESS-DYKEMA, C. (1998b). Switchboard discourse language modeling project report (Research Note No. 30). Baltimore, MD: Center for Speech and Language Processing, Johns Hopkins University.
26. KATZ, S. M. (1987). Estimation of probabilities from sparse data for the language model component of a speech recognizer. IEEE Transactions on Acoustics, Speech, and Signal Processing, 35(3), 400-401.
27. KIESSLING, A., KOMPE, R., NIEMANN, H., NÖTH, E., & BATLINER, A. (1993). "Roger," "Sorry," "I'm still listening": Dialog guiding signals in informational retrieval dialogs. In D. House & P. Touati (Eds.), ESCA, Workshop on Prosody (pp. 140-143). Lund, Sweden.
28. KITA, K., FUKUI, Y., NAGATA, M., & MORIMOTO, T. (1996). Automatic acquisition of probabilistic dialog models. In H. T. Bunnell & W. Idsardi (Eds.), Proceedings of the International Conference on Spoken Language Processing (Vol. 1, pp. 196-199). Philadephia.
29. KOMPE, R. (1997). Prosody in speech understanding systems. Berlin: Springer.
30. KOMPE, R., KIESSLING, A., NIEMANN, H., NÖTH, E., SCHUKAT-TALAMAZZINI, E., ZOTTMANN, A., & BATLINER, A. (1995). Prosodic scoring of word hypotheses graphs. In J. M. Pardo, E. Enríquez, J. Ortega, J. Ferreiros, J. Macías, & F. J. Valverde (Eds.), Proceedings of the Fourth European Conference on Speech Communication and Technology (Vol. 2, pp. 1333-1336). Madrid.
31. KOOPMANS-van BEINUM, F. J., & van DONZEL, M. E. (1996). Relationship between discourse structure and dynamic speech rate. In H. T. Bunnell & W. Idsardi (Eds.), Proceedings of the International Conference on Spoken Language Processing (Vol. 3, pp. 1724-1727). Philadephia.
32. LEVELT, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.
33. LITMAN, D. J., & PASSONNEAU, R. J. (1995). Combining multiple knowledge sources for discourse segmentation. Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics (pp. 108-115). Cambridge, MA.
34. MAST, M., KOMPE, R., HARBECK, S., KIESSLING, A., NIEMANN, H., NÖTH, E., SCHUKATTALAMAZZINI, E. G., & WARNKE, V. (1996). Dialog act classification with the help of prosody. In H. T. Bunnell & W. Idsardi (Eds.), Proceedings of the International Conference on Spoken Language Processing (Vol. 3, pp. 1732-1735). Philadelphia.
35. METEER, M., TAYLOR, A., MacINTYRE, R., & RUKMINI, I (1995). Dysfluency Annotation Stylebook for the Switchboard Corpus. Linguistic Data Consortium. (Revised June 1995, by Ann Taylor.) 〈ftp://ftp.cis.upenn.edu/pub/treebank/swbd/doc/DFL-book.ps.gz〉
36. MORGAN, N., & FOSSLER-LUSSIER, E. (1998). Combining multiple estimators of speaking rate. Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing (Vol. 2, pp. 729-732). Seattle, Washington.
37. MORGAN, N., FOSLER, E., & MIRGHAFORI, N. (1997). Speech recognition using on-line estimation of speaking rate. In G. Kokkinakis, N. Fakotakis, & E. Dermatas (Eds.), Proceedings of the Fifth European Conference on Speech Communication and Technology (Vol. 4, pp. 2079-2082). Rhodes, Greece.
38. NAGATA, M., & MORIMOTO, T. (1994). First steps toward statistical modeling of dialog to predict the speech act type of the next utterance. Speech Communication, 15, 193-203.
39. NAKAJIMA, S., & ALLEN, J. F. (1993). A study on prosody and discourse structure in cooperative dialogs. Phonetica, 50, 197-210.
40. NAKAJIMA, S., & TSUKADA, H. (1997). Prosodic features of utterances in task-oriented dialogs. In Y. Sagisaka, N. Campbell, & N. Higuchi (Eds.), Computing prosody: Computational models for processing spontaneous speech (pp. 81-94). New York: Springer.
41. NEUMEYER, L., & WEINTRAUB, M. (1994). Microphone-independent robust signal processing using probabilistic optimum filtering. In Proceedings of the ARPA Workshop on Human Language Technology (pp. 336-341). Plainsboro, NJ.
42. NEUMEYER, L., & WEINTRAUB, M. (1995). Robust speech recognition in noise using adaptation and mapping techniques. Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing. (Vol. 1, pp. 141-144). Detroit.
43. NÖTH, E. (1991). Prosodische Information in der automatischen Spracherkennung - Berechnung und Anwendung. Tübingen: Niemeyer.
44. OSTENDORF, M., & ROSS, K. (1997). Multilevel recognition of intonation labels. In Y. Sagisaka, N. Campbell, & N. Higuchi (Eds.), Computing prosody: Computational models for processing spontaneous speech (pp. 291-308). New York: Springer.
45. QUIRK, R., GREENBAUM, S., LEECH, G., & SVARTVIK, J. (1985). A comprehensive grammar of the English language. London: Longman.
46. REITHINGER, N., ENGEL, R., KIPP, M., & KLESEN, M. (1996). Predicting dialog acts for a speech-to-speech translaton system. In H. T. Bunnell & W. Idsardi (Eds.), Proceedings of the International Conference on Spoken Language Processing (Vol.2, pp. 654-657). Philadephia.
47. REITHINGER, N., & KLESEN, M. (1997). Dialog act classification using language models. In G. Kokkinakis, N. Fakotakis, & E. Dermatas (Eds.), Proceedings of the Fifth European Conference on Speech Communication and Technology (Vol. 4, pp. 2235-2238). Rhodes, Greece.
48. REITHINGER, N., & MAIER, E. (1995). Utilizing statistical dialogue act processing in Verbmobil. Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics (pp. 116-121). Cambridge, MA.
49. SAMUEL, K., CARBERRY, S., & VIJAY-SHANKER, K. (1998). Computing dialog acts from features with transformation-based learning. In J. Chu-Carroll & N. Green (Eds.), Applying machine learning to discourse processing. Papers from the 1998 AAAI Spring Symposium. Technical Report SS-98-01 (pp. 90-97). Menlo Park, CA: AAAI Press.
50. SHRIBERG, E., BATES, R., & STOLCKE, A. (1997). A prosody-only decision-tree model for disfluency detection. In G. Kokkinakis, N. Fakotakis, & E. Dermatas (Eds.), Proceedings of the Fifth European Conference on Speech Communication and Technology (Vol. 5, pp. 2383-2386). Rhodes, Greece.
51. SIEGEL, S., & CASTELLAN, N. J., Jr. (1988). Nonparametric statistics for the behavioral sciences (Second ed). New York: McGraw-Hill.
52. STOLCKE, A., & SHRIBERG, E. (1996). Automatic linguistic segmentation of conversational speech. In H. T. Bunnell & W. Idsardi (Eds.), Proceedings of the International Conference on Spoken Language Processing (Vol. 2, pp. 1005-1008). Philadelphia.
53. STOLCKE, A., SHRIBERG, E., BATES, R., COCCARO, N., JURAFSKY, D., MARTIN, R., METEER, M., RIES, K., TAYLOR, P., & van ESS-DYKEMA, C. (1998). Dialog act modeling for conversational speech. In J. Chu-Carroll & N. Green (Eds.), Applying Machine Learning to Discourse Processing. Papers from the 1998 AAAI Spring Symposium. Technical Report SS-98-01 (pp. 98-105). Menlo Park, CA: AAAI Press.
54. SUHM, B., & WAIBEL, A. (1994). Toward better language models for spontaneous speech. Proceedings of the International Conference on Spoken Language Processing (Vol. 2, pp. 831-834). Yokohama.
55. SWERTS, M. (1997). Prosodic features at discourse boundaries of different strength. Journal of the Acoustical Society of America, 101, 514-521.
56. SWERTS, M., & OSTENDORF, M. (1997). Prosodic and lexical indications of discourse structure in human-machine interactions. Speech Communication, 22(1), 25-41.
57. TAYLOR, P. A., & BLACK, A. W. (1994). Synthesizing conversational intonation from a linguistically rich input. Proceedings of the second ESCA/IEEE Workshop on Speech Synthesis (pp. 175-178). New York.
58. TAYLOR, P. A., KING, S., ISARD, S. D., WRIGHT, H., & KOWTKO, J. (1997). Using intonation to constrain language models in speech recognition. In G. Kokkinakis, N. Fakotakis, & E. Dermatas (Eds.), Proceedings of the Fifth European Conference on Speed, Communication and Technology (Vol. 5, pp. 2763-2766). Rhodes, Greece.
59. TAYLOR, P. A., KING, S., ISARD, S. D., WRIGHT, H. (1998). Intonation and dialog context as constraints for speech recognition. Language and Speech, 41, 493-512.
60. TERRY, M., SPARKS, R., & OBENCHAIN, P. (1994). Automated query identification in English dialogue. Proceedings of the International Conference on Spoken Language Processing (Vol. 2, pp. 891-894). Yokohama.
61. TRAUM, D., & HEEMAN, P. (1996). Utterance units and grounding in spoken dialogue. In H. T. Bunnell & W. Idsardi (Eds.), Proceedings of the International Conference on Spoken Language Processing (Vol. 3, pp. 1884-1887). Philadephia.
62. VAISSIÈRE, J. (1983). Language-independent prosodic features. In A. Cutler & D. R. Ladd (Eds.), Prosody: Models and measurements (pp. 53-66). Berlin: Springer.
63. VEIEEEUX, N. M., & OSTENDORF, M. (1993). Prosody/parse scoring and its application in ATIS. Proceedings of the ARPA Workshop on Human Language Technology (pp. 335-340). Princeton, NJ.
64. WAIBEE, A. (1988). Prosody and speech recognition. San Mateo, CA: Morgan Kaufmann.
65. WARNKE, V., KOMPE, R., NIEMANN, H., & NÖTH, E. (1997). Integrated dialog act segmentation and classification using prosodic features and language models. In G. Kokkinakis, N. Fakotakis, & E. Dermatas (Eds.), Proceedings of the Fifth European Conference on Speech Communication and Technology (Vol. 1, pp. 207-210). Rhodes, Greece.
66. WEBER, E. (1993). Varieties of questions in English conversation. Amsterdam: John Benjamins.
67. WIGHTMAN, C. W., & OSTENDORF, M. (1994). Automatic labeling of prosodic patterns. IEEE Transactions on Speech and Audio Processing, 2, 469-481.
68. WOSZCZYNA, M., & WAIBEL, A. (1994). Inferring linguistic structure in spoken language. Proceedings of the International Conference on Spoken Language Processing (Vol. 2, pp. 847-850). Yokohama.
69. YAMAOKA, T., & IIDA, H. (1991). Dialog interpretation model and its application to next utterance predication for spoken language processing. Proceedings of the Second European Conference on Speech Communication and Technology (Vol 2, pp. 849-852). Geneva, Italy.
70. YOSHIMURA, T., HAYAMIZU, S., OHMURA, H., & TANAKA, K. (1996). Pitch pattern clustering of user utterances in human-machine dialogue. In H. T. Bunnell & W. Idsardi (Eds.), Proceedings of the International Conference on Spoken Language Processing (pp. 837-840). Philadephia.