Enriching speech recognition with automatic detection of sentence boundaries and disfluencies

IEEE Transactions on Audio, Speech and Language Processing

Volumn 14, Issue 5, 2006, Pages 1526-1539

  Liu, Yang ^a,b   Shriberg, Elizabeth ^c,d   Stolcke, Andreas ^a,c,d   Hillard, Dustin ^e   Ostendorf, Mari ^e   Harper, Mary ^f,g  

^a IEEE   (United States)

^b The University of Texas at Dallas   (United States)

^c SRI INTERNATIONAL   (United States)

^d INTERNATIONAL COMPUTER SCIENCE INSTITUTE   (United States)

^e UNIVERSITY OF WASHINGTON   (United States)

^f UNIVERSITY OF MARYLAND   (United States)

^g Purdue University   (United States)

Author keywords

Conditional random field; Confusion network; Disfluency; Maximum entropy; Metadata extraction; Prosody; Punctuation; Rich transcription; Sentence boundary

Indexed keywords

CONDITIONAL RANDOM FIELDS; CONFUSION NETWORKS; MAXIMUM ENTROPY; METADATA EXTRACTION; PROSODY; PUNCTUATION; RICH TRANSCRIPTION; SENTENCE BOUNDARY;

BROADCASTING; HIDDEN MARKOV MODELS; INFORMATION ANALYSIS; KNOWLEDGE ACQUISITION; METADATA; WORD PROCESSING;

SPEECH RECOGNITION;

EID: 34047266607     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2006.878255     Document Type: Article

References (69)

1. D. Jones, F. Wolf, E. Gibson, E. Williams, E. Fedorenko, D. Reynolds, and M. Zissman, "Measuring the readability of automatic speech-to-text transcripts," in Proc. Eurospeech, 2003, pp. 1585-1588.
2. P. Heeman and J. Allen, "Speech repairs, intonational phrases and discourse markers: Modeling speakers' utterances in spoken dialogue," Comput. Ling., vol. 25, pp. 527-571, 1999.
3. J. Kim and P. C. Woodland, "The use of prosody in a combined system for punctuation generation and speech recognition," in Proc. Eurospeech, 2001, pp. 2757-2760.
4. Y. Gotoh and S. Renais, "Sentence boundary detection in broadcast speech transcripts," in Proc. ISCA Workshop: Automatic Speech Recognition: Challenges for the New Millennium ASR-2000, 2000, pp. 228-235.
5. R. Kompe, Prosody in Speech Understanding System. New York: Springer-Verlag, 1996.
6. M. Snover, B. Dorr, and R. Schwartz, "A lexically-driven algorithm for disfluency detection," in Proc. HLT/NAACL, 2004, pp. 157-160.
7. J. Kim, "Automatic detection of sentence boundaries, disflucncies, and conversational fillers in spontaneous speech," M.S. thesis, Univ. Washington, Seattle, 2004.
8. M. Johnson and E. Charniak, "A TAG-based noisy channel model of speech repairs," in Proc. ACL, 2004, pp. 33-39.
9. M. Gregory, M. Johnson, and E. Charniak, "Sentence-internal prosody does not help parsing the way punctuation does," in Proc. HLT/NAACL, 2004, pp. 81-88.
10. J. G. Kahn, M. Ostendorf, and C. Chelba, "Parsing conversational speech using enhanced segmentation," in Proc. HLT/NAACL, 2004, pp. 121-128.
11. M. Harper, B. Dorr, B. Roark, J. Hale, Z. Shafran, Y. Liu, M. Lease, M. Snover, L. Young, R. Stewart, and A. Krasnyanskaya. (2005) Final report: parsing speech and structural event detection. [Online] Available: http://www.clsp.jhu. edu/ws2005/groups/eventdetect/documents/final-report.pdf
12. S. Coquoz, "Broadcast News Segmentation Using MDE and STT Information to Improve Speech Recognition," Int. Computer Sci. Inst., Berkeley, CA, Tech. Rep., 2004.
13. S. Strassel, "Simple metadata annotation specification V6.2," LDC, [Online]. Available: http://www.ldc.upenn.edu/Projects/MDE/Guide-Iines/ SimpleMDE_V6.2.pdf, 2004.
14. Nat. Inst. Stand. Technol., Gaithersburg, MD. (2004). [Online] Available: http://www.nist.gov/speech/tests/rt/rt2004/fall/
15. _, (2000) Significance Tests for ASR. [Online] Available: http://www.nist.gov/speech/tests/sigtests/sigtests.htm
16. E. Shriberg, A. Stolcke, D. Hakkani-Tur, and G. Tur, "Prosody-based automatic segmentation of speech into sentences and topics," Speech Commun., pp. 127-154, 2000.
17. H. Christensen, Y. Gotoh, and S. Renal, "Punctuation annotation using statistical prosody models," in ISCA Workshop on Prosody in Speech Recognition and Understanding, 2001, pp. 35-40.
18. A. Stolcke, E. Shriberg, R. Bates, M. Ostendorf, D. Hakkani, M. Plauche, G. Tur, and Y. Lu, "Automatic detection of sentence boundaries and disiluencies based on recognized words," in Proc. ICSLP, 1998, pp. 2247-2250.
19. A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. V. Ess-Dykema, and M. Meteer, "Dialogue act modeling for automatic tagging and recognition of conversational speech," Comput. Ling., vol. 26, pp. 339-373, 2000.
20. J. Ang, R. Dhilon, A. Krupski, E. Shriberg, and A. Stolcke, "Prosody-based automatic detection of annoyance and frustration in human-computer dialog," in Proc. ICSLP, 2002, pp. 2037-2040.
21. D. Wang and S. S. Narayanan, "A multi-pass linear fold algorithm for sentence boundary detection using prosodic cues," in Proc. ICASSP, 2004, pp. 525-528.
22. J. Huang and G. Zweig, "Maximum entropy model for punctuation annotation from speech," in Proc. ICSLP, 2002, pp. 917-920.
23. C. Zong and F. Ren, "Chinese utterance segmentation in spoken language translation," in Proc. 4th Int. Conf. Comput. Ling. Intell. Text Process., 2003, pp. 516-525.
24. J. Kolar, J. Svec, and J. Psutka, "Automatic punctuation annotation in czech broadcast news speech," in Proc. 9th Conf. Speech Comput., 2004, pp. 319-325.
25. E. Charniak and M. Johnson, "Edit detection and parsing for transcribed speech," in Proc. NAACL, 2001, pp. 118-126.
26. M. Honal and T. Schultz, "Automatic disfluency removal on recognized spontaneous speech - Rapid adaptation to speaker dependent disfluencies," in Proc. ICASSP, 2005, pp. 969-972.
27. _, "Corrections of disfluencies in spontaneous speech using a noisy-channel approach," in Proc. Eurospeech, 2003, pp. 2781-2784.
28. C. Nakatani and J. Hirschberg, "A corpus-based study of repair cues in spontaneous speech," J. Acoust. Soc. Amer., pp. 1603-1616, 1994.
29. E. Shriberg, "Phonetic consequences of speech disfluency," in Proc. Int. Conf. Phonetics Sci., 1999, pp. 619-622.
30. R. Lickley, "Juncture cues to disfluency," in Proc. ICSLP, 1996, pp. 2478-2481.
31. G. Savova and J. Bachenko, "Prosodic features of four types of disfluencies," in Proc. DiSS, 2003, pp. 91-94.
32. E. Shriberg and A. Stolcke, "A prosody-only decision-tree model for disfluency detection," in Proc. Eurospeech, 1997, pp. 2383-2386.
33. J. Kim, S. Schwarm, and M. Ostendorf, "Detecting structural meta-data with decision trees and transformation-based learning," in Proc. HLT/NAACL, 2004, pp. 137-144.
34. B. Kingsbury, L. Mangu, D. Povey, G. Saon, H. Soltau, G. Zweig, A. Stolcke, R. Gadde, W. Wang, J. Zheng, B. Chen, and Q. Zhu, "The bicoastal IBM/SRI CTS STT system," in Rich Transcription (RT-04F) Workshop, Nov. 2004.
35. P. Woodland, H. Chan, G. Evermann, M. Gales, D. Kim, X. Liu, D. Mrva, K. Sim, L. Wang, K. Yu, J. Makhoul, R. Schwartz, L. Nguyen, S. Masoukas, B. Xiang, M. Afify, S. Abdou, J. Gauvain, L. Lamel, H. Schwenk, G. Adda, F. Lefevre, D. Vergyri, W. Wang, J. Zheng, A. Venkataraman, R. R. Gadde, and A. Stolcke, "SupcrEARS: Multi-site broadcast news system," in Rich Transcription (RT-04F) Workshop, Nov. 2004.
36. W. N. Campbell, "Durational cues to prominence and grouping," in Proc. ECSA Workshop on Prosody, Lund, Sweden, 1993, pp. 38-41.
37. J. R. De Pijper and A. A. Sanderman, "On the perceptual strength of prosodic boundaries and its relation to suprasegmental cues," J. Acoust. Soc. Amer., vol. 96, no. 4, pp. 2037-2047, Oct. 1994.
38. D. Hirst, "Peak, boundary and cohesion characteristics of prosodie grouping," in Proc. ECSA Workshop on Prosody, Lund, Sweden, 1993, pp. 32-37.
39. P. J. Price, M. Ostendorf, S. Shattuck-Hufnagel, and C. Fong, "The use of prosody in syntactic disambiguation," J. Acoust. Soc. Amer., vol. 90, no. 6, pp. 2956-2970, 1991.
40. D. R. Scott, "Duration as a cue to the perception of a phrase boundary," J. Acoust. Soc. Amer., vol. 71, no. 4, pp. 996-1007, 1982.
41. M. Swerts, "Prosodic features at discourse boundaries of different strength," J. Acoust. Soc. Amer., vol. 101, no. 1, pp. 514-521, Jan. 1997.
42. K. Sonmez, E. Shriberg, L. Heck, and M. Weintraub, "Modeling dynamic prosodic variation for speaker verification," in Proc. ICSLP, 1998, pp. 3189-3192.
43. C. Wooters, J. Fung, B. Peskin, and X. Anguera, "Toward robust speaker segmentation: the ICSI-SRI fall 2004 diarization system," in Rich Transcription (RT-04F) Workshop, 2004.
44. A. Sankar, L. Heck, and A. Stolcke, "Acoustic modeling for the SRI Hub4 partitioned evaluation continuous speech recognition system," in Proc. DARPA Speech Recognition Workshop. Chantilly, VA, Feb. 1997, pp. 127-132. [Online]. Available: http://www.nist.gov/speech/proc/darpa97/html/ sankar1/sankar1.htm.
45. W. Buntime and R. Caruana, Introduction to IND Version 2.1 and Recursive Partitioning. Moffett Field, CA: NASA Ames Research Center, 1992.
46. T. Mitchell, Machine tearning. New York: McGraw-Hill, 1997.
47. A. Stolcke and E. Shriberg, "Automatic linguistic segmentation of conversational speech," in Proc. ICSLP, 1996, pp. 1005-1008.
48. A. Stolcke, "SRILM - An extensible language modeling toolkit," in Proc. ICSLP, 2002, pp. 901-904.
49. S. F. Chen and J. T. Goodman, "An empirical study of smoothing techniques for language modeling," Comput. Sci. Group, Harvard University, Cambridge, MA, Tech. Rep., 1998.
50. A. L. Berger, S. A. D. Pietra, and V. J. D. Pietra, "A maximum entropy approach to natural language processing," Comput. Ling., vol. 22, pp. 39-72, 1996.
51. Y. Liu, A. Stolcke, E. Shriberg, and M. Harper, "Comparing and combining generative and posterior probability models: Some advances in sentence boundary detection in speech," in Proc. EMNLP, 2004, pp. 64-71.
52. R. H. Ryrd, P. Lu, and J. Nocedal, "A limited memory algorithm for bound constrained optimization," SIAM J. Sci. Statist. Comput., vol. 16, no. 5, pp. 1190-1208, 1995.
53. S. Chen and R. Rosenfeld, "A Gaussian prior for smoothing maximum entropy models," Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep., 1999.
54. J. Lafferty, A. McCallum, and F. Pereira, "Conditional random field: probabilistic models for segmenting and labeling sequence data," in Proc. ICML, 2001, pp. 282-289.
55. A. McCallum. (2002) Mallet: A machine learning for language toolkit. [Online] Available: http://mallet.cs.umass.edu
56. F. Sha and F. Pereira, "Shallow parsing with conditional random fields," in Proc. HLT/NAACL, 2003, pp. 134-141.
57. A. McCallum and W. Li, "Early results for named entity recognition with conditional random fields," in Proc. CoNLL, 2003, pp. 188-191.
58. T. Brants, "TnT a statistical part-of-speech tageer," in Proc. 6th Applied NLP Conf., 2000, pp. 224-231.
59. P. F. Brown, V. J. D. Pietra, P. V. DeSouza, J. C. Lai, and R. L. Mercer, "Class-based n-gram models of natural language," Comput. Ling., pp. 467-479, 1992.
60. G. Ngai and R. Florian, "Transformation-based learning in the fast lane," in Proc. NAACL, Jun. 2001, pp. 40-47.
61. Y. Liu, N. Chawla, M. Harper, E. Shriberg, and A. Stolcke, "A study in machine learning from imbalanced data for sentence boundary detection in speech," Comput. Speech Lang., pp. 468-494, 2006, to be published.
62. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, no. 2, pp. 123-140, 1996.
63. Y. Liu, "Structural event detection for rich transcription of speech," Ph.D. dissertation, Purdue Univ., West Lafayette, IN, 2004.
64. Y. Liu, E. Shriberg, and A. Stolcke, "Automatic disfluency identification in conversational speech using multiple knowledge sources," in Proc. Eurospeech, 2003, pp. 957-960.
65. H. H. Clark and T. Wasow, "Repeating words in spontaneous speech," Cognitive Psych., pp. 201-242, 1998.
66. Y. Liu, "Word fragment identification using acoustic-prosodic features in conversational speech," in Proc. HLT Student Workshop, 2003, pp. 37-42.
67. D. Hillard, M. Ostendorf, A. Stolcke, Y. Liu, and E. Shriberg, "Improving automatic sentence boundary detection with confusion networks," in Proc. HLT/NAACL, 2004, pp. 69-72.
68. L. Mangu, E. Brill, and A. Stolcke, "Finding consensus in speech recognition: word error minimization and other applications of confusion networks," Comput. Speech Lang., pp. 373-400, 2000.
69. M. Johnson, E. Charniak, and M. Lease, "Research on edit detection," in Proc. Rich Transcription (RT-04F) Workshop, 2004.