A study in machine learning from imbalanced data for sentence boundary detection in speech

Computer Speech and Language

Volumn 20, Issue 4, 2006, Pages 468-494

  Liu, Yang ^a,c   Chawla, Nitesh V ^b   Harper, Mary P ^c   Shriberg, Elizabeth ^a,d   Stolcke, Andreas ^a,d  

^a INTERNATIONAL COMPUTER SCIENCE INSTITUTE   (United States)

^b University of Notre Dame   (United States)

^c Purdue University   (United States)

^d SRI INTERNATIONAL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL METHODS; DECISION THEORY; LEARNING SYSTEMS; MARKOV PROCESSES; MATHEMATICAL MODELS; SPEECH ANALYSIS;

AREA UNDER THE CURVE (AUC); HIDDEN MARKOV MODEL (HMM); PROSODIC; RECEIVER OPERATING CHARACTERISTICS (ROC);

SPEECH RECOGNITION;

EID: 33746529930     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2005.06.002     Document Type: Article

References (62)

1. Beeferman, D., Berger, A., Lafferty, J., 1998. Cyperpunc: A lightweight punctuation annotation system for speech. In: Proceedings of the International Conference of Acoustics, Speech, and Signal Processing, 1998.
2. Bishop C. Neural Networks for Pattern Recognition (1995), Cambridge University Press, Cambridge, UK
3. Bradley A.P. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition 30 6 (1997) 1145-1159
4. Breiman L. Bagging predictors. Machine Learning 24 2 (1996) 123-140
5. Buntime W., and Caruana R. Introduction to IND version 2.1 and Recursive Partitioning (1992), NASA Ames Research Center, Moffett Field, CA
6. Campbell, W.N., 1993. Durational cues to prominence and grouping. In: Proceedings of ECSA Workshop on Prosody, Lund, Sweden, pp. 33-41.
7. Chan, P., Stolfo, S., 1998. Toward scalable learning with non-uniform class and cost distributions: A case study in credit card fraud detection. In: Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, pp. 164-168.
8. Chawla N.V., Bowyer K.W., Hall L.O., and Kegelmeyer W.P. Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research (2002) 321-357
9. Chawla, N.V., Japkowicz, N., Kolcz, A., (August 2003). Workshop on learning from imbalanced datasets II. In: Proceedings of the 20th International Conference on Machine Learning.
10. Chawla N.V., Moore T.E., Hall L.O., Bowyer K.W., Kegelmeyer W.P., and Springer L. Distributed learning with bagging-like performance. Pattern Recognition Letters 24 1-3 (2003) 455-471
11. Chawla, N.V., 2003. C4.5 and imbalanced datasets: Investigating the effect of sampling method, probabilistic estimate, and decision tree structure. In: Proceedings of the ICML'03 Workshop on Class Imbalances.
12. Chen, C.J., 1999. Speech recognition with automatic punctuation. In: Proceedings of the European Conference on Speech Communication and Technology, pp. 447-450.
13. Christensen, H., Gotoh, Y., Renal, S., 2001. Punctuation annotation using statistical prosody models. In: ISCA Workshop on Prosody in Speech Recognition and Understanding.
14. DARPA, 2003. Information Processing Technology Office, Effective, Affordable, Reusable Speech-to-text (EARS). Available from: .
15. De Pijper J.R., and Sanderman A.A. On the perceptual strength of prosodic boundaries and its relation to suprasegmental cues. Journal of the Acoustical Society of America 96 4 (1994) 2037-2047
16. Dietterich T.G. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40 2 (2000) 139-157
17. Drummond, D., Holte, R., 2003. C4.5, class imbalance, and cost sensitivity: Why under-sampling beats over-sampling. In: Proceedings of ICML'03 Workshop on Learning from Imbalanced Datasets.
18. Duda R.O., and Hart P.E. Pattern Recognition and Scene Analysis (1973), Wiley, New York
19. Ferrer, L., 2002. Prosodic features for the Switchboard database, Tech. rep., SRI International.
20. Freund, Y., Schapire, R., 1996. Experiments with a new boosting algorithm. In: Machine Learning: Proceedings of the Thirteenth National Conference, pp. 148-156.
21. Freund Y. Boosting a weak learning algorithm by majority. Information and Computation (1996) 256-285
22. Gotoh, Y., Renals, S., 2000. Sentence boundary detection in broadcast speech transcripts. In: Proceedings of ISCA Workshop: Automatic Speech Recognition: Challenges for the New Millennium ASR-2000, pp. 228-235.
23. Hand D. Construction and Assessment of Classification Rules (1997), Wiley, Chichester
24. Hirst, D., 1993. Peak, boundary and cohesion characteristics of prosodic grouping. In: Proceedings of ECSA Workshop on Prosody, Lund, Sweden, pp. 32-37.
25. Huang, J., Zweig, G., 2002. Maximum entropy model for punctuation annotation from speech. In: Proceedings of the International Conference on Spoken Language Processing, pp. 917-920.
26. Japkowicz N., and Stephen S. The class imbalance problem: A systematic study. Intelligent Data Analysis 6 5 (2002) 429-450
27. Kim, J., Woodland, P.C., 2001. The use of prosody in a combined system for punctuation generation and speech recognition. In: Proceedings of the European Conference on Speech Communication and Technology, pp. 2757-2760.
28. Kompe R. Prosody in Speech Understanding System (1996), Springer
29. Kubat, M., Matwin, S., 1997. Addressing the curse of imbalanced training sets. In: Proceedings of the International Conference on Machine Learning, pp. 179-186.
30. Kubat, M., Holte, R., Matwin, S., 1997. Learning when negative examples abound. In: Proceedings of European Conference on Machine Learning, pp. 146-153.
31. Laurikkaka, J., 2001. Improving identification of difficult small classes by balancing class distribution, Tech. rep., Department of Computer and Information Science, University of Tampere, Finland.
32. Lee S. Noisy replication in skewed binary classification. Computational Statistics and Data Analysis 34 (2000) 165-191
33. Lickley, R., Bard, E., 1996. On not recognizing disfluencies in dialog. In: Proceedings of the International Conference on Spoken Language Processing, pp. 1876-1879.
34. Ling, C., Li, C., 1998. Data mining for direct marketing problems and solutions. In: Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, pp. 73-79.
35. Liu, Y., Shriberg, E., Stolcke, A., 2003. Automatic disfluency identification in conversational speech using multiple knowledge sources. In: Proceedings of the European Conference on Speech Communication and Technology, pp. 957-960.
36. Liu, Y., Stolcke, A., Shriberg, E., Harper, M., 2004. Comparing and combining generative and posterior probability models: Some advances in sentence boundary detection in speech. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing.
37. Liu, Y., Shriberg, E., Stolcke, A., Peskin, B., Harper, M., The ICSI/SRI/UW RT04 structural metadata extraction system. In: EARS Rich Transcription Workshop, 2004.
38. Liu, Y., Shriberg, E., Stolcke, A., Harper, M., 2004. Using machine learning to cope with imbalanced classes in natural speech: Evidence from sentence boundary and disfluency detection. In: Proceedings of the International Conference on Spoken Language Processing.
39. Liu, Y., 2004. Structural event detection for rich transcription of speech, Ph.D. thesis, Purdue University.
40. Nakatani C., and Hirschberg J. A corpus-based study of repair cues in spontaneous speech. Journal of the Acoustical Society of America (1994) 1603-1616
41. National Institute of Standards and Technology, 2003. RT-03F evaluation, http://www.nist.gov/speech/tests/rt/rt2003/fall/rt03f-evaldisc doc/index.htm.
42. National Institute of Standards and Technology, (Nov. 2003) RT-03F workshop agenda and presentations, http://www.nist.gov/speech/tests/rt/rt2003/fall/presentations/.
43. Ostendorf, M., Hillard, D., 2004. Scoring structural MDE: Towards more meaningful error rates. In: EARS Rich Transcription Workshop.
44. Palmer, D.D., Hearst, M.A., 1994. Adaptive sentence boundary disambiguation. In: Proceedings of the Fourth ACL Conference on Applied Natural Language Processing, pp. 78-83.
45. Potisuk, S., 1995. Prosodic disambiguation in automatic speech understanding of Thai, Ph.D. thesis, Purdue University.
46. Price P.J., Ostendorf M., Shattuck-Hufnagel S., and Fong C. The use of prosody in syntactic disambiguation. Journal of the Acoustical Society of America 90 6 (1991) 2956-2970
47. Provost F., and Fawcett T. Robust classification for imprecise environments. Machine Learning 42 3 (2001) 203-231
48. Rabiner L.R., and Juang B.H. An introduction to hidden Markov models. IEEE ASSP Magazine 3 1 (1986) 4-16
49. Reynar, J., Ratnaparkhi, A., 1997. A maximum entropy approach to identifying sentence boundaries. In: Proceedings of the Fifth Conference on Applied Natural Language Processing, Washington, DC, pp. 16-19.
50. Schmid, H., 2000. Unsupervised learning of period disambiguation for tokenization, University of Stuttgart, Internal Report.
51. Scott D.R. Duration as a cue to the perception of a phrase boundary. Journal of the Acoustical Society of America 71 4 (1982) 996-1007
52. Shriberg E., Stolcke A., Hakkani-Tur D., and Tur G. Prosody-based automatic segmentation of speech into sentences and topics. Speech Communication (2000) 127-154
53. Sonmez, K., Shriberg, E., Heck, L., Weintraub, M., 1998. Modeling dynamic prosodic variation for speaker verification. In: Proceedings of the International Conference on Spoken Language Processing, pp. 3189-3192.
54. Stevenson, M., Gaizauskas, R., 2000. Experiments on sentence boundary detection. In: Proceedings of the North American Chapter of the Association for Computational Linguistics annual meeting, pp. 24-30.
55. Stolcke, A., Shriberg, E., 1996. Automatic linguistic segmentation of conversational speech. In: Proceedings of the International Conference on Spoken Language Processing, pp. 1005-1008.
56. Stolcke, A., Bratt, H., Butzberger, J., Franco, H., Rao Gadde, V.R., Plauché, M., Richey, C., Shriberg, E., Sönmez, K., Weng, F., Zheng, J., 2000. The SRI March 2000 Hub-5 conversational speech transcription system. In: Proceedings of NIST Speech Transcription Workshop, College Park, MD, 2000. URL http://www.nist.gov/speech/publications/tw00/html/cts80/cts80.htm.
57. Strassel, S., Walker, C., 2003. Data and annotation issues in RT-03. In: EARS Rich Transcription Workshop.
58. Strassel, S., 2003. Simple Metadata Annotation Specification V5.0, Linguistic Data Consortium. URL http://www.ldc.upenn.edu/projects/MDE/Guidelines/SimpleMDE_V5.0.pdf.
59. Swerts M. Prosodic features at discourse boundaries of different strength. Journal of the Acoustical Society of America 101 1 (1997) 514-521
60. Wang, D., Narayanan, S.S., 2004. A multi-pass linear fold algorithm for sentence boundary detection using prosodic cues. In: Proceedings of the International Conference of Acoustics, Speech, and Signal Processing.
61. Weiss G., and Provost F. Learning when training data are costly: The effect of class distribution on tree induction. Artificial Intelligence Research (2003) 315-354
62. Wrede, B., Shriberg, E., 2003. Spotting "hotspots" in meetings: Human judgments and prosodic cues. In: Proceedings of the European Conference on Speech Communication and Technology, pp. 2805-2808.