An iterative relative entropy minimization-based data selection approach for n-gram model adaptation

IEEE Transactions on Audio, Speech and Language Processing

Volumn 17, Issue 1, 2009, Pages 13-23

  Sethy, Abhinav ^a   Georgiou, Panayiotis G ^a   Ramabhadran, Bhuvana ^a   Narayanan, Shrikanth ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Data selection; Language model adaptation; Relative entropy

Indexed keywords

COMMUNITY IS; DATA SELECTION; DIALOG SYSTEMS; LANGUAGE MODEL; LANGUAGE MODEL ADAPTATION; LANGUAGE MODELING; LARGE VOCABULARY; MEDICAL DOMAINS; N-GRAM LANGUAGE MODELS; N-GRAM MODELS; PERFORMANCE IMPROVEMENTS; RELATIVE ENTROPY; RELATIVE-ENTROPY MINIMIZATION; SPECIFIC NATURE; SPEECH RECOGNITION SYSTEMS; STATE OF THE ART; SUBSET SELECTION; TEXT MATERIALS; WORD ERROR RATE;

COMPUTATIONAL LINGUISTICS; DATA REDUCTION; ENTROPY; NATURAL LANGUAGE PROCESSING SYSTEMS;

SPEECH RECOGNITION;

EID: 70350780149     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2008.2006654     Document Type: Article

References (40)

1. M. Lapata and F. Keller, "Web-based models for natural language processing," ACM Trans. Speech Lang. Process., vol.2, pp. 1-31, 2005.
2. F. Keller, M. Lapata, and O. Ourioupina, "Using the web to overcome data sparseness," in Proc. EMNLP, 2002, pp. 230-237.
3. P. Resnik and N. A. Smith, "The web as a parallel corpus," Comput. Linguist., vol.29, pp. 349-380, 2003.
4. D. Graff, "English Gigaword," LDC Catalog ID: LDC2003T05, 2003.
5. T. Ng, M. Ostendorf, M.-Y. Hwang, M. Siu, I. Bulyko, and X. Lei, "Web-data augmented language model for Mandarin speech recognition," in Proc. ICASSP, 2005, pp. 589-592.
6. R. Sarikaya, A. Gravano, and Y. Gao, "Rapid language model development using external resources for new spoken dialog domains," in Proc. ICASSP, 2005, pp. 573-576.
7. K. Nigam, A. K. McCallum, S. Thrun, and T. Mitchell, "Text classification from labeled and unlabeled documents using EM," J. Mach. Learn., vol.39, pp. 103-134, 2000.
8. X. Zhu, "Semi-supervised learning literature survey," Univ. of Wisconsin- Madison, Comput. Sci., Tech. Rep. 1530, Dec. 2005 [Online]. Available: http://www.cs.wisc.edu/jerryzhu/pub/ssl-survey.pdf.
9. A. Sethy, P. G. Georgiou, and S. Narayanan, "Text data acquisition for domain-specific language models," in Proc. EMNLP, 2006, pp. 382-389.
10. S. Narayanan, S. Ananthakrishnan, R. Belvin, E. Ettaile, S. Ganjavi, P. Georgiou, C. Hein, S. Kadambe, K. Knight, D. Marcu, H. Neely, N. Srinivasamurthy, D. Traum, and D.Wang, "Transonics: A speech to speech system for English-Persian interactions," in Proc. IEEE ASRU, 2003, pp. 1-6.
11. B. Ramabhadran, O. Siohan, L. Mangu, G. Zweig, M. Westphal, H. Schulz, and A. Soneiro, "The IBM 2006 speech transcription system for European parliamentary speeches," in Proc. ICSLP, 2006, pp. IV33-IV36.
12. T. Misu and T. Kawahara, "A bootstrapping approach for developing language model of new spoken dialogue systems by selecting web texts," in Proc. ICSLP, 2006, pp. 9-12.
13. A. Sethy, P. Georgiou, and S. Narayanan, "Building topic specific language models from web-data using competitive models," in Proc. Eurospeech, 2005, pp. 1293-1296.
14. K. Weilhammer, M. N. Stuttlem, and S. Young, "Bootstrapping language models for dialogue systems," in Proc. ICSLP, 2006, pp. 1482-1485.
15. B. Liu, Y. Dai, X. Li, W. S. Lee, and P. Yu, "Building text classifiers using positive and unlabeled examples," in Proc. ICDM, 2003, pp. 179-189.
16. A. Stolcke, "Entropy-based pruning of backoff language models," in Proc.DARPA Broadcast News Transcription and UnderstandingWorkshop, 1998, pp. 270-274.
17. L. Lee, "Measures of distributional similarity," in Proc. ACL, 1999, pp. 25-32.
18. S. F. Chen and J. Goodman, "An empirical study of smoothing techniques for language modeling," in Proc. ACL, 1996, pp. 310-318.
19. J. Reynar and A. Ratnaparkhi, "A maximum entropy approach to identifying sentence boundaries," in Proc. 5th Conf. Appl. Natural Lang. Process., 1997, pp. 16-19.
20. "TC-STAR: Technology and Corpora for Speech to Speech Translation." [Online]. Available: http://www.tc-star.org
21. B. Pellom, "SONIC: The University of Colorado Continuous Speech Recognizer," Univ. of Colorado, Boulder, Tech. Rep. TR-CSLR-2001-2101, Jan. 2001.
22. D. B. Paul and J. M. Baker, "The design for thewall street journal-based csr corpus," in Proc.Workshop on Speech and Natural Lang., 1992, pp. 889-902.
23. C. Cieri, D. Miller, and K. Walker, "The Fisher Corpus: A resource for the next generations of speech-to-text," in Proc. LREC, 2004, pp. 69-71.
24. D. Graff, K. Walker, and A. Canavan, "Switchboard-2 Phase II," LDC Catalog ID: LDC99S79, 1999.
25. K. Seymore, S. Chen, S.-J. Doh, E. Gouvea, B. Raj, M. Ravishankar, R. Rosenfeld, M. Siegler, R. Stern, and E. Thayer, "The 1997 CMU Sphinx-3 english broadcast news transcription system," in Proc. 1998 DARPA Speech Recognition Workshop, 1998, pp. 55-89.
26. E. Ettelaie, S. Gandhe, P. Georgiou, K. Knight, D. Marcu, S. Narayanan, D. Traum, and R. Belvin, "Transonics: A practical speech-to-speech translator for English-Farsi medical dialogues," in Proc. ACL, 2005, pp. 89-92.
27. A. Sethy, B. Ramabhadran, and S. Narayanan, "Measuring convergence in language model estimation using relative entropy," in Proc. ICSLP, 2004, pp. 1057-1060.
28. I. Bulyko, M. Ostendorf, and A. Stolcke, "Getting more mileage from web text sources for conversational speech language modeling using class-dependent mixtures," in Proc. HLT, 2003, pp. 7-9.
29. O. Cetin and A. Stolcke, "Language Modeling in the ICSI-SRI Spring 2005 Meeting Speech Recognition Evaluation," ICSI, Tech. Rep. TR-05-006, July 2005.
30. R. Iyer, M. Ostendorf, and M. Meteer, "Analyzing and predicting language model improvements," in Proc. ASRU, 1997, pp. 254-261.
31. "Fisher-Related Conversational Style Web Collection," [Online]. Available: http://ssli.ee.washington.edu/projects/ears/Web-Data/web\-data\- collection.html.
32. S. Chen, B. Kingsbury, L. Mangu, D. Povey, G. Saon, H. Soltau, and G. Zweig, "Advances in speech transcription at IBM under the DARPA EARS program," IEEE Trans. Audio, Speech, Lang. Process., vol.14, no.5, pp. 1596-1608, Sep. 2006.
33. L. Mangu, E. Brill, and A. Stolcke, "Finding consensus among words: Lattice-based word error minimization," in Proc. Eurospeech, 1999, pp. 495-498.
34. F. Huang, Y. Zhang, and S. Vogel, "Mining key phrase translations from web corpora," in Proc. EMNLP, 2005, pp. 483-490.
35. J. Bellegarda, "Large vocabulary speech recognition with multispan statistical language models," IEEE Trans. Speech Audio Process., vol.8, no.1, pp. 76-84, Jan. 2000.
36. B. J. Hsu and J. Glass, "Style and topic language model adaptation using HMM-LDA," in Proc. EMNLP, 2006, pp. 373-381.
37. T. Brants and A. Franz, "Web 1T 5-Gram Version 1," LDC Catalog ID: LDC2006T13, 2006.
38. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley-Interscience, Aug. 1991.
39. M. Mohri, "Finite-state transducers in language and speech processing," Comput. Linguist., vol.23, pp. 269-311, 1997.
40. R. C. Carrasco, "Accurate computation of the relative entropy between stochastic regular grammars," RAIRO (Theoretical Informatics and Applications), vol.31, pp. 437-444, 1997