Distribution-based pruning of backoff language models

Proceedings of the Annual Meeting of the Association for Computational Linguistics

Volumn 2000-October, Issue , 2000, Pages

  Gao, Jianfeng ^a   Lee, Kai Fu ^a  

^a MICROSOFT RESEARCH ASIA   (China)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS;

% REDUCTIONS; BACKOFFS; CONVENTIONAL APPROACH; CUT-OFF METHODS; LANGUAGE MODEL; N-GRAMS; TRAINING DATA; WORD PERPLEXITY;

PROBABILITY DISTRIBUTIONS;

EID: 85149146278     PISSN: 0736587X     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (11)

1. F. Jelinek, "Self-organized language modeling for speech recognition", in Readings in Speech Recognition, A. Waibel and K.F. Lee, eds., Morgan-Kaufmann, San Mateo, CA, 1990, pp. 450-506.
2. ndInternational Conference on Research and Development in Information Retrieval, Berkeley, CA, 1999, pp. 214-221.
3. V.W. Zue, "Navigating the information superhighway using spoken language interfaces", IEEE Expert, S. M. Katz, "Estimation of probabilities from sparse data for the language model component of a speech recognizer", IEEE Transactions on Acoustic, Speech and Signal Processing, ASSP-35(3): 400-401, March, 1987.
4. K. Seymore, R. Rosenfeld, "Scalable backoff language models", in Porc. International Conference on Speech and Language Processing, Vol1. Philadelphia, PA, 1996, pp. 232-235
5. A. Stolcke, "Entropy-based Pruning of Backoff Language Models" in Proc. DRAPA News Transcriptionand Understanding Workshop, Lansdowne, VA. 1998. pp. 270-274
6. rdedition, 1974.
7. G. Salton, and J. M. Michael, "Introduction to Modern Information Retrieval", New York: McGraw-Hill, 1983.
8. S. M. Katz, "Distribution of content words and phrases in text and language modeling", Natural Language Engineering, 1996(2): 15-59
9. C. D. Manning, and H. Schütze, "Foundations of Statistical Natural Language Processing", The MIT Press, 1999.
10. S. Harter, "A probabilistic approach to automatic keyword indexing: Part II. An algorithm for probabilistic indexing", Journal of the American Society for Information Science, 1975(26): 280-289
11. J. Platt, "How to Implement SVMs", IEEE Intelligent System Magazine, Trends and Controversies, Marti Hearst, ed., vol 13, no 4, 1998.