On growing and pruning Kneser-Ney smoothed N-gram models

IEEE Transactions on Audio, Speech and Language Processing

Volumn 15, Issue 5, 2007, Pages 1617-1624

  Siivola, Vesa ^a   Hirsimäki, Teemu ^a   Virpioja, Sami ^a  

^a AALTO UNIVERSITY   (Finland)

Author keywords

Modeling; Natural languages; Smoothing methods; Speech recognition

Indexed keywords

BACK OFFS; FINNISH; GOOD-TURING; GROWING AND PRUNING; LANGUAGE MODELS; LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION; MODEL ORDERS; MODELING; N-GRAM MODELS; NATURAL LANGUAGES; PRUNING ALGORITHMS; PRUNING METHODS; SMOOTHING METHODS; TEXT CORPORA; TRAINING DATUM;

ALGORITHMS; COMPUTATIONAL LINGUISTICS; CONTINUOUS SPEECH RECOGNITION; QUERY LANGUAGES; SPEECH ANALYSIS;

PROBABILITY DISTRIBUTIONS;

EID: 58349107420     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2007.896666     Document Type: Article

References (28)

1. K. Seymore and R. Rosenfeld, "Scalable backoff language models," in Proc. ICSLP, 1996, pp. 232-235.
2. R. Kneser, "Statistical language modeling using a variable context length," in Proc. ICSLP, 1996, pp. 494-497.
3. A. Stolcke, "Entropy-based pruning of backoff language models," in Proc. DARPA Broadcast News Transcription Understanding Workshop, 1998, pp. 270-274.
4. S. Chen and J. Goodman, "An empirical study of smoothing techniques for language modeling," Comput. Speech Lang., vol. 13, no. 4, pp. 359-393, Oct. 1999.
5. J. Goodman, "Abit of progress in language modeling," Comput. Speech Lang., vol. 15, no. 4, pp. 403-434, Oct. 2001.
6. R. Kneser and H. Ney, "Improved backing-off for m-gram language modeling," in Proc. ICASSP, 1995, pp. 181-184.
7. J. Goodman and J. Gao, "Language model size reduction by pruning and clustering," in Proc. ICSLP, 2000, pp. 110-113.
8. E. Ristad and R. Thomas, "New techniques for context modeling," in Meeting Assoc. Computat. Ling., 1995, pp. 220-227.
9. M. Siu and M. Ostendorf, "Variable N-grams and extensions for conversational speech language modeling," IEEE Trans. Speech Audio Process., vol. 8, no. 1, pp. 63-75, Jan. 2000.
10. T. R. Niesler and P. C. Woodland, "Variable-length category n-gram language models," Comput. Speech Lang., vol. 13, no. 1, pp. 99-124, Jan. 1999.
11. V. Siivola and B. Pellom, "Growing an N-gram model," in Proc. Interspeech, 2005, pp. 1309-1312.
12. H. Yamamoto, S. Isogai, and Y. Sagisaka, "Multi-class composite n-gram language model," Speech Commun., vol. 41, no. 2-3, pp. 369-379, Oct. 2003.
13. S. Deligne and F. Bimbot, "Inference of variable-length linguistic and acoustic units by multigrams," Speech Commun., vol. 23, no. 3, pp. 223-241, 1997.
14. M. Creutz and K. Lagus, "Unsupervised discovery of morphemes," in Proc. Workshop Morphol. Phonol. Learning ACL-02, 2002, pp. 21-30.
15. S. Virpioja and M. Kurimo, "Compact n-gram models by incremental growing and clustering of histories," in Proc. Interspeech, 2006, pp. 1037-1040.
16. A. Bonafonte and J. Mariño, "Language modeling using x-grams," in Proc. ICSLP, 1996, pp. 394-397.
17. S. Chen and R. Rosenfeld, "A survey of smoothing techniques for ME models," IEEE Trans. Speech Audio Process., vol. 8, no. 1, pp. 37-50, Jan. 2000.
18. F. James, Modified Kneser-Ney Smoothing of N-Gram Models Res. Inst. Adv. Comput. Sci., Tech. Rep. 00.07, Oct. 2000.
19. E. W. D. Whittaker and B. Raj, "Quantization-based language model compression," in Proc. Eurospeech, 2001, pp. 33-36.
20. B. Raj and E. W. D. Whittaker, "Lossless compression of language model structure and word identifiers," in Proc. ICASSP, 2003, pp. 388-391.
21. quot;Finnish Text Collection," 2004, collection of Finnish text documents from years 1990-2000. Compiled by Department of General Linguistics, University of Helsinki, Linguistics and Language Technology Department, University of Joensuu, Research Institute for the Languages of Finland, and CSC.[Online]. Available: http://www.csc.fi/kielipankki/
22. T. Hirsimäki, M. Creutz, V. Siivola, M. Kurimo, S. Virpioja, and J. Pylkkönen, "Unlimited vocabulary speech recognition with morph language models applied to Finnish," Comput. Speech Lang., vol. 20, no. 4, pp. 515-541, Oct. 2006.
23. M. Kurimo, A. Puurula, E. Arisoy, V. Siivola, T. Hirsimäki, J. Pylkkönen, T. Alumae, and M. Saraclar, "Unlimited vocabulary speech recognition for agglutinative languages," in Proc. HLT-NAACL, 2006, pp. 487-494.
24. M. Creutz and K. Lagus, "Unsupervised morpheme segmentation and morphology induction from text corpora Using Morfessor 1.0," Publications Comput. Inf. Sci., Helsinki Univ.Technol., Tech. Rep. A81, 2005.
25. A. Stolcke, "SRILM-An extensible language modeling toolkit," in Proc. ICSLP, 2002, pp. 901-904.
26. D. Graff, J. Kong, K. Chen, and K. Maeda, English Gigaword Second Edition. Philadelphia, PA: Linguistic Data Consortium, 2005.
27. D. Iskra, B. Grosskopf, K. Marasek, H. van den Heuvel, F. Diehl, and A. Kiessling, "SPEECON-Speech databases for consumer devices: Database specification and validation," in Proc. LREC, 2002, pp. 329-333.
28. J. Pylkkönen, "New pruning criteria for efficient decoding," in Proc. Interspeech, 2005, pp. 581-584.