A study of n-gram and decision tree letter language modeling methods

Speech Communication

Volumn 24, Issue 3, 1998, Pages 171-192

  Potamianos, Gerasimos ^a   Jelinek, Frederick ^b  

^a AT AND T LABS RESEARCH   (United States)

^b JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

Back off language model; Brown corpus; Decision trees; Deleted interpolation; Language modeling; Laws of succession; n grams; Smoothing

Indexed keywords

ALGORITHMS; DECISION THEORY; INTERPOLATION; TREES (MATHEMATICS);

DECISION TREES; LANGUAGE MODELS;

SPEECH RECOGNITION;

EID: 0032089995     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0167-6393(98)00018-1     Document Type: Article

References (24)

1. Bahl, L.R., Jelinek, F., Mercer, R.L., 1983. A maximum likelihood approach to continuous speech recognition. IEEE Trans. Pattern Anal. Machine Intell. 5, 179-190.
2. Bahl, L.R., Brown, P.F., deSouza, P.V., Mercer, R.L., 1989. A tree-based statistical language model for natural language speech recognition. IEEE Trans. Acoust. Speech Signal Process. 37, 1001-1008.
3. Bahl, L.R., Brown, P.F., deSouza, P.V., Mercer, R.L., Nahamoo, D., 1991. A fast algorithm for deleted interpolation. In: Proc. Europ. Conf. Speech Comm. Tech., Genova, pp. 1209-1212.
4. Bell, T.C., Cleary, J.G., Witten, I.H., 1990. Text Compression. Prentice-Hall, Englewood Cliffs, NJ.
5. Betz, M., Hild, H., 1995. Language models for a spelled letter recognizer. In: Proc. Internat. Conf. Acoust. Speech Signal Process., Detroit, pp. 856-859.
6. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J., 1984. Classification and Regression Trees. Chapman and Hall, New York.
7. Brown, P.F., Della Pietra, S.A., Della Pietra, V.J., Lai, J.C., Mercer, R.L., 1992a. An estimate of an upper bound for the entropy of English. Comput. Linguistics 18, 31-40.
8. Brown, P.F., Della Pietra, V.J., deSouza, P.V., Lai, J.C., Mercer, R.L., 1992b. Class-based n-gram models of natural language. Comput. Linguistics 18, 467-479.
9. Chen, S.F., Goodman, J., 1996. An empirical study of smoothing techniques for language modeling. In: Proc. 34th Ann. Meeting Assoc. Comput. Linguistics, Santa Cruz, pp. 310-318.
10. Chou, P.A., 1991. Optimal partitioning for classification and regression trees. IEEE Trans. Pattern Anal. Machine Intell. 13, 340-354.
11. Efron, B., 1982. The jackknife, the bootstrap and other resampling plans. In: CBMS-NSF Reg. Conf. Ser. Appl. Math., SIAM, Philadelphia, PA.
12. Good, I.J., 1953. The population frequencies of species and the estimation of population parameters. Biometrika 40, 237-264.
13. Jelinek, F., 1997. Statistical Methods for Speech Recognition. MIT Press, Cambridge, MA.
14. Jelinek, F., Mercer, R.L., 1980. Interpolated estimation of Markov source parameters from sparse data. In: Gelsema, E.S., Kanal, L.N. (Eds.), Pattern Recognition in Practice. North-Holland, Amsterdam, pp. 381-397.
15. Katz, S.M., 1987. Estimation of probabilities from sparse data for the language model component of a speech recognizer. IEEE Trans. Acoust. Speech Signal Process. 35, 400-401.
16. Kucera, H., Francis, W., 1967. Computational Analysis of Present-Day American English. Brown Univ. Press, Providence, RI.
17. Ney, H., Essen, U., Kneser, R., 1995. On the estimation of 'small' probabilities by leaving-one-out. IEEE Trans. Pattern Anal. Machine Intell. 17, 1202-1212.
18. Potamianos, G., Jelinek, F., 1997. A study of n-gram and decision tree letter language modeling methods. CLSP Research Notes 13, Center for Language and Speech Processing, The Johns Hopkins University, Baltimore, MD.
19. Press, W.H., Flannery, B.P., Teukolsky, S.A., Vetterling, W.T., 1988. Numerical Recipes in C. The Art of Scientific Computing. Cambridge Univ. Press, Cambridge.
20. Ristad, E.S., 1995. A natural law of succession. Research Report CS-TR-495-95, Department of Computer Science, Princeton University, Princeton, NJ.
21. Ristad, E.S., Thomas, R.G., 1995. New techniques for context modeling. In: Proc. 33rd Ann. Meeting Assoc. Comput. Linguistics, Cambridge, pp. 220-227.
22. Ristad, E.S., Thomas, R.G., 1997a. Hierarchical non-emitting Markov models. In: Proc. 35th Ann. Meeting Assoc. Comput. Linguistics, Madrid, pp. 381-385.
23. Ristad, E.S., Thomas, R.G., 1997b. Hierarchical non-emitting Markov models. Research Report CS-TR-544-97, Department of Computer Science, Princeton University, Princeton, NJ.
24. Shannon, C.E., 1951. Prediction and entropy of printed English. Bell Syst. Tech. J. 30, 50-64.