Statistical language model adaptation: Review and perspectives

Speech Communication

Volumn 42, Issue 1, 2004, Pages 93-108

  Bellegarda, Jerome R ^a  

^a APPLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACOUSTIC NOISE; FINITE AUTOMATA; LINGUISTICS; MAXIMUM LIKELIHOOD ESTIMATION; PROBABILITY DENSITY FUNCTION; RANDOM PROCESSES; SEMANTICS; STATISTICAL METHODS; SYNTACTICS; TRANSDUCERS;

LANGUAGE MODELING; STATISTICAL LANGUAGE MODELS (SLM);

SPEECH RECOGNITION;

EID: 0346008165     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2003.08.002     Document Type: Conference Paper

References (75)

1. Adda, G., Jardino, M., Gauvain, J.L., 1999. Language modeling for broadcast news transcription. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Vol. 4. Budapest, Hungary, September 1999, pp. 1759-1762.
2. Bahl L.R., Jelinek F., Mercer R.L. A maximum likelihood approach to continuous speech recognition. IEEE Trans. Pattern Anal. Mach. Intel. PAMI-5(2):1983;179-190.
3. Bellegarda, J.R., 1998a. Exploiting both local and global constraints for multi-span statistical language modeling. In: Proc. 1998 Internat. Conf. Acoust. Speech Signal Process., Vol. 2. Seattle, WA, May 1998, pp. 677-680.
4. Bellegarda J.R. A multi-span language modeling framework for large vocabulary speech recognition. IEEE Trans. Speech Audio Proc. 6(5):1998;456-467.
5. Bellegarda J.R. Large vocabulary speech recognition with multi-span statistical language models. IEEE Trans. Speech Audio Proc. 8(1):2000;76-84.
6. Bellegarda J.R. Exploiting latent semantic information in statistical language modeling. Proc. IEEE. 88(8):2000;1279-1296.
7. Bellegarda, J.R., 2001. A novel approach to the adaptation of latent semantic information. In: Proc. 2001 ISCA Workshop on Adaptation Methods, elsewhere in these Proceedings.
8. Bellegarda J.R., Nahamoo D. Tied mixture continuous parameter modeling for speech recognition. IEEE Trans. Acoust. Speech Signal Process. ASSP-38(12):1990;2033-2045.
9. Berger, A., Miller, R., 1998. Just-in-time language modelling. In: Proc. 1998 Internat. Conf. Acoust. Speech Signal Process., Vol. 2. Seattle, WA, May 1998, pp. 705-709.
10. Bertoldi, N., Brugnara, F., Cettolo, M., Federico, M., Giuliani, D., 2001. From broadcast news to spontaneous dialogue transcription: portability issues. In: Proc. 2001 Internat. Conf. Acoust. Speech Signal Process., Salt Lake City, UT, May 2001.
11. Besling, S., Meier, H.G., 1995. Language model speaker adaptation. In: Proc. 1995 Euro. Conf. Speech Comm. Technol., Madrid, Spain, September 1995, pp. 1755-1758.
12. Chelba, C., 2001. Portability of syntactic structure for language modeling. In: Proc. 2001 Internat. Conf. Acoust. Speech Signal Process., Salt Lake City, UT, May 2001.
13. Chelba C., Jelinek F. Structured language modeling. Computer, Speech, and Language. 14(4):2000;283-332.
14. Chelba, C., Engle, D., Jelinek, F., Jimenez, V., Khudanpur, S., Mangu, L., Printz, H., Ristad, E.S., Rosenfeld, R., Stolcke, A., Wu, D., 1997. Structure and performance of a dependency language model. In: Proc. 1997 Euro. Conf. Speech Comm. Technol., Vol. 5. Rhodes, Greece, September 1997, pp. 2775-2778.
15. Chen, L., Huang, T., 1999. An improved MAP method for language model adaptation. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Vol. 5. Budapest, Hungary, September 1999, pp. 1923-1926.
16. Chen S.F., Rosenfeld R. A survey of smoothing techniques for ME models. IEEE Trans. Speech Audio Proc. 8(1):2000;37-50.
17. Chen, S.F., Seymore, K., Rosenfeld, R., 1998. Topic adaptation for language modeling using unnormalized exponential models. In: Proc. 1998 Internat. Conf. Acoust. Speech Signal Process., Vol. 2. Seattle, WA, May 1998, pp. 681-684.
18. Church K.W. Phonological Parsing in Speech Recognition. 1987;Kluwer Academic Publishers, New York.
19. Clarkson, P.R., Robinson, A.J., 1997. Language model adaptation using mixtures and an exponentially decaying cache. In: Proc. 1997 Internat. Conf. Acoust. Speech Signal Proc., Munich, Germany, May 1997, pp. 799-802.
20. Coccaro, N., Jurafsky, D., 1998. Towards better integration of semantic predictors in statistical language modeling. In: Proc. Internat. Conf. Spoken Language Process., Sydney, Australia, December 1998, pp. 2403-2406.
21. Darroch J.N., Ratcliff D. Generalized iterative scaling for log-linear models. Ann. Math. Statist. 43(5):1972;1470-1480.
22. Deerwester S., Dumais S.T., Furnas G.W., Landauer T.K., Harshman R. Indexing by latent semantic analysis. J. Amer. Soc. Inform. Sci. 41:1990;391-407.
23. Della Pietra S., Della Pietra V., Lafferty J. Inducing features of random fields. IEEE Trans. Pattern Anal. Mach. Intel. PAMI-19(1):1997;1-13.
24. Della Pietra, S., Della Pietra, V., Mercer, R., Roukos, S., 1992. Adaptive language model estimation using minimum discrimination estimation. In: Proc. 1992 Internat. Confer. Acoust. Speech Signal Process., Vol. I. San Francisco, CA, April 1992, pp. 633-636.
25. Donnelly, P.G., Smith, F.J., Sicilia, E., Ming, J., 1999. Language modelling with hierarchical domains. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Budapest, Hungary, Vol. 4, September 1999, pp. 1575-1578.
26. Federico, M., 1996. Bayesian estimation methods for N-gram language model adaptation. In: Proc. 1996 Internat. Conf. Spoken Language Process., Philadelphia, PA, October 1996, pp. 240-243.
27. Federico, M., 1999. Efficient language model adaptation through MDI estimation. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Vol. 4. Budapest, Hungary, September 1999, pp. 1583-1586.
28. Federico M., de Mori R. Ponting K. Language Model Adaptation. 1999;Springer-Verlag, New York.
29. Galescu, L., Allen, J., 2000. Hierarchical statistical language models: experiments on in-domain adaptation. In: Proc. 2000 Internat. Conf. Spoken Language Proc., Beijing, China, October 2000, pp. I186-I189.
30. Gildea, D., Hoffman, T., 1999. Topic-based language modeling using EM. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Vol. 5. Budapest, Hungary, September 1999, pp. 2167-2170.
31. Gorin A.L. On automated language acquisition. J. Acoust. Soc. Amer. 97:1995;3441-3461.
32. Gretter, R., Riccardi, G., 2001. On-line learning of language models with word error probability distributions. In: Proc. 2001 Int. Conf. Acoust. Speech Signal Process., Salt Lake City, UT, May 2001.
33. Hofmann, T., 1999a. Probabilistic latent semantic analysis. In: Proc. Fifteenth Conf. Uncertainty in AI, Stockholm, Sweden, July 1999.
34. Hofmann T. Probabilistic topic maps: navigating through large text collections. Lecture Notes Comp. Science, No. 1642. 1999;161-172 Springer-Verlag, Heidelberg.
35. Iyer R., Ostendorf M. Modeling long distance dependencies in language: Topic mixtures versus dynamic cache models. IEEE Trans. Speech Audio Process. 7(1):1999.
36. Iyer R., Ostendorf M., Rohlicek J.R. Language modeling with sentence-level mixtures. Proc. ARPA Speech and Natural Language Workshop. 1994;82-86 Morgan Kaufmann Publishers.
37. Janiszek, D., de Mori, R., Bechet, F., 2001. Data augmentation and language model adaptation. In: Proc. 2001 Internat. Conf. Acoust. Speech Signal Process., Salt Lake City, UT, May 2001.
38. Jardino, M., 1996. Multilingual stochastic n-gram class language models. In: Proc. 1996 Internat. Conf. Acoust. Speech Signal Proc., Atlanta, GA, May 1996, pp. I161-I163.
39. Jelinek F. The development of an experimental discrete dictation recognizer. Proc. IEEE. 73(11):1985;1616-1624.
40. Jelinek, F., Chelba, C., 1999. Putting language into language modeling. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Vol. 1. Budapest, Hungary, September 1999, pp. KN1-KN5.
41. Kellner, A., 1998. Initial language models for spoken dialogue systems. In: Proc. 1998 Internat. Conf. Acoust. Speech Signal Process., Vol. 1. Seattle, WA, May 1998, pp. 185-188.
42. Kneser, R., Peters, J., 1997. Semantic clustering for adaptative language modeling. In: Proc. 1997 Internat. Conf. Acoust., Speech Signal Process., Vol. 2. Munich, Germany, May 1997, pp. 779-782.
43. Kneser, R., Steinbiss, V., 1993. On the dynamic adaptation of stochastic language models. In: Proc. 1993 Internat. Conf. Acoust. Speech Signal Process., Vol. II, Minneapolis, MN, May 1993, pp. 586-588.
44. Kneser, R., Peters, J., Klakow, D., 1997. Language model adaptation using dynamic marginals. In: Proc. 1997 Euro. Conf. Speech Comm. Technol., Vol. 4. Rhodes, Greece, September 1997, pp. 1971-1974.
45. Kuhn R., de Mori R. A Cache-based natural language method for speech recognition. IEEE Trans. Pattern Anal. Mach. Intel. PAMI-12(6):1990;570-582.
46. Lafferty J.D., Suhm B. Cluster expansion and iterative scaling for maximum entropy language models. Hanson K., Silver R. Maximum Entropy and Bayesian Methods. 1995;Kluwer Academic Publishers, Norwell, MA.
47. Lau, R., Rosenfeld, R., Roukos, S., 1993. Trigger-based language models: a maximum entropy approach. In: Proc. 1993 Internat. Conf. Acoust. Speech Signal Process., Minneapolis, MN, May 1993, pp. II45-II48.
48. Lefevre, F., Gauvain, J.L., Lamel, L., 2001. Towards task-independent speech recognition. In: Proc. 2001 Internat. Conf. Acoust. Speech Signal Process., Salt Lake City, UT, May 2001.
49. Martin, S.C., Liermann, J., Ney, H., 1997. Adaptive topic-dependent language modelling using word-based varigrams. In: Proc. 1997 Euro. Conf. Speech Comm. Technol., Vol. 3. Rhodes, Greece, September 1997, pp. 1447-1450.
50. Masataki, H., Sagisaka, Y., Tawahara, T., 1997. Task adaptation using MAP estimation in N-gram language model. In: Proc. 1997 Internat. Conf. Acoust. Speech Signal Process., Vol. 1. Munich, Germany, May 1997, pp. 783-786.
51. Mohri M. Minimization algorithms for sequential transducers. Theor. Comp. Sci. 234(1-2):2000;177-201.
52. Mood A., Graybill F., Boes D. Introduction to the Theory of Statistics. 1974;McGraw-Hill, New York.
53. Nasr, A., Esteve, Y., Bechet, F., Spriet, T., de Mori, R., 1999. A language model combining N-grams and stochastic finite state automata. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Vol. 5. Budapest, Hungary, September 1999, pp. 2175-2178.
54. Niesler, T., Woodland, P., 1996. A Variable-length category-based N-gram language model. In: Proc. 1996 Internat. Conf. Acoust. Speech Signal Process., Atlanta, GA, May 1996, pp. I164-I167.
55. Ohtsuki, K., Furui, S., Sakurai, N., Iwasaki, A., Zhang, Z.-P., 1999. Recent advances in Japanese broadcast news transcription. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Vol. 2. Budapest, Hungary, September 1999, pp. 671-674.
56. Pereira F.C., Riley M. Speech recognition by composition of weighted finite automata. Roche E., Schabes Y. Finite-State Language Processing. 1997;431-453 MIT Press, Cambridge, MA.
57. Peters, J., Klakow, D., 1999. Compact maximum entropy language models. In: Proc. 1999 Autom. Speech Reco. Understanding Workshop, Keystone, CO, December 1999, pp. I253-I256.
58. Rabiner L.R., Juang B.H., Lee C.-H. An overview of automatic speech recognition. Lee C.-H., Soong F.K., Paliwal K.K. Automatic Speech and Speaker Recognition - Advanced Topics. 1996;1-30 Kluwer Academic Publishers, Boston, MA. (Chapter 1).
59. Rao, P.S., Dharanipragada, S., Roukos, S., 1997. MDI adaptation of language models across corpora. In: Proc. 1997 Euro. Conf. Speech Comm. Technol., Vol. 4. Rhodes, Greece, September 1997, pp. 1979-1982.
60. Reichl, W., 1999. Language model adaptation using maximum discrimination information. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Vol. 4. Budapest, Hungary, September 1999, pp. 1791-1794.
61. Riccardi G., Gorin A.L. Stochastic language adaptation over time and state in natural spoken dialog systems. IEEE Trans. Speech Audio Proc. 8(1):2000;3-10.
62. Rosenfeld, R., 1995. Optimizing lexical and N-gram coverage via judicious use of linguistic data. In: Proc. 1995 Euro. Conf. on Speech Comm. Technol., Madrid, Spain, September 1995, pp. 1763-1766.
63. Rosenfeld R. A maximum entropy approach to adaptive statistical language modeling. Computer Speech and Language. Vol. 10:1996;187-228 Academic Press, London.
64. Rosenfeld, R., 1997. A whole sentence maximum entropy language model. In: Proc. 1997 Autom. Speech Reco. Understanding Workshop, Santa Barbara, CA, pp. 230-237.
65. Rosenfeld R. Two decades of statistical language modeling: Where do we go from here. Proc. IEEE. 88(8):2000;1270-1278.
66. Rosenfeld R., Chen S.F., Zhu X. Whole sentence exponential language models: a vehicle for linguistic-statistical integration. Computer Speech and Language. 15(1):2001;. Academic Press, London.
67. Schwartz, R., Imai, T., Kubala, F., Nguyen, L., Makhoul, J., 1997. A maximum likelihood model for topic classification of broadcast news. In: Proc. 1997 Euro. Conf. Speech Comm. Technol., Vol. 3. Rhodes, Greece, September 1997, pp. 1455-1458.
68. Seymore, K., Rosenfeld, R., 1997. Using story topics for language model adaptation. In: Proc. 1997 Euro. Conf. Speech Comm. Technol., Vol. 4. Rhodes, Greece, September 1997, pp. 1987-1990.
69. Souvignier B., Kellner A., Rueber B., Schramm H., Seide F. The thoughtful elephant: Strategies for spoken dialog systems. IEEE Trans. Speech Audio Proc. 8(1):2000;51-62.
70. Wu, J., Khudanpur, S., 1999. Combining nonlocal, syntactic and N-gram dependencies in language modeling. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Vol. 5. Budapest, Hungary, September 1999, pp. 2179-2182.
71. 3 Inform. Control. 10:1967;198-208.
72. Zhang, R., Black, E., Finch, A., 1999. Using detailed linguistic structure in language modeling. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Vol. 4. Budapest, Hungary, September 1999, pp. 1815-1818.
73. Zhu, X.J., Chen, S.F., Rosenfeld, R., 1999b. Linguistic features for whole sentence maximum entropy language models. In: Proc. 1999 Euro. Conf. Speech Comm. Technol., Vol. 4. Budapest, Hungary, September 1999, pp. 1807-1810.
74. Zhu, X., Rosenfeld, R., 2001. Improving trigram language modeling with the world wide web. In: Proc. 2001 Internat. Conf. Acoust., Speech, Signal Proc., Salt Lake City, UT, May 2001.
75. Zue V., Seneff S., Glass J.R., Polifroni J., Pao C., Hazen T.J., Hetherington L. JUPITER: A telephone-based conversational interface for weather information. IEEE Trans. Speech Audio Proc. 8(1):2000;85-96.