Topic-dependent-class-based n-gram language model

IEEE Transactions on Audio, Speech and Language Processing

Volumn 20, Issue 5, 2012, Pages 1513-1525

  Naptali, Welly ^a   Tsuchiya, Masatoshi ^b   Nakagawa, Seiichi ^b  

^a KYOTO UNIVERSITY   (Japan)

^b TOYOHASHI UNIVERSITY OF TECHNOLOGY   (Japan)

Author keywords

Language model; N gram; Perplexity; Speech recognition; Topic dependent

Indexed keywords

DATA MINING; SEMANTICS; SPEECH RECOGNITION; STATISTICS;

AUTOMATIC SPEECH RECOGNITION; LANGUAGE MODEL; LATENT DIRICHLET ALLOCATIONS; N-GRAM LANGUAGE MODELS; N-GRAMS; PERPLEXITY; TOPIC DEPENDENT; TOPIC-DEPENDENT LANGUAGE MODELING;

COMPUTATIONAL LINGUISTICS;

EID: 84901319731     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2012.2183870     Document Type: Article

References (39)

1. R. Barzilay and M. Elhadad, "Using lexical chains for text summarization," in Proc. ACL Workshop Intell. Scalable Text Summariz., 1997, pp. 10-17.
2. J. R. Bellegarda, "Statistical language model adaptation: Review and perspectives," Speech Commun., vol. 42, no. 1, pp. 93-108, 2004.
3. J. R. Bellegarda, "Latent semantic mapping," IEEE Signal Process. Mag., vol. 22, no. 5, pp. 70-80, Sep. 2005.
4. D. M. Blei, A. Y. Ng, M. I. Jordan, and J. Lafferty, "Latent dirichlet allocation," J. Mach. Learn. Res., vol. 3, pp. 993-1022, 2003.
5. C. Bouras and V. Tsogkas, "Improving text summarization using noun retrieval techniques," in Knowl.-Based Intell. Inf. Eng. Syst., ser. Lecture Notes in Computer Science. Berlin/Heidelberg, Germany: Springer, 2008, vol. 5178, pp. 593-600.
6. S. Broman and M. Kurimo, "Methods for combining language models in speech recognition," in Proc. Interspeech '05, Sep. 2005, pp. 1317-1320.
7. K. Chen, "Topic identification in discourse," in Proc. 7th Conf. Eur. Chap. Assoc. Comput. Linguist., Dublin, Ireland, Mar. 1995, pp. 267-271, Morgan Kaufmann.
8. S. Chen, "Performance prediction for exponential language models," Tech. Rep. RC 24671, IBM Research, Tech. Rep., Oct. 2008.
9. J. T. Chien and C. H. Chueh, "Latent dirichlet language model for speech recognition," in Proc. Spoken Lang. Technol. Workshop, Dec. 2008, pp. 201-204.
10. J. T. Chien and C. H. Chueh, "Dirichlet class language models for speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 3, pp. 482-495, Mar. 2011.
11. C. H. Chueh and J. T. Chien, "Topic cache language model for speech recognition," in Proc. ICASSP, 2010, pp. 5194-5197.
12. P. Clarkson and R. Rosenfeld, "Language model adaptation using mixtures and an exponentially decaying cache," in Proc. ICASSP, 1997, pp. 799-802.
13. P. Clarkson and R. Rosenfeld, "Statistical language modeling using the CMU-Cambridge toolkit," in Proc. 5th Eur. Conf. Speech Commun. Technol., 1997, pp. 2707-2710.
14. I. S. Dhillon, J. Fan, and Y. Guan, "Efficient clustering of very large document collections," in Data Mining for Scientific and Engineering Applications, V. K. R. Grossman, C. Kamath, and R. Namburu, Eds. Norwell, MA: Kluwer, 2001, pp. 357-381, invited book chapter.
15. Y. Fujii, K. Yamamoto, and S. Nakagawa, "Large vocabulary speech recognition system: SPOJUS++," in Proc. 11th WSEAS Int. Conf. Multimedia Syst. Signal Process. (MUSP'11), Mar. 2011, pp. 110-118.
16. J. Geiss, "Latent semantic sentence clustering for multi-document summarization," Tech. Rep., Univ. of Cambridge, Jul. 2011.
17. D. Gildea and T. Hofmann, "Topic-based language models using EM," in Proc. Eurospeech, 1999, pp. 2167-2170.
18. V. Hatzivassiloglou, J. L. Klavans, and E. Eskin, "Detecting text similarity over short passages: Exploring linguistic feature combinations via machine learning," in Proc. Joint SIGDAT Conf. Empirical Methods Natural Lang. Process. and Very Large Corpora, 1999, pp. 203-212.
19. R. Iyer and M. Ostendorf, "Modeling long distance dependence in language: Topic mixtures vs. dynamic cache models," IEEE Trans. Speech Audio Process., vol. 7, no. 1, pp. 236-239, Jan. 1996.
20. R. Iyer, M. Ostendorf, and J. R. Rohlicek, "Language modeling with sentence-level mixtures," in HLT'94: Proc. Workshop Human Lang. Technol., Morristown, NJ, 1994, pp. 82-87, Association for Computational Linguistics.
21. F. Jelinek and R. L. Mercer, "Interpolated estimation of markov source parameters from sparse data," in Proc. Workshop Pattern Recognition in Practice, Amsterdam, The Netherlands, May 1980, North-Holland.
22. D. Klakow and J. Peters, "Testing the correlation of word error rate and perplexity," Speech Commun., vol. 38, no. 1-2, pp. 19-28, 2002.
23. R. Kneser and V. Steinbiss, "On the dynamic adaptation of stochastic LM," in Proc. ICASSP, 1993, vol. 2, pp. 586-589.
24. R. Kneser, J. Peters, and D. Klakow, "Language model adaptation using dynamic marginals," in Proc. Eurospeech, Sep. 1997, vol. 4, pp. 1971-1974.
25. R. Kuhn and R. de Mori, "A cache based natural language model for speech recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 6, pp. 570-583, Jun. 1992.
26. F. Liu and Y. Liu, "Unsupervised language model adaptation incorporating named entity information," in Proc. 45th Annu. Meeting Assoc. Comput. Linguist., Prague, Czech Republic, Jun. 2007, pp. 672-679, Association for Computational Linguistics.
27. Y. Liu and F. Liu, "Unsupervised language model adaptation via topic modeling based on named entity hypotheses," in Proc. ICASSP, 2008, pp. 4921-4924.
28. S. Nakagawa and I. Murase, "Relationship among phoneme/word recognition rate, perplexity and sentence recognition and comparison of language models," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 1992, vol. 1, pp. 589-592.
29. W. Naptali, M. Tsuchiya, and S. Nakagawa, "Topic-dependent language model with voting on noun history," ACM Trans. Asian Lang. Inf. Process. (TALIP), vol. 9, no. 2, pp. 1-31, 2010.
30. H. Strik, C. Cucchiarini, and J. M. Kessens, "Comparing the recognition performance of csrs: In search of an adequate metric and statistical significance test," in Proc. ICSLP '00, Beijing, China, 2000, pp. 740-744.
31. H. Strik, C. Cucchiarini, and J. M. Kessens, "Comparing the performance of two CSRS: How to determine the significance level of the differences," in Proc. Eurospeech, Aalborg, Denmark, 2001, vol. 3, pp. 2091-2094.
32. Y. C. Tam and T. Schultz, "Language model adaptation using variational Bayes inference," in Proc. Interspeech 2005, Sep. 2005, pp. 5-8.
33. Y. C. Tam and T. Schultz, "Correlated latent semantic model for unsupervised language model adaptation," in Proc. ICASSP, Apr. 2007, vol. 4, pp. 41-44.
34. K. Vertanen, Baseline WSJ acoustic models for HTK and Sphinx: Training recipes and recognition experiments Cavendish Laboratory, Univ. of Cambridge, Tech. Rep., 2006 [Online]. Available: Http://www.keithv.com/pub/baselinewsj
35. P. Xu and F. Jelinek, "Random forests and the data sparseness problem in language modeling," Comput. Speech Lang., vol. 21, no. 1, pp. 105-152, 2007.
36. R. B. Yates and B. R. Neto, Modern Information Retrieval, 1st ed. Reading, MA: Addison Wesley, 1999.
37. S. J. Young, G. Evermann, M. J. F. Gales, T. Hain, D. Kershaw, G. Moore, J. J. Odell, D. G. Ollason, D. Povey, V. Valtchev, and P. C. Woodland, "The HTK Book Version 3.4 Manual,". Cambridge, 2006.
38. J. Zhang, L. Wang, and S. Nakagawa, "LVCSR based on context dependent syllable acoustic models," in Proc. Asian Workshop Speech Sci. Technol., 2008, pp. 81-86, ser. SP2007-200.
39. I. Zitouni, "Backoff hierarchical class n-gram language models: Effectiveness to model unseen events in speech recognition," Comput. Speech Lang., vol. 21, no. 1, pp. 88-104, 2007.