A fast re-scoring strategy to capture long-distance dependencies

EMNLP 2011 - Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

Volumn , Issue , 2011, Pages 1116-1127

  Deoras, Anoop ^a   Mikolov, Tomáš ^b   Church, Kenneth ^a  

^a JOHNS HOPKINS UNIVERSITY   (United States)

^b BRNO UNIVERSITY OF TECHNOLOGY   (Czech Republic)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATIC SPEECH RECOGNITION; BROADCAST NEWS; HILL-CLIMBING METHODS; LANGUAGE MODEL; LONG SPAN; LONG-DISTANCE DEPENDENCIES; MACHINE TRANSLATIONS; N-BEST LIST; N-GRAMS; NATURAL LANGUAGES; WORD ERROR RATE REDUCTIONS;

COMPUTATIONAL LINGUISTICS; ITERATIVE DECODING; NATURAL LANGUAGE PROCESSING SYSTEMS; OPTICAL CHARACTER RECOGNITION; RECURRENT NEURAL NETWORKS; SPEECH TRANSMISSION;

SPEECH RECOGNITION;

EID: 80053284315     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (35)

1. Cyril Allauzen, Michael Riley, Johan Schalkwyk, Wojciech Skut, and Mehryar Mohri. 2007. OpenFst: A General and Efficient Weighted Finite-State Transducer Library. In Proceedings of the Ninth International Conference on Implementation and Application of Automata, (CIAA 2007), volume 4783 of Lecture Notes in Computer Science, pages 11-23. Springer.
2. J. R. Bellegarda. 2000. Exploiting latent semantic information in statistical language modeling. Proceedings of IEEE, 88(8):1279-1296.
3. Yoshua Bengio, Réjean Ducharme, and Pascal Vincent. 2001. A Neural Probabilistic Language Model. In Proceedings of Advances in Neural Information Processing Systems.
4. Peter Beyerlein. 1998. Discriminative Model Combination. In Proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
5. Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning. Springer.
6. Ciprian Chelba and Frederick Jelinek. 2000. Structured Language Modeling. Computer Speech and Language, 14(4):283-332.
7. S. F. Chen, L. Mangu, B. Ramabhadran, R. Sarikaya, and A. Sethy. 2009. Scaling shrinkage-based language models. In Proc. of IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pages 299-304.
8. Noam Chomsky. 1957. Syntactic Structures. The Hague: Mouton.
9. Yen-Lu Chow and Richard Schwartz. 1989. The N-Best algorithm: an efficient procedure for finding top N sentence hypotheses. In Proceedings of the workshop on Speech and Natural Language, HLT '89, pages 199-202, Stroudsburg, PA, USA. Association for Computational Linguistics.
10. Kenneth Church. 2012. A Pendulum Swung Too Far. Linguistic Issues in Language Technology - LiLT. to appear.
11. T.M. Cover and J.A. Thomas. 1991. Elements of Information Theory. John Wiley and Sons, Inc. N.Y.
12. Anoop Deoras and Frederick Jelinek. 2009. Iterative Decoding: A Novel Re-Scoring Framework for Confusion Networks. In Proc. of IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pages 282-286.
13. Anoop Deoras, Denis Filimonov, Mary Harper, and Fred Jelinek. 2010. Model Combination for Speech Recognition using Empirical Bayes Risk Minimization. In Proc. of IEEE Workshop on Spoken Language Technology (SLT).
14. Anoop Deoras, Tomáš Mikolov, Stefan Kombrink, Martin Karafiát, and Sanjeev Khudanpur. 2011. Variational Approximation of Long-Span Language Models for LVCSR. In Proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
15. Jeffery Elman. 1990. Finding Structure in Time. In Cognitive Science, volume 14, pages 179-211.
16. Denis Filimonov and Mary Harper. 2009. A Joint Language Model with Fine-grain Syntactic Tags. In Proc. of 2009 Conference on Empirical Methods in Natural Language Processing.
17. V. Goel and W. Byrne. 2000. Minimum Bayes Risk Automatic Speech Recognition. Computer, Speech and Language.
18. Rukmini Iyer and Mari Ostendorf. 1999. Modeling Long Distance Dependence in Language: Topic Mixtures Versus Dynamic Cache Models. IEEE Transactions on Speech and Audio Processing, 7(1):30-39.
19. Zhifei Li and Jason Eisner. 2009. First- and second-order expectation semirings with applications to minimumrisk training on translation forests. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 40-51, Singapore, August.
20. Lidia Luminita Mangu. 2000. Finding consensus in speech recognition. Ph.D. thesis, The Johns Hopkins University. Adviser-Brill, Eric.
21. Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan "Honza" Černocký, and Sanjeev Khudanpur. 2010. Recurrent Neural Network Based Language Model. In Proc. of the ICSLP-Interspeech.
22. Tomáš Mikolov, Anoop Deoras, Stefan Kombrink, Lukáš Burget, and Jan "Honza" Černocký. 2011a. Empirical Evaluation and Combination of Advanced Language Modeling Techniques. In Proc. of Interspeech.
23. Tomáš Mikolov, Stefan Kombrink, Lukáš Burget, Jan "Honza" Černocký, and Sanjeev Khudanpur. 2011b. Extensions of Recurrent Neural Network Language Model. In Proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
24. Mehryar Mohri and Michael Riley. 2002. An Efficient Algorithm for the N-Best-Strings Problem. In Proceedings of the International Conference on Spoken Language Processing (ICSLP).
25. M. Mohri, F.C.N. Pereira, and M. Riley. 2000. The design principles of a weighted finite-state transducer library. Theoretical Computer Science, 231:17-32.
26. A. Ogawa, K. Takeda, and F. Itakura. 1998. Balancing Acoustic and Linguistic Probabilities. In Proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
27. F. Richardson, M. Ostendorf, and J.R. Rohlicek. 1995. Lattice-based search strategies for large vocabulary speech recognition. In Proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
28. Brian Roark. 2001. Probabilistic top-down parsing and language modeling. Computational Linguistics, 27(2):249-276.
29. Roni Rosenfeld, Stanley F. Chen, and Xiaojin Zhu. 2001. Whole-Sentence Exponential Language Models: a Vehicle for Linguistic-Statistical Integration. Computer Speech and Language, 15(1).
30. Roni Rosenfeld. 1997. A Whole Sentence Maximum Entropy Language Model. In Proc. of IEEE workshop on Automatic Speech Recognition and Understanding (ASRU), Santa Barbara, California, December.
31. D.E. Rumelhart, G. E. Hinton, and R.J. Williams. 1986. Learning representations by back-propagating errors. Nature, 323:533-536.
32. Holger Schwenk. 2007. Continuous space language models. Computer Speech and Language, 21(3):492-518.
33. C. E. Shannon. 1948. A Mathematical Theory of Communication. The Bell System Technical Journal, 27:379-423, 623-656.
34. H. Soltau, G. Saon, and B. Kingsbury. 2010. The IBM Attila speech recognition toolkit. In Proc. of IEEE Workshop on Spoken Language Technology (SLT).
35. Wen Wang and Mary Harper. 2002. The SuperARV language model: investigating the effectiveness of tightly integrating multiple knowledge sources. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).