Round-robin duel discriminative language models

IEEE Transactions on Audio, Speech and Language Processing

Volumn 20, Issue 4, 2012, Pages 1244-1255

  Oba, Takanobul ^a,b   Hori, Takaaki ^a   Nakamura, Atsushi ^a   Ito, Akinori ^b  

^a Nippon Telegraph and Telephone Corporation   (Japan)

^b TOHOKU UNIVERSITY   (Japan)

Author keywords

Discriminative language model; error correction; round robin duel discrimination (R2D2)

Indexed keywords

CONDITIONAL RANDOM FIELD; CONVENTIONAL METHODS; DISCRIMINATIVE APPROACH; DISCRIMINATIVE TRAINING; ERROR RATE; FEATURE CONSTRUCTION; GLOBAL OPTIMUM; LANGUAGE MODEL; LOGLINEAR MODEL; OBJECTIVE FUNCTIONS; PARAMETER ESTIMATION METHOD; QUASI-NEWTON METHODS; RE-RANKING; ROUND ROBIN;

COMPUTATIONAL LINGUISTICS; ERROR CORRECTION; IMAGE SEGMENTATION; NEWTON-RAPHSON METHOD; PARAMETER ESTIMATION; REGRESSION ANALYSIS; SPEECH RECOGNITION;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 84857478465     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2011.2174225     Document Type: Article

References (32)

1. P. F. Brown, P. V. deSouza, R. L. Mercer, V. J. D. Pietra, and J. C. Lai, "Class-based n-gram models of natural language," Comput. Linguist., vol. 18, no. 4, pp. 467-479, 1992.
2. C. Hori, M. Katoh, A. Ito, and M. Kohda, "Construction and evaluation of language models based on stochastic context-free grammar for speech recognition," Syst. Comput. Japan, vol. 33, pp. 48-59, 2002. (Pubitemid 35312162)
3. A. Ito, C. Hori, M. Katoh, and M. Kohda, "Language modeling by stochastic dependency grammar for Japanese speech recognition," in Proc. ICSLP, 2000, vol. 1, pp. 246-249.
4. J. R. Bellegarda, "Exploiting latent semantic information in statistical language modeling," Proc. IEEE, vol. 88, no. 8, pp. 1279-1296, Aug. 2000.
5. D. Gildea and T. Hofmann, "Topic-based language models using EM," in Proc. Eurospeech, 1999, pp. 2167-2170.
6. D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent Dirichlet allocation," J. Mach. Learn. Res., vol. 3, pp. 993-1022, 2003.
7. S. M. Katz, "Estimation of probabilities from sparse data for the language model component of a speech recognizer," IEEE Trans. Acoust., Speech, Signal Process., vol. 35, no. 3, pp. 400-401, Mar. 1987.
8. K. W. Church and W. A. Gale, "A comparison of the enhanced good-turing and deleted estimation methods for estimating probabilities of English bigrams," Comput. Speech Lang., vol. 5, no. 1, pp. 19-54, 1991.
9. I. H. Witten and T. C. Bel, "Source models for natural language text," Int. J. Man-Mach Studies, vol. 32, pp. 545-579, 1990.
10. R. Kneser and H. Ney, "Improved backing-off for m-gram language modeling," in Proc. ICASSP, 1995, pp. 181-184.
11. B. Roark, M. Saraclar, and M. Collins, "Corrective language modeling for large vocabulary ASR with the perceptron algorithm," in Proc. ICASSP, 2004, pp. 749-752.
12. B. Roark, M. Saraclar, M. Collins, and M. Johnson, "Discriminative language modeling with conditional random fields and the perceptron algorithm," in Proc. ACL, 2004, pp. 47-54.
13. B. Roark, M. Saraclar, and M. Collins, "Discriminative n-gram language modeling," Comput. Speech Lang., vol. 21, no. 2, pp. 373-392, 2007. (Pubitemid 44709838)
14. Z. Zhou, J. Gao, F. K. Soong, and H. Meng, "A comparative study of discriminative methods for reranking LVCSR n-best hypotheses in domain adaptation and generalization," in Proc. ICASSP, 2006, pp. 141-144.
15. T. Oba, T. Hori, and A. Nakamura, "An approach to efficient generation of high-accuracy and compact error-corrective models for speech recognition," in Proc. Interspeech, 2007, pp. 1753-1756.
16. V. Magdin and H. Jiang, "Large margin estimation of n-gram language models for speech recognition via linear programming," in Proc. ICASSP, 2010, pp. 5398-5401.
17. M. Collins, B. Roark, and M. Saraclar, "Discriminative syntactic language modeling for speech recognition," in Proc. ACL, 2005, pp. 507-514.
18. I. Shafran and K. Hall, "Corrective models for speech recognition of inflected languages," in Proc. EMNLP, Jul. 2006, pp. 390-398.
19. M. Lehr and I. Shafran, "Learning a discriminative weighted finite-state transducer for speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 5, pp. 1360-1367, 2011.
20. B. R. E. Arisoy, M. Saraclar, and I. Shafran, "Syntactic and sub-lexical features for Turkish discriminative language models," in Proc. ICASSP, 2010, pp. 5538-5541.
21. M. Collins and T. Koo, "Discriminative reranking for natural language parsing," Comput. Linguist., vol. 31, no. 1, pp. 25-70, 2005.
22. T. Oba, T. Hori, and A. Nakamura, "A comparative study on methods of weighted language model training for reranking LVCSR n-best hypotheses," in Proc. ICASSP, 2010, pp. 5126-5129.
23. L. R. Bahi, P. F. Brown, P. V. de Souza, and R. L. Mercer, "Maximum mutual information estimation of hidden Markov model parameters for speech recognition," in Proc. ICASSP, 1986, vol. 11, pp. 49-52.
24. Y. Freund, R. Iyer, R. E. Schapire, and Y. Singer, "An efficient boosting algorithm for combining preferences," J. Mach. Learn. Res., vol. 4, pp. 933-969, 2003.
25. R. McDonald, K. Crammer, and F. Pereira, "Online large-margin training of dependency parsers," in Proc. ACL, 2004, pp. 91-98.
26. N. Singh-Miller and M. Collins, "Trigger-based language modeling using a loss-sensitive perceptron algorithm," in Proc. ICASSP, 2006, pp. 141-144.
27. C. Cortes and M. Mohri, "AUC optimization vs. error rate minimization," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2003.
28. K. Maekawa, H. Koiso, S. Furui, and H. Isahara, "Spontaneous speech corpus of Japanese," in Proc. ICLRE, 2000, pp. 947-952.
29. J. Glass, T. J. Hazen, S. Cyphers, I. Malioutov, D. Huynh, and R. Barzilay, "Recent progress in the MIT spoken lecture processing project," in Proc. Interspeech, 2007, pp. 2553-2556.
30. E. McDermott, T. J. Hazen, J. L. Roux, A. Nakamura, and S. Kata-giri, "Discriminative training for large vocabulary speech recognition using minimum classification error," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 1, pp. 203-223, Jan. 2007.
31. T. Hori and A. Nakamura, "Generalized fast on-the-fly composition algorithm for WFST-based speech recognition," in Proc. Interspeech, 2005, pp. 284-289.
32. D. C. Liu and J. Nocedal, "On the limited memory BFGS method for large scale optimization," Math. Program., vol. 45, pp. 503-528, 1989. (Pubitemid 20660315)