Discriminative language modeling with linguistic and statistically derived features

IEEE Transactions on Audio, Speech and Language Processing

Volumn 20, Issue 2, 2012, Pages 540-550

  Arisoy, Ebru ^a   Saraçlar, Murat ^b   Roark, Brian ^c   Shafran, Izhak ^c  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b BOGAZICI UNIVERSITY   (Turkey)

^c OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

Author keywords

Discriminative training; language modeling; morphologically rich languages; speech recognition

Indexed keywords

AGGLUTINATIVE LANGUAGE; AUTOMATIC SPEECH RECOGNITION SYSTEM; DISCRIMINATIVE TRAINING; FEATURE SETS; FEATURE WEIGHT; LANGUAGE MODEL; LANGUAGE MODELING; LINGUISTIC FEATURES; LINGUISTIC INFORMATION; N-BEST HYPOTHESIS; N-GRAM LANGUAGE MODELS; OVERLAPPING FEATURES; PARAMETER ESTIMATE; SEMANTIC INFORMATION; SENSITIVE FEATURES; STATISTICAL APPROACH; STATISTICAL FEATURES; TRAINING EXAMPLE; TURKISHS;

COMPUTATIONAL LINGUISTICS; FEATURE EXTRACTION; LAGRANGE MULTIPLIERS; NATURAL LANGUAGE PROCESSING SYSTEMS; PARAMETER ESTIMATION; SEMANTICS;

SPEECH RECOGNITION;

EID: 83755187965     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2011.2162323     Document Type: Article

References (48)

1. E. Arisoy, M. Saraçlar, B. Roark, and I. Shafran, "Syntactic and sub-lexical features for Turkish discriminative language models," in Proc. ICASSP, Dallas, TX, 2010, pp. 5538-5541.
2. E. Arisoy, B. Roark, I. Shafran, and M. Saraçlar, "Discriminative n-gram language modeling for Turkish," in Proc. Interspeech, Brisbane, Australia, 2008, pp. 825-828.
3. J. A. Bilmes and K. Kirchhoff, "Factored language models and generalized parallel backoff," in Proc. NAACL, Edmonton, AB, Canada, 2003, pp. 4-6.
4. R. Rosenfeld, "Adaptive statistical language modeling: A maximum entropy approach," Ph.D. dissertation, Carnegie Mellon Univ., Pittsburgh, PA, 1994.
5. S. Khudanpur and J. Wu, "Maximum entropy techniques for exploiting syntactic, semantic and collocational dependencies in language modeling," Comput. Speech Lang., vol. 14, pp. 355-372, 2000.
6. H. Erdogan, R. Sarikaya, S. F. Chen, Y. Gao, and M. Picheny, "Using semantic analysis to improve speech recognition performance," Comput. Speech Lang., vol. 19, pp. 321-343, 2005. (Pubitemid 40467094)
7. R. Sarikaya, M. Afify, Y. Deng, H. Erdogan, and Y. Gao, "Joint morphological-lexical language modeling for processing morphologically rich languages with application to dialectal Arabic," IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 7, pp. 1330-1339, Sep. 2008.
8. R. Rosenfeld, S. F. Chen, and X. Zhu, "Whole-sentence exponential language models: A vehicle for linguistic-statistical integration," Comput. Speech Lang., vol. 15, no. 1, pp. 55-73, 2001. (Pubitemid 32334275)
9. B. Roark, M. Saraçlar, and M. Collins, "Discriminative n-gram language modeling," Comput. Speech Lang., vol. 21, no. 2, pp. 373-392, 2007. (Pubitemid 44709838)
10. B. Roark, M. Saraçlar, M. J. Collins, and M. Johnson, "Discriminative language modeling with conditional random fields and the perceptron algorithm," in Proc. ACL, Barcelona, Spain, 2004, pp. 47-54.
11. M. Collins, M. Saraçlar, and B. Roark, "Discriminative syntactic language modeling for speech recognition," in Proc. ACL, Ann Arbor, MI, 2005, pp. 507-514.
12. T.Oba, T. Hori, and A. Nakamura, "Round-robin discrimination model for reranking ASR hypotheses," in Proc. Interspeech, Makuhari, Japan, 2010, pp. 5126-5129.
13. M. Saraclar and B. Roark, "Utterance classification with discriminative language modeling," Speech Commun., vol. 48, no. 3-4, pp. 276-287, Mar.-Apr. 2006. (Pubitemid 43207401)
14. M. Collins, "Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms," in Proc. EMNLP, Philadelphia, PA, 2002, pp. 1-8.
15. T. Oba, T. Hori, and A. Nakamura, "A comparative study on methods of weighted language model training for reranking LVCSR n-best hypotheses," in Proc. ICASSP, Dallas, TX, 2010, pp. 2446-2449.
16. I. Shafran and K. Hall, "Corrective models for speech recognition of inflected languages," in Proc. EMNLP, Sydney, Australia, 2006, pp. 390-398.
17. N. Singh-Miller and M. Collins, "Trigger-based language modeling using a loss-sensitive perceptron algorithm," in Proc. ICASSP, Honolulu, HI, 2007, pp. 25-28.
18. M. Lehr and I. Shafran, "Discriminatively estimated joint acoustic, duration and language model for speech recognition," in Proc. ICASSP, Dallas, TX, 2010, pp. 5542-5545.
19. E. Charniak and M. Johnson, "Coarse-to-fine n-best parsing and maxent discriminative reranking," in Proc. ACL, Ann Arbor, MI, 2005, pp. 173-180.
20. M. Colins and T. Koo, "Discriminative reranking for natural language parsing," Comput. Linguist., vol. 31, no. 1, pp. 25-70, 2005.
21. H.-K. J. Kuo, E. Fosler-Lussier, H. Jiang, and C.-H. Lee, "Discriminative training of language models for speech recognition," in Proc. ICASSP, Orlando, FL, 2002, pp. 325-328.
22. A. Rastrow, A. Sethy, and B. Ramabhadran, "Constrained discriminative training of n-gram language models," in Proc. IEEE Workshop ASRU, Merano, Italy, 2009, pp. 311-316.
23. S. Yaman, L. Deng, D. Yu, Y.-Y. Wang, and A. Acero, "An integrative and discriminative technique for spoken utterance classification," IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 6, pp. 1207-1214, 2008.
24. T. Alumäe, "Methods for Estonian large vocabulary speech recognition," Ph.D. dissertation, Tallinn Univ. of Technol., Tallinn, Estonia, 2006.
25. T. Hirsimäki, M. Creutz, V. Siivola, M. Kurimo, S. Virpioja, and J. Pylkkönen, "Unlimited vocabulary speech recognition with morph language models applied to Finnish," Comput. Speech Lang., vol. 20, no. 4, pp. 515-541, 2006. (Pubitemid 44142005)
26. P. Mihajlik, T. Fegyò, Z. Täske, and P. Ircing, "A morpho-graphemic approach for the recognition of spontaneous speech in agglutinative languages-Like Hungarian," in Proc. Interspeech, Antwerp, Belgium, Aug. 2007, pp. 1497-1500.
27. E. Arisoy, D. Can, S. Parlak, H. Sak, and M. Saraçlar, "Turkish broadcast news transcription and retrieval," IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 5, pp. 874-883, 2009.
28. M. Creutz, T. Hirsimäki, M. Kurimo, A. Puurula, J. Pylkkönen, V. Siivola, M. Varjokallio, E. Arisoy, M. Saraçlar, and A. Stolcke, "Morph-based speech recognition and modeling of out-of-vocabulary words across languages," ACM Trans. Speech Lang. Process., vol. 5, no. 1, pp. 1-29, 2007.
29. K. Çarki, P. Geutner, and T. Schultz, "Turkish LVCSR: Towards better speech recognition for agglutinative languages," in Proc. ICASSP,?Istanbul, Turkey, 2000, pp. 1563-1566.
30. D. Z. Hakkani-Tür, "Statistical language modeling for agglutinative languages," Ph.D. dissertation, Bilkent Univ., Ankara, Turkey, 2000.
31. E. Mengüşoglu and O. Deroo, "Turkish LVCSR: Database preparation and language modeling for an agglutinative language," in Proc. ICASSP, Student Forum, Salt Lake City, UT, 2001.
32. E. Arisoy, H. Dutagaci, and L. M. Arslan, "A unified language model for large vocabulary continuous speech recognition of Turkish," Signal Process., vol. 86, no. 10, pp. 2844-2862, 2006.
33. K. Hacioglu, B. Pellom, T. Çiloglu, Ö. Öztürk, M. Kurimo, and M. Creutz, "On lexicon creation for Turkish LVCSR," in Proc. Eu-rospeech, Geneva, Switzerland, 2003, pp. 1165-1168.
34. M. Creutz and K. Lagus, "Unsupervised morpheme segmentation and morphology induction from text corpora using Morfessor 1.0," Helsinki Univ. of Technol., Publications in Computer and Information Sci. Rep. A81, Mar. 2005.
35. H. Erdogan, O. Büyük, and K. Oflazer, "Incorporating language constraints in sub-word based speech recognition," in Proc. ASRU, San Juan, Puerto Rico, 2005, pp. 98-103.
36. E. Arisoy and M. Saraçlar, "Lattice extension and vocabulary adaptation for Turkish LVCSR," IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 1, pp. 163-173, Jan. 2009.
37. K. Oflazer, "Two-level description of Turkish morphology," Literary Linguist. Comput., vol. 9, no. 2, pp. 137-148, 1994.
38. H.Sak,T. Güngör, and M.Saraçlar, "Morphological disambiguation of Turkish text with perceptron algorithm," in Proc. CICLing LNCS 4394, 2007, pp. 107-118. (Pubitemid 350259176)
39. D. Hakkani-Tür, K. Oflazer, and G. Tür, "Statistical morphological disambiguation for agglutinative languages," J. Comput. Humanities, vol. 36, no. 4, pp. 381-410, 2002. (Pubitemid 44947889)
40. G. Eryigit, J. Nivre, and K. Oflazer, "Dependency parsing of Turkish," Comput. Linguist., vol. 34, no. 3, pp. 357-389, 2008.
41. C. Chelba and F. Jelinek, "Structured language modeling," Comput. Speech Lang., vol. 14, no. 4, pp. 283-332, 2000.
42. B. Roark, "Probabilistic top-down parsing and language modeling," Computational Linguistics, vol. 27, no. 2, pp. 249-276, 2001. (Pubitemid 33597448)
43. P. F. Brown, V. J. D. Pietra, P. V. deSouza, J. C. Lai, and R. L. Mercer, "Class-based n-gram models of natural language," Comput. Linguist., vol. 18, no. 4, pp. 467-479, Dec. 1992.
44. C. Manning and H. Schütze, Foundations of Statistical Natural Language Processing. Cambridge, MA: MIT Press, 1999.
45. M. Hearst, "Texttiling: Segmenting text into multi-paragraph subtopic passages," Comput. Linguist., vol. 23, no. 1, pp. 33-64, Mar. 1997. (Pubitemid 127458657)
46. H. Sak, T. Gungor, and M. Saraclar, "Resources for Turkish morphological processing," Lang. Resources Eval., vol. 45, no. 2, pp. 249-261, 2011.
47. A. Stolcke, "SRILM-An extensible language modeling toolkit," in Proc. ICSLP, Denver, CO, 2002, vol. 2, pp. 901-904.
48. M. Bacchiani, B. Roark, and M. Saraçlar, "Language model adaptation with MAP estimation and the perceptron algorithm," in Proc. HLT-NAACL, Boston, MA, 2004, pp. 21-24.