Bayesian learning of a language model from continuous speech

IEICE Transactions on Information and Systems

Volumn E95-D, Issue 2, 2012, Pages 614-625

  Neubig, Graham ^a   Mimura, Masato ^a   Mori, Shinsuke ^a   Kawahara, Tatsuya ^a  

^a KYOTO UNIVERSITY   (Japan)

Author keywords

Automatic speech recognition; Bayesian learning; Language modeling; Weighted finite state transducers

Indexed keywords

COMPUTATIONAL LINGUISTICS; DEEP NEURAL NETWORKS; LEARNING ALGORITHMS; MODELING LANGUAGES; SAMPLING; SPEECH; TRANSDUCERS;

AUTOMATIC SPEECH RECOGNITION; BAYESIAN LEARNING; CONTINUOUS SPEECH; LANGUAGE MODEL; LINGUISTIC CONSTRAINTS; NON-PARAMETRIC BAYESIAN; STATISTICAL FRAMEWORK; WEIGHTED FINITE-STATE TRANSDUCERS;

SPEECH RECOGNITION;

EID: 84856378396     PISSN: 09168532     EISSN: 17451361     Source Type: Journal    
DOI: 10.1587/transinf.E95.D.614     Document Type: Article

References (40)

1. D. Tannen, Spoken and Written Language: Exploring Orality and Literacy, ABLEX, 1982.
2. Y. Akita and T. Kawahara, "Statistical transformation of language and pronunciation models for spontaneous speech recognition," IEEE Trans. Audio Speech Language Process., vol.18, no.6, pp.1539-1549, 2010.
3. I. Bazzi and J. Glass, "Learning units for domain-independent outof-vocabulary word modelling," Proc. 7th European Conference on Speech Communication and Technology (EuroSpeech), pp.61-64, 2001.
4. T. Hirsimaki, M. Creutz, V. Siivola, M. Kurimo, S. Virpioja, and J. Pylkkonen, "Unlimited vocabulary speech recognition with morph language models applied to finnish," Comput. Speech Lang., vol.20, no.4, pp.515-541, 2006. (Pubitemid 44142005)
5. S. Abney and S. Bird, "The human language project: Building a universal corpus of the world's languages," Proc. 48th Annual Meeting of the Association for Computational Linguistics, pp.88-97, Uppsala, Sweden, July 2010.
6. D. Roy and A. Pentland, "Learning words from sights and sounds: A computational model," Cognitive Science, vol.26, no.1, pp.113-146, 2002.
7. C. de Marcken, "The unsupervised acquisition of a lexicon from continuous speech," tech. rep., Massachusetts Institute of Technology, Cambridge, MA, USA, 1995.
8. S. Deligne and F. Bimbot, "Inference of variable-length linguistic and acoustic units by multigrams," Speech Commun., vol.23, no.3, pp.223-241, 1997. (Pubitemid 127433795)
9. A. Gorin, D. Petrovska-Delacretaz, G. Riccardi, and J. Wright, "Learning spoken language without transcriptions," Proc. 1999 IEEE Automatic Speech Recognition and UnderstandingWorkshop, 1999.
10. J. Driesen and H.V. Hamme, "Improving the multigram algorithm by using lattices as input," Proc. 9th Annual Conference of the International Speech Communication Association (InterSpeech), 2008.
11. L. ten Bosch and B. Cranen, "A computational model for unsupervised word discovery," Proc. 8th Annual Conference of the International Speech Communication Association (InterSpeech), pp.1481-1484, 2007.
12. A. Park and J. Glass, "Unsupervised pattern discovery in speech," IEEE Trans. Audio Speech Language Process., vol.16, no.1, 2008.
13. A. Jansen, K. Church, and H. Hermansky, "Towards spoken term discovery at scale with zero resources," Proc. 11th Annual Conference of the International Speech Communication Association (Inter-Speech), 2010.
14. N. Iwahashi, "Language acquisition through a human-robot interface by combining speech, visual, and behavioral information," Inf. Sci., vol.156, no.1-2, pp.109-121, 2003.
15. C. Yu and D.H. Ballard, "A multimodal learning interface for grounding spoken language in sensory perceptions," ACM Trans. Applied Perception, vol.1, pp.57-80, July 2004.
16. D. Mochihashi, T. Yamada, and N. Ueda, "Bayesian unsupervised word segmentation with nested Pitman-Yor modeling," Proc. 47th Annual Meeting of the Association for Computational Linguistics, 2009.
17. Y.W. Teh, "A Bayesian interpretation of interpolated kneser-ney," tech. rep., School of Computing, National Univ. of Singapore, 2006.
18. J.T. Goodman, "A bit of progress in language modeling," Comput. Speech Lang., vol.15, no.4, pp.403-434, 2001.
19. S.F. Chen and J. Goodman, "An empirical study of smoothing techniques for language modeling," Proc. 34th Annual Meeting of the Association for Computational Linguistics, 1996.
20. D.J.C. Mackay and L.C.B. Petoy, "A hierarchical Dirichlet language model," Natural Language Engineering, vol.1, pp.1-19, 1995.
21. J. Pitman and M. Yor, "The two-parameter Poisson-Dirichlet distri bution derived from a stable subordinator," The Annals of Probability, vol.25, no.2, pp.855-900, 1997.
22. M. Mohri, F. Pereira, and M. Riley, "Speech recognition with weighted finite-state transducers," in Handbook on speech processing and speech communication, Part E: Speech recognition, 2008.
23. F. Bimbot, R. Pieraccini, E. Levin, and B. Atal, "Variable-length sequence modeling: Multigrams," IEEE Signal Process. Lett., vol.2, no.6, pp.111-113, 1995.
24. M.R. Brent, "An efficient, probabilistically sound algorithm for segmentation and word discovery," Mach. Learn., vol.34, pp.71-105, 1999.
25. S. Goldwater, T.L. Griffiths, and M. Johnson, "A Bayesian framework for word segmentation: Exploring the effects of context," Cognition, vol.112, no.1, pp.21-54, 2009.
26. H. Poon, C. Cherry, and K. Toutanova, "Unsupervised morphological segmentation with log-linear models," Proc. North American Chapter of the Association for Computational Linguistics -Human Language Technology (NAACL HLT), pp.209-217, 2009.
27. S. Geman and D. Geman, "Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images," IEEE Trans. Pattern Anal. Mach. Intell., vol.6, no.6, pp.721-741, 1984. (Pubitemid 15453722)
28. D.J. MacKay, Information theory, inference, and learning algorithms, pp.357-386, Cambridge University Press, 2003.
29. C.S. Jensen, U. Kjaerulff, and A. Kong, "Blocking Gibbs sampling in very large probabilistic expert systems," Int. J. Human Comput. Studies, vol.42, no.6, pp.647-666, 1995.
30. J. Fiscus, "A post-processing system to yield reduced word error rates: Recognizer output voting error reduction (ROVER)," Proc. 1997 IEEE Automatic Speech Recognition and Understanding Workshop, 1997.
31. L. Lamel, J. Gauvain, and G. Adda, "Lightly supervised and unsupervised acoustic model training," Comput. Speech Lang., vol.16, pp.115-129, 2002.
32. J.R. Glass, Finding acoustic regularities in speech: Application to phonetic recognition, Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, 1988.
33. G. Neubig and S. Mori, "Word-based partial annotation for efficient corpus construction," Proc. 7th International Conference on Language Resources and Evaluation, 2010.
34. T. Schultz and A. Waibel, "Language-independent and languageadaptive acoustic modeling for speech recognition," Speech Commun., vol.35, no.1, pp.31-52, 2001.
35. L. Pearl, S. Goldwater, and M. Steyvers, "How ideal are we? incorporating human limitations into Bayesian models of word segmentation," Proc. 34th Annual Boston University Conference on Child Language Development, pp.315-326, Somerville, MA, 2010.
36. F.R. McInnes and S. Goldwater, "Unsupervised extraction of recurring words from infant-directed speech," Proc. 33rd Annual Conference of the Cognitive Science Society, 2011.
37. O. Räsänen, "A computational model of word segmentation from continuous speech using transitional probabilities of atomic acoustic events," Cognition, vol.120, no.2, pp.149-176, 2011.
38. P.D. Eimas, E.R. Siqueland, P. Jusczyk, and J. Vigorito, "Speech perception in infants," Science, vol.171, no.3968, p.303, 1971.
39. J. Van Gael, Y. Saatci, Y. Teh, and Z. Ghahramani, "Beam sampling for the infinite hiddenMarkov model," Proc. 25th International Conference on Machine Learning, 2008.
40. A. Asuncion, P. Smyth, and M. Welling, "Asynchronous distributed learning of topic models," Proc. 22nd Annual Conference on Neural Information Processing Systems, vol.21, 2008.