K-component recurrent neural network language models using curriculum learning

2013 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2013 - Proceedings

Volumn , Issue , 2013, Pages 1-6

  Shi, Yangyang ^a   Larson, Martha ^a   Jonker, Catholijn M ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Curriculum Learning; Language Models; Latent Dirichlet Allocation; Recurrent Neural Networks; Socio situational setting; Topics

Indexed keywords

ADDITIONAL EXPERIMENTS; LANGUAGE MODEL; LATENT DIRICHLET ALLOCATION; LONG-DISTANCE DEPENDENCIES; N-GRAM LANGUAGE MODELS; RELATIVE CONTRIBUTION; SOCIO-SITUATIONAL SETTINGS; TOPICS;

COMPUTATIONAL LINGUISTICS; FRACTURE MECHANICS; RECURRENT NEURAL NETWORKS; SEMANTICS; SPEECH RECOGNITION; STATISTICS;

CURRICULA;

EID: 84893696196     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ASRU.2013.6707696     Document Type: Conference Paper

References (17)

1. R. Rosenfeld, Two decades of statistical language modeling: Where do we go from here?, Proceedings of the IEEE, vol. 88, no. 8, pp. 1270-1278, 2000.
2. Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur, Recurrent neural network based language model, in Proceedings of Interspeech, 2010, pp. 1045-1048.
3. Yoshua Bengio, Jerome Louradour, Ronan Collobert, and Jason Weston, Curriculum learning, in Proceedings of International Conference on Machine Learning. 2009, pp. 41-48, ACM.
4. Jeffrey L. Elman, Learning and development in neural networks: The importance of starting small, Cognition, vol. 48, no. 1, pp. 71 - 99, 1993.
5. Yangyang Shi, PascalWiggers, and Catholijn M. Jonker, Socio-situational setting classification based on language use, in IEEE workshop on automatic speech recognition and understanding, 2011.
6. Rukmini Iyer, Mari Ostendorf, and J. Robin Rohlicek, Language modeling with sentence-level mixtures, in Proceedings of the workshop on Human Language Technology, 1994, pp. 82-87.
7. R. Iyer and M. Ostendorf, Modeling long distance dependence in language: Topic mixtures vs. dynamic cache models, in International Conference on Spoken Language Processing, 1996, vol. 1, pp. 236-239.
8. David M. Blei, Andrew Y. Ng, and Michael I. Jordan, Latent dirichlet allocation, Journal of Machine Learning Research, vol. 3, pp. 993-1022, Mar. 2003.
9. Yoshua Bengio, Rejean Ducharme, Pascal Vincent, and Christian Janvin, A neural probabilistic language model, Journal of Machine Learning Research, vol. 3, pp. 1137-1155, 2003.
10. T. Mikolov, S. Kombrink, L. Burget, J.H. Cernocky, and S. Khudanpur, Extensions of recurrent neural network language model, in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 2011, pp. 5528 -5531.
11. Tomas Mikolov, Anoop Deoras, Stefan Kombrink, Lukas Burget, and Jan ernock, Empirical evaluation and combination of advanced language modeling techniques, in Proceedings of Interspeech, 2011, pp. 605- 608.
12. Tomas Mikolov and Geoffrey Zweig, Context dependent recurrent neural network language model, in IEEE Workshop on Spoken Language Technology, 2012, pp. 234-239.
13. Yangyang Shi, Martha Larson, Pascal Wiggers, and Catholijn M Jonker, K-component adaptive recurrent neural network language models, in The proceeding of International conference of Text, Speech and Dialogue, 2013.
14. Tomas Mikolov, Anoop Deoras, Daniel Povey, Lukas Burget, and Jan Cernocky, Strategies for training large scale neural network language models, in IEEE Workshop on Automatic Speech Recognition and Understanding, 2011, pp. 196-201.
15. Nelleke Oostdijk, Wim Goedertier, Frank Van Eynde, Louis Boves, Jean pierre Martens, Michael Moortgat, and Harald Baayen, Experiences from the spoken dutch corpus project, in Araujo (eds), Proceedings of the Third International Conference on Language Resources and Evaluation, 2002, pp. 340-347.
16. A. van den Bosch, Scalable classification-based word prediction and confusible correction, Traitement Automatique des Langues, vol. 46, no. 2, pp. 39-63, 2006.
17. Wen Wang and Mary P. Harper, The superarv language model: Investigating the effectiveness of tightly integrating multiple knowledge sources, in Proceedings of Conference of Empirical Methods in Natural Language Processing, 2002, pp. 238-247.