Hierarchical probabilistic neural network language model

AISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics

Volumn , Issue , 2005, Pages 246-252

  Morin, Frederic ^a   Bengio, Yoshua ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

CONDITIONAL PROBABILITIES; CONTINUOUS SPACES; HIER-ARCHICAL CLUSTERING; HIERARCHICAL DECOMPOSITIONS; IMPORTANCE SAMPLING METHOD; LANGUAGE MODEL; LANGUAGE MODELING; N-GRAM MODELS; PRIOR KNOWLEDGE; PROBABILISTIC NEURAL NETWORKS; SPEECH RECOGNIZER; STATISTICAL LANGUAGE MODELING; TRAINING EXAMPLE; WORDNET SEMANTICS;

NETWORK ARCHITECTURE; NEURAL NETWORKS; SEMANTICS;

COMPUTATIONAL LINGUISTICS;

EID: 34547997987     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Baker, D. and McCallum, A. (1998). Distributional clustering of words for text classification. In SIGIR'98.
2. Bengio, Y., Ducharme, R., and Vincent, P. (2001). A neural probabilistic language model. In Leen, T., Dietterich, T., and Tresp, V., editors, Advances in Neural Information Processing Systems 13, pages 933-938. MIT Press.
3. Bengio, Y., Ducharme, R., Vincent, P., and Jauvin, C. (2003). A neural probabilistic language model. Journal of Machine Learning Research, 3:1137-1155.
4. Bengio, Y. and Senécal, J.-S. (2003). Quick training of probabilistic neural nets by importance sampling. In Proceedings of AISTATS'2003.
5. Berger, A., Della Pietra, S., and Della Pietra, V. (1996). A maximum entropy approach to natural language processing. Computational Linguistics, 22:39-71.
6. Brown, P., Pietra, V. D., DeSouza, P., Lai, J., and Mercer, R. (1992). Class-based n-gram models of natural language. Computational Linguistics, 18:467-479.
7. Deerwester, S., Dumais, S., Furnas, G., Landauer, T., and Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6):391-407.
8. Elman, J. (1990). Finding structure in time. Cognitive Science, 14:179-211.
9. Fellbaum, C. (1998). WordNet: An Electronic Lexical Database. MIT Press.
10. Goodman, J. (2001a). A bit of progress in language modeling. Technical Report MSR-TR-2001-72, Microsoft Research.
11. Goodman, J. (2001b). Classes for fast maximum entropy training. In International Conference on Acoustics, Speech, and Signal Processing, Utah.
12. Hinton, G. (1986). Learning distributed representations of concepts. In Proceedings of the Eighth Annual Conference of the Cognitive Science Society, pages 1-12, Amherst 1986. Lawrence Erlbaum, Hillsdale.
13. Hinton, G. (2000). Training products of experts by minimizing contrastive divergence. Technical Report GCNU TR 2000-004, Gatsby Unit, University College London.
14. Jelinek, F. and Mercer, R. L. (1980). Interpolated estimation of Markov source parameters from sparse data. In Gelsema, E. S. and Kanal, L. N., editors, Pattern Recognition in Practice. North-Holland, Amsterdam.
15. Katz, S. M. (1987). Estimation of probabilities from sparse data for the language model component of a speech recognizer. IEEE Transactions on Acoustics, Speech, and Signal Processing, ASSP-35(3):400-401.
16. Miikkulainen, R. and Dyer, M. (1991). Natural language processing with modular neural networks and distributed lexicon. Cognitive Science, 15:343-399.
17. Ney, H. and Kneser, R. (1993). Improved clustering techniques for class-based statistical language modelling. In European Conference on Speech Communication and Technology (Eurospeech), pages 973-976, Berlin.
18. Niesler, T., Whittaker, E., and Woodland, P. (1998). Comparison of part-of-speech and automatically derived category-based language models for speech recognition. In International Conference on Acoustics, Speech, and Signal Processing, pages 177-180.
19. Pereira, F., Tishby, N., and Lee, L. (1993). Distributional clustering of english words. In 30th Annual Meeting of the Association for Computational Linguistics, pages 183-190, Columbus, Ohio.
20. Salton, G. and Buckley, C. (1988). Term weighting approaches in automatic text retrieval. Information Processing and Management, 24(5):513-523.
21. Schmidhuber, J. (1996). Sequential neural text compression. IEEE Transactions on Neural Networks, 7(1):142-146.
22. Schutze, H. (1993). Word space. In Giles, C., Hanson, S., and Cowan, J., editors, Advances in Neural Information Processing Systems 5, pages pp. 895-902, San Mateo CA. Morgan Kaufmann.
23. Schwenk, H. (2004). Efficient training of large neural networks for language modeling. In IEEE joint conference on neural networks.
24. Schwenk, H. and Gauvain, J.-L. (2002). Connectionist language modeling for large vocabulary continuous speech recognition. In International Conference on Acoustics, Speech, and Signal Processing, pages 765-768, Orlando, Florida.
25. Stolcke, A. (2002). SRILM - an extensible language modeling toolkit. In Proceedings of the International Conference on Statistical Language Processing, Denver, Colorado.
26. Xu, P., Emami, A., and Jelinek, F. (2003). Training connectionist models for the structured language model. In Empirical Methods in Natural Language Processing, EMNLP'2003.
27. Xu, W. and Rudnicky, A. (2000). Can artificial neural network learn language models. In International Conference on Statistical Language Processing, pages M1-13, Beijing, China.