Learning to transduce with unbounded memory

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 1828-1836

  Grefenstette, Edward ^a   Hermann, Karl Moritz ^a   Suleyman, Mustafa ^a   Blunsom, Phil ^b  

^a DEEPMIND   (United Kingdom)

^b UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

BACTERIOPHAGES; INFORMATION SCIENCE;

CONTINUOUSLY DIFFERENTIABLE; GENERALISATION; MACHINE TRANSLATIONS; NATURAL LANGUAGES; RECURRENT NETWORKS; UNBOUNDED MEMORY;

RECURRENT NEURAL NETWORKS;

EID: 84965153738     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. Ilya Sutskever, Oriol Vinyals, and Quoc V. V. Le. Sequence to sequence learning with neural networks. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 3104-3112. Curran Associates, Inc., 2014.
2. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014.
3. G Z Sun, C Lee Giles, H H Chen, and Y C Lee. The neural network pushdown automaton: Model, stack and learning simulations. 1998.
4. Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. CoRR, abs/1410.5401, 2014.
5. Alex Graves. Supervised Sequence Labelling with Recurrent Neural Networks, volume 385 of Studies in Computational Intelligence. Springer, 2012.
6. Markus Dreyer, Jason R. Smith, and Jason Eisner. Latent-variable modeling of string transductions with finite-state methods. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP'08, pages 1080-1089, Stroudsburg, PA, USA, 2008. Association for Computational Linguistics.
7. Cyril Allauzen, Michael Riley, Johan Schalkwyk, Wojciech Skut, and Mehryar Mohri. Open-FST: A general and efficient weighted finite-state transducer library. In Implementation and Application of Automata, volume 4783 of Lecture Notes in Computer Science, pages 11-23. Springer Berlin Heidelberg, 2007.
8. Dekai Wu. Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. Computational linguistics, 23(3):377-403, 1997.
9. Alex Graves. Sequence transduction with recurrent neural networks. In Representation Learning Worksop, ICML. 2012.
10. Sreerupa Das, C Lee Giles, and Guo-Zheng Sun. Learning context-free grammars: Capabilities and limitations of a recurrent neural network with an external stack memory. In Proceedings of The Fourteenth Annual Conference of Cognitive Science Society. Indiana University, 1992.
11. Sreerupa Das, C Lee Giles, and Guo-Zheng Sun. Using prior knowledge in a {NNPDA} to learn context-free languages. Advances in neural information processing systems, 1993.
12. Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. Weakly supervised memory networks. CoRR, abs/1503.08895, 2015.
13. Wojciech Zaremba and Ilya Sutskever. Reinforcement learning neural turing machines. arXiv preprint arXiv:1505.00521, 2015.
14. Armand Joulin and Tomas Mikolov. Inferring algorithmic patterns with stack-augmented recurrent nets. arXiv preprint arXiv:1503.01007, 2015.
15. Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), pages 807-814, 2010.
16. Alfred V Aho and Jeffrey D Ullman. The theory of parsing, translation, and compiling. Prentice-Hall, Inc., 1972.
17. Dekai Wu and Hongsing Wong. Machine translation with a stochastic grammatical channel. In Proceedings of the 17th international conference on Computational linguistics-Volume 2, pages 1408-1415. Association for Computational Linguistics, 1998.
18. Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 4, 2012.
19. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. Understanding the exploding gradient problem. Computing Research Repository (CoRR) abs/1211.5063, 2012.
20. Zhi-Hua Zhou, Jianxin Wu, and Wei Tang. Ensembling neural networks: many could be better than all. Artificial intelligence, 137(1):239-263, 2002.