End-to-end memory networks

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 2440-2448

  Sukhbaatar, Sainbayar ^a   Szlam, Arthur ^b   Weston, Jason ^b   Fergus, Rob ^b  

^a POLYTECHNIC UNIVERSITY   (United States)

^b FACEBOOK AI RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SCIENCE; NATURAL LANGUAGE PROCESSING SYSTEMS;

ATTENTION MODEL; END TO END; EXTERNAL MEMORY; LANGUAGE MODEL; MEMORY NETWORK; QUESTION ANSWERING; TREEBANKS;

MODELING LANGUAGES;

EID: 84965143740     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. C. G. Atkeson and S. Schaal. Memory-based neural networks for robot learning. Neurocomputing, 9:243-269, 1995.
2. D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR), 2015.
3. Y. Bengio, R. Ducharme, P. Vincent, and C. Janvin. A neural probabilistic language model. J. Mach. Learn. Res., 3:1137-1155, Mar. 2003.
4. J. Chung, ç. Gülçehre, K. Cho, and Y. Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint: 1412.3555, 2014.
5. S. Das, C. L. Giles, and G.-Z. Sun. Learning context-free grammars: Capabilities and limitations of a recurrent neural network with an external stack memory. In In Proceedings of The Fourteenth Annual Conference of Cognitive Science Society, 1992.
6. J. Goodman. A bit of progress in language modeling. CoRR, cs. CL/0108005, 2001.
7. A. Graves. Generating sequences with recurrent neural networks. arXiv preprint: 1308.0850, 2013.
8. A. Graves, G. Wayne, and I. Danihelka. Neural turing machines. arXiv preprint: 1410.5401, 2014.
9. K. Gregor, I. Danihelka, A. Graves, and D. Wierstra. DRAW: A recurrent neural network for image generation. CoRR, abs/1502.04623, 2015.
10. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.
11. A. Joulin and T. Mikolov. Inferring algorithmic patterns with stack-augmented recurrent nets. NIPS, 2015.
12. J. Koutník, K. Greff, F. J. Gomez, and J. Schmidhuber. A clockwork RNN. In ICML, 2014.
13. M. P. Marcus, M. A. Marcinkiewicz, and B. Santorini. Building a large annotated corpus of english: The Penn Treebank. Comput. Linguist., 19(2):313-330, June 1993.
14. T. Mikolov. Statistical language models based on neural networks. Ph. D. thesis, Brno University of Technology, 2012.
15. T. Mikolov, A. Joulin, S. Chopra, M. Mathieu, and M. Ranzato. Learning longer memory in recurrent neural networks. arXiv preprint: 1412.7753, 2014.
16. M. C. Mozer and S. Das. A connectionist symbol manipulator that discovers the structure of context-free languages. NIPS, pages 863-863, 1993.
17. B. Peng, Z. Lu, H. Li, and K. Wong. Towards Neural Network-based Reasoning. ArXiv preprint: 1508.05508, 2015.
18. J. Pollack. The induction of dynamical recognizers. Machine Learning, 7(2-3):227-252, 1991.
19. K. Steinbuch and U. Piske. Learning matrices and their applications. IEEE Transactions on Electronic Computers, 12:846-862, 1963.
20. M. Sundermeyer, R. Schlüter, and H. Ney. LSTM neural networks for language modeling. In Interspeech, pages 194-197, 2012.
21. W. K. Taylor. Pattern recognition by means of automatic analogue apparatus. Proceedings of The Institution of Electrical Engineers, 106:198-209, 1959.
22. J. Weston, A. Bordes, S. Chopra, and T. Mikolov. Towards AI-complete question answering: A set of prerequisite toy tasks. arXiv preprint: 1502.05698, 2015.
23. J. Weston, S. Chopra, and A. Bordes. Memory networks. In International Conference on Learning Representations (ICLR), 2015.
24. K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. ArXiv preprint: 1502.03044, 2015.
25. W. Zaremba, I. Sutskever, and O. Vinyals. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329, 2014.