Associative long short-term memory

33rd International Conference on Machine Learning, ICML 2016

Volumn 4, Issue , 2016, Pages 2929-2938

  Danihelka, Ivo ^a   Wayne, Greg ^a   Una, Benigno ^a   Kalchbrenner, Nal ^a   Graves, Alex ^a  

^a DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; ASSOCIATIVE PROCESSING; BRAIN; COMPLEX NETWORKS; HOLOGRAPHY; LEARNING SYSTEMS; RECURRENT NEURAL NETWORKS;

ASSOCIATIVE MEMORY; COMPLEX-VALUED; HOLOGRAPHIC REDUCED REPRESENTATIONS; LIMITED CAPACITY; LONG SHORT TERM MEMORY; NETWORK PARAMETERS;

SEARCH ENGINES;

EID: 84999054163     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. Arjovsky, Martin, Shah, Amar, and Bengio, Yoshua. Unitary evolution recurrent neural networks. arXiv preprint arXiv:1511.06464, 2015.
2. Gers, Felix A, Schmidhuber, Jiirgen, and Cummins, Fred. Learning to forget: Continual prediction with lstm. Neural computation, 12(10):2451-2471, 2000.
3. Graves, Alan, Mohamed, Abdel-rahman, and Hinton, Geoffrey. Speech recognition with deep recurrent neural networks. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, pp. 6645-6649. IEEE, 2013.
4. Graves, Alex. Generating sequences with recurrent neural networks. arXiv preprint arXiv: 1308.0850, 2013.
5. Graves, Alex, Wayne, Greg, and Danihelka, Ivo. Neural turing machines. arXiv preprint arXiv:1410.5401, 2014.
6. Grefenstette, Edward, Hermann, Karl Moritz, Suleyman, Mustafa, and Blunsom, Phil. Learning to transduce with unbounded memory. In Advances in Neural Information Processing Systems, pp. 1819-1827, 2015.
7. Hochreiter, Sepp and Schmidhuber, Jiirgen. Long short- term memory. Neural computation, 9(8): 1735-1780, 1997.
8. Hutter, Marcus. The human knowledge compression contest. 2006. URL http://prize.hutterl.net.
9. Joulin, Armand and Mikolov, Tomas. Inferring algorithmic patterns with stack-augmented recurrent nets. In Advances in Neural Information Processing Systems 28, pp. 190-198. 2015.
10. Jozefowicz, Rafal, Zaremba, Wojciech, and Sutskever, Ilya. An empirical exploration of recurrent network architectures. In Proceedings of the 32nd International Conference on Machine Learning (ICML-15), pp. 2342- 2350, 2015.
11. Kanerva, Pentti. Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors. Cognitive Computation, 1(2): 139-159, 2009.
12. Kingma, Diederik and Ba, Jimmy. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
13. Plate, Tony. Holographic reduced representation: Distributed representation for cognitive structures. 2003.
14. Russakovsky, Olga, Deng, Jia, Su, Hao, Krause, Jonathan, Satheesh, Sanjeev, Ma, Sean, Huang, Zhiheng, Karpa- thy, Andrej, Khosla, Aditya, Bernstein, Michael, Berg, Alexander C., and Fei-Fei, Li. Image Net Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211-252, 2015. doi:10.1007/sl1263-015-0816-y.
15. Sukhbaatar, Sainbayar, Weston, Jason, Fergus, Rob, et al. End-to-end memory networks. In Advances in Neural Information Processing Systems, pp. 2431-2439, 2015.
16. Sutskever, Ilya, Martens, James, and Hinton, Geoffrey E. Generating text with recurrent neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-U), pp. 1017-1024, 2011.
17. Sutskever, Ilya, Vinyals, Oriol, and Le, Quoc VV. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pp. 3104-3112, 2014.
18. Williams, Ronald J and Peng, Jing. An efficient gradient- based algorithm for on-line training of recurrent network trajectories. Neural computation, 2(4):490-501, 1990.
19. Zaremba, Wojciech and Sutskever, Ilya. Reinforcement learning neural turing machines. arXiv preprint arXiv: 1505.00521, 2015.