Modeling deep temporal dependencies with recurrent "Grammar Cells"

Advances in Neural Information Processing Systems

Volumn 3, Issue January, 2014, Pages 1925-1933

  Michalski, Vincent ^a   Memisevic, Roland ^b   Konda, Kishore ^a  

^a GOETHE UNIVERSITY   (Germany)

^b UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SCIENCE; LINEAR TRANSFORMATIONS; MATHEMATICAL TRANSFORMATIONS; TIME SERIES;

AUTO ENCODERS; LINEAR MODELING; MULTIPLE LAYERS; PREDICTION ACCURACY; RECURRENT NETWORKS;

RECURRENT NEURAL NETWORKS;

EID: 84937955008     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. R. Memisevic and G. E. Hinton. Unsupervised learning of image transformations. In Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, 2007.
2. B. A. Olshausen, C. Cadieu, J. Culpepper, and D. K. Warland. Bilinear models of natural images. 2007.
3. G. W. Taylor, G. E. Hinton, and S. T. Roweis. Two distributed-state models for generating high-dimensional time series. The Journal of Machine Learning Research, 12:1025-1068, 2011.
4. P. J. Werbos. Generalization of backpropagation with application to a recurrent gas market model. Neural Networks, 1(4):339-356, 1988.
5. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.
6. S. Hochreiter. Untersuchungen zu dynamischen neuronalen netzen. diploma thesis, institut für informatik, lehrstuhl prof. brauer, technische universität münchen. 1991.
7. I. Sutskever, J. Martens, and G. E. Hinton. Generating text with recurrent neural networks. In Proceedings of the 2011 International Conference on Machine Learning, 2011.
8. J. Schmidhuber. Learning complex, extended sequences using the principle of history compression. Neural Computation, 4(2):234-242, 1992.
9. R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.
10. J. Luttinen, T. Raiko, and A. Ilin. Linear state-space model with time-varying dynamics. In Machine Learning and Knowledge Discovery in Databases, pages 338-353. Springer, 2014.
11. V. Michalski. Neural networks for motion understanding: Diploma thesis. Master's thesis, Goethe-Universität Frankfurt, Frankfurt, Germany, 2013.
12. P. Földiák. Learning invariance from transformation sequences. Neural Computation, 3(2):194-200, 1991.
13. L. Wiskott and T. Sejnowski. Slow feature analysis: Unsupervised learning of invariances. Neural computation, 14(4):715-770, 2002.
14. R. Memisevic. Learning to relate images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1829-1846, 2013.
15. R. Memisevic and G. E. Hinton. Learning to represent spatial transformations with factored higher-order boltzmann machines. Neural Computation, 22(6):1473-1492, 2010.
16. R. Memisevic. Gradient-based learning of higher-order image features. In 2011 IEEE International Conference on Computer Vision, pages 1591-1598. IEEE, 2011.
17. Y. LeCun, F. J. Huang, and L. Bottou. Learning methods for generic object recognition with invariance to pose and lighting. In Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition, 2001.
18. D. Martin, Fowlkes C., D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of the Eigth IEEE International Conference on Computer Vision, Volume 2, pages 416-423, July 2001.
19. G. W. Taylor, G. E. Hinton, and S. T. Roweis. Modeling human motion using binary latent variables. In Advances in Neural Information Processing Systems 20, pages 1345-1352, 2007.
20. R. Memisevic and G. Exarchakis. Learning invariant features by harnessing the aperture problem. In Proceedings of the 30th International Conference on Machine Learning, 2013.
21. I. Sutskever, G. E. Hinton, and G. W. Taylor. The recurrent temporal restricted boltzmann machine. In Advances in Neural Information Processing Systems 21, pages 1601-1608, 2008.
22. Y. Bengio. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2(1):1-127, 2009. Also published as a book. Now Publishers, 2009.