Sequential neural models with stochastic layers

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 2207-2215

  Fraccaro, Marco ^a   Sønderby, Søren Kaae ^b   Paquet, Ulrich ^c   Winther, Ole ^a,b  

^a TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

^b UNIVERSITY OF COPENHAGEN   (Denmark)

^c DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER MUSIC; RECURRENT NEURAL NETWORKS; STATE SPACE METHODS; STOCHASTIC SYSTEMS; STORMS; UNCERTAINTY ANALYSIS;

POLYPHONIC MUSIC; POSTERIOR DISTRIBUTIONS; RECURSIVE STRUCTURE; STATE - SPACE MODELS; STATE OF THE ART; STOCHASTIC LAYERS; STOCHASTIC RECURRENT NEURAL NETWORK; VARIATIONAL INFERENCE;

STOCHASTIC MODELS;

EID: 85019203505     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. E. Archer, I. M. Park, L. Buesing, J. Cunningham, and L. Paninski. Black box variational inference for state space models. arXiv:1511.07367, 2015.
2. F. Bastien, P. Lamblin, R. Pascanu, J. Bergstra, I. Goodfellow, A. Bergeron, N. Bouchard, D. Warde-Farley, and Y. Bengio. Theano: new features and speed improvements. arXiv:1211.5590, 2012.
3. J. Bayer and C. Osendorfer. Learning stochastic recurrent networks. arXiv:1411.7610, 2014.
4. N. Boulanger-Lewandowski, Y. Bengio, and P. Vincent. Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription. arXiv:1206.6392, 2012.
5. K. Cho, B. Van Merriënboer, Ç. Gülçehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP, pages 1724-1734, 2014.
6. J. Chung, C. Gulcehre, K. Cho, and Y. Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555, 2014.
7. J. Chung, K. Kastner, L. Dinh, K. Goel, A. C. Courville, and Y. Bengio. A recurrent latent variable model for sequential data. In NIPS, pages 2962-2970, 2015.
8. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39(1), 1977.
9. S. Dieleman, J. Schlüter, C. Raffel, E. Olson, S. K. Sønderby, D. Nouri, E. Battenberg, and A. van den Oord. Lasagne: First release, 2015.
10. A. Doucet, N. de Freitas, and N. Gordon. An introduction to sequential Monte Carlo methods. In Sequential Monte Carlo Methods in Practice, Statistics for Engineering and Information Science. 2001.
11. O. Fabius and J. R. van Amersfoort. Variational recurrent auto-encoders. arXiv:1412.6581, 2014.
12. Z. Gan, C. Li, R. Henao, D. E. Carlson, and L. Carin. Deep temporal sigmoid belief networks for sequence modeling. In NIPS, pages 2458-2466, 2015.
13. D. Geiger, T. Verma, and J. Pearl. Identifying independence in Bayesian networks. Networks, 20:507-534, 1990.
14. K. Gregor, I. Danihelka, A. Graves, and D. Wierstra. DRAW: A recurrent neural network for image generation. In ICML, 2015.
15. S. Gu, Z. Ghahramani, and R. E. Turner. Neural adaptive sequential Monte Carlo. In NIPS, pages 2611-2619, 2015.
16. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, Nov. 1997.
17. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. An introduction to variational methods for graphical models. Machine Learning, 37(2):183-233, 1999.
18. S. King and V. Karaiskos. The Blizzard challenge 2013. In The Ninth Annual Blizzard Challenge, 2013.
19. D. Kingma and M. Welling. Auto-encoding variational Bayes. In ICLR, 2014.
20. R. G. Krishnan, U. Shalit, and D. Sontag. Deep Kalman filters. arXiv:1511.05121, 2015.
21. A. Mnih and K. Gregor. Neural variational inference and learning in belief networks. arXiv:1402.0030, 2014.
22. J. W. Paisley, D. M. Blei, and M. I. Jordan. Variational Bayesian inference with stochastic search. In ICML, 2012.
23. D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In ICML, pages 1278-1286, 2014.
24. S. Roweis and Z. Ghahramani. A unifying review of linear Gaussian models. Neural Computation, 11(2):305-45, 1999.