Structured inference networks for nonlinear state space models

31st AAAI Conference on Artificial Intelligence, AAAI 2017

Volumn , Issue , 2017, Pages 2101-2109

  Krishnan, Rahul G ^a   Shalit, Uri ^a   Sontag, David ^a  

^a NEW YORK UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION ALGORITHMS; ARTIFICIAL INTELLIGENCE; DEEP LEARNING; DEEP NEURAL NETWORKS; INFERENCE ENGINES; RECURRENT NEURAL NETWORKS; STATE SPACE METHODS;

INFERENCE NETWORK; NONLINEAR STATE SPACE MODELS; POSTERIOR DISTRIBUTIONS; PROBABILISTIC INTERPRETATION; REAL-WORLD DATASETS; SIMPLE FUNCTIONAL FORMS; STRUCTURED VARIATIONAL APPROXIMATIONS; UNIFIED ALGORITHM;

LEARNING ALGORITHMS;

EID: 85030484471     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. Archer, E.; Park, I. M.; Buesing, L.; Cunningham, J.; and Paninski, L. 2015. Black box variational inference for state space models. arXiv preprint arXiv:1511.07367.
2. Bayer, J., and Osendorfer, C. 2014. Learning stochastic recurrent networks. arXiv preprint arXiv:1411.7610.
3. Boulanger-lewandowski, N.; Bengio, Y.; and Vincent, P. 2012. Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription. In ICML 2012.
4. Briegel, T., and Tresp, V. 1999. Fisher scoring and a mixture of modes approach for approximate inference and learning in nonlinear state space models. In NIPS 1999.
5. Chung, J.; Gulcehre, C.; Cho, K.; and Bengio, Y. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555.
6. Chung, J.; Kastner, K.; Dinh, L.; Goel, K.; Courville, A.; and Bengio, Y. 2015. A recurrent latent variable model for sequential data. In NIPS 2015.
7. Duckworth, D. 2016. Kalman filter, kalman smoother, and em library for python. https://pykalman.github.io/. Accessed: 2016-02-24.
8. Fabius, O., and van Amersfoort, J. R. 2014. Variational recurrent auto-encoders. arXiv:1412.6581.
9. Fraccaro, M.; Sønderby, S. K.; Paquet, U.; and Winther, O. 2016. Sequential neural models with stochastic layers. In NIPS 2016.
10. Gan, Z.; Li, C.; Henao, R.; Carlson, D. E.; and Carin, L. 2015. Deep temporal sigmoid belief networks for sequence modeling. In NIPS 2015.
11. Gao, Y.; Archer, E.; Paninski, L.; and Cunningham, J. P. 2016. Linear dynamical neural population models through nonlinear embeddings. In NIPS 2016.
12. Ghahramani, Z., and Roweis, S. T. 1999. Learning nonlinear dynamical systems using an EM algorithm. In NIPS 1999.
13. Gregor, K.; Danihelka, I.; Graves, A.; Rezende, D. J.; and Wierstra, D. 2015. DRAW: A recurrent neural network for image generation. In ICML 2015.
14. Gu, S.; Ghahramani, Z.; and Turner, R. E. 2015. Neural adaptive sequential monte carlo. In NIPS 2015.
15. Hinton, G. E.; Dayan, P.; Frey, B. J.; and Neal, R. M. 1995. The "wake-sleep" algorithm for unsupervised neural networks. Science 268.
16. Johnson, M. J.; Duvenaud, D.; Wiltschko, A. B.; Datta, S. R.; and Adams, R. P. 2016. Structured VAEs: Composing probabilistic graphical models and variational autoencoders. In NIPS 2016.
17. Kaae Sønderby, C.; Raiko, T.; Maaløe, L.; Kaae Sønderby, S.; and Winther, O. 2016. How to Train Deep Variational Autoencoders and Probabilistic Ladder Networks. arXiv e-prints.
18. Kingma, D., and Ba, J. 2015. Adam: A method for stochastic optimization. In ICLR 2015.
19. Kingma, D. P., and Welling, M. 2014. Auto-encoding variational bayes. In ICLR 2014.
20. Mnih, A., and Gregor, K. 2014. Neural variational inference and learning in belief networks. In ICML 2014.
21. Pearl, J. 2009. Causality. Cambridge university press.
22. Raiko, T., and Tornio, M. 2009. Variational bayesian learning of nonlinear hidden state-space models for model predictive control. Neurocomputing 72(16):3704-3712.
23. Raiko, T.; Tornio, M.; Honkela, A.; and Karhunen, J. 2006. State inference in variational bayesian nonlinear state-space models. In International Conference on ICA and Signal Separation 2006.
24. Rezende, D. J., and Mohamed, S. 2015. Variational inference with normalizing flows. In ICML 2015.
25. Rezende, D. J.; Mohamed, S.; and Wierstra, D. 2014. Stochastic backpropagation and approximate inference in deep generative models. In ICML 2014.
26. Schön, T. B.; Wills, A.; and Ninness, B. 2011. System identification of nonlinear state-space models. Automatica 47(1):39-49.
27. Theano Development Team. 2016. Theano: A Python framework for fast computation of mathematical expressions. abs/1605.02688.
28. Tran, D.; Ranganath, R.; and Blei, D. M. 2016. The variational gaussian process. In ICLR 2016.
29. Valpola, H., and Karhunen, J. 2002. An unsupervised ensemble learning method for nonlinear dynamic state-space models. Neural computation 14(11):2647-2692.
30. Wan, E. A., and Nelson, A. T. 1996. Dual kalman filtering methods for nonlinear prediction, smoothing and estimation. In NIPS 1996.
31. Wan, E.; Van Der Merwe, R.; et al. 2000. The unscented kalman filter for nonlinear estimation. In AS-SPCC 2000.
32. Watter, M.; Springenberg, J. T.; Boedecker, J.; and Riedmiller, M. 2015. Embed to control: A locally linear latent dynamics model for control from raw images. In NIPS 2015.