Factored temporal sigmoid belief networks for sequence learning

33rd International Conference on Machine Learning, ICML 2016

Volumn 3, Issue , 2016, Pages 1937-1946

  Song, Jiaming ^a   Gan, Zhe ^b   Carin, Lawrence ^b  

^a TSINGHUA UNIVERSITY   (China)

^b Duke University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; BAYESIAN NETWORKS; BLENDING; LEARNING ALGORITHMS; LEARNING SYSTEMS; SUPERVISED LEARNING;

CLASSIFICATION PERFORMANCE; GENERATIVE MODEL; MULTIPLE SEQUENCES; SEMI- SUPERVISED LEARNING; SEQUENCE CLASSIFICATION; SEQUENCE LEARNING; SIDE INFORMATION; TRANSITION MATRICES;

CLASSIFICATION (OF INFORMATION);

EID: 84998827890     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (35)

1. Bayer, J. and Osendorfer, C. Learning stochastic recurrent networks. In arXiv: 1411.7610, 2014.
2. Boulanger-Lewandowski, N., Bengio, Y., and Vincent, P. Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription. In ICML, 2012.
3. Chung, J., Gulcehre, C, Cho, K., and Bengio, Y. Gated feedback recurrent neural networks. In ICML, 2015a.
4. Chung, J., Kastner, K., Dinh, L., Goel, K., Courville, A., and Bengio, Y A recurrent latent variable model for sequential data. In NIPS, 2015b.
5. Fabius, O., van Amersfoort, J. R., and Kingma, D. P. Variational recurrent auto-encoders. In arXiv: 1412.6581, 2014.
6. Gammerman, A., Vovk, V, and Vapnik, V. Learning by transduction. In UAI, 1998.
7. Gan, Z., Chen, C, Henao, R., Carlson, D., and Carin, L. Scalable deep poisson factor analysis for topic modeling. In ICML, 2015a.
8. Gan, Z., Henao, R., Carlson, D., and Carin, L. Learning deep sigmoid belief networks with data augmentation. In AISTATS, 2015b.
9. Gan, Z., Li, C, Henao, R., Carlson, D., and Carin, L. Deep temporal sigmoid belief networks for sequence modeling. In NIPS, 2015c.
10. Graves, A. Generating sequences with recurrent neural networks. In arXiv: 1308.0850, 2013.
11. Hinton, G. E. Training products of experts by minimizing contrastive divergence. In Neural computation, 2002.
12. Hinton, G. E., Dayan, P., Frey, B. J., and Neal, R. M. The "wake-sleep" algorithm for unsupervised neural networks. In Science, 1995.
13. Hinton, G. E., Osindero, S., and Teh, Y. W. A fast learning algorithm for deep belief nets. In Neural Computation, 2006.
14. Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., and Saul, L. K. An introduction to variational methods for graphical models. In Machine Learning, 1999.
15. Kingma, D. P. and Welling, M. Auto-encoding variational bayes. In ICLR, 2014.
16. Kingma, D. P., Mohamed, S., Rezende, D. J., and Welling, M. Semi-supervised learning with deep generative models. In NIPS, 2014.
17. Kiros, R., Salakhutdinov, R., and Zemel, R. Multimodal neural language models. In ICML, 2014a.
18. Kiros, R., Zemel, R., and Salakhutdinov, R. R. A multiplicative model for learning distributed text-based attribute representations. In NIPS, 2014b.
19. Le, Q. V. and Mikolov, T. Distributed representations of sentences and documents. In ICML, 2014.
20. Li, C, Zhu, J., Shi, T., and Zhang, Bo. Max-margin deep generative models. In NIPS, 2015.
21. Liu, Y., Niculescu-Mizil, A., Lozano, A. C, and Lu, Y. Learning temporal causal graphs for relational timeseries analysis. In ICML, 2010.
22. Memisevic, R. and Hinton, G. Unsupervised learning of image transformations. In CVPR, 2007.
23. Mikolov, T, Sutskever, I., Chen, K., Corrado, G., and Dean, J. Distributed representations of words and phrases and their compositionality. In NIPS, 2013.
24. Mittelman, R., Kuipers, B., Savarese, S., and Lee, H. Structured recurrent temporal restricted boltzmann machines. In ICML, 2014.
25. Mnih, A. and Gregor, K. Neural variational inference and learning in belief networks. In ICML, 2014.
26. Neal, R. M. Connectionist learning of belief networks. In Artificial intelligence, 1992.
27. Rezende, D. J., Mohamed, S., and Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. In ICML, 2014.
28. Salakhutdinov, R. and Hinton, G. Deep boltzmann machines. In AISTATS, 2009.
29. Srivastava, N., Mansimov, E., and Salakhutdinov, R. Unsupervised learning of video representations using lstms. In ICML, 2015.
30. Sutskever, I. and Hinton, G. E. Learning multilevel distributed representations for high-dimensional sequences. In AISTATS, 2007.
31. Sutskever, I., Hinton, G., and Taylor, G. The recurrent temporal restricted boltzmann machine. In NIPS, 2009.
32. Sutskever, I., Martens, J., and Hinton, G. E. Generating text with recurrent neural networks. In ICML, 2011.
33. Taylor, G., Hinton, G., and Roweis, S. Modeling human motion using binary latent variables. In NIPS, 2006.
34. Taylor, G. W. and Hinton, G. E. Factored conditional restricted boltzmann machines for modeling motion style. In ICML, 2009.
35. Tieleman, T. and Hinton, G. E. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. In COURSERA: Neural Networks for Machine Learning, 2012.