Structured recurrent temporal restricted Boltzmann machines

31st International Conference on Machine Learning, ICML 2014

Volumn 5, Issue , 2014, Pages 3620-3628

  Mittelman, Roni ^a   Kuipers, Benjamin ^a   Savarese, Silvio ^b   Lee, Honglak ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

^b Stanford University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; GRAPHIC METHODS; LEARNING SYSTEMS; SPEECH RECOGNITION;

BENCHMARK DATASETS; DEPENDENCY STRUCTURES; FULL CONNECTIVITIES; GAUSSIAN GRAPHICAL MODELS; GRAPHICAL LASSOS; PREDICTION PERFORMANCE; RESTRICTED BOLTZMANN MACHINE; TIME SERIES MODELS;

TOPOLOGY;

EID: 84919828137     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Banerjee, O., Ghaoui, L. E., and D'aspremont, A. Model selection through sparse maximum Ukelihood estimation for multivariate Gaussian or binary data. Journal of Machine Learning Research, 9:485-516, 2008.
2. Bengio, Y., Boulanger-Lewandowski, N., and Pascanu, R. Advances in optimizing recurrent networks. In ICASSP, 2013.
3. Boulanger-Lewandowski, N., Bengio, Y, and Vincent, R Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription. In ICML, 2012.
4. Courville, A. C, Bergstra, J., and Bengio, Y. Unsupervised models of images by spike and-slab RBMs. In ICML, 2011a.
5. Courville, A. C, Bergstra, J., and Bengio, Y. A spike and slab restricted Boltzmann machine. In AISTATS, 2011b.
6. Dahl., G. E., Ranzato, M., Rahman, M. A., and Hinton, G. E. Phone recognition with the mean-covariance restricted Boltzmann machine. In NIPS, 2010.
7. Doshi, E, Wingate, D., Tenenbaum, J. B., and Roy, N. Infinite dynamic Bayesian networks. In ICML, 2011.
8. Eriedman, J., Hastie, T., and Tibshirani, R. Sparse inverse covariance estimation with the graphical lasso. Biostatistics, (5):432441, 2008.
9. Hinton, G. E. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8): 1771-1800, 2002.
10. Hinton, G. E. and Salakhutdinov, R. Reducing the dimensionality of data with neural networks. Science, 313: 504-507, 2006.
11. Lauritzen, S. L. Graphical Models. Oxford University Press, 1996.
12. Lee, H., Ekanadham, C, and Ng, A. Y. Sparse deep belief net model for visual area V2. In NIPS. 2008.
13. Martens, J. and Sutskever, I. Learning recurrent neural networks with Hessian-free optimization. In ICML, 2011.
14. Mohan, K., Chung, M., Han, S., Witten, D., Lee, S. I., and M, E. Structured learning of Gaussian graphical models. In NIPS. 2012.
15. Pascanu, R., Mikolov, T., and Bengio, Y. On the difficulty of training recurrent neural networks. In ICML, volume 28,2013.
16. Prabhakar, K., Oh, S. M., Wang, R, Abowd., G. D., and Rehg, J. M. Temporal causality for the analysis of visual events. In CVPR, 2010.
17. Rabiner, L. and Juang, B.-H. An introduction to hidden markov models. ASSP Magazine, IEEE, 3(1):4-16, 1986.
18. Ranzato, M. and Hinton, G. E. Modeling pixel means and covariances using factorized third-order bohzmann machines. In CVPR, 2010.
19. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. Learning internal representations by error propagation. In Rumelhart, D. E. and Mcclelland, J. L. (eds.). Parallel Distributed Processing: Explorations in the Microstruc-ture of Cognition, Volume I: Foundations, pp. 318-362. MIT Press, Cambridge, MA, 1986.
20. Smolensky, P. Information processing in dynamical systems: Foundations of harmony theory. In Rumelhart, D. E., McClelland, J. L., et al. (eds.). Parallel Distributed Processing: Volume 1: Foundations, pp. 194-281. MIT Press, Cambridge, 1987.
21. Songsiri, J. and Vandenberghe, L. Topology selection in graphical models of autoregressive processes. Journal of Machine Learning Research, 11:2671-2705, 2010.
22. Sutskever, I. and Hinton, G. E. Learning multilevel distributed representations for high-dimensional sequences. In AISTATS, 2007.
23. Sutskever, I., Hinton, G. E., and Graham, T. W. The recurrent temporal restricted Boltzmann machine. In NIPS, 2008.
24. Sutskever, I., Martens, J., Dahl, G. E., and Hinton, G. E. On the importance of initialization and momentum in deep learning. In ICML, 2013.
25. Taylor, G. W., Hinton, G. E., and Roweis, S. T. Two distributed-state models for generating high-dimensional time series. Journal of Machine Learning Research, 12: 1025-1068, 2011.