Discovering hidden factors of variation in deep networks

3rd International Conference on Learning Representations, ICLR 2015 - Workshop Track Proceedings

Volumn , Issue , 2015, Pages

  Cheung, Brian ^a   Livezey, Jesse A ^a   Bansal, Arjun K ^b   Olshausen, Bruno A ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b Nervana Systems   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOENCODERS; CROSS-COVARIANCE; DEEP ARCHITECTURES; HANDWRITING STYLES; HANDWRITTEN DIGIT; REGULARIZATION TERMS; TORONTO;

DEEP LEARNING;

EID: 85083952152     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (30)

1. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097–1105.
2. David Eigen, Dilip Krishnan, and Rob Fergus, “Restoring an image taken through a window covered with dirt or rain,” in Computer Vision (ICCV), 2013 IEEE International Conference on. IEEE, 2013, pp. 633–640.
3. Joshua B Tenenbaum and William T Freeman, “Separating style and content with bilinear models,” Neural computation, vol. 12, no. 6, pp. 1247–1283, 2000.
4. David B Grimes and Rajesh PN Rao, “Bilinear sparse coding for invariant vision,” Neural computation, vol. 17, no. 1, pp. 47–73, 2005.
5. Bruno A Olshausen, Charles Cadieu, Jack Culpepper, and David K Warland, “Bilinear models of natural images,” in Electronic Imaging 2007. International Society for Optics and Photonics, 2007, pp. 649206–649206.
6. Geoffrey E Hinton, Alex Krizhevsky, and Sida D Wang, “Transforming auto-encoders,” in Artificial Neural Networks and Machine Learning–ICANN 2011, pp. 44–51. Springer, 2011.
7. Roland Memisevic and Geoffrey E Hinton, “Learning to represent spatial transformations with factored higher-order boltzmann machines,” Neural Computation, vol. 22, no. 6, pp. 1473–1492, 2010.
8. Pietro Berkes, Richard E Turner, and Maneesh Sahani, “A structured model of video reproduces primary visual cortical organisation,” PLoS computational biology, vol. 5, no. 9, pp. e1000495, 2009.
9. Charles F Cadieu and Bruno A Olshausen, “Learning intermediate-level representations of form and motion from natural movies,” Neural computation, vol. 24, no. 4, pp. 827–866, 2012.
10. Scott Reed, Kihyuk Sohn, Yuting Zhang, and Honglak Lee, “Learning to disentangle factors of variation with manifold interaction,” in Proceedings of The 31st International Conference on Machine Learning. ACM, 2014, p. 14311439.
11. Ruslan Salakhutdinov and Geoffrey E Hinton, “Learning a nonlinear embedding by preserving class neighbourhood structure,” in International Conference on Artificial Intelligence and Statistics, 2007, pp. 412–419.
12. Diederik P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling, “Semi-supervised learning with deep generative models,” in Advances in Neural Information Processing Systems, 2014, pp. 3581–3589.
13. Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot, and Yoshua Bengio, “Contractive auto-encoders: Explicit invariance during feature extraction,” in Proceedings of the 28th International Conference on Machine Learning (ICML-11), 2011, pp. 833–840.
14. Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion,” The Journal of Machine Learning Research, vol. 11, pp. 3371–3408, 2010.
15. Quoc V Le, “Building high-level features using large scale unsupervised learning,” in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013, pp. 8595–8598.
16. Yoshua Bengio, Pascal Lamblin, Dan Popovici, and Hugo Larochelle, “Greedy layer-wise training of deep networks,” Advances in neural information processing systems, vol. 19, pp. 153, 2007.
17. Salah Rifai, Yoshua Bengio, Aaron Courville, Pascal Vincent, and Mehdi Mirza, “Disentangling factors of variation for facial expression recognition,” in Computer Vision–ECCV 2012, pp. 808–822. Springer, 2012.
18. Matthew D Zeiler, “Adadelta: An adaptive learning rate method,” arXiv preprint arXiv:1212.5701, 2012.
19. Ian J Goodfellow, David Warde-Farley, Pascal Lamblin, Vincent Dumoulin, Mehdi Mirza, Razvan Pascanu, James Bergstra, Frédéric Bastien, and Yoshua Bengio, “Pylearn2: a machine learning research library,” arXiv preprint arXiv:1308.4214, 2013.
20. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio, “Generative adversarial nets,” in Advances in Neural Information Processing Systems, 2014, pp. 2672–2680.
21. Yann LeCun and Corinna Cortes, “The mnist database of handwritten digits,” 1998.
22. Ian J Goodfellow, David Warde-farley, Mehdi Mirza, Aaron Courville, and Yoshua Bengio, “Maxout networks,” in Proceedings of the 30th International Conference on Machine Learning. ACM, 2013, pp. 1319–1327.
23. J. Susskind, A. Anderson, and G. E. Hinton, “The toronto face database,” Tech. Rep., University of Toronto, 2010.
24. Ralph Gross, Iain Matthews, Jeffrey Cohn, Takeo Kanade, and Simon Baker, “Multi-pie,” Image and Vision Computing, vol. 28, no. 5, pp. 807–813, 2010.
25. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich, “Going deeper with convolutions,” arXiv preprint arXiv:1409.4842, 2014.
26. Jascha Sohl-Dickstein, Ben Poole, and Surya Ganguli, “Fast large-scale optimization by unifying stochastic gradient and quasi-newton methods,” in Proceedings of the 31st International Conference on Machine Learning (ICML-14), 2014, pp. 604–612.
27. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” The Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929–1958, 2014.
28. Olivier Breuleux, Yoshua Bengio, and Pascal Vincent, “Quickly generating representative samples from an rbm-derived process,” Neural Computation, vol. 23, no. 8, pp. 2058–2073, 2011.
29. Yoshua Bengio, Gregoire Mesnil, Yann Dauphin, and Salah Rifai, “Better mixing via deep representations,” in Proceedings of The 30th International Conference on Machine Learning, 2013, pp. 552–560.
30. Yoshua Bengio, Eric Laufer, Guillaume Alain, and Jason Yosinski, “Deep generative stochastic networks trainable by backprop,” in Proceedings of the 31st International Conference on Machine Learning (ICML-14), 2014, pp. 226–234.