Deep predictive coding networks for video prediction and unsupervised learning

5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

Volumn , Issue , 2017, Pages

  Lotter, William ^a   Kreiman, Gabriel ^a   Cox, David ^a  

^a HARVARD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FORECASTING; LEARNING ALGORITHMS; MACHINE LEARNING; NETWORK CODING; NETWORK LAYERS; NEURAL NETWORKS; OBJECT RECOGNITION; UNSUPERVISED LEARNING; VIDEO RECORDING; VIDEO SIGNAL PROCESSING;

IMPLICIT LEARNING; INTERNAL REPRESENTATION; NEUROSCIENCE LITERATURE; OBJECT AND SCENES; OBJECT PARAMETERS; PREDICTIVE CODING; PREDICTIVE NEURAL NETWORK; VIDEO PREDICTION;

DEEP LEARNING;

EID: 85088229806     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (59)

1. Pulkit Agrawal, João Carreira, and Jitendra Malik. Learning to see by moving. CoRR, 2015.
2. Andre M. Bastos, W. Martin Usrey, Rick A. Adams, George R. Mangun, Pascal Fries, and Karl J. Friston. Canonical microcircuits for predictive coding. Neuron, 2012.
3. Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. CoRR, 2015.
4. Yoshua Bengio. How auto-encoders could provide credit assignment in deep networks via target propagation. CoRR, 2014.
5. Riccardo Biasini, George Hotz, Sam Khalandovsky, Eder Santana, and Niel van der Westhuizen. Comma.ai research, 2016. URL https://github.com/commaai/research.
6. Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D. Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, Xin Zhang, Jake Zhao, and Karol Zieba. End to end learning for self-driving cars. CoRR, 2016.
7. Bert De Brabandere, Xu Jia, Tinne Tuytelaars, and Luc Van Gool. Dynamic filter networks. CoRR, 2016.
8. Rakesh Chalasani and Jose C. Principe. Deep predictive coding networks. CoRR, 2013.
9. François Chollet. Comma.ai, 2016. URL http://keras.io/.
10. Andy Clark. Whatever next? predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 2013.
11. Piotr Dollár, Christian Wojek, Bernt Schiele, and Pietro Perona. Pedestrian detection: A benchmark. In CVPR, 2009.
12. Tobias Egner, Jim M. Monti, and Christopher Summerfield. Expectation and surprise determine neural population responses in the ventral visual stream. J Neurosci, 2010.
13. Chelsea Finn, Ian J. Goodfellow, and Sergey Levine. Unsupervised learning for physical interaction through video prediction. CoRR, 2016.
14. Peter Földiák. Learning invariance from transformation sequences. Neural Computation, 1991.
15. Karl Friston. A theory of cortical responses. Philos Trans R Soc Lond B Biol Sci, 2005.
16. Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. International Journal of Robotics Research (IJRR), 2013.
17. Dileep George and Jeff Hawkins. A hierarchical bayesian model of invariant pattern recognition in the visual cortex. In Proceedings of the International Joint Conference on Neural Networks. IEEE, 2005.
18. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS. 2014.
19. Ross Goroshin, Joan Bruna, Jonathan Tompson, David Eigen, and Yann LeCun. Unsupervised learning of spatiotemporally coherent metrics. CoRR, 2015a.
20. Ross Goroshin, Michaël Mathieu, and Yann LeCun. Learning to linearize under uncertainty. CoRR, 2015b.
21. Sepp Hochreiter and Jurgen Schmidhuber. Long short-term memory. Neural Computation, 1997.
22. Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014.
23. Kevin Jarrett, Koray Kavukcuoglu, MarcAurelio Ranzato, and Yann LeCun. What is the best multistage architecture for object recognition? In ICCV. 2009.
24. Ryota Kanai, Yutaka Komura, Stewart Shipp, and Karl Friston. Cerebral hierarchies: predictive processing, precision and the pulvinar. Philos Trans R Soc Lond B Biol Sci, 2015.
25. Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, 2014.
26. Tejas D. Kulkarni, Will Whitney, Pushmeet Kohli, and Joshua B. Tenenbaum. Deep convolutional inverse graphics network. CoRR, 2015.
27. Tai Sing Lee and David Mumford. Hierarchical bayesian inference in the visual cortex. J Opt Soc Am A Opt Image Sci Vis, 2003.
28. William Lotter, Gabriel Kreiman, and David Cox. Unsupervised learning of visual structure using predictive generative networks. CoRR, 2015.
29. Davide Maltoni and Vincenzo Lomonaco. Semi-supervised tuning from temporal coherence. CoRR, 2015.
30. Michaël Mathieu, Camille Couprie, and Yann LeCun. Deep multi-scale video prediction beyond mean square error. ICLR, 2016.
31. Vincent Michalski, Roland Memisevic, and Kishore Konda. Modeling deep temporal dependencies with recurrent”grammar cells”. In NIPS. 2014.
32. Hossein Mohabi, Ronan Collobert, and Jason Weston. Deep learning from temporal coherence in video. In ICML. 2009.
33. Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, Richard L. Lewis, and Satinder P. Singh. Action-conditional video prediction using deep networks in atari games. CoRR, 2015.
34. Randall C. O'Reilly, Dean Wyatte, and John Rohrlich. Learning through time in the thalamocortical loops. CoRR, 2014.
35. Rasmus Berg Palm. Prediction as a candidate for learning deep hierarchical models of data. Master's thesis, Technical University of Denmark, 2012.
36. Viorica Patraucean, Ankur Handa, and Roberto Cipolla. Spatio-temporal video autoencoder with differentiable memory. CoRR, 2015.
37. Mohammad Pezeshki, Linxi Fan, Philemon Brakel, Aaron C. Courville, and Yoshua Bengio. Deconstructing the ladder network architecture. CoRR, 2015.
38. Nicolas Pinto, David Doukhan, James J. DiCarlo, and David D. Cox. A high-throughput screening approach to discovering good forms of biologically inspired visual representation. PLoS Comput Biol, 2009.
39. Marc'Aurelio Ranzato, Arthur Szlam, Joan Bruna, Michaël Mathieu, Ronan Collobert, and Sumit Chopra. Video (language) modeling: a baseline for generative models of natural videos. CoRR, 2014.
40. Rajesh P. N. Rao and Dana H. Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 1999.
41. Rajesh P. N. Rao and T. J. Sejnowski. Predictive sequence learning in recurrent neocortical circuits. NIPS, 2000.
42. Antti Rasmus, Harri Valpola, Mikko Honkala, Mathias Berglund, and Tapani Raiko. Semi-supervised learning with ladder network. CoRR, 2015.
43. Eder Santana and George Hotz. Learning a driving simulator. CoRR, 2016.
44. Andrew Saxe, Maneesh Bhand, Zhenghao Chen, Pang Wei Koh, Bipin Suresh, and Andrew Y. Ng. On random weights and unsupervised feature learning. In Workshop: Deep Learning and Unsupervised Feature Learning (NIPS). 2010.
45. Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. CoRR, 2015.
46. Singular Inversions, Inc. FaceGen. http://facegen.com.
47. William R. Softky. Unsupervised pixel-prediction. NIPS, 1996.
48. M. W. Spratling. Unsupervised learning of generative and discriminative weights encoding elementary image components in a predictive coding model of cortical function. Neural Computation, 2012.
49. Nitish Srivastava, Elman Mansimov, and Ruslan Salakhutdinov. Unsupervised learning of video representations using lstms. CoRR, 2015.
50. Christopher Summerfield, Tobias Egner, Matthew Greene, Etienne Koechlin, Jennifer Mangels, and Joy Hirsch. Predictive codes for forthcoming perception in the frontal cortex. Science, 314, 2006.
51. Lin Sun, Kui Jia, Tsung-Han Chan, Yuqiang Fang, Gang Wang, and Shuicheng Yan. Dl-sfa: Deeply-learned slow feature analysis for action recognition. CVPR, 2014.
52. Lucas Theis, Aaron van den Oord, and Matthias Bethge. A note on the evaluation of generative models. ICLR, 2016.
53. Harri Valpola. From neural pca to deep unsupervised learning. CoRR, 2015.
54. Carl Vondrick, Hamed Pirsiavash, and Antonio Torralba. Generating videos with scene dynamics. CoRR, 2016.
55. Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using videos. CoRR, 2015.
56. Zhou Wang, Alan Bovik, Hamid Sheikh, and Eero Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 2004.
57. William F. Whitney, Michael Chang, Tejas D. Kulkarni, and Joshua B. Tenenbaum. Understanding visual concepts with continuation learning. CoRR, 2016.
58. Laurenz Wiskott and Terrence J. Sejnowski. Learning invariance from transformation sequences. Neural Computation, 2002.
59. Tianfan Xue, Jiajun Wu, Katherine L. Bouman, and William T. Freeman. Visual dynamics: Probabilistic future frame synthesis via cross convolutional networks. CoRR, 2016.