Self-supervised video representation learning with odd-one-out networks

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 5729-5738

  Fernando, Basura ^a   Bilen, Hakan ^b   Gavves, Efstratios ^c   Gould, Stephen ^a  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); LEARNING SYSTEMS; NEURAL NETWORKS; VIDEO RECORDING;

ACTION CLASSIFICATIONS; ACTION RECOGNITION; CONVOLUTIONAL NEURAL NETWORK; MANUAL ANNOTATION; STATE-OF-THE-ART METHODS; SUPERVISED LEARNING METHODS; TEMPORAL REPRESENTATIONS; VIDEO REPRESENTATIONS;

COMPUTER VISION;

EID: 85041924012     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.607     Document Type: Conference Paper

References (46)

1. S. Abu-El-Haija, N. Kothari, J. Lee, P. Natsev, G. Toderici, B. Varadarajan, and S. Vijayanarasimhan. YouTube-8M: A large-scale video classification benchmark. arXiv preprint arXiv:1609.08675, 2016. 1
2. P. Agrawal, J. Carreira, and J. Malik. Learning to see by moving. In ICCV, 2015. 1, 2
3. Y. Bengio and J. S. Bergstra. Slow, decorrelated features for pretraining complex cell-like networks. In NIPS, 2009. 1, 2
4. H. Bilen, B. Fernando, E. Gavves, and A. Vedaldi. Action recognition with dynamic image networks. CoRR, abs/1612.00738, 2016. 5
5. H. Bilen, B. Fernando, E. Gavves, A. Vedaldi, and S. Gould. Dynamic image networks for action recognition. In CVPR, 2016. 1, 2, 3, 5, 6, 8
6. H. Bourlard and Y. Kamp. Auto-association by multilayer perceptrons and singular value decomposition. Biological cybernetics, 59(4-5):291-294, 1988. 1, 2
7. C. Cadieu and B. A. Olshausen. Learning transformational invariants from natural movies. In NIPS, 2008. 2
8. R. S. Cruz, B. Fernando, A. Cherian, and S. Gould. Visual permutation learning. In CVPR, 2017. 1, 2
9. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009. 8
10. C. Doersch, A. Gupta, and A. A. Efros. Unsupervised visual representation learning by context prediction. In ICCV, 2015. 1, 2
11. J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. CoRR, abs/1310.1531, 2013. 6
12. A. Dosovitskiy, J. T. Springenberg, M. Riedmiller, and T. Brox. Discriminative unsupervised feature learning with convolutional neural networks. In NIPS, 2014. 1, 2
13. B. Fernando, P. Anderson, M. Hutter, and S. Gould. Discriminative hierarchical rank pooling for activity recognition. In CVPR, 2016. 2
14. B. Fernando, E. Gavves, J. Oramas, A. Ghodrati, and T. Tuytelaars. Modeling video evolution for action recognition. In CVPR, 2015. 2
15. B. Fernando, E. Gavves, J. Oramas, A. Ghodrati, and T. Tuytelaars. Rank pooling for action recognition. TPAMI, PP(99):1-1, 2016. 2, 3, 5
16. B. Fernando and S. Gould. Learning end-to-end video classification with rank-pooling. In ICML, 2016. 3
17. R. Hadsell, S. Chopra, and Y. LeCun. Dimensionality reduction by learning an invariant mapping. In CVPR, 2006. 8
18. S. Herath, M. Harandi, and F. Porikli. Going deeper into action recognition: A survey. Image and Vision Computing, 60:4-21, 2017. 3
19. G. E. Hinton and T. J. Sejnowski. Learning and releaming in boltzmann machines. Parallel distributed processing: Explorations in the microstructure of cognition, 1:282-317, 1986. 2
20. G. E. Hinton and R. S. Zemel. Autoencoders, minimum description length, and helmholtz free energy. NIPS, 1994. 1, 2
21. M. Jain, J. C. van Gemert, H. Jégou, P. Bouthemy, and C. G. M. Snoek. Action localization by tubelets from motion. In CVPR, 2014. 3
22. S. Ji, W. Xu, M. Yang, and K. Yu. 3d convolutional neural networks for human action recognition. PAMI, 35(1):221-231, 2013. 5
23. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. Large-scale video classification with convolutional neural networks. In CVPR, 2014. 1
24. A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, 2012. 3
25. H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre. Hmdb: a large video database for human motion recognition. In ICCV, 2011. 6
26. B. G. V. Kumar, G. Carneiro, and I. D. Reid. Learning local image descriptors with deep siamese and triplet convolutional networks by minimising global loss functions. CoRR, 2015. 2
27. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998. 1
28. H. Lee, P. Pham, Y. Largman, and A. Y. Ng. Unsupervised feature learning for audio classification using convolutional deep belief networks. In NIPS, 2009. 2
29. Y. Li, W. Li, V. Mahadevan, and N. Vasconcelos. Vlad3: Encoding dynamics of deep features for action recognition. In CVPR, 2016. 2
30. Z. Li, E. Gavves, M. Jain, and C. G. M. Snoek. Videolstm convolves, attends and flows for action recognition. arXiv preprint arXiv:1607.01794, 2016. 3
31. I. Misra, C. L. Zitnick, and M. Hebert. Unsupervised learning using sequential verification for action recognition. arXiv preprint arXiv:1603.08561, 2016. 2, 7, 8
32. H. Mobahi, R. Collobert, and J. Weston. Deep learning from temporal coherence in video. In ICML, 2009. 1, 2, 8
33. L. C. Pickup, Z. Pan, D. Wei, Y. Shih, C. Zhang, A. Zisser-man, B. Schölkopf, and W. T. Freeman. Seeing the arrow of time. In CVPR, 2014. 1, 2
34. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV, 115(3):211-252, 2015. 1
35. K. Soomro, A. R. Zamir, and M. Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012. 5
36. N. Srivastava, E. Mansimov, and R. Salakhutdinov. Unsupervised learning of video representations using lstms. In ICML, 2015. 2
37. L. Sun, K. Jia, D.-Y. Yeung, and B. E. Shi. Human action recognition using factorized spatio-temporal convolutional networks. In ICCV, 2015. 2, 3, 5
38. I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. In NIPS, 2014. 5
39. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri. Learning spatiotemporal features with 3d convolutional networks. arXiv preprint arXiv:1412.0767, 2014. 3
40. A. Vedaldi and K. Lenc. Matconvnet - convolutional neural networks for matlab. In Proceeding of the ACM Int. Conf. on Multimedia, 2015. 6
41. H. Wang, A. Kläser, C. Schmid, and C.-L. Liu. Dense trajectories and motion boundary descriptors for action recognition. IJCV, 103:60-79, 2013. 3
42. H. Wang and C. Schmid. Action recognition with improved trajectories. In ICCV, 2013. 3
43. L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. Van Gool. Temporal segment networks: towards good practices for deep action recognition. In ECCV, 2016. 2, 3, 5, 8
44. X. Wang and A. Gupta. Unsupervised learning of visual representations using videos. In ICCV, 2015. 1, 2, 8
45. L. Wiskott and T. J. Sejnowski. Slow feature analysis: Unsupervised learning of invariances. Neural computation, 14(4):715-770, 2002. 1, 2
46. S. Zha, F. Luisier, W. Andrews, N. Srivastava, and R. Salakhutdinov. Exploiting image-trained CNN architectures for unconstrained video classification. In BMVC, 2015. 3