One-Shot Action Localization by Learning Sequence Matching Network

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn , Issue , 2018, Pages 1450-1459

  Yang, Hongtao ^a   He, Xuming ^b   Porikli, Fatih ^a  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^b SHANGHAITECH UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; SAMPLING;

DETECTION PROBLEMS; LEARNING SEQUENCES; LOCALIZATION METHOD; MATCHING NETWORKS; ONE-SHOT LEARNING; TRAINING SAMPLE; VIDEO DATASETS; VIDEO SEQUENCES;

LARGE DATASET;

EID: 85062826542     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2018.00157     Document Type: Conference Paper

References (51)

1. M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, T. Schaul, and N. de Freitas. Learning to learn by gradient descent by gradient descent. In Advances in Neural Information Processing Systems, pages 3981-3989, 2016.
2. H. Bilen, B. Fernando, E. Gavves, A. Vedaldi, and S. Gould. Dynamic image networks for action recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
3. P. Bojanowski, R. Lajugie, F. Bach, I. Laptev, J. Ponce, C. Schmid, and J. Sivic. Weakly supervised action labeling in videos under ordering constraints. In European Conference on Computer Vision, pages 628-643. Springer, 2014.
4. S. Buch, V. Escorcia, C. Shen, B. Ghanem, and J. C. Niebles. Sst: Single-stream temporal action proposals.
5. F. Caba Heilbron, J. Carlos Niebles, and B. Ghanem. Fast temporal activity proposals for efficient detection of human actions in untrimmed videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1914-1923, 2016.
6. J. Carreira and A. Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. arXiv preprint arXiv:1705.07750, 2017.
7. P. D. Human and animal cognition: Continuity and discontinuity. 104(35), 2007.
8. X. Dai, B. Singh, G. Zhang, L. S. Davis, and Y. Q. Chen. Temporal context network for activity localization in videos. arXiv preprint arXiv:1708.02349, 2017.
9. J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, June 2015.
10. B. G. Fabian Caba Heilbron, Victor Escorcia and J. C. Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 961-970, 2015.
11. C. Feichtenhofer, A. Pinz, and A. Zisserman. Convolutional two-stream network fusion for video action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1933-1941, 2016.
12. C. Finn, P. Abbeel, and S. Levine. Model-agnostic metalearning for fast adaptation of deep networks. International Conference on Machine Learning, 2017.
13. S. H. Frey and V. E. Gerry. Modulation of neural activity during observational learning of actions and their sequential orders. 26, 2006.
14. J. Gao, Z. Yang, C. Sun, K. Chen, and R. Nevatia. Turn tap: Temporal unit regression network for temporal action proposals. arXiv preprint arXiv:1703.06189, 2017.
15. R. Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 1440-1448, 2015.
16. S. Herath, M. Harandi, and F. Porikli. Going deeper into action recognition: A survey. arXiv preprint arXiv:1605.04988, 2016.
17. D.-A. Huang, L. Fei-Fei, and J. C. Niebles. Connectionist temporal modeling for weakly supervised action labeling. In European Conference on Computer Vision, pages 137-153. Springer, 2016.
18. Y.-G. Jiang, J. Liu, A. Roshan Zamir, G. Toderici, I. Laptev, M. Shah, and R. Sukthankar. THUMOS challenge: Action recognition with a large number of classes. http: //crcv.ucf.edu/THUMOS14/, 2014.
19. S. Karaman, L. Seidenari, and A. Del Bimbo. Fast saliency based pooling of fisher encoded dense trajectories. In ECCV THUMOS Workshop, volume 1, page 5, 2014.
20. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. Large-scale video classification with convolutional neural networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 1725-1732, 2014.
21. W. Kay, J. Carreira, K. Simonyan, B. Zhang, C. Hillier, S. Vijayanarasimhan, F. Viola, T. Green, T. Back, P. Natsev, et al. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950, 2017.
22. G. Koch, R. Zemel, and R. Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, volume 2, 2015.
23. C. Lea, M. D. Flynn, R. Vidal, A. Reiter, and G. D. Hager. Temporal convolutional networks for action segmentation and detection. 2017.
24. K. Li and J. Malik. Learning to optimize. arXiv preprint arXiv:1606.01885, 2016.
25. I. Lillo, J. Carlos Niebles, and A. Soto. A hierarchical posebased approach to complex action understanding using dictionaries of actionlets and motion poselets. In CVPR, pages 1981-1990, 2016.
26. T. Lin, X. Zhao, and Z. Shou. Uts at activitynet 2016. ActivityNet Large Scale Activity Recognition Challenge 2016, 2016.
27. T. Lin, X. Zhao, and Z. Shou. Temporal convolution based action proposal: Submission to activitynet 2017. arXiv preprint arXiv:1707.06750, 2017.
28. S. Ma, L. Sigal, and S. Sclaroff. Learning activity progression in lstms for activity detection and early detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1942-1950, 2016.
29. N. Mishra, M. Rohaninejad, X. Chen, and P. Abbeel. Meta-learning with temporal convolutions. arXiv preprint arXiv:1707.03141, 2017.
30. B. Ni, X. Yang, and S. Gao. Progressively parsing interactional objects for fine grained action detection. In CVPR, pages 1020-1028, 2016.
31. R. Poppe. A survey on vision-based human action recognition. Image and vision computing, 28(6):976-990, 2010.
32. S. Ravi and H. Larochelle. Optimization as a model for fewshot learning. International Conference on Learning Representations.
33. A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap. One-shot learning with memory-augmented neural networks. International Conference on Machine Learning, 2016.
34. Z. Shou, J. Chan, A. Zareian, K. Miyazawa, and S.-F. Chang. Cdc: Convolutional-de-convolutional networks for precise temporal action localization in untrimmed videos. 2017.
35. Z. Shou, D. Wang, and S.-F. Chang. Temporal action localization in untrimmed videos via multi-stage cnns. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1049-1058, 2016.
36. P. Shyam, S. Gupta, and A. Dukkipati. Attentive recurrent comparators. International Conference on Machine Learning, 2017.
37. K. Simonyan and A. Zisserman. Two-stream convolutional networks for action recognition in videos. In Advances in neural information processing systems, pages 568-576, 2014.
38. B. Singh, T. K. Marks, M. Jones, O. Tuzel, and M. Shao. A multi-stream bi-directional recurrent neural network for finegrained action detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1961-1970, 2016.
39. J. Snell, K. Swersky, and R. S. Zemel. Prototypical networks for few-shot learning. arXiv preprint arXiv:1703.05175, 2017.
40. K. Soomro, A. Roshan Zamir, and M. Shah. UCF101: A dataset of 101 human actions classes from videos in the wild. In CRCV-TR-12-01, 2012.
41. K. Soomro and M. Shah. Unsupervised action discovery and localization in videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 696-705, 2017.
42. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri. Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision, pages 4489-4497, 2015.
43. O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al. Matching networks for one shot learning. In Advances in Neural Information Processing Systems, pages 3630-3638, 2016.
44. L. Wang, Y. Qiao, and X. Tang. Action recognition and detection by combining motion and appearance features. THUMOS14 Action Recognition Challenge, 1(2):2, 2014.
45. L. Wang, Y. Xiong, D. Lin, and L. Van Gool. Untrimmednets for weakly supervised action recognition and detection. arXiv preprint arXiv:1703.03329, 2017.
46. L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. Van Gool. Temporal segment networks: Towards good practices for deep action recognition. In European Conference on Computer Vision, pages 20-36. Springer, 2016.
47. Y. Xiong, Y. Zhao, L. Wang, D. Lin, and X. Tang. A pursuit of temporal accuracy in general activity detection. arXiv preprint arXiv:1703.02716, 2017.
48. H. Xu, A. Das, and K. Saenko. R-c3d: Region convolutional 3d network for temporal activity detection. arXiv preprint arXiv:1703.07814, 2017.
49. S. Yeung, O. Russakovsky, G. Mori, and L. Fei-Fei. Endto-end learning of action detection from frame glimpses in videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2678-2687, 2016.
50. Z. Yuan, J. C. Stroud, T. Lu, and J. Deng. Temporal action localization by structured maximal sums. 2017.
51. Y. Zhao, Y. Xiong, L. Wang, Z. Wu, D. Lin, and X. Tang. Temporal action detection with structured segment networks. arXiv preprint arXiv:1704.06228, 2017.