CDC: Convolutional-de-convolutional networks for precise temporal action localization in untrimmed videos

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 1417-1426

  Shou, Zheng ^a   Chan, Jonathan ^a   Zareian, Alireza ^a   Miyazawa, Kazuyuki ^b   Chang, Shih Fu ^a  

^a Och Spine at New York Presbyterian Hospitals   (United States)

^b MITSUBISHI ELECTRIC CORPORATION   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; PATTERN RECOGNITION; SEMANTICS; SIGNAL SAMPLING;

ACTION SEMANTICS; COMPLEX BACKGROUND; CONVOLUTIONAL NETWORKS; FINE GRANULARITY; FRAMES PER SECONDS; HIGH-EFFICIENCY; STATE-OF-THE-ART SYSTEM; TEMPORAL DYNAMICS;

CONVOLUTION;

EID: 85044270610     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.155     Document Type: Conference Paper

References (79)

1. Mexaction2. http://mexculture.cnam.fr/xwiki/bin/view/Datasets/Mex+action+dataset, 2015.
2. Activitynet challenge 2016. http://activity-net.org/challenges/2016/, 2016.
3. J. K. Aggarwal and M. S. Ryoo. Human activity analysis: A review. In ACM Computing Surveys, 2011.
4. V. Badrinarayanan, A. Kendall, and R. Cipolla. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. TPAMI, 2016.
5. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Semantic image segmentation with deep con-volutional nets and fully connected crfs. In ICLR, 2015.
6. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. 2016.
7. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, 2015.
8. O. Duchenne, I. Laptev, J. Sivic, F. Bach, and J. Ponce. Automatic annotation of human actions in video. In ICCV, 2007.
9. V. Escorcia, F. C. Heilbron, J. C. Niebles, and B. Ghanem. Daps: Deep action proposals for action understanding. In ECCV, 2016.
10. C. Feichtenhofer, A. Pinz, and A. Zisserman. Convolutional two-stream network fusion for video action recognition. In CVPR, 2016.
11. A. Gaidon, Z. Harchaoui, and C. Schmid. Actom sequence models for efficient action detection. In CVPR, 2011.
12. A. Gaidon, Z. Harchaoui, and C. Schmid. Temporal localization of actions with actoms. In TPAMI, 2013.
13. C. Gan, N. Wang, Y. Yang, D.-Y. Yeung, and A. G. Hauptmann. Devnet: A deep event network for multimedia event detection and evidence recounting. In CVPR, 2015.
14. G. Gkioxari and J. Malik. Finding action tubes. In CVPR, 2015.
15. A. Gorban, H. Idrees, Y.-G. Jiang, A. R. Zamir, I. Laptev, M. Shah, and R. Sukthankar. THUMOS challenge: Action recognition with a large number of classes. http://www.thumos.info/, 2015.
16. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016.
17. F. C. Heilbron, V. Escorcia, B. Ghanem, and J. C. Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In CVPR, 2015.
18. F. C. Heilbron, J. C. Niebles, and B. Ghanem. Fast temporal activity proposals for efficient detection of human actions in untrimmed videos. In CVPR, 2016.
19. S. Hong, H. Noh, and B. Han. Decoupled deep neural network for semi-supervised semantic segmentation. In NIPS, 2015.
20. M. Jain, J. van Gemert, H. Jégou, P. Bouthemy, and C. Snoek. Action localization with tubelets from motion. In CVPR, 2014.
21. M. Jain, J. van Gemert, T. Mensink, and C. Snoek. Objects2action: Classifying and localizing actions without any video example. In ICCV, 2015.
22. M. Jain, J. van Gemert, and C. Snoek. What do 15, 000 object categories tell us about classifying and localizing actions? In CVPR, 2015.
23. H. Jégou, M. Douze, C. Schmid, and P. Pérez. Aggregating local descriptors into a compact image representation. In CVPR, 2010.
24. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Gir-shick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. In ACM MM, 2014.
25. Y.-G. Jiang, J. Liu, A. R. Zamir, G. Toderici, I. Laptev, M. Shah, and R. Sukthankar. THUMOS challenge: Action recognition with a large number of classes. http://crcv.ucf.edu/THUMOS14/, 2014.
26. S. Karaman, L. Seidenari, and A. D. Bimbo. Fast saliency based pooling of fisher encoded dense trajectories. In ECCV THUMOS Workshop, 2014.
27. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. Large-scale video classification with convo-lutional neural networks. In CVPR, 2014.
28. A. Kläser, M. Marszałek, C. Schmid, and A. Zisserman. Human focused action localization in video. In Trends and Topics in Computer Vision, 2012.
29. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
30. I. Laptev and P. Pérez. Retrieving actions in movies. In ICCV, 2007.
31. C. Lea, A. Reiter, R. Vidal, and G. D. Hager. Segmental spatiotemporal cnns for fine-grained action segmentation. In ECCV, 2016.
32. G. Lin, C. Shen, A. van den Hengel, and I. Reid. Efficient piecewise training of deep structured models for semantic segmentation. In CVPR, 2016.
33. S. Liu, X. Qi, J. Shi, H. Zhang, and J. Jia. Multi-scale patch aggregation (mpa) for simultaneous detection and segmentation. In CVPR, 2016.
34. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015.
35. M. A. M. Rohrbach, S. Amin and B. Schiele. A database for fine grained activity detection of cooking activities. In CVPR, 2012.
36. P. Mettes, J. van Gemert, and C. Snoek. Spot on: Action localization from pointly-supervised proposals. In ECCV, 2016.
37. H. Noh, S. Hong, and B. Han. Learning deconvolution network for semantic segmentation. In ICCV, 2015.
38. D. Oneata, J. Verbeek, and C. Schmid. Action and event recognition with fisher vectors on a compact feature set. In ICCV, 2013.
39. D. Oneata, J. Verbeek, and C. Schmid. The lear submission at thumos 2014. In ECCV THUMOS Workshop, 2014.
40. F. Perronnin, J. Sánchez, and T. Mensink. Improving the fisher kernel for large-scale image classification. In ECCV, 2010.
41. R. Poppe. A survey on vision-based human action recognition. In Image and vision computing, 2010.
42. M. M. Puscas, E. Sangineto, D. Culibrk, and N. Sebe. Un-supervised tube extraction using transductive learning and dense trajectories. In ICCV, 2015.
43. A. Richard and J. Gall. Temporal action detection using a statistical language model. In CVPR, 2016.
44. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJCV, 2015.
45. E. Shelhamer, J. Long, and T. Darrell. Fully convolutional networks for semantic segmentation. TPAMI, 2016.
46. Z. Shou, J. Chan, A. Zareian, K. Miyazawa, and S.-F. Chang. Cdc: Convolutional-de-convolutional networks for precise temporal action localization in untrimmed videos. arXiv preprint arXiv:1703.01515, 2017.
47. Z. Shou, D. Wang, and S.-F. Chang. Temporal action localization in untrimmed videos via multi-stage cnns. In CVPR, 2016.
48. G. A. Sigurdsson, O. Russakovsky, A. Farhadi, I. Laptev, and A. Gupta. Much ado about time: Exhaustive annotation of temporal data. In HCOMP, 2016.
49. G. A. Sigurdsson, G. Varol, X. Wang, A. Farhadi, I. Laptev, and A. Gupta. Hollywood in homes: Crowdsourcing data collection for activity understanding. In ECCV, 2016.
50. K. Simonyan and A. Zisserman. Two-stream convolutional networks for action recognition in videos. In NIPS, 2014.
51. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
52. B. Singh, T. K. Marks, M. Jones, O. Tuzel, and M. Shao. A multi-stream bi-directional recurrent neural network for finegrained action detection. In CVPR, 2016.
53. G. Singh and F. Cuzzolin. Untrimmed classification for activity detection: submission to activitynet challenge. In CVPR ActivityNet Workshop, 2016.
54. K. Soomro, H. Idrees, and M. Shah. Action localization in videos through context walk. In ICCV, 2015.
55. K. Soomro, H. Idrees, and M. Shah. Predicting the where and what of actors and actions through online action localization. In CVPR, 2016.
56. A. Stoian, M. Ferecatu, J. Benois-Pineau, and M. Crucianu. Fast action localization in large scale video archives. In TCSVT, 2015.
57. A. Stoian, M. Ferecatu, J. Benois-Pineau, and M. Crucianu. Scalable action localization with kernel-space hashing. In ICIP, 2015.
58. W. Sultani and M. Shah. What if we do not have multiple videos of the same action? - video action localization using web images. In CVPR, 2016.
59. C. Sun, S. Shetty, R. Sukthankar, and R. Nevatia. Temporal localization of fine-grained actions in videos by domain transfer from web images. In ACM MM, 2015.
60. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri. Learning spatiotemporal features with 3d convolutional networks. In ICCV, 2015.
61. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri. Deep end2end voxel2voxel prediction. In CVPR Workshop on Deep Learning in Computer Vision, 2016.
62. J. van Gemert, M. Jain, E. Gati, and C. Snoek. Apt: Action localization proposals from dense trajectories. In BMVC, 2015.
63. H. Wang, A. Kläser, C. Schmid, and C.-L. Liu. Action Recognition by Dense Trajectories. In CVPR, 2011.
64. H. Wang and C. Schmid. Action recognition with improved trajectories. In ICCV, 2013.
65. L. Wang, Y. Qiao, and X. Tang. Action recognition and detection by combining motion and appearance features. In ECCV THUMOS Workshop, 2014.
66. L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. V. Gool. Temporal segment networks: Towards good practices for deep action recognition. In ECCV, 2016.
67. R. Wang and D. Tao. Uts at activitynet 2016. In CVPR ActivityNet Workshop, 2016.
68. D. Weinland, R. Ronfard, and E. Boyer. A survey of vision-based methods for action representation, segmentation and recognition. In Computer Vision and Image Understanding, 2011.
69. P. Weinzaepfel, Z. Harchaoui, and C. Schmid. Learning to track for spatio-temporal action localization. In ICCV, 2015.
70. C. Xu and J. J. Corso. Actor-action semantic segmentation with grouping process models. In CVPR, 2016.
71. Z. Xu, Y. Yang, and A. G. Hauptmann. A discriminative cnn video representation for event detection. In CVPR, 2015.
72. S. Yeung, O. Russakovsky, N. Jin, M. Andriluka, G. Mori, and L. Fei-Fei. Every moment counts: Dense detailed labeling of actions in complex videos. arXiv preprint arXiv:1507.05738, 2015.
73. S. Yeung, O. Russakovsky, G. Mori, and L. Fei-Fei. End-to-end learning of action detection from frame glimpses in videos. In CVPR, 2016.
74. F. Yu and V. Koltun. Multi-scale context aggregation by dilated convolutions. In ICLR, 2016.
75. G. Yu and J. Yuan. Fast action proposals for human action detection and search. In CVPR, 2015.
76. J. Yuan, B. Ni, X. Yang, and A. Kassim. Temporal action localization with pyramid of score distribution features. In CVPR, 2016.
77. M. Zeiler and R. Fergus. Visualizing and understanding con-volutional networks. In ECCV, 2014.
78. M. Zeiler, D. Krishnan, G. W. Taylor, and R. Fergus. Decon-volutional networks. In CVPR, 2010.
79. S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. H. S. Torr. Conditional random fields as recurrent neural networks. In ICCV, 2015.