Temporal convolutional networks for action segmentation and detection

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 1003-1012

  Lea, Colin ^a   Flynn, Michael D ^a   Vidal, René ^a   Reiter, Austin ^a   Hager, Gregory D ^a  

^a JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; CONVOLUTION; SECURITY SYSTEMS;

ACTION SEGMENTATION; CONVOLUTIONAL NETWORKS; ENCODER-DECODER; LONG-RANGE DEPENDENCIES; SEGMENT DURATION; SPATIO TEMPORAL FEATURES; STATE OF THE ART; TEMPORAL PATTERN;

LONG SHORT-TERM MEMORY;

EID: 85030216293     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.113     Document Type: Conference Paper

References (39)

1. M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghe-mawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org. 4
2. V. Badrinarayanan, A. Handa, and R. Cipolla. Seg-net: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling. arXiv preprint arXiv:1505.07293, 2015. 2
3. G. Cheron, I. Laptev, and C. Schmid. P-cnn: Pose-based cnn features for action recognition. 2015. 2
4. F. Chollet. Keras. https://github.com/fchollet/ keras, 2015. 4
5. J. Chung, Ç. Gülçehre, K. Cho, and Y. Bengio. Gated feedback recurrent neural networks. 2015. 1
6. A. Fathi, A. Farhadi, and J. M. Rehg. Understanding egocentric activities. 2011. 2, 6
7. A. Fathi and J. M. Rehg. Modeling actions through state changes. 2013. 2
8. A. Fathi, R. Xiaofeng, and J. M. Rehg. Learning to recognize objects in egocentric activities. 2011. 2, 6
9. A. Graves, S. Fernandez, and J. Schmidhuber. Bidirectional lstm networks for improved phoneme classification and recognition. In ICANN, 2005. 2, 6
10. D.-A. Huang, L. Fei-Fei, and J. C. Niebles. Connectionist Temporal Modeling for Weakly Supervised Action Labeling, pages 137-153. Springer International Publishing, Cham, 2016. 1, 2
11. M. Jain, J. C. van Gemert, and C. G. M. Snoek. What do 15,000 object categories tell us about classifying and localizing actions? 2015. 2
12. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. Large-scale video classification with convolutional neural networks. 2014. 2
13. D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. 2014. 4
14. H. Kuehne, A. Arslan, and T. Serre. The language of actions: Recovering the syntax and semantics of goal-directed human activities. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014. 2, 4
15. H. Kuehne, J. Gall, and T. Serre. An end-to-end generative framework for video segmentation and recognition. Lake Placid, Mar 2016. 2
16. C. Lea, A. Reiter, R. Vidal, and G. D. Hager. Segmental spatio-temporal CNNs for fine-grained action segmentation. 2016. 1, 2, 4, 6, 7
17. C. Lea, R. Vidal, and G. D. Hager. Learning convolutional action primitives for fine-grained action recognition. 2016. 4, 6
18. Y. Q. Limin Wang and X. Tang. Action recognition with trajectory-pooled deep-convolutional descriptors. 2015. 7
19. J. Y. Ng, M. J. Hausknecht, S. Vijayanarasimhan, O. Vinyals, R. Monga, and G. Toderici. Beyond short snippets: Deep networks for video classification. 2015. 2
20. B. Ni, V. R. Paramathayalan, and P. Moulin. Multiple granularity analysis for fine-grained action detection. 2014. 2
21. B. Ni, X. Yang, and S. Gao. Progressively parsing inter-actional objects for fine grained action detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. 1, 2
22. R. Pascanu, T. Mikolov, and Y. Bengio. On the difficulty of training recurrent neural networks. 2013. 1
23. X. Peng and C. Schmid. Encoding feature maps of cnns for action recognition. In CVPR, THUMOS Challenge 2015 Workshop, 2015. 2
24. H. Pirsiavash and D. Ramanan. Parsing videos of actions with segmental grammars. 2014. 1
25. A. Richard and J. Gall. Temporal action detection using a statistical language model. 2016. 1, 2, 4, 6
26. M. Rohrbach, A. Rohrbach, M. Regneri, S. Amin, M. An-driluka, M. Pinkal, and B. Schiele. Recognizing fine-grained and composite activities using hand-centric features and script data. 2015. 1, 2, 4
27. K. Simonyan and A. Zisserman. Two-stream convolutional networks for action recognition in videos. 2014. 2
28. B. Singh, T. K. Marks, M. Jones, O. Tuzel, and M. Shao. A multi-stream bi-directional recurrent neural network for fine-grained action detection. 2016. 1, 2, 4, 5, 6, 7
29. S. Singh, C. Arora, and C. V. Jawahar. First person action recognition using deep learned descriptors. June 2016. 2, 6, 7
30. S. Stein and S. J. McKenna. Combining embedded accelerometers with computer vision for recognizing food preparation activities. 2013. 2, 4, 6
31. L. Sun, K. Jia, D.-Y. Yeung, and B. Shi. Human action recognition using factorized spatio-temporal convolutional networks. 2015. 2
32. J. Tompson, R. Goroshin, A. Jain, Y. LeCun, and C. Bre-gler. Efficient object localization using convolutional networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015. 4
33. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri. Learning spatiotemporal features with 3d convolutional networks. 2015. 2
34. A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. W. Senior, and K. Kavukcuoglu. Wavenet: A generative model for raw audio. CoRR, abs/1609.03499, 2016. 2, 3
35. A. van den Oord, N. Kalchbrenner, O. Vinyals, L. Espeholt, A. Graves, and K. Kavukcuoglu. Conditional image generation with pixelcnn decoders. CoRR, abs/1606.05328, 2016. 7
36. A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. J. Lang. Readings in speech recognition. chapter Phoneme Recognition Using Time-delay Neural Networks, pages 393-404. 1990. 3
37. H. Wang and C. Schmid. Action recognition with improved trajectories. 2013. 2
38. F. Yu and V. Koltun. Multi-scale context aggregation by dilated convolutions. In ICLR, 2016. 2
39. Y. Zhou, B. Ni, R. Hong, M. Wang, and Q. Tian. Interaction part mining: A mid-level approach for fine-grained action recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015. 2