Segmental spatiotemporal CNNs for fine-grained action segmentation

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 9907 LNCS, Issue , 2016, Pages 36-52

  Lea, Colin ^a   Reiter, Austin ^a   Vidal, René ^a   Hager, Gregory D ^a  

^a JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HUMAN ROBOT INTERACTION; INFERENCE ENGINES; MARKOV PROCESSES; SECURITY SYSTEMS;

ACTION CLASSIFICATIONS; ACTION RECOGNITION; ACTION SEGMENTATION; INFERENCE ALGORITHM; JOINT SEGMENTATION; ORDERS OF MAGNITUDE; SPATIO TEMPORAL FEATURES; VIDEO SURVEILLANCE;

COMPUTER VISION;

EID: 84990062965     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-319-46487-9_3     Document Type: Conference Paper

References (43)

1. Wang, H., Schmid, C.: Action recognition with improved trajectories. In: IEEE International Conference on Computer Vision (ICCV) (2013)
2. Sun, L., Jia, K., Yeung, D.Y., Shi, B.: Human action recognition using factorized spatio-temporal convolutional networks. In: IEEE International Conference on Computer Vision (ICCV) (2015)
3. Caba Heilbron, F., Escorcia, V., Ghanem, B., Carlos Niebles, J.: ActivityNet: a large-scale video benchmark for human activity understanding. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
4. Rohrbach, M., Rohrbach, A., Regneri, M., Amin, S., Andriluka, M., Pinkal, M., Schiele, B.: Recognizing fine-grained and composite activities using hand-centric features and script data. Int. J. Comput. Vis. (IJCV) 119, 346–373 (2015)
5. Fathi, A., Rehg, J.M.: Modeling actions through state changes. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)
6. Li, Y., Ye, Z., Rehg, J.M.: Delving into egocentric actions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
7. Cheron, G., Laptev, I., Schmid, C.: P-CNN: pose-based CNN features for action recognition. In: IEEE International Conference on Computer Vision (ICCV) (2015)
8. Ni, B., Paramathayalan, V.R., Moulin, P.: Multiple granularity analysis for finegrained action detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)
9. Vo, N.N., Bobick, A.F.: From stochastic grammar to Bayes network: probabilistic parsing of complex activity. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)
10. Hawkins, K.P., Bansal, S., Vo, N.N., Bobick, A.F.: Anticipating human actions for collaboration in the presence of task and sensor uncertainty. In: IEEE International Conference on Robotics and Automation (ICRA) (2014)
11. Gao, Y., Vedula, S.S., Reiley, C.E., Ahmidi, N., Varadarajan, B., Lin, H.C., Tao, L., Zappella, L., Béjar, B., Yuh, D.D., et al.: JHU-ISI gesture and skill assessment working set (JIGSAWS): a surgical activity dataset for human motion modeling. In: MICCAI Workshop: Modeling and Monitoring of Computer Assisted Interventions (M2CAI) (2014)
12. Zappella, L., Haro, B.B., Hager, G.D., Vidal, R.: Surgical gesture classification from video and kinematic data. Med. Image Anal. 17, 732–745 (2013)
13. Tao, L., Zappella, L., Hager, G.D., Vidal, R.: Surgical gesture segmentation and recognition. In: Medical Image Computing and Computer Assisted Intervention (MICCAI), pp. 339–346 (2013)
14. Lei, J., Ren, X., Fox, D.: Fine-grained kitchen activity recognition using RGB-D. In: ACM Conference on Ubiquitous Computing (UbiComp) (2012)
15. Morariu, V.I., Davis, L.S.: Multi-agent event recognition in structured scenarios. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)
16. Koppula, H.S., Gupta, R., Saxena, A.: Learning human activities and object affordances from RGB-D videos. Intl. J. Robot. Res. (IJRR) 32(8), 951–970 (2013)
17. van Kasteren, T., Englebienne, G., Kröse, B.J.: Activity recognition using semi- Markov models on real world smart home datasets. J. Ambient Intell. Smart Environ. 2, 311–325 (2010)
18. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference Learning Representations (ICLR) (2015)
19. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems (NIPS) (2012)
20. Sarawagi, S., Cohen, W.W.: Semi-Markov conditional random fields for information extraction. In: Advances in Neural Information Processing Systems (NIPS) (2004)
21. Stein, S., McKenna, S.J.: Combining embedded accelerometers with computer vision for recognizing food preparation activities. In: ACM Conference on Ubiquitous Computing (UbiComp) (2013)
22. Wang, H., Kläser, A., Schmid, C., Liu, C.L.: Action recognition by dense trajectories. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)
23. Pirsiavash, H., Ramanan, D.: Parsing videos of actions with segmental grammars. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)
24. Jain, M., van Gemert, J.C., Snoek, C.G.M.: What do 15,000 object categories tell us about classifying and localizing actions? In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
25. Pishchulin, L., Andriluka, M., Schiele, B.: Fine-grained activity recognition with holistic and pose based features. In: German Conference on Pattern Recognition (GCPR) (2014)
26. Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Largescale video classification with convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)
27. Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: Advances in Neural Information Processing Systems (NIPS), pp. 568–576 (2014)
28. Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: The IEEE International Conference on Computer Vision (ICCV) (2015)
29. Peng, X., Schmid, C.: Encoding feature maps of cnns for action recognition. In: CVPR, THUMOS Challenge 2015 Workshop (2015)
30. Ng, J.Y., Hausknecht, M.J., Vijayanarasimhan, S., Vinyals, O., Monga, R., Toderici, G.: Beyond short snippets: deep networks for video classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
31. Shi, Q., Cheng, L., Wang, L., Smola, A.: Human action segmentation and recognition using discriminative semi-Markov models. Intl. J. Comput. Vis. (IJCV) 93, 22–32 (2011)
32. Tang, K., Fei-Fei, L., Koller, D.: Learning latent temporal structure for complex event detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)
33. Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
34. Donahue, J., Hendricks, L.A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., Darrell, T.: Long-term recurrent convolutional networks for visual recognition and description. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
35. Hannun, A.Y., Case, C., Casper, J., Catanzaro, B.C., Diamos, G., Elsen, E., Prenger, R., Satheesh, S., Sengupta, S., Coates, A., Ng, A.Y.: Deep speech: scaling up end-to-end speech recognition. CoRR abs/1412.5567 (2014)
36. Abdel-Hamid, O., Deng, L., Yu, D.: Exploring convolutional neural network structures and optimization techniques for speech recognition. In: INTERSPEECH International Speech Communication Association (2013)
37. Lea, C., Vidal, R., Hager, G.D.: Learning convolutional action primitives for finegrained action recognition. In: IEEE International Conference on Robotics and Automation (ICRA) (2016)
38. Ian Goodfellow, Y.B., Courville, A.: Deep Learning. MIT Press (2016, in preparation). http://www.deeplearningbook.org/
39. Davis, J., Bobick, A.: The representation and recognition of action using temporal templates. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (1997)
40. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference Learning Representations (ICLR) (2014)
41. Chollet, F.: Keras (2015). https://github.com/fchollet/keras
42. Fathi, A., Farhadi, A., Rehg, J.M.: Understanding egocentric activities. In: IEEE International Conference on Computer Vision (ICCV) (2011)
43. Navarro, G.: A guided tour to approximate string matching. ACM Comput. Surv. 33, 31–88 (2001)