A Hierarchical Deep Temporal Model for Group Activity Recognition

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2016-December, Issue , 2016, Pages 1971-1980

  Ibrahim, Mostafa S ^a   Muralidharan, Srikanth ^a   Deng, Zhiwei ^a   Vahdat, Arash ^a   Mori, Greg ^a  

^a SIMON FRASER UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; DYNAMICS;

BASELINE METHODS; GROUP ACTIVITY RECOGNITION; LONG SHORT TERM MEMORY; TEMPORAL DYNAMICS; TEMPORAL MODELING;

PATTERN RECOGNITION;

EID: 84986332689     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2016.217     Document Type: Conference Paper

References (42)

1. M. R. Amer, P. Lei, and S. Todorovic. Hirf: Hierarchical random field for collective activity recognition in videos. In Computer Vision-ECCV 2014, pages 572-585. Springer, 2014.
2. M. R. Amer, D. Xie, M. Zhao, S. Todorovic, and S.-C. Zhu. Cost-sensitive top-down/bottom-up inference for multiscale activity recognition. In Computer Vision-ECCV 2012, pages 187-200. Springer, 2012.
3. W. Choi and S. Savarese. A unified framework for multitarget tracking and collective activity recognition. In Computer Vision-ECCV 2012, pages 215-230. Springer, 2012.
4. W. Choi, K. Shahid, and S. Savarese. What are they doing?: Collective activity classification using spatio-temporal relationship among people. In Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on, pages 1282-1289. IEEE, 2009.
5. W. Choi, K. Shahid, and S. Savarese. Learning context for collective activity recognition. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 3273-3280. IEEE, 2011.
6. M. Danelljan, G. Häger, F. Shahbaz Khan, and M. Felsberg. Accurate scale estimation for robust visual tracking, 2014. BMVC.
7. Z. Deng, M. Zhai, L. Chen, Y. Liu, S. Muralidharan, M. Roshtkhari, and G. Mori. Deep structured models for group activity recognition. In British Machine Vision Conference (BMVC), 2015.
8. C. Direkoglu and N. E. O'Connor. Team activity recognition in sports. In Computer Vision-ECCV 2012, pages 69-83. Springer, 2012.
9. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. arXiv preprint arXiv:1411.4389, 2014.
10. A. Graves and N. Jaitly. Towards end-to-end speech recognition with recurrent neural networks. In Proceedings of the 31st International Conference on Machine Learning (ICML-14), pages 1764-1772, 2014.
11. H. Hajimirsadeghi, W. Yan, A. Vahdat, and G. Mori. Visual recognition by counting instances: A multi-instance cardinality potential kernel. CVPR, 2015.
12. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.
13. S. S. Intille and A. Bobick. Recognizing planned, multiperson action. Computer Vision and Image Understanding (CVIU), 81:414-445, 2001.
14. Y. Jia. Caffe: An open source convolutional architecture or fast feature embedding, 2013. http://caffe.berkeleyvision.org/.
15. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. CVPR, 2015.
16. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. Large-scale video classification with convolutional neural networks. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 1725-1732. IEEE, 2014.
17. D. E. King. Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research, 10:1755-1758, 2009.
18. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097-1105, 2012.
19. H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre. Hmdb: a large video database for human motion recognition. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 2556-2563. IEEE, 2011.
20. S. Kwak, B. Han, and J. H. Han. Multi-agent event detection: Localization and role assignment. In CVPR, 2013.
21. T. Lan, L. Sigal, and G. Mori. Social roles in hierarchical models for human activity recognition. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 1354-1361. IEEE, 2012.
22. T. Lan, L. Sigal, and G. Mori. Social roles in hierarchical models for human activity recognition. In Computer Vision and Pattern Recognition (CVPR), 2012.
23. T. Lan, Y.Wang,W. Yang, S. Robinovitch, and G. Mori. Discriminative latent models for recognizing contextual group activities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(8):1549-1562, 2012.
24. V. I. Morariu and L. S. Davis. Multi-agent event recognition in structured scenarios. In Computer Vision and Pattern Recognition (CVPR), 2011.
25. J. Y.-H. Ng, M. Hausknecht, S. Vijayanarasimhan, O. Vinyals, R. Monga, and G. Toderici. Beyond short snippets: Deep networks for video classification. CVPR, 2015.
26. R. Poppe. A survey on vision-based human action recognition. Image and vision computing, 28(6):976-990, 2010.
27. V. Ramanathan, B. Yao, and L. Fei-Fei. Social role discovery in human events. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 2475-2482. IEEE, 2013.
28. C. Schüldt, I. Laptev, and B. Caputo. Recognizing human actions: a local svm approach. In Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on, volume 3, pages 32-36. IEEE, 2004.
29. A. G. Schwing and R. Urtasun. Fully connected deep structured networks. arXiv preprint arXiv:1503.02351, 2015.
30. T. Shu, D. Xie, B. Rothrock, S. Todorovic, and S.-C. Zhu. Joint inference of groups, events and human roles in aerial videos. In CVPR, 2015.
31. B. Siddiquie, Y. Yacoob, and L. Davis. Recognizing plays in american football videos. Technical report, Technical report, University of Maryland, 2009.
32. K. Simonyan and A. Zisserman. Two-stream convolutional networks for action recognition in videos. In Advances in Neural Information Processing Systems, pages 568-576, 2014.
33. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
34. K. Soomro, A. R. Zamir, and M. Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012.
35. E. Swears, A. Hoogs, Q. Ji, and K. Boyer. Complex activity recognition using granger constrained dbn (gcdbn) in sports and surveillance video. In Computer Vision and Pattern Recognition (CVPR), June 2014.
36. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In CVPR, 2015.
37. J. J. Tompson, A. Jain, Y. LeCun, and C. Bregler. Joint training of a convolutional network and a graphical model for human pose estimation. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 1799-1807. Curran Associates, Inc., 2014.
38. S. Venugopalan, H. Xu, J. Donahue, M. Rohrbach, R. Mooney, and K. Saenko. Translating videos to natural language using deep recurrent neural networks. arXiv preprint arXiv:1412.4729, 2014.
39. H.Wang, A. Kläser, C. Schmid, and C.-L. Liu. Action recognition by dense trajectories. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 3169-3176. IEEE, 2011.
40. D. Weinland, R. Ronfard, and E. Boyer. A survey of visionbased methods for action representation, segmentation and recognition. Computer Vision and Image Understanding, 115(2):224-241, 2011.
41. S. Yeung, O. Russakovsky, N. Jin, M. Andriluka, G. Mori, and L. Fei-Fei. Every moment counts: Dense detailed labeling of actions in complex videos. arXiv preprint arXiv:1507.05738, 2015.
42. S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. H. S. Torr. Conditional random fields as recurrent neural networks. In International Conference on Computer Vision (ICCV), 2015.