Understanding collective activities of people from videos

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 36, Issue 6, 2014, Pages 1242-1257

  Choi, Wongun ^a   Savarese, Silvio ^b  

^a NEC LABORATORIES AMERICA   (United States)

^b Stanford University   (United States)

Author keywords

Collective activity recognition; tracking; tracklet association

Indexed keywords

SEMANTICS; SURFACE DISCHARGES;

ATOMIC ACTIVITIES; COLLECTIVE ACTIVITY RECOGNITION; COLLECTIVE CLASSIFICATIONS; CONTEXTUAL INFORMATION; INTERACTION ACTIVITIES; ROBUST TRACKING; SPATIAL-TEMPORAL NEIGHBORHOODS; TRACKLET ASSOCIATIONS;

CLASSIFICATION (OF INFORMATION);

ALGORITHM; AUTOMATED PATTERN RECOGNITION; BODY POSTURE; CLASSIFICATION; HUMAN; HUMAN ACTIVITIES; IMAGE PROCESSING; PHYSIOLOGY; PROCEDURES; VIDEORECORDING;

ALGORITHMS; HUMAN ACTIVITIES; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED; POSTURE; VIDEOTAPE RECORDING;

EID: 84901835746     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2013.220     Document Type: Article

References (45)

1. M.R. Amer, D. Xie, M. Zhao, S. Todorovic, and S.-C. Zhu, "Cost- Sensitive Top-Down/Bottom-Up Inference for Multiscale Activity Recognition," Proc. 12th European Conf. Computer Vision (ECCV), pp. 187-200, 2012.
2. S. Belongie, J. Malik, and J. Puzicha, "Shape Matching and Object Recognition Using Shape Contexts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 509-522, Apr. 2002. (Pubitemid 34448769)
3. L. Breiman, and A. Cutler, "Random Forest," [online], marzec 2004.
4. C.-C. Chang and C.-J. Lin, LIBSVM: A Library for Support Vector Machines, 2001.
5. W. Choi and S. Savarese, "Multiple Target Tracking in World Coordinate with Single, Minimally Calibrated Camera," Proc. 11th European Conf. Computer Vision (ECCV), Sept. 2010.
6. W. Choi and S. Savarese, "A Unified Framework for Multi-Target Tracking and Collective Activity Recognition," Proc. 12th European Conf. Computer Vision (ECCV), 2012.
7. W. Choi, K. Shahid, and S. Savarese, "What Are They Doing?: Collective Activity Classification Using Spatio-Temporal Relationship Among People," Proc. Workshop Visual Surveillance (VSWS), 2009.
8. W. Choi, K. Shahid, and S. Savarese, "Learning Context for Collective Activity Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2011.
9. F. Cupillard, F. Bremond, and M. Thonnat, "Group Behavior Recognition with Multiple Cameras," Proc. IEEE Sixth Workshop Applications of Computer Vision (WACV), pp. 177-183, 2002.
10. N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2005.
11. P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie, "Behavior Recognition via Sparse Spatio-Temporal Features," Proc. IEEE Second Joint Int'l Workshop Visual Surveillance and Performance Evaluation of Tracking and Surveillance (VS-PETS), 2005.
12. A. Ess, B. Leibe, K. Schindler, and L. van Gool, "A Mobile Vision System for Robust Multi-Person Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2008.
13. P. Felzenszwalb and D. Huttenlocher, "Efficient Belief Propagation for Early Vision," Int'l J. Computer Vision, vol. 70, no. 1, pp. 41-54, 2006. (Pubitemid 43855866)
14. A. Gupta, P. Srinivasan, J. Shi, and L.S. Davis, "Understanding Videos, Constructing Plots Learning a Visually Grounded Storyline Model From Annotated Videos," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 2012-2019, 2009.
15. D. Hoiem, A.A. Efros, and M. Herbert, "Putting Objects in Perspective," Int'l J. Computer Vision, vol. 80, no. 1, pp. 3-15, 2008.
16. S. Intille and A. Bobick, "Recognizing Planned, Multiperson Action," Computer Vision and Image Understanding, vol. 81, no. 3, pp. 414-445, 2001. (Pubitemid 32415984)
17. T. Joachims, T. Finley, and C.-N. Yu, "Cutting-Plane Training of Structural SVMs," Machine Learning, vol. 77, 2009.
18. S.M. Khan and M. Shah, "Detecting Group Activities Using Rigidity of Formation," Proc. ACM 13th Ann. Int'l Conf. Multimedia, pp. 403-406, 2005.
19. Z. Khan, T. Balch, and F. Dellaert, "MCMC-Based Particle Filtering for Tracking a Variable Number of Interacting Targets," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 11, pp. 1805-1819, 2005. (Pubitemid 41685115)
20. T. Lan, L. Sigal, and G. Mori, "Social Roles in Hierarchical Models for Human Activity Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 1354-1361, 2012.
21. T. Lan, Y. Wang, W. Yang, and G. Mori, "Beyond Actions: Discriminative Models for Contextual Group Activities," Proc. Advances in Neural Information Processing Systems (NIPS), 2010.
22. A.H. Land and A.G. Doig, "An Automatic Method of Solving Discrete Programming Problems," Econometrica, vol. 28, 1960.
23. L. Leal-Taixe, G. Pons-Moll, and B. Rosenhahn, "Everybody Needs Somebody: Modeling Social and Grouping Behavior on a Linear Programming Multiple People Tracker," Proc. IEEE Int'l Conf. Computer Vision Workshop Modeling, Simulation and Visual Analysis of Large Crowds, 2011.
24. Y. LeCun, S. Chopra, R. Hadsell, M. Ranzato, and F. Huang, A Tutorial on Energy-Based Learning, MIT Press, 2006.
25. R. Li, R. Chellappa, and S.K. Zhou, "Learning Multi-Modal Densities on Discriminative Temporal Interaction Manifold for Group Activity Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2009.
26. J. Liu, J. Luo, and M. Shah, "Recognizing Realistic Actions from Videos 'in the Wild'," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2009.
27. G. Medioni, I. Cohen, F. Bremond, S. Hongeng, and R. Nevatia, "Event Detection and Analysis from Video Streams," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 8, pp. 873-889, Aug. 2001. (Pubitemid 32780014)
28. B. Ni, S. Yan, and A. Kassim, "Recognizing Human Group Activities with Localized Causalities," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2009.
29. J.C. Niebles, H. Wang, and L. Fei-Fei, "Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words," Int'l J. Computer Vision, vol. 79, pp. 299-318, 2008.
30. S. Pellegrini, A. Ess, K. Schindler, and L. van Gool, "You'll Never Walk Alone: Modeling Social Behavior for Multi-Target Tracking," Proc. IEEE Int'l Conf. Computer Vision (ICCV), 2009.
31. H. Pirsiavash, D. Ramanan, and C. Fowlkes, "Globally-Optimal Greedy Algorithms for Tracking a Variable Number of Objects," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2011.
32. A. O. Ramin Mehran and M. Shah, "Abnormal Crowd Behavior Detection Using Social Force Model," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2009.
33. M. Rodriguez, S. Ali, and T. Kanade, "Tracking in Unstructured Crowded Scenes," Proc. IEEE Int'l Conf. Computer Vision (ICCV), 2009.
34. M. S. Ryoo and J. K. Aggarwal, "Spatio-Temporal Relationship Match: Video Structure Comparison for Recognition of Complex Human Activities," Proc. IEEE Int'l Conf. Computer Vision (ICCV), 2009.
35. M.S. Ryoo and J.K. Aggarwal, "Stochastic Representation and Recognition of High-Level Group Activities," Int'l J. Computer Vision, vol. 93, pp. 183-200, 2010.
36. S. Savarese, A. DelPozo, J. Niebles, and L. Fei Fei, "Spatial- Temporal Correlatons for Unsupervised Action Classification," Proc. IEEE Workshop Motion and Video Computing (WMVC), 2008.
37. P. Scovanner and M. Tappen, "Learning Pedestrian Dynamics from the Real World," Proc. IEEE Int'l Conf. Computer Vision (ICCV), 2009.
38. V.K. Singh, B. Wu, and R. Nevatia, "Pedestrian Tracking by Associating Tracklets Using Detection Residuals," Proc. IEEE Workshop Motion and Video Computing (IMVC), 2008.
39. E. Swears and A. Hoogs, "Learning and Recognizing Complex Multi-Agent Activities with Applications to American Football Plays," Proc. IEEE Workshop Applications of Computer Vision (WACV), 2011.
40. J. Weston, and C. Watkins, Multi-Class Support Vector Machines, Technical Report CSD-TR-98-04, University of London, 1998.
41. B. Wu and R. Nevatia, "Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet Based Part Detectors," Int'l J. Computer Vision, vol. 75, pp. 247-266, 2007. (Pubitemid 47293421)
42. K. Yamaguchi, A.C. Berg, T. Berg, and L. Ortiz, "Who Are You With and Where Are You Going?" Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2011.
43. J.Y. Yen, "Finding the K Shortest Loopless Paths in a Network," Management Science, vol. 17, pp. 712-716, July 1971.
44. L. Zhang, Y. Li, and R. Nevatia, "Global Data Association for Multi-Object Tracking Using Network Flows," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2008.
45. B. Zhou, X. Wang, and X. Tang. "Understanding Collective Crowd Behaviors: Learning a Mixture Model of Dynamic Pedestrian Agents, Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 2871-2878, 2012.