Video-based human behavior understanding: A survey

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 23, Issue 11, 2013, Pages 1993-2008

  Borges, Paulo Vinicius Koerich ^a   Conci, Nicola ^b   Cavallaro, Andrea ^c  

^a CSIRO   (Australia)

^b UNIVERSITY OF TRENTO   (Italy)

^c QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Behavior analysis; computer vision; human detection; video analysis

Indexed keywords

APPLICATION SCENARIO; BEHAVIOR ANALYSIS; BEHAVIOR UNDERSTANDING; FUNDAMENTAL BUILDING BLOCKS; HUMAN BEHAVIOR UNDERSTANDING; HUMAN DETECTION; INDEXING AND RETRIEVAL; VIDEO ANALYSIS;

COMPUTER VISION; DATA PROCESSING; FEATURE EXTRACTION; RISK MANAGEMENT; SIGNAL PROCESSING; SURVEYS;

SOCIAL SCIENCES;

EID: 84887585310     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2013.2270402     Document Type: Article

References (187)

1. T. Xiang and S. Gong, "Video behavior profiling for anomaly detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 893-908, May 2008. (Pubitemid 351486756)
2. M. Pantic, A. Pentland, A. Nijholt, and T. Huang, "Human computing and machine understanding of human behavior: A survey," in Proc. Artif. Intell. Human Comput., pp. 47-71, 2007.
3. A. Shirai, E. Geslin, and S. Richir, "WiiMedia: Motion analysis methods and applications using a consumer video game controller," in Proc. ACM SIGGRAPH Symp. Video Games, 2007, pp. 133-140.
4. L. Yin, X. Wei, Y. Sun, J. Wang, and M. Rosato, "A 3-D facial expression database for facial behavior research," in Proc. IEEE Int. Conf. Autom. Face Gesture Recogn., Apr. 2006, pp. 211-216. (Pubitemid 44711352)
5. C. Kidd, R. Orr, G. Abowd, C. Atkeson, I. Essa, B. MacIntyre, E. Mynatt, T. Starner, and W. Newstetter, "The aware home: A living laboratory for ubiquitous computing research," in Proc. Cooperative Build. Integr. Inform., Org., Arch., Oct. 1999, pp. 191-198.
6. Z. Niu, X. Gao, and Q. Tian, "Tactic analysis based on real-world ball trajectory in soccer video," Pattern Recogn., vol. 45, no. 5, pp. 1937-1947, May 2012.
7. C. G. M. Snoek, K. E. A. van de Sande, O. de Rooij, B. Huurnink, J. C. van Gemert, J. R. R. Uijlings, J. He, X. Li, I. Everts, V. Nedovic, M. van Liempt, R. van Balen, F. Yan, M. A. Tahir, K. Mikolajczyk, J. Kittler, M. de Rijke, J. M. Geusebroek, T. Gevers, M. Worring, A. W. M. Smeulders, and D. C. Koelma, "The mediamill TRECVID 2008 semantic video search engine," 2009.
8. Z. Zhang and D. Tao, "Slow feature analysis for human action recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 3, pp. 436-450, Mar. 2012.
9. A. Yao, J. Gall, and L. Van Gool, "Coupled action recognition and pose estimation from multiple views," Int. J. Comput. Vision, vol. 100, no. 1, pp. 16-37, 2012.
10. S. Ba and J. Odobez, "Multiperson visual focus of attention from head pose and meeting contextual cues," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 1, pp. 101-116, Jan. 2011.
11. M. Enzweiler and D. M. Gavrila, "Monocular pedestrian detecion: Survey and experiments," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 12, pp. 2179-2195, Dec. 2009.
12. D. Geŕonimo, A. M. Ĺopez, A. D. Sappa, and T. Graf, "Survey of pedestrian detection for advanced driver assistance systems," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 7, pp. 1239-1258, Jul. 2010.
13. P. Dolĺar, C. Wojek, B. Schiele, and P. Perona, "Pedestrian detection: A benchmark," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recogn., Jun. 2009, pp. 304-311.
14. P. Dolĺar, C. Wojek, B. Schiele, and P. Perona, "Pedestrian detection: An evaluation of the state-of-the-art," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 4, pp. 743-761, Apr. 2012.
15. B. Morris and M. Trivedi, "A survey of vision-based trajectory learning and analysis for surveillance," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 8, pp. 1114-1127, Aug. 2008.
16. T. Moeslund, A. Hilton, and V. Kruger, "A survey of advances in vision-based human motion capture and analysis," Comput. Vision Image Understand., vol. 104, nos. 2-3, pp. 90-126, 2006. (Pubitemid 44573873)
17. R. Poppe, "A survey on vision-based human action recognition," Image Vision Comput., vol. 28, no. 6, pp. 976-990, 2010.
18. J. Aggarwal and M. Ryoo, "Human activity analysis: A review," ACM Comput. Surveys, vol. 43, no. 3, p. 16, 2011.
19. Y. Wu and T. Huang, "Vision-based gesture recognition: A review," in Proc. Gesture-Based Commun. Human Comput. Interaction, 1999, pp. 103-115.
20. P. Turaga, R. Chellappa, V. Subrahmanian, and O. Udrea, "Machine recognition of human activities: A survey," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 11, pp. 1473-1488, Nov. 2008.
21. A. Maslow, R. Frager, and J. Fadiman, Motivation and Personality, vol. 2. New York, NY, USA: Harper and Row, 1970.
22. J. Baird and D. Baldwin, "Making sense of human behavior: Action parsing and intentional inference," in Proc. Intentions Intentionality: Foundations Soc. Cogn., 2001, pp. 193-206.
23. J. Thibaut and H. Kelley, The Social Psychology of Groups. New York, NY, USA: Wiley, 1959.
24. W. Doise and A. Palmonari, Social Interaction in Individual Development, vol. 3. Cambridge, U.K.: Cambridge Univ., 1984.
25. D. Gavrila and S. Munder, "Multicue pedestrian detection and tracking from a moving vehicle," Int. J. Comput. Vision, vol. 73, no. 1, pp. 41-59, 2007. (Pubitemid 46181616)
26. A. Howard, L. Matthies, A. Huertas, M. Bajracharya, and A. Rankin, "Detecting pedestrians with stereo vision: Safe operation of autonomous ground vehicles in dynamic environments," in Proc. 13th Int. Symp. Robotics Res., 2007, pp. 26-29.
27. N. Dalal, B. Triggs, I. Rhone-Alps, and F. Montbonnot, "Histogram of oriented gradients for human detection," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recogn., vol. 1. Jun. 2005, pp. 886-893. (Pubitemid 43897286)
28. P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., vol. 1. 2001, pp. I-511-I-518.
29. S. Paisitkriangkrai, C. Shen, and J. Zhang, "Fast pedestrian detection using a cascade of boosted covariance features," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 8, pp. 1140-1151, Aug. 2008.
30. P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1627-1645, Sep. 2009.
31. M. Andriluka, S. Roth, and B. Schiele, "Pictorial structures revisited: People detection and articulated pose estimation," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2009, pp. 1014-1021.
32. B. Leibe, E. Seemann, and B. Schiele, "Pedestrian detection in crowded scenes," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recogn., vol. 1. Jun. 2005, pp. 878-885. (Pubitemid 43897285)
33. Z. Lin and L. Davis, "A pose-invariant descriptor for human detection and segmentation," in Proc. Eur. Conf. Comput. Vis., 2008, pp. 423-436.
34. M. Enzweiler and D. M. Gavrila, "Combination of feature extraction method for SVM pedestrian detection," IEEE Trans. Intell. Transport. Syst., vol. 8, no. 2, pp. 292-307, Jun. 2007.
35. P. Viola, M. Jones, and D. Snow, "Detecting pedestrians using patterns of motion and appearance," Int. J. Comput. Vis., vol. 63, no. 2, pp. 153-161, 2005. (Pubitemid 40536820)
36. INRIA. (2012) Person Dataset [Online]. Available: http://pascal. inrialpes.fr/data/human/
37. L. Pishchulin, A. Jain, C. Wojek, M. Andriluka, T. Thormahlen, and B. Schiele, "Learning people detection models from few training samples," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 1473-1480.
38. T. Bouwmans, F. E. Baf, and B. Vachon, Statistical Background Modeling for Foreground Detection: A Survey, vol. 4. Singapore: World Scientific Publishing, 2010, ch. 3, pp. 181-199.
39. B. Wu and R. Nevatia, "Detection and segmentation of multiple, partially occluded objects by grouping, merging, assigning part detection responses," Int. J. Comput. Vision, vol. 82, no. 2, pp. 185-204, Apr. 2009.
40. P. Borges, "Pedestrian detection based on blob motion statistics," IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 2, pp. 224-235, Feb. 2013.
41. J. J. Little and J. E. Boyd, "Recognizing people by their gait: The shape of motion," Videre J. Comput. Vision Res., vol. 1, no. 2, pp. 1-32, 1998.
42. P. S. Tsai, M. Shah, K. Keiter, and T. Kasparis, "Cyclic motion detection for motion based recognition," Pattern Recogn., vol. 27, no. 12, pp. 1591-1603, Dec. 1994. (Pubitemid 124017501)
43. Q. Meng, B. Li, and H. Holstein, "Recognition of human periodic movements from unstructured information using a motion-based frequency domain approach," Image Vision Comput., vol. 24, no. 8, pp. 795-809, Aug. 2006. (Pubitemid 44188981)
44. S. Johnsen and A. Tews, "Real-time object tracking and classification using a static camera," in Proc. IEEE Int. Conf. Robotics Autom. Workshop People Detect. Track., May 2009.
45. Y. Liu, X. Chen, H. Yao, X. Cui, C. Liu, and W. Gao, "Contourmotion feature (CMF): A space-time approach for robust pedestrian detection," Pattern Recogn. Lett., vol. 30, no. 2, pp. 148-156, 2009.
46. R. Cutler and L. Davis, "Robust real-time periodic motion detection, analysis, and applications," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 781-796, Aug. 2000.
47. Y. Chen, Q. Wu, and X. He, "Motion based pedestrian recognition," in Proc. Int. Congr. Image Signal Process., 2008, pp. 376-380.
48. J. E. Boyd, "Synchronization of oscillations for machine perception of gaits," Comput. Vision Image Understand., vol. 96, no. 1, pp. 35-59, Oct. 2004.
49. B. G. Quinn and E. J. Hannan, The Estimation and Tracking of Frequency. Cambridge, U.K.: Cambridge Univ. Press, 2001, pp. 143-162.
50. D. G. Manolakis, V. K. Ingle, and S. M. Kogon, Statistical and Adaptive Signal Processing. New York, NY, USA: McGraw-Hill, 2000.
51. Y. Ran, I. Weiss, Q. Zheng, and L. S. Davis, "Pedestrian detection via periodic motion analysis," Int. J. Comput. Vision, vol. 71, no. 2, pp. 143-160, Feb. 2007. (Pubitemid 44359118)
52. S. Walk, N. Majer, K. Schindler, and B. Schiele, "New features and insights for pedestrian detection," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recogn., Jun. 2010, pp. 1030-1037.
53. A. Efros, A. Berg, G. Mori, and J. Malik, "Recognizing action at a distance," in Proc. IEEE Int. Conf. Comput. Vision, Oct. 2003, pp. 726-733.
54. F. Perbet, A. Maki, and B. Stenger, "Correlated probabilistic trajectories for pedestrian motion detection," in Proc. IEEE Int. Conf. Comput. Vision, Sep.-Oct. 2009, pp. 1647-1654.
55. X. Wang, X. Ma, and W. Grimson, "Unsupervised activity perception in crowded and complicated scenes using hierarchical Bayesian models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 3, pp. 539-555, Mar. 2009.
56. M. Wang and X. Wang, "Automatic adaptation of a generic pedestrian detector to a specific traffic scene," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 3401-3408.
57. J. Rosell, G. Andreu, A. Rodas, V. Atienza, and J. Valiente, "Feature sets for people and luggage recognition in airport surveillance under real-time constraints," in Proc. VISIGRAPP, 2008, pp. 662-665.
58. X. Liu, L. Lin, S. Yan, H. Jin, and W. Tao, "Integrating spatio-temporal context with multiview representation for object recognition in visual surveillance," IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 4, pp. 393-407, Apr. 2011.
59. J. Ge, Y. Luo, and G. Tei, "Real-time pedestrian detection and tracking at nighttime for driver-assistance systems," IEEE Trans. Intell. Transport. Syst., vol. 10, no. 2, pp. 283-298, Jun. 2009.
60. T. Elguebaly and N. Bouguila, "A nonparametric Bayesian approach for enhanced pedestrian detection and foreground segmentation," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 21-26.
61. C. Dai, Y. Zheng, and X. Li, "Pedestrian detection and tracking in infrared imagery using shape and appearance," Comput. Vision Image Understand., vol. 106, nos. 2-3, pp. 288-299, 2007. (Pubitemid 46680516)
62. N. Dalal, "Finding people in images and videos," Ph.D. thesis, Inst. Nat. Polytechnique de Grenoble, Isère, France, 2006.
63. Y. Freund and R. Schapire, "A desicion-theoretic generalization of online learning and an application to boosting," in Computational Learning Theory. Berlin, Germany: Springer, 1995, pp. 23-37.
64. M. Jones and D. Snow, "Pedestrian detection using boosted features over many frames," in Proc. IEEE Int. Conf. Pattern Recogn., Dec. 2008, pp. 1-4.
65. I. Bouchrika, J. Carter, M. Nixon, R. Morzinger, and G. Thallinger, "Using gait features for improving walking people detection," in Proc. IEEE Int. Conf. Pattern Recogn., Aug. 2010, pp. 3097-3100.
66. W. Ge, R. Collins, and R. Ruback, "Vision-based analysis of small groups in pedestrian crowds," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 5, pp. 1003-1016, May 2012.
67. L. Xia, C. Chen, and J. Aggarwal, "Human detection using depth information by Kinect," in Proc. Workshop Human Activity Understand. 3-D Data Conjunction IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 15-22.
68. L. Spinello and K. O. Arras, "People detection in RGB-D data," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Sep. 2011, pp. 3838-3843.
69. MIT. (2000). CBCL Pedestrian Database [Online]. Available: http://cbcl.mit.edu/cbcl/software-datasets/PedestrianData.html
70. B. Wu and R. Nevatia, "Cluster boosted tree classifier for multiview, multipose object detection," in Proc. IEEE Int. Conf. Comput. Vision, Oct. 2007, pp. 1-8.
71. B. Wu and R. Nevatia, "Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors," in Proc. IEEE Int. Conf. Comput. Vision, vol. 1. Oct. 2005, pp. 90-97. (Pubitemid 44054945)
72. A. Ess, B. Leibe, and L. Van Gool, "Depth and appearance for mobile scene analysis," in Proc. IEEE Int. Conf. Comput. Vision, Oct. 2007, pp. 1-8.
73. PETS. (2005). Computational Vision Group-University of Reading, U.K. [Online]. Available: http://www.cvg.cs.rdg.ac.uk
74. I. Laptev, "On space-time interest points," Int. J. Comput. Vision, vol. 64, no. 2, pp. 107-123, 2005. (Pubitemid 41314645)
75. C. Schuldt, I. Laptev, and B. Caputo, "Recognizing human actions: A local SVM approach," in Proc. IEEE Int. Conf. Pattern Recogn., vol. 3. Aug. 2004, pp. 32-36.
76. J. Yamato, J. Ohya, and K. Ishii, "Recognizing human action in time-sequential images using hidden Markov model," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 1992, pp. 379-385.
77. W. Choi, K. Shahid, and S. Savarese, "Learning context for collective activity recognition," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 3273-3280.
78. L. Lin, H. Gong, L. Li, and L. Wang, "Semantic event representation and recognition using syntactic attribute graph grammar," Pattern Recogn. Lett., vol. 30, no. 2, pp. 180-186, 2009.
79. S. Sadanand and J. Corso, "Action bank: A high-level representation of activity in video," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2012, pp. 1234-1241.
80. H.W.ng, A. Klaser, C. Schmid, and C.-L. Liu, "Action recognition by dense trajectories," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 3169-3176.
81. S. Wu, O. Oreifej, and M. Shah, "Action recognition in videos acquired by a moving camera using motion decomposition of Lagrangian particle trajectories," in Proc. IEEE Int. Conf. Comput. Vision, Nov. 2011, pp. 1419-1426.
82. D. Chen, S. Shih, and H. Liao, "Human action recognition using 2-D spatio-temporal templates," in Proc. IEEE Int. Conf. Multimedia Expo., Jul. 2007, pp. 667-670.
83. P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie, "Behavior recognition via sparse spatio-temporal features," in Proc. 2nd Joint IEEE Int. Workshop Visual Surveillance Performance Eval. Tracking Surveillance., Oct. 2005, pp. 65-72. (Pubitemid 46178025)
84. J. Niebles, H.W.ng, and L. Fei-Fei, "Unsupervised learning of human action categories using spatial-temporal words," Int. J. Comput. Vision, vol. 79, no. 3, pp. 299-318, 2008.
85. H. Jhuang, T. Serre, L. Wolf, and T. Poggio, "A biologically inspired system for action recognition," in Proc. IEEE 11th Int. Conf. Comput. Vision, Oct. 2007, pp. 1-8.
86. J. Liu and M. Shah, "Learning human actions via information maximization," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2008, pp. 1-8.
87. M. Ryoo and J. Aggarwal, "Spatio-temporal relationship match: Video structure comparison for recognition of complex human activities," in Proc. IEEE Int. Conf. Comput. Vision, Sep.-Oct. 2009, pp. 1593-1600.
88. I. Laptev and T. Lindeberg, "Space-time interest points," in Proc. IEEE Int. Conf. Comput. Vision, Oct. 2003, pp. 432-439.
89. C. Harris and M. Stephens, "A combined corner and edge detector," in Proc. Alvey Vision Conf., Manchester, U.K., vol. 15. 1988, p. 50.
90. M. Bregonzio, S. Gong, and T. Xiang, "Recognising action as clouds of space-time interest points," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2009, pp. 1948-1955.
91. M. Ullah, S. Parizi, and I. Laptev, "Improving bag-of-features action recognition with non-local cues," in Proc. Brit. Mach. Vision Conf., vol. 2. 2010, p. 7.
92. D. Han, L. Bo, and C. Sminchisescu, "Selection and context for action recognition," in Proc. IEEE Int. Conf. Comput. Vision, Sep.-Oct. 2009, pp. 1933-1940.
93. J. Sun, X. Wu, S. Yan, L. Cheong, T. Chua, and J. Li, "Hierarchical spatio-temporal context modeling for action recognition," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2009, pp. 2004-2011.
94. B. Chakraborty, M. Holte, T. Moeslund, J. Gonzalez, and F. Roca, "A selective spatio-temporal interest point detector for human action recognition in complex scenes," in Proc. IEEE Int. Conf. Comput. Vision, Nov. 2011, pp. 1776-1783.
95. H.W.ng, A. Klaser, C. Schmid, and C.-L. Liu, "Dense trajectories and motion boundary descriptors for action recognition," Int. J. Comput. Vision, vol. 103, no. 1, pp. 60-79, 2013.
96. J. Liu, J. Luo, and M. Shah, "Recognizing realistic actions from videos in tH.W.ld," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2009, pp. 1996-2003.
97. M. Marszalek, I. Laptev, and C. Schmid, "Actions in context," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2009, pp. 2929-2936.
98. M. D. Rodriguez, J. Ahmed, and M. Shah, "Action MACH: A spatio-temporal maximum average correlation height filter for action recognition," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2008, pp. 1-8.
99. C. Chen and J. Aggarwal, "Modeling human activities as speech," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 3425-3432.
100. M. Ryoo, "Human activity prediction: Early recognition of ongoing activities from streaming videos," in Proc. IEEE Int. Conf. Comput. Vision, Nov. 2011, pp. 1036-1043.
101. M. S. Ryoo and J. K. Aggarwal. (2010) UT-Interaction Dataset, ICPR Contest on Semantic Description of Human Activities (SDHA) [Online]. Available: http://cvrc.ece.utexas.edu/SDHA2010/Human\ Interaction.html
102. A. Castrodad and G. Sapiro, "Sparse modeling of human actions from motion imagery," Int. J. Comput. Vision, vol. 100, no. 1, pp. 1-15, 2012.
103. W. Choi, K. Shahid, and S. Savarese, "What are they doing: Collective activity classification using spatio-temporal relationship among people," in Proc. IEEE Int. Conf. Comput. Vision Workshops, Sep.-Oct. 2009, pp. 1282-1289.
104. M. Raptis, I. Kokkinos, and S. Soatto, "Discovering discriminative action parts from mid-level video representations," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., 2012, pp. 1242-1249.
105. M. Hoai, Z. Lan, and F. De la Torre, "Joint segmentation and classification of human actions in video," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 3265-3272.
106. M. Daldoss, N. Piotto, N. Conci, and F. G. B. De Natale, "Learning and matching human activities using regular expressions," in Proc. IEEE Int. Conf. Image Process., Sep. 2010, pp. 4681-4684.
107. Z. Zhang, T. Tan, and K. Huang, "An extended grammar system for learning and recognizing complex visual events," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 240-255, Feb. 2011.
108. S. Calderara, U. Heinemann, A. Prati, R. Cucchiara, and N. Tishby, "Detecting anomalies in peoples trajectories using spectral graph analysis," Comput. Vision Image Understand., vol. 115, no. 8, pp. 1099-1111, 2011.
109. A. Gupta, P. Srinivasan, J. Shi, and L. S. Davis, "Understanding videos, constructing plots learning a visually grounded storyline model from annotated videos," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2009, pp. 2012-2019.
110. M. Raptis and L. Sigal, "Poselet key-framing: A model for human activity recognition," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., to be published, 2013.
111. L. Bourdev and J. Malik, "Poselets: Body part detectors trained using 3-D human pose annotations," in Proc. IEEE Int. Conf. Comput. Vision, Sep.-Oct. 2009, pp. 1365-1372.
112. L. Li, H. Su, E. Xing, and L. Fei-Fei, "Object bank: A high-level image representation for scene classification and semantic feature sparsification," in Proc. Adv. Neural Inf. Process. Syst., vol. 24, Dec. 2010, pp. 1378-1386.
113. R. Messing, "Human activity recognition in video: Extending statistical features across time, space and semantic context," Ph.D. dissertation, University of Rochester, Rochester, NY, USA, 2011.
114. R. Messing, C. Pal, and H. Kautz, "Activity recognition using the velocity histories of tracked keypoints," in Proc. IEEE Int. Conf. Comput.Vision, Sep.-Oct. 2009, pp. 104-111.
115. A. Fathi, X. Ren, and J. M. Rehg, "Learning to recognize objects in egocentric activities," in Proc. Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 3281-3288.
116. A. Bobick and J. Davis, "The recognition of human movement using temporal templates," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 3, pp. 257-267, Mar. 2001. (Pubitemid 32286632)
117. M. Hu, "Visual pattern recognition by moment invariants," IRE Trans. Inf. Theory, vol. 8, no. 2, pp. 179-187, 1962.
118. J. Davis and A. Tyagi, "Minimal-latency human action recognition using reliable-inference," Image Vision Comput., vol. 24, no. 5, pp. 455-472, 2006. (Pubitemid 43782937)
119. F. Nater, H. Grabner, and L. Van Gool, "Exploiting simple hierarchies for unsupervised human behavior analysis," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2010, pp. 2014-2021.
120. W. Li, Z. Zhang, and Z. Liu, "Action recognition based on a bag of 3-D points," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2010, pp. 9-14.
121. L. Wang and D. Suter, "Recognizing human activities from silhouettes: Motion subspace and factorial discriminative graphical model," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2007, pp. 1-8.
122. X. Ji, H. Liu, and Y. Li, "Viewpoint insensitive actions recognition using hidden conditional random fields," in Proc. Knowl. Based Intell. Inf. Eng. Syst., Sep. 2010, pp. 369-378.
123. O. Concha, D. Xu, R. Yi, Z. Moghaddam, and M. Piccardi, "HMMMIO: An enhanced hidden Markov model for action recognition," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 62-69.
124. V. Kellokumpu, M. Pietikainen, and J. Heikkila, "Human activity recognition using sequences of postures," in Proc. IAPR Conf. Mach. Vision Applicat., 2005, pp. 570-573.
125. Y. Ke, R. Sukthankar, and M. Hebert, "Event detection in crowded videos," in Proc. IEEE Int. Conf. Comput. Vision, Oct. 2007, pp. 1-8.
126. Y. Wang and G. Mori, "Hidden part models for human action recognition: Probabilistic vs. max-margin," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 7, pp. 1310-1323, Jul. 2011.
127. C. Sun, I. Junejo, and H. Foroosh, "Action recognition using rank-1 approximation of joint self-similarity volume," in Proc. IEEE Int. Conf. Comput. Vision, Nov. 2011, pp. 1007-1012.
128. H. Fujiyoshi and A. Lipton, "Real-time human motion analysis by image skeletonization," in Proc. IEEE Workshop Appl. Comput. Vision, Oct. 1998, pp. 15-21.
129. W. Li, Z. Zhang, and Z. Liu, "Expandable data-driven graphical modeling of human actions based on salient postures," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 11, pp. 1499-1510, Nov. 2008.
130. J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake, "Real-time human pose recognition in parts from single depth images," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 1297-1304.
131. J. Sung, C. Ponce, B. Selman, and A. Saxena, "Human activity detection from RGBD images," in Proc. AAAI Workshop Pattern, Activity Intent Recogn., 2011, pp. 47-55.
132. J. Sung, C. Ponce, B. Selman, and A. Saxena, "Unstructured human activity detection from RGBD images," in Proc. IEEE Int. Conf. Robotics Autom, May 2012, pp. 842-849.
133. M. Blank, L. Gorelick, E. Shechtman, M. Irani, and R. Basri, "Actions as space-time shapes," in Proc. IEEE Int. Conf. Comput. Vision, vol. 2. Oct. 2005, pp. 1395-1402. (Pubitemid 44042342)
134. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld, "Learning realistic human actions from movies," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2008, pp. 1-8.
135. K. Reddy and M. Shah, "Recognizing 50 human action categories of web videos," in Proc. Mach. Vision Applicat., 2012, pp. 1-11.
136. D. Weinland, R. Ronfard, and E. Boyer, "Free viewpoint action recognition using motion history volumes," Comput. Vision Image Understand., vol. 104, no. 2, pp. 249-257, 2006. (Pubitemid 44573869)
137. S. Singh, S. Velastin, and H. Ragheb, "MuHAVI: A multicamera human action video dataset for the evaluation of action recognition methods," in Proc. IEEE Int. Comput. Soc. Conf. Adv. Video Signal Based Surveillance, 2010, pp. 48-55.
138. D. Baltieri, R. Vezzani, and R. Cucchiara, "3DPEs: 3-D people dataset for surveillance and forensics," in Proc. 1st Int. ACM Workshop Multimedia Access 3-D Human Objects, Nov. 2011, pp. 59-64.
139. X. Wang, K. T. Ma, G.-W. Ng, and E. Grimson, "Trajectory analysis and semantic region modeling using a nonparametric Bayesian model," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2008, pp. 1-8.
140. EIFPD. Edinburgh Informatics Forum Pedestrian Database [Online]. Available: http://homepages.inf.ed.ac.uk/rbf/FORUMTRACKING/
141. M. W. Kadous, "Temporal classification: Extending the classification paradigm to multivariate time series," Ph.D. dissertation, University of New South Wales, Sydney, New South Wales, Australia, 2002.
142. FGnet. Two Handed Gesture Dataset [Online]. Available: http://www-prima.inrialpes.fr/FGnet/data/04-TwoHand/main.html
143. M. Holte and M. Stoerring. (2004). Pointing and Command Gestures Under Mixed Illumination Conditions: Video Sequence Dataset [Online]. Available: http://www-prima.inrialpes.fr/FGnet/data/03-Pointing/index.html
144. S. Marcel, O. Bernier, J. Viallet, and D. Collobert, "Hand gesture recognition using input-output hidden Markov models," in Proc. IEEE Int. Conf. Autom. Face Gesture Recogn., Mar. 2000, pp. 456-461.
145. FGnet. FGnet Pointing04 Benchmark Data [Online]. Available: http://www-prima.inrialpes.fr/Pointing04/data-hand.html
146. T. Kim, S. Wong, and R. Cipolla, "Tensor canonical correlation analysis for action classification," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2007, pp. 1-8.
147. ChaLearn. (2011). Gesture Dataset [Online]. Available: http://www-prima.inrialpes.fr/FGnet/data/03-Pointing/index.html
148. S. Marcel. (2000). Hand Posture and Gesture Datasets [Online]. Available: http://www-prima.inrialpes.fr/FGnet/data/10-Gesture/gestures/main.html
149. B. Ni, G. Wang, and P. Moulin, "RGBD-HUDAACT: A color-depth video database for human daily activity recognition," in Proc. Consumer Depth Cameras Computer Vision Workshop-Int. Conf. Comput. Vision. Berlin, Germany: Springer, 2013, pp. 193-208.
150. E. Hall, The Hidden Dimension, vol. 6. New York, NY, USA: Doubleday, 1966.
151. E. Hall, The Silent Language. Harpswell, ME, USA: Anchor, 1973.
152. D. Helbing and P. Molnar, "Social force model for pedestrian dynamics," Phys. Rev. E, vol. 51, no. 5, p. 4282, 1995.
153. R. Mehran, A. Oyama, and M. Shah, "Abnormal crowd behavior detection using social force model," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2009, pp. 935-942.
154. P. Scovanner and M. Tappen, "Learning pedestrian dynamics from the real world," in Proc. IEEE Int. Conf. Comput. Vision, Sep.-Oct. 2009, pp. 381-388.
155. S. Pellegrini, A. Ess, K. Schindler, and L. van Gool, "Youll never walk alone: Modeling social behavior for multitarget tracking," in Proc. IEEE Int. Conf. Comput. Vision, Oct. 2009, pp. 261-268.
156. M. Taj and A. Cavallaro, "Recognizing interactions in video," in Proc. Intell. Multimedia Anal. Security Applicat., 2010, pp. 29-57.
157. P. Rota, N. Conci, and N. Sebe, "Real time detection of social interactions in surveillance video," in Proc. 3rd Int. Workshop Anal. Retrieval Tracked Events Motion Imagery Streams, Eur. Conf. Comput. Vision, 2012, pp. 111-120.
158. G. Zen, B. Lepri, E. Ricci, and O. Lanz, "Space speaks: Towards socially and personality aware visual surveillance," in Proc. ACM MPVA, 2010, pp. 37-42.
159. M. Cristani, G. Paggetti, A. Vinciarelli, L. Bazzani, G. Menegaz, and V. Murino, "Towards computational proxemics: Inferring social relations from interpersonal distances," in Proc. SocialCom, 2011, pp. 290-297.
160. B. Ni, S. Yan, and A. Kassim, "Recognizing human group activities with localized causalities," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2009, pp. 1470-1477.
161. S. Pellegrini, A. Ess, and L. Van Gool, "Improving data association by joint modeling of pedestrian trajectories and groupings," in Proc. Eur. Conf. Comput. Vision, 2010, pp. 452-465.
162. X. Cui, Q. Liu, M. Gao, and D. Metaxas, "Abnormal detection using interaction energy potentials," in Proc. IEEE Conf. Comput. Vision Pattern Recogn., Jun. 2011, pp. 3161-3167.
163. J. Lei, X. Ren, and D. Fox, "Fine-grained kitchen activity recognition using RGB-D," in Proc. ACM Conf. Ubiquitous Comput., 2012, pp. 208-211.
164. H. Zhang and L. E. Parker, "4-dimensional local spatio-temporal features for human activity recognition," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Sep. 2011, pp. 2044-2049.
165. D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent Dirichlet allocation," J. Mach. Learn. Res., vol. 3, pp. 993-1022, 2003.
166. H. Swetha Koppula, R. Gupta, and A. Saxena, "Learning human activities and object affordances from RGB-D videos," 2013.
167. BEHAVE. (2007). The Behave Dataset[Online]. Available: http://homepages.inf.ed.ac.uk/rbf/BEHAVE
168. A. Patron-Perez. (2010). Tv Interactions Dataset [Online]. Available: http://www.robots.ox.ac.uk/vgg/data/tv human interactions/index. html
169. CMU. (2013). CMU Graphics Lab Motion Capture Database [Online]. Available: http://mocap.cs.cmu.edu/
170. A. Patron-Perez, M. Marszalek, A. Zisserman, and I. Reid, "High five: Recognising human interactions in TV shows," in Proc. British Mach. Vision Conf., Aug. 2010, pp. 1-11.
171. C. Wolf, J. Mille, E. Lombardi, O. Celiktutan, M. Jiu, M. Baccouche, E. Dellandŕea, C.-E. Bichot, C. Garcia, and B. Sankur, "The Liris human activities dataset and the ICPR 2012 human activities recognition and localization competition," Tech. Rep. LIRIS RR-2012-004, Laboratoire Informatique en Images et Systmes Information, INSA de Lyon, France, Tech. Rep., 2012.
172. M. J. Wainwright and M. I. Jordan, "Graphical models, exponential families, and variational inference," Found. Trends Mach. Learn., vol. 1, no. 1-2, pp. 1-305, Jan. 2008
173. [Online]. Available: http://dx.doi.org/10.1561/2200000001
174. K. Kitani, B. Ziebart, J. Bagnell, and M. Hebert, "Activity forecasting," in Proc. Eur. Conf. Comput. Vision. Springer, 2012.
175. M. Amer, D. Xie, M. Zhao, S. Todorovic, and S. Zhu, "Cost-sensitive top-down/bottom-up inference for multiscale activity recognition," in Proc. Eur. Conf. Comput. Vision. Springer, 2012.
176. W. Choi and S. Savarese, "A unified framework for multitarget tracking and collective activity recognition," in Proc. Eur. Conf. Comput. Vision, 2012, pp. 215-230.
177. J. Valmadre, Y. Zhu, S. Sridharan, and S. Lucey, "Efficient articulated trajectory reconstruction using dynamic programming and filters," in Proc. Eur. Conf. Comput. Vision, 2012, pp. 72-85.
178. X. Wei and J. Chai, "Modeling 3-D human poses from uncalibrated monocular images," in Proc. IEEE Int. Conf. Comput. Vision, Sep.-Oct. 2009, pp. 1873-1880.
179. Asus. (2012). Xtion Pro
180. [Online]. Available: http://www.asus.com/Multimedia/Motion Sensor/Xtion PRO/
181. Microsoft. (2013). Microsoft Kinect
182. [Online]. Available: http://www.xbox.com/en-GB/Kinect
183. L. Oliveira, U. Nunes, P. Peixoto, M. Silva, and F. Moita, "Semantic fusion of laser and vision in pedestrian detection," Pattern Recogn., vol. 43, no. 10, pp. 3648-3659, 2010.
184. A. Broggi, P. Cerri, S. Ghidoni, P. Grisleri, and H. Jung, "A new approach to urban pedestrian detection for automatic braking," IEEE Trans. Intell. Transp. Syst., vol. 10, no. 4, pp. 594-605, Dec. 2009.
185. M. Taj and A. Cavallaro, "Interaction recognition in wide areas using audiovisual sensors," in Proc. IEEE Int. Conf. Image Process., Sep.-Oct. 2012, pp. 1113-1116.
186. C. Vondrick, D. Ramanan, and D. Patterson, "Efficiently scaling up video annotation with crowdsourced marketplaces," in Proc. Eur. Conf. Comput. Vision, 2010, pp. 610-623.
187. S. Vittayakorn and J. Hays, "Quality assessment for crowdsourced object annotations," in Proc. Brit. Machine Vision Conf., 2011, pp. 109-111.