Human activity recognition from 3D data: A review

Pattern Recognition Letters

Volumn 48, Issue , 2014, Pages 70-80

  Aggarwal, J K ^a   Xia, Lu ^a  

^a UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

3D data; Computer vision; Depth image; Human activity recognition

Indexed keywords

COMPUTER VISION; PATTERN RECOGNITION;

3D DATA; 3D DATA ACQUISITION; CONVENTIONAL CAMERA; DEPTH IMAGE; HUMAN ACTIVITY RECOGNITION;

THREE DIMENSIONAL COMPUTER GRAPHICS;

EID: 84906782091     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2014.04.011     Document Type: Review

References (110)

1. J. Aggarwal, and M.S. Ryoo Human activity analysis: a review ACM Comput. Surv. (CSUR) 43 2011 16
2. M. Ahmad, S.W. Lee, Hmm-based human action recognition using multiview image sequences in: ICPR, (2006) 263-266. (Pubitemid 46531893)
3. V. Argyriou, M. Petrou, and S. Barsky Photometric stereo with an arbitrary number of illuminants CVIU 114 2010 887 900
4. F. Arman, and J. Aggarwal Model-based object recognition in dense-range images review ACM Comput. Surv. (CSUR) 25 1993 5 43
5. G. Ballin, M. Munaro, E. Menegatti, Human action recognition from rgb-d frames based on real-time 3d optical flow estimation, in: Biologically Inspired Cognitive Architectures. Springer, 2013, pp. 65-74.
6. J.L. Barron, D.J. Fleet, and S. Beauchemin Performance of optical flow techniques IJCV 12 1994 43 77
7. J. Ben-Arie, Z. Wang, P. Pandit, and S. Rajaram Human activity recognition using multidimensional indexing Pattern Anal. Mach. Intell. IEEE Trans. 24 2002 1091 1104
8. A. Blinov, and M. Petrou Reconstruction of 3-d horizons from 3-d seismic datasets Geosci. Remote Sens. 43 2005 1421 1431 (Pubitemid 40811949)
9. V. Bloom, D. Makris, V. Argyriou, G3d: a gaming action dataset and real time action recognition evaluation framework, in: CVPRW, 2012, pp. 7-12.
10. J. Cech, J. Sanchez-Riera, R. Horaud, Scene flow estimation by growing correspondence seeds, in: CVPR, IEEE. 2011, pp. 3129-3136.
11. R. Chaudhry, A. Ravichandran, G. Hager, R. Vidal, Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions, in: CVPR, IEEE, 2009, pp. 1932-1939.
12. L. Chen, H. Wei, and J.M. Ferryman A survey of human motion analysis using depth imagery Pattern Recognit. Lett. 2013
13. L.C. Chen, J.W. Hsieh, C.H. Chuang, C.Y. Huang, and D.Y. Chen Occluded human action analysis using dynamic manifold model ICPR, IEEE 2012 1245 1248
14. M.Y. Chen, A. Hauptmann, Mosift: Recognizing human actions in surveillance videos, 2009.
15. Z. Cheng, L. Qin, Y. Ye, Q. Huang, Q. Tian, Human daily action analysis with multi-view and color-depth data, in: ECCV Workshops and Demonstrations, Springer, 2012, pp. 52-61.
16. A. Chiuso, P. Favaro, H. Jin, and S. Soatto Structure from motion causally integrated over time Pattern Anal. Mach. Intell. IEEE Trans. 24 2002 523 535 (Pubitemid 34448770)
17. W.J. Christmas, J. Kittler, and M. Petrou Structural matching in computer vision using probabilistic relaxation PAMI 17 1995 749 764
18. I. Cohen, and H. Li Inference of human postures by classification of 3d human body shape AMFG Workshop, IEEE 2003 74 81
19. N. Dalal, B. Triggs, C. Schmid, Human detection using oriented histograms of flow and appearance, in: ECCV. Springer, 2006, pp. 428-441. (Pubitemid 44029745)
20. T. Darrell, G. Gordon, M. Harville, and J. Woodfill Integrated person tracking using stereo, color, and pattern detection IJCV 37 2000 175 185
21. J.W. Davis, A.F. Bobick, The representation and recognition of human movement using temporal templates, in: CVPR, IEEE, 1997, pp. 928-934.
22. L. Deng, H. Leung, N. Gu, Y. Yang, Generalized model-based human motion recognition with body partition index maps, in: Computer Graphics Forum, Wiley Online Library, 2012, pp. 202-215.
23. U.R. Dhond, and J.K. Aggarwal Structure from stereo-a review Syst. Man Cybern. IEEE Trans. 19 1989 1489 1510
24. P. Dollár, V. Rabaud, G. Cottrell, and S. Belongie Behavior recognition via sparse spatio-temporal features PETS 2005 65 72 (Pubitemid 46178025)
25. M. Domínguez-Morales, A. Jiménez-Fernández, R. Paz-Vicente, A. Linares-Barranco, G. Jiménez-Moreno, 2012. Stereo matching: From the basis to neuromorphic engineering.
26. S.R. Fanello, I. Gori, G. Metta, and F. Odone Keep it simple and sparse: real-time action recognition J. Mach. Learn. Res. 14 2013 2617 2640
27. P. Favaro, S. Soatto, Learning shape from defocus, in: ECCV. Springer, 2002, pp. 735-745.
28. H. Fujiyoshi, A.J. Lipton, Real-time human motion analysis by image skeletonization, in: Applications of Computer Vision, 1998. WACV'98. Proceedings., Fourth IEEE Workshop on, IEEE, 1998, pp. 15-21.
29. D. Gehrig, T. Schultz, Selecting relevant features for human motion recognition, in: ICPR, IEEE, 2008.
30. N. Georgis, M. Petrou, and J. Kittler Error guided design of a 3d vision system PAMI 20 1998 366 379 (Pubitemid 128741982)
31. J.J. Gibson, The perception of the visual world, 1950.
32. R. Grzeszcuk, G. Bradski, M.H. Chu, and J.Y. Bouguet Stereo based gesture recognition invariant to 3d pose and lighting CVPR 2000 826 833
33. M. Harville, D. Li, Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera, in: CVPR, 2004.
34. A. Hernández-Vela, M. Bautista, X. Perez-Sala, V. Ponce, X. Baró, O. Pujol, C. Angulo, S. Escalera, Bovdw: Bag-of-visual-and-depth- words for gesture recognition, in: ICPR, 2012.
35. C. Hesher, A. Srivastava, G. Erlebacher, A novel technique for face recognition using range imaging Signal processing and its applications, Seventh international symposium on, IEEE, 2003, pp. 201-204.
36. M.B. Holte, T.B. Moeslund, and P. Fihl View-invariant gesture recognition using 3d optical flow and harmonic motion context CVIU 114 2010 1353 1361
37. M.B. Holte, T.B. Moeslund, N., Nikolaidis, I. Pitas, 3d human action recognition for multi-view camera systems, in: 3DIMPVT, 2011, pp. 342-349.
38. B.K. Horn, and B.G. Schunck Determining optical flow Artif. Intell. 17 1981 185 203
39. F. Huguet, F. Devernay, A variational method for scene flow estimation from stereo sequences, in: ICCV, IEEE, 2007, pp. 1-7.
40. A. Jalal, M.Z. Uddin, J.T. Kim, and T.S. Kim Recognition of human home activities via depth silhouettes and R transformation for smart homes Indoor Built Environ. 2011 1 7
41. H. Jin, P. Favaro, A variational approach to shape from defocus, in: ECCV. Springer, 2002, pp. 18-30.
42. G. Johansson, Visual motion perception. Scientific American, 1975.
43. T. Kanade, A. Yoshida, K. Oda, H. Kano, M. Tanaka, A stereo machine for video-rate dense depth mapping and its new applications, in: CVPR, IEEE. 1996, pp. 196-202.
44. A. Klaser, M. Marszalek, A spatio-temporal descriptor based on 3d-gradients, in: BMVC, 2008.
45. V.A. Kovalev, M. Petrou, and Y.S. Bondar Texture anisotropy in 3-d images Image Proces. IEEE Trans. 8 1999 346 360
46. D. Kulić, W. Takano, and Y. akamura Incremental learning, clustering and hierarchy formation of whole body motion patterns using adaptive hidden markov chains Int. J. Rob. Res. 27 2008 761 784
47. A. Kurakin, Z. Zhang, Z. Liu, A real time system for dynamic hand gesture recognition with a depth sensor, 2012.
48. I. Laptev On space-time interest points Int. J. Comput. Vis. 64 2005 107 123 (Pubitemid 41314645)
49. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld Learning realistic human actions from movies CVPR 2008 1 8
50. J. Lei, X. Ren, D. Fox, Fine-grained kitchen activity recognition using rgb-d 2012.
51. A. Letouzey, B. Petit, E. Boyer, M. Team, Scene flow from depth and color images, in: Jesse Hoey, Stephen McKenna and Emanuele Trucco, Proceedings of the British Machine Vision Conference, pages, 2011, pp. 46-1.
52. W. Li, Z. Zhang, and Z. Liu Action recognition based on a bag of 3d points CVPR Workshop 1 2010 9 14
53. A.M. Loh, The recovery of 3-D structure using visual texture patterns. University of Western Australia, 2006.
54. B.D. Lucas, T. Kanade, et al., An iterative image registration technique with an application to stereo vision., in: IJCAI, 1981, pp. 674-679. (Pubitemid 14478744)
55. F. Lv, and R. Nevatia Recognition and segmentation of 3-d human action using hmm and multi-class adaboost ECCV 2006 359 372 (Pubitemid 44029643)
56. S. Malassiotis, N. Aifanti, M. Strintzis, A gesture recognition system using 3d data, in: 3D Data Processing Visualization and Transmission, 2002. Proceedings. First International Symposium on, IEEE, 2002, pp. 190-193.
57. S.Z. Masood, C. Ellis, A. Nagaraja, M.F. Tappen, J. LaViola, and R. Sukthankar Measuring and reducing observational latency when recognizing actions ICCV Workshops, IEEE 2011 422 429
58. Y. Matsumoto, and A. Zelinsky An algorithm for real-time stereo vision implementation of head pose and gaze direction measurement Autom. Face Gesture Recognit. IEEE 2000 499 504
59. R. Muñoz-Salinas, E. Aguirre, M. García-Silvente, People detection and tracking using stereo vision and color. Image and Vision Computing 25, 2007, 995-1007. (Pubitemid 46528595)
60. B. Ni, G. Wang, and P. Moulin Rgbd-hudaact: a color-depth video database for human daily activity recognition ICCV Workshop 2011 1147 1153
61. A. Ogale, A. Karapurkar, G. Guerra-Filho, Y. Aloimonos, View-invariant identification of pose sequences for action recognition, in: VACE, Citeseer, 2004.
62. A. Oikonomopoulos, I. Patras, and M. Pantic Spatiotemporal salient points for visual recognition of human actions Syst. Man Cybern. Part B: Cybern. IEEE Trans. 36 2005 710 719
63. V. Parameswaran, and R. Chellappa View invariance for human action recognition IJCV 66 2006 83 101 (Pubitemid 43004673)
64. R. Poppe A survey on vision-based human action recognition Image Vision Comput. 28 2010 976 990
65. E. Prados, and O. Faugeras Unifying approaches and removing unrealistic assumptions in shape from shading: mathematics can help ECCV. Springer 2004 141 154
66. E. Prados, and O. Faugeras Shape from shading: a well-posed problem? CVPR, IEEE 2005 870 877
67. P. Pritchett, and A. Zisserman Wide baseline stereo matching ICCV, IEEE 1998 754 760
68. M.C. Roh, H.K. Shin, and S.W. Lee View-independent human action recognition with volume motion template on single stereo camera Pattern Recognit. Lett. 31 2010 639 647
69. D.B. Russakoff, and M. Herman Head tracking using stereo Mach. Vis. Appl. 13 2002 164 173 (Pubitemid 41200800)
70. R. Rusu, N. Blodow, and M. Beetz Fast point feature histograms (fpfh) for 3d registration ICRA 2009 3212 3217
71. B. Sabata, and J. Aggarwal Estimation of motion from a pair of range images: A review CVGIP: Image Understanding 54 1991 309 324
72. B. Sabata, and J. Aggarwal Surface correspondence and motion computation from a pair of range images Comput. Vision Image Understanding 63 1996 232 250 (Pubitemid 126374133)
73. H. Saito, and M. Mori Application of genetic algorithms to stereo matching of images Pattern Recognit. Lett. 16 1995 815 821
74. P. Scovanner, S. Ali, M. Shah, A 3-dimensional sift descriptor and its application to action recognition, in: Proceedings of the 15th international conference on Multimedia, ACM, 2007, pp. 357-360.
75. E. Seemann, K. Nickel, and R. Stiefelhagen Head pose estimation using stereo vision for human-robot interaction Autom. Face Gesture Recognit. IEEE 2004 626 631
76. S. Sempena, N. Maulidevi, and P. Aryan Human action recognition using dynamic time warping ICEEI, IEEE 2011 1 5
77. Y. Shen, H. Foroosh, View-invariant action recognition using fundamental ratios, in: CVPR, IEEE, 2008.
78. J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, A., Blake, Real-time human pose recognition in parts from a single depth image. CVPR, 2011.
79. I. Stamos, and P. Allen 3-d model construction using range and image data CVPR, IEEE 2000 531 536
80. J. Sung, C. Ponce, B. Selman, A. Saxena, Human activity detection from RGBD images. PAIR, 2011.
81. J. Sung, C. Ponce, B. Selman, and A. Saxena Unstructured human activity detection from rgbd images ICRA, IEEE 2012 842 849
82. A. Swadzba, N. Beuter, J. Schmidt, and G. Sagerer Tracking objects in 6d for reconstructing static scenes CVPRW, IEEE 2008 1 7
83. P. Turaga, R. Chellappa, V.S. Subrahmanian, and O. Udrea Machine recognition of human activities: A survey Circuits Syst. Video Technol. IEEE Trans. 18 2008 1473 1488
84. M.Z. Uddin, N.D. Thang, J.T. Kim, and T.S. Kim Human activity recognition using body joint-angle features and hidden markov model ETRI J. 33 2011 569 579
85. R. Urtasun, P. Fua, 3d tracking for gait characterization and recognition, in: Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on, IEEE, 2004, pp. 17-22.
86. S. Vedula, S. Baker, P. Rander, R. Collins, and T. Kanade Three-dimensional scene flow ICCV, IEEE 1999 722 729 (Pubitemid 32079937)
87. A. Vieira, E. Nascimento, G. Oliveira, Z. Liu, and M. Campos Stop: Space-time occupancy patterns for 3d action recognition from depth map sequences Progress Pattern Recognit. Image Anal. Comput. Vision Appl. 2012 252 259
88. H. Wang, M.M., Ullah, A. Klaser, I. Laptev, C. Schmid, et al., Evaluation of local spatio-temporal features for action recognition, in: BMVC, 2009.
89. J. Wang, Z. Liu, J. Chorowski, Z. Chen, Y. Wu, Robust 3d action recognition with random occupancy patterns, in: ECCV. Springer, 2012a, pp. 872-885.
90. J. Wang, Z. Liu, Y. Wu, and J. Yuan Mining actionlet ensemble for action recognition with depth cameras CVPR 2012 1290 1297
91. A. Wedel, T. Brox, T. Vaudrey, C. Rabe, U. Franke, and D. Cremers Stereoscopic scene flow computation for 3d motion understanding IJCV 95 2011 29 51
92. D. Weinland, E. Boyer, and R. Ronfard Action recognition from arbitrary views using 3d exemplars ICCV, IEEE 2007 1 7
93. D. Weinland, M. Özuysal, and P. Fua Making action recognition robust to occlusions and viewpoint changes ECCV. Springer 2010 635 648
94. G. Willems, T. Tuytelaars, and L. Van Gool An efficient dense and scale-invariant spatio-temporal interest point detector ECCV 2008 650 663
95. C. Wolf, J. Mille, E. Lombardi, O. Celiktutan, M. B. Jiu, E. Dellandrea, C.E., Bichot, C. Garcia, B. Sankur, The liris human activities dataset and the icpr 2012 human activities recognition and localization competition. Technical Report RR-LIRIS-2012-004, LIRIS Laboratory.
96. D. Wu, F. Zhu, and L. Shao One shot learning gesture recognition from rgbd images CVPRW, IEEE 2012 7 12
97. L. Xia, J. Aggarwal, Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera, in: CVPR, IEEE, 2013.
98. L. Xia, C.C. Chen, and J. Aggarwal View invariant human action recognition using histograms of 3d joints CVPRW, IEEE 2012 20 27
99. C. Yang, and G. Medioni Object modelling by registration of multiple range images Image Vision Comput. 10 1992 145 155
100. X. Yang, Y. Tian, Eigenjoints-based action recognition using naïve-bayes-nearest-neighbor, in: CVPRW, IEEE, 2012, pp. 14-19.
101. X. Yang, C. Zhang, Y. Tian, Recognizing actions using depth motion maps-based histograms of oriented gradients, 2012.
102. A. Yao, J. Gall, G. Fanelli, L. Van Gool, Does human action recognition benefit from pose estimation?, in: BMVC, 2011.
103. A. Yao, J. Gall, and L. Van Gool A hough transform-based voting framework for action recognition CVPR, IEEE 2010 2061 2068
104. E. Yu, J. Aggarwal, Human action recognition with extremities as semantic posture representation, in: Computer Vision and Pattern Recognition Workshops, 2009. CVPR Workshops 2009. IEEE Computer Society Conference on, IEEE, 2009, pp. 1-8.
105. Y.K. Yu, K.H. Wong, S.H. Or, and M.M.Y. Chang Robust 3-d motion tracking from stereo images: A model-less method Instrum. Meas. IEEE Trans. 57 2008 622 630 (Pubitemid 351359257)
106. K. Yun, J. Honorio, D. Chattopadhyay, T.L. Berg, and D. Samaras Two-person interaction detection using body-pose features and multiple instance learning CVPRW, IEEE 2012 28 35
107. C. Zhang, Y. Tian, Rgbd camera-based daily living activity recognition, 2012.
108. H. Zhang, and L. Parker 4-dimensional local spatio-temporal features for human activity recognition IROS 2011 2044 2049
109. Y. Zhao, Z. Liu, L. Yang, and H. Cheng Combing rgb and depth map features for human activity recognition APSIPA ASC, IEEE 2012 1 4
110. Mitiche, Amar and Aggarwal, Jagdishkumar Keshoram, Computer Vision Analysis of Image Motion by Variational Methods, Springer, 2013.