Space-time representation of people based on 3D skeletal data: A review

Computer Vision and Image Understanding

Volumn 158, Issue , 2017, Pages 85-105

  Han, Fei ^a   Reily, Brian ^a   Hoff, William ^a   Zhang, Hao ^a  

^a Department of Metallurgical and Materials Engineering   (United States)

Author keywords

3D visual perception; Human representation; Skeleton data; Space time features; Survey

Indexed keywords

SURVEYING; SURVEYS; VISION;

FEATURE ENGINEERINGS; FUTURE RESEARCH DIRECTIONS; HUMAN REPRESENTATION; ON-LINE PERFORMANCE; SKELETON DATA; SPACE-TIME FEATURES; SPACE-TIME REPRESENTATIONS; VISUAL PERCEPTION;

MUSCULOSKELETAL SYSTEM;

EID: 85012894624     PISSN: 10773142     EISSN: 1090235X     Source Type: Journal    
DOI: 10.1016/j.cviu.2017.01.011     Document Type: Article

References (234)

1. Aggarwal, J., Xia, L., Human activity recognition from 3D data: a review. Pattern. Recognit. Lett. 48:0 (2014), 70–80.
2. Aggarwal, J.K., Ryoo, M.S., Human activity analysis: a review. ACM Comput. Surv., 43(3), 2011, 16.
3. Akhter, I., Black, M.J., Pose-conditioned joint angle limits for 3D human pose reconstruction. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
4. Amor, B., Su, J., Srivastava, A., Action recognition using rate-invariant analysis of skeletal shape trajectories. IEEE Trans. Pattern Anal. Mach. Intell. 38:1 (2016), 1–13.
5. Andriluka, M., Roth, S., Schiele, B., Pictorial structures revisited people detection and articulated pose estimation. IEEE Conference on Computer Vision and Pattern Recognition, 2009.
6. Anirudh, R., Turaga, P., Su, J., Srivastava, A., Elastic functional coding of human actions: From vector-fields to latent variables. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
7. Azary, S., Savakis, A., Grassmannian sparse representations and motion depth surfaces for 3D action recognition. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2013.
8. Baak, A., Muller, M., Bharaj, G., Seidel, H.-P., Theobalt, C., A data-driven approach for real-time full body pose reconstruction from a depth camera. IEEE International Conference on Computer Vision, 2011.
9. Bălan, A.O., Sigal, L., Black, M.J., Davis, J.E., Haussecker, H.W., Detailed human shape and pose from images. IEEE Conference on Computer Vision and Pattern Recognition, 2007.
10. Barbosa, B.I., Cristani, M., Del Bue, A., Bazzani, L., Murino, V., Re-identification with RGB-D sensors. International Workshop on Re-Identification, 2012.
11. Barnachon, M., Bouakaz, S., Boufama, B., Guillou, E., Ongoing human action recognition with motion capture. Pattern. Recognit. 47:1 (2014), 238–247.
12. Belagiannis, V., Amin, S., Andriluka, M., Schiele, B., Navab, N., Ilic, S., 3D pictorial structures for multiple human pose estimation. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
13. Bengio, Y., Courville, A., Vincent, P., Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35:8 (2013), 1798–1828.
14. BESL, P., MCKAY, N., A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14:2 (1992), 239–256.
15. Bloom, V., Argyriou, V., Makris, D., Dynamic feature selection for online action recognition. Human Behavior Understanding, 2013, 64–76.
16. Bloom, V., Argyriou, V., Makris, D., G3Di: A gaming interaction dataset with a real time detection and evaluation framework. Workshops on Computer Vision on European Conference on Computer Vision, 2014.
17. Bloom, V., Makris, D., Argyriou, V., G3D: a gaming action dataset and real time action recognition evaluation framework. Workshops on IEEE Conference on Computer Vision and Pattern Recognition, 2012.
18. Borges, P.V.K., Conci, N., Cavallaro, A., Video-based human behavior understanding: a survey. IEEE Trans. Circuits Syst. Video Technol. 23:11 (2013), 1993–2008.
19. Boubou, S., Suzuki, E., Classifying actions based on histogram of oriented velocity vectors. J. Intell. Inf. Syst. 44:1 (2015), 49–65.
20. Bourdev, L., Malik, J., Poselets: Body part detectors trained using 3D human pose annotations. IEEE International Conference on Computer Vision, 2009.
21. Brdiczka, O., Langet, M., Maisonnasse, J., Crowley, J.L., Detecting human behavior models from multimodal observation in a smart home. IEEE Trans. Autom. Sci. Eng. 6:4 (2009), 588–597.
22. Broadbent, E., Stafford, R., MacDonald, B., Acceptance of healthcare robots for the older population: review and future directions. Int. J. Soc. Robot. 1:4 (2009), 319–330.
23. Brooks, A.L., Czarowicz, A., Markerless motion tracking: MS Kinect & Organic Motion OpenStage®. International Conference on Disability, Virtual Reality and Associated Technologies, 2012.
24. Burenius, M., Sullivan, J., Carlsson, S., 3D pictorial structures for multiple view articulated pose estimation. IEEE Conference on Computer Vision and Pattern Recognition, 2013.
25. Campbell, L., Bobick, A., Recognition of human body motion using phase space constraints. IEEE International Conference on Computer Vision, 1995.
26. Cavazza, J., Zunino, A., Biagio, M.S., Murino, V., Kernelized covariance for action recognition. International Conference on Pattern Recognition, 2016.
27. Chaaraoui, A.A., Climent-Pérez, P., Flórez-Revuelta, F., A review on vision techniques applied to human behaviour analysis for ambient-assisted living. Expert Syst. Appl. 39:12 (2012), 10873–10888.
28. Chaaraoui, A.A., Padilla-López, J.R., Climent-Pérez, P., Flórez-Revuelta, F., Evolutionary joint selection to improve human action recognition with RGB-D devices. Expert Syst. Appl. 41:3 (2014), 786–794.
29. Charles, J., Everingham, M., Learning shape models for monocular human pose estimation from the Microsoft Xbox Kinect. IEEE International Conference on Computer Vision, 2011.
30. Chaudhry, R., Ofli, F., Kurillo, G., Bajcsy, R., Vidal, R., Bio-inspired dynamic 3D discriminative skeletal features for human action recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2013.
31. Chen, C., Jafari, R., Kehtarnavaz, N., UTD-MAD: A multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor. IEEE International Conference on Image Processing, 2015.
32. Chen, G., Giuliani, M., Clarke, D., Gaschler, A., Knoll, A., Action recognition using ensemble weighted multi-instance learning. IEEE International Conference on Robotics and Automation, 2014.
33. Chen, L., Wei, H., Ferryman, J., A survey of human motion analysis using depth imagery. Pattern Recognit. Lett. 34:15 (2013), 1995–2006.
34. Chen, X., Koskela, M., Online RGB-D gesture recognition with extreme learning machines. ACM International Conference on Multimodal Interaction, 2013.
35. Cippitelli, E., Gasparrini, S., De Santis, A., Montanini, L., Raffaeli, L., Gambi, E., Spinsante, S., Comparison of RGB-D mapping solutions for application to food intake monitoring. Ambient Assisted Living, 2015, 295–305.
36. Cippitelli, E., Gasparrini, S., Gambi, E., Spinsante, S., A human activity recognition system using skeleton data from RGBD sensors. Comput. Intell. Neurosci., 2016, 2016.
37. Cippitelli, E., Gasparrini, S., Gambi, E., Spinsante, S., Wahsleny, J., Orhany, I., Lindhy, T., Time synchronization and data fusion for RGB-depth cameras and inertial sensors in AAL applications. Workshop on IEEE International Conference on Communication, 2015.
38. Demircan, E., Kulic, D., Oetomo, D., Hayashibe, M., Human movement understanding. IEEE Robot. Autom. Mag. 22:3 (2015), 22–24.
39. Devanne, M., Wannous, H., Berretti, S., Pala, P., Daoudi, M., Del Bimbo, A., Space-time pose representation for 3D human action recognition. New Trends in Image Analysis and Processing, 2013, 456–464.
40. Devanne, M., Wannous, H., Berretti, S., Pala, P., Daoudi, M., Del Bimbo, A., 3-D human action recognition by shape analysis of motion trajectories on Riemannian manifold. IEEE Trans. Cybern. 45:7 (2015), 1340–1352.
41. Devanne, M., Wannous, H., Pala, P., Berretti, S., Daoudi, M., Del Bimbo, A., Combined shape analysis of human poses and motion units for action segmentation and recognition. IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, 2015.
42. Ding, M., Fan, G., Articulated and generalized Gaussian kernel correlation for human pose estimation. IEEE Trans. Image Process. 25:2 (2016), 776–789.
43. Dong, J., Chen, Q., Shen, X., Yang, J., Yan, S., Towards unified human parsing and pose estimation. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
44. Du, Y., Wang, W., Wang, L., Hierarchical recurrent neural network for skeleton based action recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
45. Elhayek, A., de Aguiar, E., Jain, A., Tompson, J., Pishchulin, L., Andriluka, M., Bregler, C., Schiele, B., Theobalt, C., Efficient ConvNet-based marker-less motion capture in general scenes with a low number of cameras. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
46. Ellis, C., Masood, S.Z., Tappen, M.F., Laviola Jr, J.J., Sukthankar, R., Exploring the trade-off between accuracy and observational latency in action recognition. Int. J. Comput. Vis. 101:3 (2013), 420–436.
47. Escalera, S., Gonzàlez, J., Baró, X., Reyes, M., Lopes, O., Guyon, I., Athitsos, V., Escalante, H., Multi-modal gesture recognition challenge 2013: dataset and results. ACM on International Conference on Multimodal Interaction, 2013.
48. Evangelidis, G., Singh, G., Horaud, R., Skeletal quads: human action recognition using joint quadruples. International Conference on Pattern Recognition, 2014.
49. Eweiwi, A., Cheema, M.S., Bauckhage, C., Gall, J., Efficient pose-based action recognition. Asian Conference on Computer Vision, 2015.
50. Fan, X., Zheng, K., Lin, Y., Wang, S., Combining local appearance and holistic view: dual-source deep neural networks for human pose estimation. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
51. Fan, Z., Li, G., Haixian, L., Shu, G., Jinkui, L., Star skeleton for human behavior recognition. International Conference on Audio, Language and Image Processing, 2012.
52. Fischler, M.A., Elschlager, R.A., The representation and matching of pictorial structures. IEEE Trans. Comput.(1), 1973, 67–92.
53. Fothergill, S., Mentis, H., Kohli, P., Nowozin, S., Instructing people for training gestural interactive systems. The SIGCHI Conference on Human Factors in Computing Systems, 2012.
54. Fu, Q., Santello, M., Tracking whole hand kinematics using extended Kalman filter. Annual International Conference on Engineering in Medicine and Biology Society, 2010.
55. Fujita, M., Digital creatures for future entertainment robotics. IEEE International Conference on Robotics and Automation, 2000.
56. Gall, J., Stoll, C., De Aguiar, E., Theobalt, C., Rosenhahn, B., Seidel, H.-P., Motion capture using joint skeleton tracking and surface estimation. IEEE Conference on Computer Vision and Pattern Recognition, 2009.
57. Ganapathi, V., Plagemann, C., Koller, D., Thrun, S., Real time motion capture using a single time-of-flight camera. IEEE Conference on Computer Vision and Pattern Recognition, 2010.
58. Ganapathi, V., Plagemann, C., Koller, D., Thrun, S., Real-time human pose tracking from range data. European Conference on Computer Vision, 2012.
59. ® sensor. Sensors 14:2 (2014), 2756–2775.
60. Ge, W., Collins, R.T., Ruback, R.B., Vision-based analysis of small groups in pedestrian crowds. IEEE Trans. Pattern Anal. Mach. Intell. 34:5 (2012), 1003–1016.
61. Giachetti, A., Fornasa, F., Parezzan, F., Saletti, A., Zambaldo, L., Zanini, L., Achilles, F., Ichim, A., Tombari, F., Navab, N., et al. Shrec’16 track: retrieval of human subjects from depth sensor data. Eurographics Workshop on 3D Object Retrieval, 2016.
62. Girshick, R., Shotton, J., Kohli, P., Criminisi, A., Fitzgibbon, A., Efficient regression of general-activity human poses from depth images. IEEE International Conference on Computer Vision, 2011.
63. Gong, D., Medioni, G., Dynamic manifold warping for view invariant action recognition. IEEE International Conference on Computer Vision, 2011.
64. Gong, D., Medioni, G., Zhao, X., Structured time series analysis for human action segmentation and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36:7 (2014), 1414–1427.
65. Gould, S., Russakovsky, O., Goodfellow, I., Baumstarck, P., STAIR Vision Library. Technical Report, 2011.
66. Gowayyed, M.A., Torki, M., Hussein, M.E., El-Saban, M., Histogram of oriented displacements (HOD): Describing trajectories of human joints for action recognition. International Joint Conference on Artificial Intelligence, 2013.
67. Green, S., Billinghurst, M., Chen, X., Chase, G., Human-robot collaboration: a literature review and augmented reality approach in design. Int. J. Adv. Rob. Syst. 5:1 (2008), 1–18.
68. Grest, D., Woetzel, J., Koch, R., Nonlinear body pose estimation from depth images. Pattern Recognition, 2005, 285–292.
69. Gu, Y., Do, H., Ou, Y., Sheng, W., Human gesture recognition through a Kinect sensor. IEEE International Conference on Robotics and Biomimetics, 2012.
70. Guerra-Filho, G., Aloimonos, Y., Understanding visuo-motor primitives for motion synthesis and analysis. Comput. Animat. Virtual Worlds 17:3–4 (2006), 207–217.
71. Gupta, A., Martinez, J.L., Little, J.J., Woodham, R.J., 3D pose from motion for cross-view action recognition via non-linear circulant temporal encoding. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
72. Han, F., Reardon, C., Parker, L., Zhang, H., Minimum uncertainty latent variable models for robot recognition of sequential human activities. IEEE International Conference on Robotics and Automation, 2017.
73. Han, F., Yang, X., Reardon, C., Zhang, Y., Zhang, H., Simultaneous feature and body-part learning for real-time robot awareness of human behaviors. IEEE International Conference on Robotics and Automation, 2017.
74. Han, J., Shao, L., Xu, D., Shotton, J., Enhanced computer vision with Microsoft Kinect sensor: a review. IEEE Trans. Cybern. 43:5 (2013), 1318–1334.
75. Han, L., Wu, X., Liang, W., Hou, G., Jia, Y., Discriminative human action recognition in the learned hierarchical manifold space. Image Vis. Comput. 28:5 (2010), 836–849.
76. Herda, L., Urtasun, R., Fua, P., Hierarchical implicit surface joint limits for human body tracking. Comput. Vision Image Understanding 99:2 (2005), 189–209.
77. Holt, B., Ong, E.-J., Cooper, H., Bowden, R., Putting the pieces together: Connected poselets for human pose estimation. Workshops on IEEE International Conference on Computer Vision, 2011.
78. Hu, J.-F., Zheng, W.-S., Lai, J., Zhang, J., Jointly learning heterogeneous features for RGB-D activity recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
79. Huang, C.-H., Boyer, E., Navab, N., Ilic, S., Human shape and pose tracking using keyframes. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
80. Huang, D., Yao, S., Wang, Y., De La Torre, F., Sequential max-margin event detectors. European Conference on Computer Vision, 2014.
81. Huang, D.-A., Kitani, K.M., Action-reaction: forecasting the dynamics of human interaction. European Conference on Computer Vision, 2014.
82. Huang, Y., Wu, Z., Wang, L., Tan, T., Feature coding in image classification: a comprehensive study. IEEE Trans. Pattern Anal. Mach. Intell. 36:3 (2014), 493–506.
83. Hussein, M.E., Torki, M., Gowayyed, M.A., El-Saban, M., Human action recognition using a temporal hierarchy of covariance descriptors on 3D joint locations. International Joint Conference on Artificial Intelligence, 2013.
84. Ichim, A.E., Tombari, F., Semantic parametric body shape estimation from noisy depth sequences. Rob. Auton. Syst. 75 (2016), 539–549.
85. Ikemura, S., Fujiyoshi, H., Real-time human detection using relational depth similarity features. Asian Conference on Computer Vision, 2011.
86. Ionescu, C., Carreira, J., Sminchisescu, C., Iterated second-order label sensitive pooling for 3D human pose estimation. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
87. Ionescu, C., Papava, D., Olaru, V., Sminchisescu, C., Human3.6m: large scale datasets and predictive methods for 3D human sensing in natural environments. IEEE Trans. Pattern Anal. Mach. Intell. 36:7 (2014), 1325–1339.
88. Ji, X., Liu, H., Advances in view-invariant human motion analysis: a review. IEEE Trans. Syst. Man Cybernet., Part C: Appl. Rev. 40:1 (2010), 13–24.
89. Jiang, X., Zhong, F., Peng, Q., Qin, X., Online robust action recognition based on a hierarchical model. Vis. Comput. 30:9 (2014), 1021–1033.
90. Jin, S.-Y., Choi, H.-J., Essential body-joint and atomic action detection for human activity recognition using longest common subsequence algorithm. Workshops on Asian Conference on Computer Vision, 2013.
91. Johansson, G., Visual perception of biological motion and a model for its analysis. Perception Psychophysics 14:2 (1973), 201–211.
92. Johnson, S., Everingham, M., Clustered pose and nonlinear appearance models for human pose estimation. British Machine Vision Conference, 2010.
93. Jun, B., Choi, I., Kim, D., Local transform features and hybridization for accurate face and human detection. IEEE Trans. Pattern Anal. Mach. Intell. 35:6 (2013), 1423–1436.
94. Junejo, I.N., Dexter, E., Laptev, I., Perez, P., View-independent action recognition from temporal self-similarities. IEEE Trans. Pattern Anal. Mach. Intell. 33:1 (2011), 172–185.
95. Jung, H.-J., Hong, K.-S., Enhanced sequence matching for action recognition from 3D skeletal data. Asian Conference on Computer Vision, 2015.
96. Jung, H.Y., Lee, S., Heo, Y.S., Yun, I.D., Random tree walk toward instantaneous 3D human pose estimation. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
97. Kakadiaris, I., Metaxas, D., Model-based estimation of 3D human motion. IEEE Trans. Pattern Anal. Mach. Intell. 22:12 (2000), 1453–1459.
98. Kang, K.I., Freedman, S., Matarić, M.J., Cunningham, M.J., Lopez, B., A hands-off physical therapy assistance robot for cardiac patients. International Conference on Rehabilitation Robotics, 2005.
99. Kapsouras, I., Nikolaidis, N., Action recognition on motion capture data using a dynemes and forward differences representation. J. Vis. Commun. Image Represent. 25:6 (2014), 1432–1445.
100. Karcher, H., Riemannian center of mass and mollifier smoothing. Commun. Pure Appl. Math. 30:5 (1977), 509–541.
101. Kazemi, V., Burenius, M., Azizpour, H., Sullivan, J., Multi-view body part recognition with random forests. British Machine Vision Conference, 2013.
102. Ke, S.-R., Thuc, H.L.U., Lee, Y.-J., Hwang, J.-N., Yoo, J.-H., Choi, K.-H., A review on video-based human activity recognition. Computers 2:2 (2013), 88–131.
103. Kerola, T., Inoue, N., Shinoda, K., Spectral graph skeletons for 3D action recognition. Asian Conference on Computer Vision, 2014.
104. Koppula, H., Saxena, A., Learning spatio-temporal structure from RGB-D videos for human activity detection and anticipation. International Conference on Machine Learning, 2013.
105. Koppula, H.S., Gupta, R., Saxena, A., Learning human activities and object affordances from RGB-D videos. Int. J. Rob. Res. 32:8 (2013), 951–970.
106. Kviatkovsky, I., Rivlin, E., Shimshoni, I., Online action recognition using covariance of shape and motion. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
107. LaViola, J.J., 3D gestural interaction: the state of the field. Int. Sch. Res. Notices, 2013, 2013.
108. Lazebnik, S., Schmid, C., Ponce, J., Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. IEEE Conference on Computer Vision and Pattern Recognition, 2006.
109. Le, Q.V., Zou, W.Y., Yeung, S.Y., Ng, A.Y., Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. IEEE Conference on Computer Vision and Pattern Recognition, 2011.
110. Lee, M.W., Nevatia, R., Dynamic human pose estimation using Markov chain Monte Carlo approach. IEEE Workshops on Application of Computer Vision, 2005.
111. Lehrmann, A.M., Gehler, P.V., Nowozin, S., A non-parametric Bayesian network prior of human pose. IEEE International Conference on Computer Vision, 2013.
112. Li, S.-Z., Yu, B., Wu, W., Su, S.-Z., Ji, R.-R., Feature learning based on SAE–PCA network for human gesture recognition in RGBD images. Neurocomputing 151 (2015), 565–573.
113. Li, W., Zhang, Z., Liu, Z., Action recognition based on a bag of 3D points. Workshops on IEEE Conference on Computer Vision and Pattern Recognition, 2010.
114. Lillo, I., Soto, A., Niebles, J.C., Discriminative hierarchical modeling of spatio-temporally composable human activities. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
115. Liu, A.-A., Xu, N., Su, Y.-T., Lin, H., Hao, T., Yang, Z.-X., Single/multi-view human action recognition via regularized multi-task learning. Neurocomputing 151 (2015), 544–553.
116. Liu, Y., Stoll, C., Gall, J., Seidel, H.-P., Theobalt, C., Markerless motion capture of interacting characters using multi-view image segmentation. IEEE Conference on Computer Vision and Pattern Recognition, 2011.
117. Lopez, J. R. P., Chaaraoui, A. A., Revuelta, F. F., 2014. A discussion on the validation tests employed to compare human action recognition methods using the MSR Action3D dataset. Preprint arXiv:1407.7390.
118. Lun, R., Zhao, W., A survey of applications and human motion recognition with Microsoft Kinect. Int. J. Pattern Recognit. Artif. Intell., 29(5), 2015.
119. Luo, J., Wang, W., Qi, H., Group sparsity and geometry constrained dictionary learning for action recognition from depth maps. IEEE International Conference on Computer Vision, 2013.
120. Lv, F., Nevatia, R., Recognition and segmentation of 3-D human action using HMM and multi-class adaboost. European Conference on Computer Vision, 2006.
121. Marinoiu, E., Papava, D., Sminchisescu, C., Pictorial human spaces: How well do humans perceive a 3D articulated pose?. IEEE International Conference on Computer Vision, 2013.
122. Masood, S.Z., Ellis, C., Nagaraja, A., Tappen, M.F., Jr., J.J.L., Sukthankar, R., Measuring and reducing observational latency when recognizing actions. Workshop on IEEE International Conference on Computer Vision, 2011.
123. Meshry, M., Hussein, M.E., Torki, M., Linear-time online action detection from 3D skeletal data using bags of gesturelets. IEEE Winter Conference on Applications of Computer Vision, accepted, 2016, IEEE.
124. Miranda, L., Vieira, T., Martinez, D., Lewiner, T., Vieira, A.W., Campos, M.F., Real-time gesture recognition from depth data through key poses learning and decision forests. SIBGRAPI Conference on Graphics, Patterns and Images, 2012.
125. Moeslund, T.B., Granum, E., A survey of computer vision-based human motion capture. Comput. Vision Image Understanding 81:3 (2001), 231–268.
126. Moeslund, T.B., Hilton, A., Krüger, V., A survey of advances in vision-based human motion capture and analysis. Comput. Vision Image Understanding 104:2 (2006), 90–126.
127. Mondada, F., Bonani, M., Raemy, X., Pugh, J., Cianci, C., Klaptocz, A., Magnenat, S., Zufferey, J.-C., Floreano, D., Martinoli, A., The e-puck, a robot designed for education in engineering. Conference on Autonomous Robot Systems and Competitions, 2009.
128. Muller, M., Roder, T., Clausen, M., Eberhardt, B., Kruger, B., Weber, A., Documentation Mocap Database HDM05. Technical Report, 2007, Universität Bonn.
129. Munaro, M., Fossati, A., Basso, A., Menegatti, E., Van Gool, L., One-shot person re-identification with a consumer depth camera. Person Re-Identification, 2014, 161–181.
130. Munsell, B.C., Temlyakov, A., Qu, C., Wang, S., Person identification using full-body motion and anthropometric biometrics from Kinect videos. European Conference on Computer Vision, 2012.
131. Negin, F., Ozdemir, F., Akgul, C.B., Yuksel, K.A., Ercil, A., A decision forest based feature selection framework for action recognition from RGB-Depth cameras. International Conference on Image Analysis and Recognition, 2013.
132. Nie, X., Xiong, C., Zhu, S.-C., Joint action recognition and pose estimation from video. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
133. Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., Bajcsy, R., Berkeley MHAD: A comprehensive multimodal human action database. IEEE Workshop on Applications of Computer Vision, 2013.
134. Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., Bajcsy, R., Sequence of the most informative joints (SMIJ): A new representation for human skeletal action recognition. J. Vis. Commun. Image Represent. 25:1 (2014), 24–38.
135. Ohn-Bar, E., Trivedi, M.M., Joint angles similarities and HOG2 for action recognition. Workshop on IEEE Conference on Computer Vision and Pattern Recognition, 2013.
136. Okada, K., Ogura, T., Haneda, A., Fujimoto, J., Gravot, F., Inaba, M., Humanoid motion generation system on HRP2-JSK for daily life environment. IEEE International Conference on Mechatronics and Automation, 2005.
137. Oreifej, O., Liu, Z., HON4D: histogram of oriented 4D normals for activity recognition from depth sequences. IEEE Conference on Computer Vision and Pattern Recognition, 2013.
138. Paiement, A., Tao, L., Hannuna, S., Camplani, M., Damen, D., Mirmehdi, M., Online quality assessment of human movement from skeleton data. British Machine Vision Conference, 2014.
139. Parameswaran, V., Chellappa, R., View invariance for human action recognition. Int. J. Comput. Vis. 66:1 (2006), 83–101.
140. Patsadu, O., Nukoolkit, C., Watanapa, B., Human gesture recognition using Kinect camera. International Joint Conference on Computer Science and Software Engineering, 2012.
141. Plagemann, C., Ganapathi, V., Koller, D., Thrun, S., Real-time identification and localization of body parts from depth images. IEEE International Conference on Robotics and Automation, 2010.
142. Pons-Moll, G., Fleet, D.J., Rosenhahn, B., Posebits for monocular human pose estimation. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
143. Poppe, R., A survey on vision-based human action recognition. Image Vis. Comput. 28:6 (2010), 976–990.
144. Rahmani, H., Mahmood, A., Huynh, D.Q., Mian, A., HOPC: histogram of oriented principal components of 3D pointclouds for action recognition. European Conference on Computer Vision, 2014.
145. Rahmani, H., Mahmood, A., Mian, A., Huynh, D., Real time action recognition using histograms of depth gradients and random decision forests. IEEE Winter Conference on Applications of Computer Vision, 2014.
146. Rahmani, H., Mian, A., Learning a non-linear knowledge transfer model for cross-view action recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
147. Reyes, M., Domínguez, G., Escalera, S., Feature weighting in dynamic timewarping for gesture recognition in depth data. Workshops on IEEE International Conference on Computer Vision, 2011.
148. Ruffieux, S., Lalanne, D., Mugellini, E., Khaled, O.A., A survey of datasets for human gesture recognition. Human-Computer Interaction. Advanced Interaction Modalities and Techniques, 2014, 337–348.
149. Sadeghipour, A., Morency, L.-P., Kopp, S., Gesture-based object recognition using histograms of guiding strokes. British Machine Vision Conference, 2012.
150. Salakhutdinov, R., Tenenbaum, J.B., Torralba, A., Learning with hierarchical-deep models. IEEE Trans. Pattern Anal. Mach. Intell. 35:8 (2013), 1958–1971.
151. Schwarz, L.A., Mkhitaryan, A., Mateus, D., Navab, N., Human skeleton tracking from depth data using geodesic distances and optical flow. Image Vis. Comput. 30:3 (2012), 217–226.
152. Seidenari, L., Varano, V., Berretti, S., Del Bimbo, A., Pala, P., Recognizing actions from depth cameras as weakly aligned multi-part bag-of-poses. Workshop on IEEE Conference on Computer Vision and Pattern Recognition, 2013.
153. Shahroudy, A., Ng, T.-T., Yang, Q., Wang, G., Multimodal multipart learning for action recognition in depth videos. IEEE Trans. Pattern Anal. Mach. Intell 38:10 (2016), 2123–2129.
154. Shahroudy, A., Wang, G., Ng, T.-T., Multi-modal feature fusion for action recognition in RGB-D sequences. International Symposium on Communications, Control and Signal Processing, 2014.
155. Sharaf, A., Torki, M., Hussein, M.E., El-Saban, M., Real-time multi-scale action detection from 3D skeleton data. IEEE Winter Conference on Applications of Computer Vision, 2015.
156. Sheikh, Y., Sheikh, M., Shah, M., Exploring the space of a human action. IEEE International Conference on Computer Vision, 2005.
157. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A., Real-time human pose recognition in parts from single depth images. IEEE Conference on Computer Vision and Pattern Recognition, 2011.
158. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A., Real-time human pose recognition in parts from single depth images. IEEE Conference on Computer Vision and Pattern Recognition, 2011.
159. Siddharth, N., Barbu, A., Siskind, J.M., Seeing what you're told: Sentence-guided activity recognition in video. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
160. Sigal, L., Balan, A.O., Black, M.J., Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int. J. Comput. Vis. 87:1–2 (2010), 4–27.
161. Slama, R., Wannous, H., Daoudi, M., Grassmannian representation of motion depth for 3D human gesture and action recognition. International Conference on Pattern Recognition, 2014.
162. Slama, R., Wannous, H., Daoudi, M., Srivastava, A., Accurate 3D action recognition using learning on the Grassmann manifold. Pattern Recognit. 48:2 (2015), 556–567.
163. Spinello, L., Arras, K.O., People detection in RGB-D data. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011.
164. Su, J., Srivastava, A., de Souza, F.D., Sarkar, S., Rate-invariant analysis of trajectories on Riemannian manifolds with application in visual speech recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
165. Sun, M., Kohli, P., Shotton, J., Conditional regression forests for human pose estimation. IEEE Conference on Computer Vision and Pattern Recognition, 2012.
166. Sun, X., Wei, Y., Liang, S., Tang, X., Sun, J., Cascaded hand pose regression. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
167. Sung, J., Ponce, C., Selman, B., Saxena, A., Human activity detection from RGBD images. Workshops on AAAI Conference on Artificial Intelligence, 2011.
168. Sung, J., Ponce, C., Selman, B., Saxena, A., Unstructured human activity detection from RGBD images. IEEE International Conference on Robotics and Automation, 2012.
169. Tang, D., Chang, H.J., Tejani, A., Kim, T.-K., Latent regression forest: Structured estimation of 3D articulated hand posture. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
170. Tao, L., Vidal, R., Moving poselets: A discriminative and interpretable skeletal motion representation for action recognition. Workshops on IEEE International Conference on Computer Vision, 2015.
171. Taylor, J., Shotton, J., Sharp, T., Fitzgibbon, A., The vitruvian manifold: Inferring dense correspondences for one-shot human pose estimation. IEEE Conference on Computer Vision and Pattern Recognition, 2012.
172. Tenorth, M., Bandouch, J., Beetz, M., The TUM kitchen data set of everyday manipulation activities for motion tracking and action recognition. Workshops on IEEE International Conference on Computer Vision, 2009.
173. Tobon, R., The Mocap Book: A Practical Guide to the Art of Motion Capture, 2010, Foris Force.
174. Tompson, J.J., Jain, A., LeCun, Y., Bregler, C., Joint training of a convolutional network and a graphical model for human pose estimation. Annual Conference on Neural Information Processing Systems, 2014.
175. Toshev, A., Szegedy, C., DeepPose: Human pose estimation via deep neural networks. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
176. Vantigodi, S., Babu, R.V., Real-time human action recognition from motion capture data. National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics, 2013.
177. Vantigodi, S., Radhakrishnan, V.B., Action recognition from motion capture data using meta-cognitive RBF network classifier. International Conference on Intelligent Sensors, Sensor Networks and Information Processing, 2014.
178. Vemulapalli, R., Arrate, F., Chellappa, R., Human action recognition by representing 3D skeletons as points in a lie group. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
179. Vieira, A.W., Nascimento, E.R., Oliveira, G.L., Liu, Z., Campos, M.F., STOP: space-time occupancy patterns for 3D action recognition from depth map sequences. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, 2012, 252–259.
180. Wang, C., Wang, Y., Lin, Z., Yuille, A.L., Gao, W., Robust estimation of 3D human poses from single images. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
181. Wang, C., Wang, Y., Yuille, A.L., An approach to pose-based action recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2013.
182. Wang, J., Liu, Z., Chorowski, J., Chen, Z., Wu, Y., Robust 3D action recognition with random occupancy patterns. European Conference on Computer Vision, 2012.
183. Wang, J., Liu, Z., Wu, Y., Yuan, J., Mining actionlet ensemble for action recognition with depth cameras. IEEE Conference on Computer Vision and Pattern Recognition, 2012.
184. Wang, J., Liu, Z., Wu, Y., Yuan, J., Learning actionlet ensemble for 3D human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36:5 (2014), 914–927.
185. Wang, J., Nie, X., Xia, Y., Wu, Y., Zhu, S.-C., Cross-view action modeling, learning, and recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
186. Wang, L., Zhang, J., Zhou, L., Tang, C., Li, W., Beyond covariance: feature representation with nonlinear kernel matrices. IEEE International Conference on Computer Vision, 2015.
187. Wei, P., Zhao, Y., Zheng, N., Zhu, S.-C., Modeling 4D human-object interactions for event and object recognition. IEEE International Conference on Computer Vision, 2013.
188. Wei, P., Zheng, N., Zhao, Y., Zhu, S.-C., Concurrent action detection with structural prediction. IEEE International Conference on Computer Vision, 2013.
189. Wu, C., Zhang, J., Savarese, S., Saxena, A., Watch-n-Patch: unsupervised understanding of actions and relations. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
190. Wu, D., Shao, L., Leveraging hierarchical parametric networks for skeletal joints action segmentation and recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
191. Xia, L., Chen, C.-C., Aggarwal, J., View invariant human action recognition using histograms of 3D joints. Workshops on IEEE Conference on Computer Vision and Pattern Recognition, 2012.
192. Xu, C., Cheng, L., Efficient hand pose estimation from a single depth image. IEEE International Conference on Computer Vision, 2013.
193. Xu, N., Liu, A., Nie, W., Wong, Y., Li, F., Su, Y., Multi-modal & multi-view & interactive benchmark dataset for human action recognition. Annual ACM Conference on Multimedia Conference, 2015.
194. Yamane, Y., Carlson, E.T., Bowman, K.C., Wang, Z., Connor, C.E., A neural code for three-dimensional object shape in macaque inferotemporal cortex. Nat. Neurosci. 11:11 (2008), 1352–1360.
195. Yang, X., Tian, Y., EigenJoints-based action recognition using Näive-Bayes-Nearest-Neighbor. Workshops on IEEE Conference on Computer Vision and Pattern Recognition, 2012.
196. Yang, X., Tian, Y., Effective 3D action recognition using eigenjoints. J. Vis. Commun. Image Represent. 25:1 (2014), 2–11.
197. Yang, Y., Ramanan, D., Articulated pose estimation with flexible mixtures-of-parts. IEEE Conference on Computer Vision and Pattern Recognition, 2011.
198. Yao, A., Gall, J., Fanelli, G., Gool, L.V., Does human action recognition benefit from pose estimation?. British Machine Vision Conference, 2011.
199. Yao, A., Gall, J., Van Gool, L., Coupled action recognition and pose estimation from multiple views. Int. J. Comput. Vis. 100:1 (2012), 16–37.
200. Yao, B., Fei-Fei, L., Action recognition with exemplar based 2.5D graph matching. European Conference on Computer Vision, 2012.
201. Ye, M., Wang, X., Yang, R., Ren, L., Pollefeys, M., Accurate 3D pose estimation from a single depth image. IEEE International Conference on Computer Vision, 2011.
202. Ye, M., Zhang, Q., Wang, L., Zhu, J., Yang, R., Gall, J., A survey on human motion analysis from depth data. Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, 2013, 149–187.
203. Yilma, A., Shah, M., Recognizing human actions in videos acquired by uncalibrated moving cameras. IEEE International Conference on Computer Vision, 2005.
204. Yu, G., Liu, Z., Yuan, J., Discriminative orderlet mining for real-time recognition of human-object interaction. Asian Conference on Computer Vision, 2014.
205. Yun, K., Honorio, J., Chattopadhyay, D., Berg, T., Samaras, D., Two-person interaction detection using body-pose features and multiple instance learning. Workshops on IEEE Conference on Computer Vision and Pattern Recognition, 2012.
206. Zanfir, M., Leordeanu, M., Sminchisescu, C., The moving pose: An efficient 3D kinematics descriptor for low-latency action recognition and detection. IEEE International Conference on Computer Vision, 2013.
207. Zeiler, M.D., Fergus, R., Visualizing and understanding convolutional networks. European Conference on Computer Vision, 2014.
208. Zhang, C., Tian, Y., RGB-D camera-based daily living activity recognition. J. Comput. Vis. Image Process., 2(4), 2012, 12.
209. Zhang, H., Parker, L.E., 4-dimensional local spatio-temporal features for human activity recognition. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011.
210. Zhang, H., Parker, L.E., Bio-inspired predictive orientation decomposition of skeleton trajectories for real-time human activity prediction. International Conference on Robotics and Automation, 2015.
211. Zhang, Q., Song, X., Shao, X., Shibasaki, R., Zhao, H., Unsupervised skeleton extraction and motion capture from 3D deformable matching. Neurocomputing 100 (2013), 170–182.
212. Zhao, X., Li, X., Pang, C., Zhu, X., Sheng, Q.Z., Online human gesture recognition from motion data streams. ACM International Conference on Multimedia, 2013.
213. Zhou, F., De la Torre, F., Spatio-temporal matching for human detection in video. European Conference on Computer Vision, 2014.
214. Zhou, F., De la Torre, F., Hodgins, J.K., Hierarchical aligned cluster analysis for temporal clustering of human motion. IEEE Trans. Pattern Anal. Mach. Intell. 35:3 (2013), 582–596.
215. Zhou, H., Hu, H., Human motion tracking for rehabilitationa survey. Biomed Signal Process. Control 3:1 (2008), 1–18.
216. Zhu, W., Lan, C., Xing, J., Li, Y., Shen, L., Zeng, W., Xie, X., Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks. AAAI Conference on Artificial Intelligence, to appear, 2016.
217. Zhu, Y., Dariush, B., Fujimura, K., Controlled human pose estimation from depth image streams. IEEE Conference on Computer Vision and Pattern Recognition, 2008.
218. Zou, W., Wang, B., Zhang, R., Human action recognition by mining discriminative segment with novel skeleton joint feature. Advances in Multimedia Information Processing, 2013, 517–527.
219. CMU Graphics Lab Motion Capture Database. 2001. http://mocap.cs.cmu.edu.
220. CMU Multi-Modal Activity Database. 2010. http://kitchen.cs.cmu.edu.
221. ASUS Xtion PRO LIVE. 2011. https://www.asus.com/3D-Sensor/Xtion_PRO/.
222. PrimeSense. 2011. https://en.wikipedia.org/wiki/PrimeSense.
223. Microsoft Kinect. 2012. https://dev.windows.com/en-us/kinect.
224. MSRC-12 Kinect Gesture Dataset. 2012. http://research.microsoft.com/en-us/um/cambridge/projects/msrc12.
225. NITE. 2012. http://openni.ru/files/nite/.
226. Package for orientation matrix approach. 2012. https://github.com/jysung/activity_detection.
227. The OpenKinect Library. 2012. http://www.openkinect.org.
228. Package for node feature map approach. 2013. https://github.com/hemakoppula/human_activity_labeling.
229. Package for temporal relational features approach. 2013. https://github.com/hemakoppula/human_activity_anticipation.
230. The OpenNI Library. http://structure.io/openni. 2013.
231. Kinect v2 SDK. 2014. https://dev.windows.com/en-us/kinect/tools.
232. Package for lie group manifold approach. http://ravitejav.weebly.com/kbac.html. 2014.
233. Package for MPJPE approach. 2014. http://vision.imar.ro/human3.6m/description.php.
234. Package for Ker-RP approach. 2015. http://www.uow.edu.au/~leiw/share_files/Kernel_representation_Code_for_release.zip.