R3DG features: Relative 3D geometry-based skeletal representations for human action recognition

Computer Vision and Image Understanding

Volumn 152, Issue , 2016, Pages 155-166

  Vemulapalli, Raviteja ^a   Arrate, Felipe ^b   Chellappa, Rama ^a  

^a UNIVERSITY OF MARYLAND   (United States)

^b DEPARTAMENTO DE FÍSICA   (Chile)

Author keywords

Action recognition; Dual quaternions; Lie groups; Quaternions; Skeletal representations; Special Euclidean group; Special orthogonal group

Indexed keywords

ALGORITHMS; COST EFFECTIVENESS; GESTURE RECOGNITION; IMAGE RECOGNITION; LIE GROUPS; MATHEMATICAL TRANSFORMATIONS; MUSCULOSKELETAL SYSTEM;

ACTION RECOGNITION; DUAL QUATERNION; QUATERNIONS; SKELETAL REPRESENTATIONS; SPECIAL EUCLIDEAN GROUP; SPECIAL ORTHOGONAL GROUP;

MOTION ESTIMATION;

EID: 84963976003     PISSN: 10773142     EISSN: 1090235X     Source Type: Journal    
DOI: 10.1016/j.cviu.2016.04.005     Document Type: Article

References (75)

1. Aggarwal, J.K., Ryoo, M.S., Human activity analysis: a review. ACM Comput. Surv. 43:3 (2011), 16:1–16:43.
2. Anirudh, R., Turaga, P., Su, J., Srivastava, A., Elastic functional coding of human actions: from vector-fields to latent variables. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
3. Belta, C., Kumar, V., An SVD-based projection method for interpolation on SE(3). IEEE Trans. Rob. Autom. 18:3 (2002), 334–345.
4. Bloom, V., Makris, D., Argyriou, V., G3D: a gaming action dataset and real time action recognition evaluation framework. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2012.
5. Broida, T.J., Chandrashekhar, S., Chellappa, R., Recursive 3-D motion estimation from a monocular image sequence. IEEE Trans. Aerosp. Electron. Syst. 26 (1990), 639–656.
6. Campbell, L.W., Bobick, A.F., Recognition of human body motion using phase space constraints. IEEE International Conference on Computer Vision, 1995.
7. Chaaraoui, A.A., Padilla-López, J.R., Flórez-Revuelta, F., Fusion of skeletal and silhouette-based features for human action recognition with RGB-D devices. IEEE International Conference on Computer Vision Workshops, 2013.
8. Chaudhry, R., Ofli, F., Kurillo, G., Bajcsy, R., Vidal, R., Bio-inspired dynamic 3D discriminative skeletal features for human action recognition. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2013.
9. Chen, C., Jafari, R., Kehtarnavaz, N., Action recognition from depth sequences using depth motion maps-based local binary patterns. IEEE Winter Conference on Applications of Computer Vision, 2015.
10. Chen, G., Giuliani, M., Clarke, D., Gaschler, A., Knoll, A., Action recognition using ensemble weighted multi-instance learning. IEEE International Conference on Robotics and Automation, 2014.
11. Chen, L., Wei, H., Ferryman, J.M., A survey of human motion analysis using depth imagery. Pattern Recognit. Lett. 34:15 (2013), 1995–2006.
12. Devanne, M., Wannous, H., Berretti, S., Pala, P., Daoudi, M., Bimbo, A.D., 3D human action recognition by shape analysis of motion trajectories on Riemannian manifold. IEEE Trans. Cybern. 45:7 (2014), 1340–1352.
13. Du, Y., Wang, W., Wang, L., Hierarchical recurrent neural network for skeleton based action recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
14. Ellis, C., Masood, S.Z., Tappen, M.F., LaViola, J.J., Sukthankar, R., Exploring the trade-off between accuracy and observational latency in action recognition. Int. J. Comput. Vis. 101:3 (2013), 420–436.
15. Evangelidis, G., Singh, G., Horaud, R., Skeletal quads: human action recognition using joint quadruples. International Conference on Pattern Recognition, 2014.
16. Gavrila, D.M., Davis, L.S., Towards 3-D model-based tracking and recognition of human movement: a multi-view approach. International Workshop on Automatic Face and Gesture Recognition, 1995.
17. Gowayyed, M.A., Torki, M., Hussein, M.E., El-Saban, M., Histogram of oriented displacements (HOD): describing trajectories of human joints for action recognition. International Joint Conference on Artificial Intelligence, 2013.
18. Gu, J., Ding, X., Wang, S., Wu, Y., Action and gait recognition from recovered 3-D human joints. IEEE Trans. Syst Man Cybern Part B 40:4 (2010), 1021–1033.
19. Hall, B., Lie Groups, Lie Algebras, and Representations: An Elementary Introduction, 2003, Springer.
20. Hu, J.-F., Zheng, W.-S., Lai, J., Zhang, J., Jointly learning heterogeneous features for RGB-D activity recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
21. Hussein, M.E., Torki, M., Gowayyed, M.A., El-Saban, M., Human action recognition using a temporal hierarchy of covariance descriptors on 3D joint locations. International Joint Conference on Artificial Intelligence, 2013.
22. Jaakkola, T.S., Haussler, D., Exploiting generative models in discriminative classifiers. Neural Information Processing Systems, 1998.
23. Johansson, G., Visual perception of biological motion and a model for its analysis. Percept. Psychophys. 14:2 (1973), 201–211.
24. Kavan, L., Collins, S., Zára, J., O'Sullivan, C., Geometric skinning with approximate dual quaternion blending. ACM Trans. Graphics 27:4 (2008), 105:1–105:23.
25. Kenwright, B., A Beginners Guide to Dual-Quaternions: What They are, how they work, and how to use them for 3D character hierarchies. International Conference on Computer Graphics, Visualization and Computer Vision, 2012.
26. Kong, Y., Fu, Y., Bilinear Heterogeneous Information Machine for RGB-D Action Recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
27. Kulkarni, K., Evangelidis, G., Cech, J., Horaud, R., Continuous Action Recognition Based on Sequence Alignment. Int. J. Comput. Vis. 112:1 (2015), 90–114.
28. Li, W., Zhang, Z., Liu, Z., Action Recognition Based on a Bag of 3D Points. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2010.
29. Lu, C., Jia, J., Tang, C., Range-Sample Depth Feature for Action Recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
30. Lv, F., Nevatia, R., Recognition and Segmentation of 3D Human Action Using HMM and Multi-class AdaBoost. ECCV, 2006.
31. Marr, D., Nishihara, H.K., Representation and recognition of the spatial organization of three-dimensional shapes. Proceedings of the Royal Society of London, Series B, Biological Sciences 200:1140 (1978), 269–294.
32. McCarthy, J.M., Introduction to Theoretical Kinematics, 1990, MIT Press.
33. Moeslund, T.B., Hilton, A., Krüger, V., A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104:2-3 (2006), 90–126.
34. Müller, M., Information Retrieval for Music and Motion, 2007, Springer-Verlag, New York.
35. Müller, M., Röder, T., Clausen, M., Efficient content-based retrieval of motion capture data. ACM Trans. Graphics 24:3 (2005), 677–685.
36. Murray, R.M., Li, Z., Sastry, S.S., A Mathematical Introduction to Robotic Manipulation, 1994, CRC press.
37. Nie, S., Ji, Q., Capturing global and local dynamics for human action recognition. International Conference on Pattern Recognition, 2014.
38. Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., Bajcsy, R., Sequence of the most informative joints (SMIJ): a new representation for human skeletal action recognition. J. Vis. Commun. Image Represent. 25:1 (2014), 24–38.
39. 2 for action recognition. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2013.
40. Oreifej, O., Liu, Z., HON4D: histogram of oriented 4D normals for activity recognition from depth sequences. IEEE Conference on Computer Vision and Pattern Recognition, 2013.
41. Park, F.C., Ravani, B., Smooth invariant interpolation of rotations. ACM Trans. Graphics 16:3 (1997), 277–295.
42. Presti, L.L., Cascia, M.L., S., S., Camps, O.I., Gesture modeling by hanklet-based hidden Markov model. Asian Conference on Computer Vision, 2014.
43. Rahmani, H., Mahmood, A., Huynh, D.Q., Mian, A.S., HOPC: histogram of oriented principal components of 3D pointclouds for action recognition. European Conference on Computer Vision, 2014.
44. Reyes, M., Dominguez, G., Escalera, S., Feature weighting in dynamic time warping for gesture recognition in depth data. IEEE International Conference on Computer Vision Workshops, 2011.
45. Seidenari, L., Varano, V., Berretti, S., Bimbo, A.D., Pala, P., Recognizing actions from depth cameras as weakly aligned multi-part bag-of-poses. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2013.
46. Shao, Z., Li, Y.F., A new descriptor for multiple 3D motion trajectories recognition. IEEE International Conference on Robotics and Automation, 2013.
47. Sheikh, Y., Sheikh, M., Shah, M., Exploring the space of a human action. IEEE International Conference on Computer Vision, 2005.
48. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A., Real-time human pose recognition in parts from a single depth image. IEEE Conference on Computer Vision and Pattern Recognition, 2011.
49. Sung, J., Ponce, C., Selman, B., Saxena, A., Unstructured human activity detection from RGBD images. IEEE International Conference on Robotics and Automation, 2012.
50. Turaga, P.K., Chellappa, R., Subrahmanian, V.S., Udrea, O., Machine recognition of human activities: a survey. IEEE Trans. Circ. Syst. Vid. Technol. 18:11 (2008), 1473–1488.
51. Veeraraghavan, A., Srivastava, A., Roy-Chowdhury, A.K., Chellappa, R., Rate-invariant recognition of humans and their activities. IEEE Trans. Image Process. 18:6 (2009), 1326–1339.
52. Vemulapalli, R., Arrate, F., Chellappa, R., Human action recognition by representing 3D skeletons as points in a lie group. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
53. Wang, C., Wang, Y., Yuille, A.L., An approach to pose-based action recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2013.
54. Wang, J., Liu, Z., Chorowski, J., Chen, Z., Wu, Y., Robust 3D action recognition with random occupancy patterns. European Conference on Computer Vision, 2012.
55. Wang, J., Liu, Z., Wu, Y., Yuan, J., Mining actionlet ensemble for action recognition with depth cameras. IEEE Conference on Computer Vision and Pattern Recognition, 2012.
56. Wang, J., Wu, Y., learning maximum margin temporal warping for action recognition. IEEE International Conference on Computer Vision, 2013.
57. Wei, P., Zheng, N., Zhao, Y., Zhu, S.C., Concurrent action detection with structural prediction. IEEE International Conference on Computer Vision, 2013.
58. Xia, L., Aggarwal, J.K., Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera. IEEE Conference on Computer Vision and Pattern Recognition, 2013.
59. Xia, L., Chen, C.C., Aggarwal, J.K., View invariant human action recognition using histograms of 3D joints. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2012.
60. Yacoob, Y., Black, M.J., Parameterized modeling and recognition of activities. IEEE International Conference on Computer Vision, 1998.
61. Yang, X., Tian, Y., Effective 3D action recognition using Eigen joints. J. Vis. Commun. Image Represent. 25:1 (2014), 2–11.
62. Yang, X., Tian, Y., Super normal vector for activity recognition using depth sequences. IEEE Conference on Computer Vision and Pattern Recognition, 2014.
63. Yang, X., Zhang, C., Tian, Y., Recognizing actions using depth motion maps-based histograms of oriented gradients. ACM Multimedia Conference, 2012.
64. Yao, A., Gall, J., Fanelli, G., Gool, L.V., Does human action recognition benefit from pose estimation?. British Machine Vision Conference, 2011.
65. Yao, A., Gall, J., Gool, L.V., Coupled action recognition and pose estimation from multiple views. Int. J. Comput. Vis. 100:1 (2012), 16–37.
66. Ye, M., Zhang, Q., Wang, L., Zhu, J., Yang, R., Gall, J., A survey on human motion analysis from depth data. Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, 2013, Springer, 149–187.
67. Young, G.S.J., Chellappa, R., 3D motion estimation using a sequence of noisy stereo images: models, estimation, and uniqueness results. IEEE Trans. Pattern Anal. Mach. Intell. 12:8 (1990), 735–759.
68. Yun, K., Honorio, J., Chattopadhyay, D., Berg, T.L., Samaras, D., Two-person interaction detection using body-pose features and multiple instance learning. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2012.
69. Zanfir, M., Leordeanu, M., Sminchisescu, C., The moving pose: an efficient 3D kinematics descriptor for low-latency action recognition and detection. IEEE International Conference on Computer Vision, 2013.
70. Zatsiorsky, V.M., Kinematics of Human Motion, 1997, Human Kinetics Publishers.
71. Zefran, M., Kumar, V., Two methods for interpolating rigid body motions. IEEE International Conference on Robotics and Automation, 1998.
72. Zefran, M., Kumar, V., Croke, C., Choice of Riemannian metrics for rigid body kinematics. ASME Design Engineering Technical Conference and Computers in Engineering Conference, 1996.
73. Zhang, C., Tian, Y., Histogram of 3D facets: A depth descriptor for human action and hand gesture recognition. Comput. Vis. Image Underst. 139 (2015), 29–39.
74. Zhu, Y., Chen, W., Guo, G., Fusing spatiotemporal features and joints for 3D action recognition. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2013.
75. Zhu, Y., Chen, W., Guo, G., Evaluating spatiotemporal interest point features for depth-based action recognition. Image Vis. Comput. 32:8 (2014), 453–464.