Joint Distance Maps Based Action Recognition with Convolutional Neural Networks

IEEE Signal Processing Letters

Volumn 24, Issue 5, 2017, Pages 624-628

  Li, Chuankun ^a   Hou, Yonghong ^a   Wang, Pichao ^b   Li, Wanqing ^b  

^a TIANJIN UNIVERSITY   (China)

^b UNIVERSITY OF WOLLONGONG   (Australia)

Author keywords

Action recognition; convolutional neural networks (ConvNets); joint distance maps (JDM)

Indexed keywords

MUSCULOSKELETAL SYSTEM; NEURAL NETWORKS;

ACTION RECOGNITION; COLOR TEXTURE IMAGES; COLOR VARIATIONS; CONVOLUTIONAL NEURAL NETWORK; DISCRIMINATIVE FEATURES; DISTANCE MAP; SPATIOTEMPORAL INFORMATION; TEMPORAL INFORMATION;

CONVOLUTION;

EID: 85018500124     PISSN: 10709908     EISSN: None     Source Type: Journal    
DOI: 10.1109/LSP.2017.2678539     Document Type: Article

References (28)

1. W. Li, Z. Zhang, and Z. Liu, "Action recognition based on a bag of 3D points," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Workshops, 2010, pp. 9-14.
2. P. Wang, W. Li, Z. Gao, Y. Zhang, C. Tang, and P. Ogunbona, "Scene flow to action map: A new representation for RGB-D based action recognition with convolutional neural networks," in Proc. Comput. Vis. Pattern Recognit., 2017, pp. 1-10.
3. J. Shotton et al., "Real-time human pose recognition in parts from single depth images," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2011, pp. 1297-1304.
4. X. Yang and Y. L. Tian, "Eigenjoints-based action recognition using naivebayes- nearest-neighbor," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Workshops, 2012, pp. 14-19.
5. J. Wang, Z. Liu, Y. Wu, and J. Yuan, "Mining actionlet ensemble for action recognition with depth cameras," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2012, pp. 1290-1297.
6. L. Xia, C.-C. Chen, and J. Aggarwal, "View invariant human action recognition using histograms of 3D joints," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, 2012, pp. 20-27.
7. M. Zanfir, M. Leordeanu, and C. Sminchisescu, "The moving pose: An efficient 3D kinematics descriptor for low-latency action recognition and detection," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 2752-2759.
8. Z. Shao and Y. Li, "A new descriptor for multiple 3D motion trajectories recognition," in Proc. IEEE Int. Conf. Robot. Autom., 2013, pp. 4749-4754.
9. M. A. Gowayyed, M. Torki, M. E. Hussein, and M. El-Saban, "Histogram of oriented displacements (HOD): Describing trajectories of human joints for action recognition," in Proc. Int. Joint Conf. Artif. Intell., 2013, pp. 1351-1357.
10. R. Chaudhry, F. Ofli, G. Kurillo, R. Bajcsy, and R. Vidal, "Bio-inspired dynamic 3D discriminative skeletal features for human action recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshops, 2013, pp. 471-478.
11. Y. Zhu, W. Chen, and G. Guo, "Fusing spatiotemporal features and joints for 3D action recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, 2013, pp. 486-491.
12. R. Vemulapalli, F. Arrate, and R. Chellappa, "Human action recognition by representing 3D skeletons as points in a lie group," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 588-595.
13. E. Ohn-Bar and M. Trivedi, "Joint angles similarities and HOG2 for action recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, 2013, pp. 465-470.
14. P. Wang, W. Li, P. Ogunbona, Z. Gao, and H. Zhang, "Mining mid-level features for action recognition based on effective skeleton representation," in Proc. IEEE Int. Conf. Digital lmage Comput., Tech. Appl., 2014, pp. 1-8.
15. Y. Du, W. Wang, and L. Wang, "Hierarchical recurrent neural network for skeleton based action recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 1110-1118.
16. V. Veeriah, N. Zhuang, and G.-J. Qi, "Differential recurrent neural networks for action recognition," in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 4041-4049.
17. W. Zhu et al., "Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks," in Proc. AAAI Conf. Artif. Intell., 2016, pp. 3697-3704.
18. A. Shahroudy, J. Liu, T.-T. Ng, and G. Wang, "NTU RGB+ D: A large scale dataset for 3Dhuman activity analysis," in Proc. IEEE Conf.Comput. Vis. Pattern Recog., 2016, pp. 1010-1019.
19. J. Liu, A. Shahroudy, D. Xu, and G. Wang, "Spatio-temporal LSTM with trust gates for 3D human action recognition," in Proc. Eur. Conf. Comput. Vis., 2016, pp. 816-833.
20. Y. Du, Y. Fu, and L. Wang, "Skeleton based action recognition with convolutional neural network," in Proc. Asian Conf. Pattern Recognit., 2016, pp. 579-583.
21. P. Wang, Z. Li, Y. Hou, and W. Li, "Action recognition based on joint trajectory maps using convolutional neural networks," in Proc. ACM Multimedia Conf., 2016, pp. 102-106.
22. C. Chen, R. Jafari, and N. Kehtarnavaz, "UTD-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor," in Proc. IEEE Int. Conf. Image Process., 2015, pp. 168-172.
23. P. Wang, W. Li, Z. Gao, C. Tang, J. Zhang, and P. O. Ogunbona, "ConvNets-based action recognition from depth maps through virtual cameras and pseudocoloring," in Proc. ACM Multimedia Conf., 2015, pp. 1119-1122.
24. P. Wang, W. Li, Z. Gao, J. Zhang, C. Tang, and P. Ogunbona, "Action recognition from depth maps using deep convolutional neural networks," IEEE Trans. Human-Mach. Syst., vol. 46, no. 4, pp. 498-509, Aug. 2016.
25. Y. Hou, Z. Li, P. Wang, and W. Li, "Skeleton optical spectra based action recognition using convolutional neural networks," IEEE Trans. Circuits Syst. Video Technol., to be published.
26. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Adv. Neural Inform. Process. Syst., 2012, pp. 1097-1105.
27. L. Zhou, W. Li, Y. Zhang, P. Ogunbona, D. T. Nguyen, and H. Zhang, "Discriminative key pose extraction using extended LC-KSVD for action recognition," in Proc. IEEE Int. Conf. Digital lmage Comput., Tech. Appl., 2014, pp. 1-8.
28. M. E. Hussein, M. Torki, M. A. Gowayyed, and M. El-Saban, "Human action recognition using a temporal hierarchy of covariance descriptors on 3D joint locations." in Proc. Int. Joint Conf. Artif. Intell., 2013, pp. 2466-2472.