Large-scale Continuous Gesture Recognition Using Convolutional Neural Networks

Proceedings - International Conference on Pattern Recognition

Volumn 0, Issue , 2016, Pages 13-18

  Wang, Pichao ^a   Li, Wanqing ^a   Liu, Song ^a   Zhang, Yuyao ^a   Gao, Zhimin ^a   Ogunbona, Philip ^a  

^a UNIVERSITY OF WOLLONGONG   (Australia)

Author keywords

Convolutional neural network; Depth map sequence; Depth motion map; Gesture recognition

Indexed keywords

CLASSIFICATION (OF INFORMATION); CONVOLUTION; IMAGE ENHANCEMENT; NEURAL NETWORKS; PATTERN RECOGNITION;

CONVNET; CONVOLUTIONAL NEURAL NETWORK; DEPTH MAP; FINE TUNING; JACCARD INDEX; TEMPORAL INFORMATION;

GESTURE RECOGNITION;

EID: 85019114823     PISSN: 10514651     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICPR.2016.7899600     Document Type: Conference Paper

References (40)

1. W. Li, Z. Zhang, and Z. Liu, "Action recognition based on a bag of 3D points, " in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2010, pp. 9-14.
2. J. Wang, Z. Liu, Y. Wu, and J. Yuan, "Mining actionlet ensemble for action recognition with depth cameras, " in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 1290-1297.
3. X. Yang, C. Zhang, and Y. Tian, "Recognizing actions using depth motion maps-based histograms of oriented gradients, " in Proc. ACM international conference on Multimedia (ACM MM), 2012, pp. 1057-1060.
4. O. Oreifej and Z. Liu, "HON4D: Histogram of oriented 4D normals for activity recognition from depth sequences, " in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 716-723.
5. X. Yang and Y. Tian, "Super normal vector for activity recognition using depth sequences, " in Proc. IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 804-811.
6. P. Wang, W. Li, P. Ogunbona, Z. Gao, and H. Zhang, "Mining mid-level features for action recognition based on effective skeleton representation, " in Proc. International Conference on Digital Image Computing: Techniques and Applications (DICTA), 2014, pp. 1-8.
7. C. Lu, J. Jia, and C.-K. Tang, "Range-sample depth feature for action recognition, " in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 772-779.
8. R. Vemulapalli and R. Chellappa, "Rolling rotations for recognizing human actions from 3d skeletal data, " in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1-9.
9. J. Zhang, W. Li, P. O. Ogunbona, P. Wang, and C. Tang, "Rgb-d-based action recognition datasets: A survey, " Pattern Recognition, vol. 60, pp. 86-105, 2016.
10. P. Wang, W. Li, Z. Gao, C. Tang, J. Zhang, and P. O. Ogunbona, "Convnets-based action recognition from depth maps through virtual cameras and pseudocoloring, " in Proc. ACM international conference on Multimedia (ACM MM), 2015, pp. 1119-1122.
11. P. Wang, W. Li, Z. Gao, J. Zhang, C. Tang, and P. Ogunbona, "Action recognition from depth maps using deep convolutional neural networks, " Human-Machine Systems, IEEE Transactions on, vol. 46, no. 4, pp. 498-509, 2016.
12. P. Wang, Z. Li, Y. Hou, and W. Li, "Action recognition based on joint trajectory maps using convolutional neural networks, " in Proc. ACM international conference on Multimedia (ACM MM), 2016, pp. 1-5.
13. P. Wang, W. Li, S. Liu, Z. Gao, C. Tang, and P. Ogunbona, "Largescale isolated gesture recognition using convolutional neural networks, " in Proceedings of ICPRW, 2016.
14. Y. Hou, Z. Li, P. Wang, and W. Li, "Skeleton optical spectra based action recognition using convolutional neural networks, " in Circuits and Systems for Video Technology, IEEE Transactions on, 2016, pp. 1-5.
15. Y. Du, W. Wang, and L. Wang, "Hierarchical recurrent neural network for skeleton based action recognition, " in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1110-1118.
16. V. Veeriah, N. Zhuang, and G.-J. Qi, "Differential recurrent neural networks for action recognition, " in Proc. IEEE International Conference on Computer Vision (ICCV), 2015, pp. 4041-4049.
17. W. Zhu, C. Lan, J. Xing, W. Zeng, Y. Li, L. Shen, and X. Xie, "Cooccurrence feature learning for skeleton based action recognition using regularized deep LSTM networks, " in The 30th AAAI Conference on Artificial Intelligence (AAAI), 2016.
18. A. Shahroudy, J. Liu, T.-T. Ng, and G. Wang, "NTU RGB+ D: A large scale dataset for 3D human activity analysis, " in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
19. F. Jiang, S. Zhang, S. Wu, Y. Gao, and D. Zhao, "Multi-layered gesture recognition with kinect, " Journal of Machine Learning Research, vol. 16, no. 2, pp. 227-254, 2015.
20. J. Wan, Q. Ruan, W. Li, and S. Deng, "One-shot learning gesture recognition from rgb-d data using bag of features, " Journal of Machine Learning Research, vol. 14, pp. 2549-2582, 2013.
21. J. Wan, V. Athitsos, P. Jangyodsuk, H. J. Escalante, Q. Ruan, and I. Guyon, "Csmmi: Class-specific maximization of mutual information for action and gesture recognition, " IEEE Transactions on Image Processing, vol. 23, no. 7, pp. 3152-3165, July 2014.
22. H. J. Escalante, I. Guyon, V. Athitsos, P. Jangyodsuk, and J. Wan, "Principal motion components for one-shot gesture recognition, " Pattern Analysis and Applications, pp. 1-16, 2015.
23. G. D. Forney, "The viterbi algorithm, " Proceedings of the IEEE, vol. 61, no. 3, pp. 268-278, March 1973.
24. Y. M. Lui, "Human gesture recognition on product manifolds, " Journal of Machine Learning Research, vol. 13, no. 1, pp. 3297-3321, Nov 2012.
25. D. Wu, F. Zhu, and L. Shao, "One shot learning gesture recognition from rgbd images, " in 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, June 2012, pp. 7-12.
26. W. Li, Z. Zhang, and Z. Liu, "Expandable data-driven graphical modeling of human actions based on salient postures, " Circuits and Systems for Video Technology, IEEE Transactions on, vol. 18, no. 11, pp. 1499-1510, 2008.
27. J. Yue-Hei Ng, M. Hausknecht, S. Vijayanarasimhan, O. Vinyals, R. Monga, and G. Toderici, "Beyond short snippets: Deep networks for video classification, " in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 4694-4702.
28. K. Simonyan and A. Zisserman, "Two-stream convolutional networks for action recognition in videos, " in Proc. Annual Conference on Neural Information Processing Systems (NIPS), 2014, pp. 568-576.
29. S. Ji, W. Xu, M. Yang, and K. Yu, "3D convolutional neural networks for human action recognition, " Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 35, no. 1, pp. 221-231, 2013.
30. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, "Learning spatiotemporal features with 3D convolutional networks, " in Proc. IEEE International Conference on Computer Vision (ICCV), 2015, pp. 4489-4497.
31. J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, "Long-term recurrent convolutional networks for visual recognition and description, " in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 2625-2634.
32. H. Bilen, B. Fernando, E. Gavves, A. Vedaldi, and S. Gould, "Dynamic image networks for action recognition, " in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
33. B. R. Abidi, Y. Zheng, A. V. Gribok, and M. A. Abidi, "Improving weapon detection in single energy X-ray images through pseudocoloring, " Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, vol. 36, no. 6, pp. 784-796, 2006.
34. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks, " in Proc. Annual Conference on Neural Information Processing Systems (NIPS), 2012, pp. 1106-1114.
35. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. B. Girshick, S. Guadarrama, and T. Darrell, "Caffe: Convolutional architecture for fast feature embedding. " in Proc. ACM international conference on Multimedia (ACM MM), 2014, pp. 675-678.
36. H. J. Escalante, V. Ponce-Lpez, J. Wan, M. A. Riegler, B. Chen, A. Claps, S. Escalera, I. Guyon, X. Bar, P. Halvorsen, H. Mller, and M. Larson, "Chalearn joint contest on multimedia challenges beyond visual analysis: An overview, " in Proceedings of ICPRW, 2016.
37. I. Guyon, V. Athitsos, P. Jangyodsuk, and H. J. Escalante, "The chalearn gesture dataset (CGD 2011), " Machine Vision and Applications, vol. 25, no. 8, pp. 1929-1951, 2014.
38. J. Wan, G. Guo, and S. Z. Li, "Explore efficient local features from rgbd data for one-shot learning gesture recognition, " IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 8, pp. 1626-1639, Aug 2016.
39. X. Chai, Z. Liu, F. Yin, Z. Liu, and X. Chen, "Two streams recurrent neural networks for large-scale continuous gesture recognition, " in Proceedings of ICPRW, 2016.
40. N. C. Camgoz, S. Hadfield, O. Koller, and R. Bowden, "Using convolutional 3d neural networks for user-independent continuous gesture recognition, " in Proceedings of ICPRW, 2016.