TriViews: A general framework to use 3D depth data effectively for action recognition

Journal of Visual Communication and Image Representation

Volumn 26, Issue , 2015, Pages 182-191

  Chen, Wenbin ^a   Guo, Guodong ^a  

^a West Virginia University   (United States)

Author keywords

3D depth data; Action recognition; Fusion; Kinect; PFA; Public databases; RGB D sensor; TriViews framework

Indexed keywords

FUSION REACTIONS; GRAPHIC METHODS; MUSCULOSKELETAL SYSTEM; TRAJECTORIES;

3D DEPTH DATA; ACTION RECOGNITION; KINECT; PFA; PUBLIC DATABASE; RGB-D SENSORS; TRIVIEWS FRAMEWORK;

SENSOR DATA FUSION;

EID: 84919914603     PISSN: 10473203     EISSN: 10959076     Source Type: Journal    
DOI: 10.1016/j.jvcir.2014.11.008     Document Type: Article

References (51)

1. P. Turaga, R. Chellappa, V. Subrahmanian, and O. Udrea Machine recognition of human activities: a survey IEEE Trans. Circ. Syst. Video Technol. 18 11 2008 1473 1488
2. R. Poppe A survey on vision-based human action recognition Image Vis. Comput. 28 6 2010 976 990
3. J. Aggarwal, and M. Ryoo Human activity analysis: a review ACM Comput. Surv. (CSUR) 43 3 2011 16:1 16:43
4. D. Weinland, R. Ronfard, and E. Boyer A survey of vision-based methods for action representation, segmentation and recognition Comput. Vis. Image Underst. 115 2 2011 224 241
5. J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, A. Blake, Real-time human pose recognition in parts from single depth images, in: IEEE Conf. on Computer Vision and Pattern Recognition, 2011, pp. 1-8.
6. A. Shabani, D. Clausi, J. Zelek, Evaluation of local spatio-temporal salient feature detectors for human action recognition, in: 9th Conf. on Computer and Robot Vision (CRV), 2012, pp. 468-475.
7. H. Wang, M. Ullah, A. Kläser, I. Laptev, C. Schmid, et al.; Evaluation of local spatio-temporal features for action recognition, in: BMVC 2009-British Machine Vision Conference, 2009, pp. 124.1-124.11.
8. Y. Zhu, W. Chen, G.-D. Guo, Fusing spatiotemporal features and joints for 3D action recognition, in: IEEE Computer Vision and Pattern Recognition Workshops (CVPRW), 2013, pp. 486-491.
9. Y. Zhao, Z. Liu, L. Yang, H. Cheng, Combing RGB and depth map features for human activity recognition, in: Signal Information Processing Association Annual Summit and Conference (APSIPA ASC), Asia-Pacific, 2012, pp. 1-4.
10. B. Ni, G. Wang, and P. Moulin RGBD-HuDaAct: a color-depth video database for human daily activity recognition Computer Vision Workshops (ICCV Workshops) 2011 IEEE 1147 1153
11. H. Zhang, L. Parker, 4-Dimensional local spatio-temporal features for human activity recognition, in: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2011, pp. 2044-2049.
12. P. Dollár, V. Rabaud, G. Cottrell, S. Belongie, Behavior recognition via sparse spatio-temporal features, in: 2nd Joint IEEE Int'l Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2005, pp. 65-72.
13. L. Xia, J. Aggarwal, Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera, in: IEEE Conf. on Computer Vision and Pattern Recognition, 2013, pp. 2834-2841.
14. Y. Zhu, W. Chen, and G.-D. Guo Evaluating spatiotemporal interest point features for depth-based action recognition Image Vis. Comput. 2014
15. R. Messing, C. Pal, and H. Kautz Activity recognition using the velocity histories of tracked keypoints 12th International Conference on Computer Vision 2009 IEEE 104 111
16. B. Lucas, T. Kanade, et al.; An iterative image registration technique with an application to stereo vision, in: Proceedings of the 1981 DARPA Image Understanding Workshop, 1981, pp. 121-130.
17. J. Sun, X. Wu, S. Yan, L.-F. Cheong, T.-S. Chua, J. Li, Hierarchical spatio-temporal context modeling for action recognition, in: IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 2004-2011.
18. M. Raptis, and S. Soatto Tracklet descriptors for action modeling and video analysis Computer Vision - ECCV 2010 Springer 577 590
19. H. Wang, A. Klaser, C. Schmid, C.-L. Liu, Action recognition by dense trajectories, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 3169-3176.
20. Y.-G. Jiang, Q. Dai, X. Xue, W. Liu, and C.-W. Ngo Trajectory-based modeling of human actions with motion reference points Computer Vision - ECCV 2012 Springer 425 438
21. W. Li, Z. Zhang, Z. Liu, Action recognition based on a bag of 3D points, in: IEEE Computer Vision and Pattern Recognition Workshops (CVPRW), 2010, pp. 9-14.
22. A. Vieira, E. Nascimento, G. Oliveira, Z. Liu, and M. Campos Stop: space-time occupancy patterns for 3D action recognition from depth map sequences Prog. Pattern Recogn. Image Anal. Comput. Vis. Appl. 2012 252 259
23. J. Wang, Z. Liu, J. Chorowski, Z. Chen, Y. Wu, Robust 3D action recognition with random occupancy patterns, in: Computer Vision - ECCV, 2012, pp. 872-885.
24. X. Yang, C. Zhang, and Y. Tian Recognizing actions using depth motion maps-based histograms of oriented gradients Proceedings of the 20th ACM International Conference on Multimedia 2012 ACM 1057 1060
25. O. Oreifej, Z. Liu, HON4D: histogram of oriented 4D normals for activity recognition from depth sequences, 2013, pp. 716-723.
26. L. Xia, C. Chen, J. Aggarwal, View invariant human action recognition using histograms of 3D joints, in: IEEE Computer Vision and Pattern Recognition Workshops (CVPRW), 2012, pp. 20-27.
27. L. Miranda, T. Vieira, D. Martinez, T. Lewiner, A.W. Vieira, M.F.M. Campos, Real-time gesture recognition from depth data through key poses learning and decision forests, in: 25th SIBGRAPI Conference on Graphics, Patterns and Images, vol. 0, 2012, pp. 268-275.
28. I. Theodorakopoulos, D. Kastaniotis, G. Economou, and S. Fotopoulos Pose-based human action recognition via sparse representation in dissimilarity space J. Vis. Commun. Image Represent. 25 1 2013 12 23
29. E. Pekalska, Dissimilarity Representations in Pattern Recognition. Concepts, Theory and Applications, Ph. D. Thesis, Delft University of Technology, Delft, The Netherlands, 2005.
30. X. Yang, Y. Tian, Eigenjoints-based action recognition using Naive-Bayes-nearest-neighbor, in: IEEE Computer Vision and Pattern Recognition Workshops (CVPRW), 2012, pp. 14-19.
31. M. Devanne, H. Wannous, S. Berretti, P. Pala, M. Daoudi, and A. Del Bimbo Space-time pose representation for 3D human action recognition ICIAP Workshop on Social Behaviour Analysis 2013 Springer Berlin Heidelberg 456 464
32. S. Joshi, E. Klassen, A. Srivastava, I. Jermyn, A novel representation for riemannian analysis of elastic curves in rn, in: IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1-7.
33. J. Wang, Z. Liu, Y. Wu, J. Yuan, Mining actionlet ensemble for action recognition with depth cameras, in: IEEE Conf. on Computer Vision and Pattern Recognition, 2012, pp. 1290-1297.
34. J. Sung, C. Ponce, B. Selman, A. Saxena, Unstructured human activity detection from RGBD images, in: IEEE Int'l Conf. on Robotics and Automation (ICRA), 2012, pp. 842-849.
35. L. Breiman Random forests Mach. Learn. 45 1 2001 5 32
36. G.-D. Guo, Y. Fu, C. Dyer, T. Huang, A probabilistic fusion approach to human age prediction, in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2008, pp. 1-6.
37. I. Laptev, T. Lindeberg, Velocity adaptation of space-time interest points, in: Proc. of the 17th Int'l Conf. on Pattern Recognition, vol. 1, 2004, pp. 52-56.
38. C. Harris, M. Stephens, A combined corner and edge detector, in: Alvey Vision Conference, vol. 15, Manchester, UK, 1988, pp. 147-151.
39. G. Willems, T. Tuytelaars, L. Van Gool, An efficient dense and scale-invariant spatio-temporal interest point detector, in: European Conf. on Computer Vision, 2008, pp. 650-663.
40. I. Laptev, M. Marszalek, C. Schmid, B. Rozenfeld, Learning realistic human actions from movies, in: IEEE Conf. on Computer Vision and Pattern Recognition, 2008, pp. 1-8.
41. A. Klaser, M. Marszałek, and C. Schmid A spatio-temporal descriptor based on 3D-gradients British Machine Vision Conference 2008 BMVA Press 99.1 99.10
42. G. Farnebäck Two-frame motion estimation based on polynomial expansion Image Analysis 2003 Springer 363 370
43. X. Yang, and Y. Tian Effective 3D action recognition using eigenjoints J. Vis. Commun. Image Represent. 25 1 2014 2 11
44. S. Azary, A. Savakis, Grassmannian sparse representations and motion depth surfaces for 3D action recognition, in: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2013, pp. 492-499.
45. H. Soh, and Y. Demiris Iterative temporal learning and prediction with the sparse online echo state gaussian process International Joint Conference on Neural Networks (IJCNN) 2012 IEEE 1 8
46. C. Wang, Y. Wang, A. Yuille, An approach to pose-based action recognition, in: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 915-922.
47. A. Chaaraoui, J. Padilla-López, P. Climent-Pérez, and F. Flórez-Revuelta Evolutionary joint selection to improve human action recognition with RGB-D devices Expert Syst. Appl. 41 3 2014 786 794
48. E. Ohn-Bar, M. Trivedi, Joint angles similarities and HOG2 for action recognition, 2013.
49. L. Seidenari, V. Varano, S. Berretti, P. Pala, A. Del Bimbo, Recognizing actions from depth cameras as weakly aligned multi-part bag-of-poses, 2013.
50. L. Liu, and L. Shao Learning discriminative representations from RGB-D video data Proc. Int. Joint Conf. on Artificial Intelligence (IJCAI) 2013 AAAI Press 1493 1500
51. B. Ni, P. Moulin, Manipulation pattern discovery: a nonparametric bayesian approach, in: IEEE Int'l Conf. on Computer Vision, 2013, pp. 1361-1368.