Accurate 3D action recognition using learning on the Grassmann manifold

Pattern Recognition

Volumn 48, Issue 2, 2015, Pages 556-567

  Slama, Rim ^a,b   Wannous, Hazem ^a,b   Daoudi, Mohamed ^b,c   Srivastava, Anuj ^d  

^a UNIV LILLE   (France)

^b ECOLE CENTRALE DE LILLE   (France)

^c INSTITUT MINES TELECOM   (France)

^d FLORIDA STATE UNIVERSITY   (United States)

Author keywords

Classification; Depth images; Grassmann manifold; Human action recognition; Observational latency; Skeleton

Indexed keywords

CLASSIFICATION (OF INFORMATION); DYNAMICAL SYSTEMS; MUSCULOSKELETAL SYSTEM; SUPPORT VECTOR MACHINES; VECTOR SPACES;

DEPTH IMAGE; GRASSMANN MANIFOLD; HUMAN-ACTION RECOGNITION; OBSERVATIONAL LATENCY; SKELETON;

LEARNING ALGORITHMS;

EID: 84911003814     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2014.08.011     Document Type: Article

References (58)

1. S. Fothergill, H. Mentis, P. Kohli, S. Nowozin, Instructing people for training gestural interactive systems, in: CHI Conference on Human Factors in Computing Systems, New York, NY, USA, 2012, pp. 1737-1746.
2. W. Lao, J. Han, P. de With, Automatic video-based human motion analyzer for consumer surveillance system, IEEE Trans. Consum. Electron. 55 (2009) 591-598.
3. A. Jalal, M. Uddin, T.S. Kim, Depth video-based human activity recognition system using translation and scaling invariant features for life logging at smart home, IEEE Trans. Consum. Electron. 58 (2012) 863-871.
4. R. Poppe, A survey on vision-based human action recognition, Image Vis. Comput. 28 (2010) 976-990.
5. P. Turaga, R. Chellappa, V. S. Subrahmanian, O. Udrea, Machine recognition of human activities: a survey, in: IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, Piscataway, NJ, USA, 2008, pp. 1473-1488.
6. J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, A. Blake, Real-time human pose recognition in parts from single depth images, Mach. Learn. Comput. Vis. 411 (2013) 119-135.
7. C.-S. Lee, A.M. Elgammal, Modeling view and posture manifolds for tracking, IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
8. Y.M. Lui, Advances in matrix manifolds for computer vision, Image Vis. Comput. 30 (2012) 380-388.
9. M.T. Harandi, C. Sanderson, S. Shirazi, B.C. Lovell, Kernel analysis on Grassmann manifolds for action recognition, Pattern Recognit. Lett. 34 (2013) 1906-1915.
10. M. Bregonzio, T. Xiang, S. Gong, Fusing appearance and distribution information of interest points for action recognition, Pattern Recognit. 45 (2012) 1220-1234.
11. S. OHara, Y.M. Lui, B.A. Draper, Using a product manifold distance for unsupervised action recognition, Image Vis. Comput. 30 (2012) 206-216.
12. J. Aggarwal, M. Ryoo, Human activity analysis: a review, ACM Comput. Surv. 43 (2011) 1-43.
13. D. Weinland, R. Ronfard, E. Boyer, A survey of vision-based methods for action representation, segmentation and recognition, Comput. Vis. Image Underst. 115 (2011) 224-241.
14. A. Veeraraghavan, A. Roy-Chowdhury, R. Chellappa, Matching shape sequences in video with applications in human movement analysis, IEEE Trans. Pattern Anal. Mach. Intell. 27 (2005) 1896-1909.
15. M.F. Abdelkader, W. Abd-Almageed, A. Srivastava, R. Chellappa, Silhouette-based gesture and action recognition via modeling trajectories on Riemannian shape manifolds, Comput. Vis. Image Underst. 115 (2011) 439-455.
16. D. Gong, G. Medioni, Dynamic manifold warping for view invariant action recognition, in: IEEE International Conference on Computer Vision, Barcelona, Spain, 2011, pp. 571-578.
17. D. Gong, G. Medioni, X. Zhao, Structured time series analysis for human action segmentation and recognition, IEEE Trans. Pattern Anal. Mach. Intell., vol. PP, 2014, pp. 1-1.
18. P. Turaga, A. Veeraraghavan, A. Srivastava, R. Chellappa, Statistical computations on Grassmann and Stiefel manifolds for image and video-based recognition, IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, 2011, pp. 2273-2286.
19. P. Turaga, R. Chellappa, Locally time-invariant models of human activities using trajectories on the Grassmannian, in: IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 2435-2441.
20. K. Guo, P. Ishwar, J. Konrad, Action recognition in video by sparse representation on covariance manifolds of silhouette tunnels, Recogn. Patterns Signals Speech Images Videos, 6388 (2010) 294-305.
21. Y.M. Lui, J.R. Beveridge, Tangent bundle for human action recognition, in: IEEE International Conference on Automatic Face and Gesture Recognition, 2011, pp. 97-102.
22. Y.M. Lui, Tangent bundles on special manifolds for action recognition, IEEE Trans. Circuits Syst. Video Technol. 22 (2012) 930-942.
23. S. Shirazi, M.T. Har, C.S.A. Alavi, B.C. Lovell, Clustering on Grassmann manifolds via kernel embedding with application to action analysis, in: International Conference on Image Processing, 2012, pp. 781-784.
24. M.T. Harandi, C. Sanderson, S. Shirazi, B.C. Lovell, Kernel analysis on Grassmann manifolds for action recognition, Pattern Recognit. Lett., 34 (2013) 1906-1915.
25. J. Gall, A. Yao, L. Van Gool, 2D action recognition serves 3D human pose estimation, in: European Conference on Computer Vision, vol. 6313, 2010, pp. 425-438.
26. L. Chen, H. Wei, J. Ferryman, A survey of human motion analysis using depth imagery, in: Pattern Recognit. Lett. 34 (2013) 1995-2006.
27. M. Ye, Q. Zhang, L. Wang, J. Zhu, R. Yang, J. Gall, A survey on human motion analysis from depth data, in: Time-of-Flight and Depth Imaging, Sensors, Algorithms, and Applications, vol. 8200, 2013, pp. 149-187.
28. W. Li, Z. Zhang, Z. Liu, Action recognition based on a bag of 3D points, in: IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2010, pp. 9-14.
29. B. Ni, G. Wang, P. Moulin, Rgbd-hudaact: a color-depth video database for human daily activity recognition, in: International Conference on Computer Vision Workshops, 2011, pp. 1147-1153.
30. L. Xia, J. Aggarwal, Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera, in: IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 2834-2841.
31. A.W. Vieira, E.R. Nascimento, G.L. Oliveira, Z. Liu, M.F. Campos, STOP: space-time occupancy patterns for 3D action recognition from depth map sequences, in: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, vol. 7441, 2012, pp. 252-259.
32. J. Wang, Z. Liu, J. Chorowski, Z. Chen, Y. Wu, Robust 3D action recognition with random occupancy patterns, in: European Conference on Computer Vision, 2012, pp. 872-885.
33. X. Yang, C. Zhang, Y. Tian, Recognizing actions using depth motion maps-based histograms of oriented gradients, in: International Conference on ACM Multimedia, New York, NY, USA, 2012, pp. 1057-1060.
34. O. Oreifej, Z. Liu, Hon4d: histogram of oriented 4d normals for activity recognition from depth sequences, in: IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, 2013, pp. 716-723.
35. L. Xia, C.-C. Chen, J.K. Aggarwal, View invariant human action recognition using histograms of 3D joints, in: Computer Vision and Pattern Recognition Workshops, 2012, pp. 20-27.
36. X. Yang, Y. Tian, Eigenjoints based action recognition using naive Bayes nearest neighbor, in: Computer Vision and Pattern Recognition Workshops, 2012, pp. 14-19.
37. T. Giorgino, Computing and visualizing dynamic time warping alignments in R: the dtw package, J. Stat. Softw. 31 (2009) 1-24.
38. M. Reyes, G. Dominguez, S. Escalera, Feature weighting in dynamic timewarping for gesture recognition in depth data, in: IEEE International Conference on Computer Vision Workshops, 2011, pp. 1182-1188.
39. S. Sempena, N. Maulidevi, P. Aryan, Human action recognition using dynamic time warping, in: International Conference on Electrical Engineering and Informatics, 2011, pp. 1-5.
40. M. Bautista, A. Hernandez-Vela, V. Ponce, X. Perez-Sala, X. Bar, O. Pujol, C. Angulo, S. Escalera, Probability-based dynamic time warping for gesture recognition on RGB-D data, Adv. Depth Image Anal. Appl. 7854 (2013) 126-135.
41. C. Ellis, S.Z. Masood, M.F. Tappen, J.J. Laviola, Jr., R. Sukthankar, Exploring the trade-off between accuracy and observational latency in action recognition, Int. J. Comput. Vis. 101 (2013) 420-436.
42. M. Barnachon, S. Bouakaz, B. Boufama, E. Guillou, Ongoing human action recognition with motion capture, Pattern Recognit. 47 (2014) 238-247.
43. M. Tenorth, J. Bandouch, M. Beetz, The TUM kitchen data set of everyday manipulation activities for motion tracking and action recognition, in: International Conference on Computer Vision Workshops, 2009, pp. 1089-1096.
44. S. Azary, A. Savakis, A spatiotemporal descriptor based on radial distances and 3D joint tracking for action classification, in: IEEE International Conference on Image Processing, 2012, pp. 769-772.
45. J. Wang, Z. Liu, Y. Wu, J. Yuan, Mining actionlet ensemble for action recognition with depth cameras, in: IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 1290-1297.
46. S. Althloothi, M.H. Mahoor, X. Zhang, R.M. Voyles, Human activity recognition using multi-features and multiple kernel learning, Pattern Recognit. 47 (2014) 1800-1812.
47. M. Devanne, H. Wannous, S. Berretti, P. Pala, M. Daoudi, A. Del Bimbo, Space-time pose representation for 3D human action recognition, in: Workshop on Social Behaviour Analysis ICIAP, vol. 8158, 2013, pp. 456-464.
48. S. Azary, A. Savakis, Grassmannian sparse representations and motion depth surfaces for 3D action recognition, in: IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2013, pp. 492-499.
49. X. Zhang, Y. Yang, L. Jiao, F. Dong, Manifold-constrained coding and sparse representation for human action recognition, Pattern Recognit. 46 (2013) 1819-1831.
50. R. Li, P. Turaga, A. Srivastava, R. Chellappa, Differential geometric representations and algorithms for some pattern recognition and computer vision problems, Pattern Recognit. Lett. 43 (2014) 3-16.
51. H. Wang, C. Yuan, G. Luo, W. Hu, C. Sun, Action recognition using linear dynamic systems, Pattern Recognit. 46 (2013) 1710-1718.
52. G. Doretto, A. Chiuso, Y.N. Wu, S. Soatto, Dynamic textures, Int. J. Comput. Vis. 51 (2003) 91-109.
53. A. Bissacco, A. Chiuso, Y. Ma, S. Soatto, Recognition of human gaits, in: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2001, pp. 52-57.
54. A. Edelman, T.A. Arias, S.T. Smith, The geometry of algorithms with orthogonality constraints, SIAM J. Matrix Anal. Appl. 20 (1998) 303-353.
55. A. Srivastava, E. Klassen, S. Joshi, I. Jermyn, Shape analysis of elastic curves in Euclidean spaces, IEEE Trans. Pattern Anal. Mach. Intell. 33 (2011) 1415-1428.
56. S. Kurtek, A. Srivastava, E. Klassen, Z. Ding, Statistical modeling of curves using shapes and related features, J. Am. Stat. Assoc. 107 (2012) 1152-1165.
57. C.-C. Chang, C.-J. Lin, LIBSVM: a library for support vector machines, ACM Trans. Intell. Syst. Technol. 2 (2011) 1-27.
58. L. Xia, C.-C. Chen, J.K. Aggarwal, View invariant human action recognition using histograms of 3D joints, in: IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2012, pp. 20-27.