3-D Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian Manifold

IEEE Transactions on Cybernetics

Volumn 45, Issue 7, 2015, Pages 1340-1352

  Devanne, Maxime ^a,b   Wannous, Hazem ^a   Berretti, Stefano ^b   Pala, Pietro ^b   Daoudi, Mohamed ^a   Del Bimbo, Alberto ^b  

^a ECOLE DES MINES DE DOUAI   (France)

^b UNIVERSITY OF FLORENCE   (Italy)

Author keywords

3 D human action; activity recognition; Riemannian shape space; temporal modeling

Indexed keywords

GEOMETRY; GESTURE RECOGNITION; HOSIERY MANUFACTURE; IMAGE RECOGNITION; NEAREST NEIGHBOR SEARCH; SECURITY SYSTEMS; TRAJECTORIES;

ACTIVITY RECOGNITION; COMPACT REPRESENTATION; DESIGN AND DEVELOPMENT; HUMAN ACTIONS; HUMAN-ACTION RECOGNITION; SHAPE SPACE; STATE-OF-THE-ART METHODS; TEMPORAL MODELING;

MOTION ESTIMATION;

ACTIMETRY; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; HUMAN; MACHINE LEARNING; MOVEMENT (PHYSIOLOGY); PHOTOGRAPHY; PHYSIOLOGY; PROCEDURES; THREE DIMENSIONAL IMAGING; VIDEORECORDING; WHOLE BODY IMAGING;

ACTIGRAPHY; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; MACHINE LEARNING; MOVEMENT; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAPHY; VIDEO RECORDING; WHOLE BODY IMAGING;

EID: 84933048626     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCYB.2014.2350774     Document Type: Article

References (41)

1. (2013). Microsoft Kinect [Online]. Available: http://www.microsoft.com/ en-us/kinectforwindows/
2. (2013). ASUS Xtion PRO LIVE [Online]. Available: http://www.asus.com/Multimedia/Xtion-PRO/
3. J. Shotton et al., "Real-time human pose recognition in parts from single depth images," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Colorado Springs, CO, USA, Jun. 2011, pp. 1-8.
4. S. Hadfield and R. Bowden, "Kinecting the dots: Particle based scene flow from depth sensors," in Proc. Int. Conf. Comput. Vis., Barcelona, Spain, Nov. 2011, pp. 2290-2295.
5. Z. Ren, J. Yuan, and Z. Zhang, "Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera," in Proc. ACM Int. Conf. Multimedia, Scottsdale, AR, USA, Nov. 2011, pp. 1093-1096.
6. S. Berretti, A. Del Bimbo, and P. Pala, "Superfaces: A super-resolution model for 3D faces," in Proc. Workshop Non-Rigid Shape Anal. Deformable Image Alignment, Florence, Italy, Oct. 2012, pp. 73-82.
7. J. Han, L. Shao, D. Xu, and J. Shotton, "Enhanced computer vision with Microsoft Kinect sensor: A review," IEEE Trans. Cybern., vol. 43, no. 5, pp. 1318-1334, Oct. 2013.
8. L. Liu and L. Shao, "Learning discriminative representations from RGB-D video data," in Proc. 23rd Int. Joint Conf. Artif. Intell. (IJCAI), 2013, pp. 1493-1500.
9. L. Xia, C.-C. Chen, and J. K. Aggarwal, "View invariant human action recognition using histograms of 3D joints," in Proc. Workshop Human Activity Underst. 3D Data, Providence, RI, USA, Jun. 2012, pp. 20-27.
10. X. Yang and Y. Tian, "EigenJoints-based action recognition using naive- Bayes-nearest-neighbor," in Proc. Workshop Human Activity Underst. 3D Data, Providence, RI, USA, Jun. 2012, pp. 14-19.
11. C. Ellis, S. Z. Masood, M. F. Tappen, J. J. La Viola, Jr., and R. Sukthankar, "Exploring the trade-off between accuracy and observational latency in action recognition," Int. J. Comput. Vis., vol. 101, no. 3, pp. 420-436, 2013.
12. W. Li, Z. Zhang, and Z. Liu, "Action recognition based on a bag of 3D points," in Proc. Workshop Human Commun. Behav. Anal., San Francisco, CA, USA, Jun. 2010, pp. 9-14.
13. X. Yang, C. Zhang, and Y. Tian, "Recognizing actions using depth motion maps-based histograms of oriented gradients," in Proc. ACM Int. Conf. Multimedia, Nara, Japan, Oct. 2012, pp. 1057-1060.
14. A. W. Vieira, E. R. Nascimento, G. L. Oliveira, Z. Liu, and M. F. Campos, "STOP: Space-time occupancy patterns for 3D action recognition from depth map sequences," in Proc. Iberoamerican Congr. Pattern Recognit., Buenos Airies, Argentina, Sep. 2012, pp. 252-259.
15. J. Wang, Z. Liu, J. Chorowski, Z. Chen, and Y. Wu, "Robust 3D action recognition with random occupancy patterns," in Proc. Eur. Conf. Comput. Vis., Florence, Italy, Oct. 2012, pp. 1-8.
16. L. Xia and J. K. Aggarwal, "Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera," in Proc. CVPR Workshop Human Activity Underst. 3D Data, Portland, OR, USA, Jun. 2013, pp. 2834-2841.
17. O. Oreifej and Z. Liu, "HON4D: Histogram of oriented 4D normals for activity recognition from depth sequences," in Proc. Int. Conf. Comput. Vision Pattern Recognit., Portland, OR, USA, Jun. 2013, pp. 716-723.
18. B. Ni, Y. Pei, P. Moulin, and S. Yan, "Multi-level depth and image fusion for human activity detection," IEEE Trans. Cybern., vol. 43, no. 5, pp. 1383-1394, Oct. 2013.
19. J. Wang, Z. Liu, Y. Wu, and J. Yuan, "Mining actionlet ensemble for action recognition with depth cameras," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Providence, RI, USA, Jun. 2012, pp. 1-8.
20. E. Ohn-Bar and M. M. Trivedi, "Joint angles similarities and HOG2 for action recognition," in Proc. Comput. Vision Pattern Recognit. Workshop (CVPRW) Human Activity Underst. 3D Data, Portland, OR, USA, Jun. 2013, pp. 465-470.
21. D. Weinland, R. Ronfard, and E. Boyer, "A survey of vision-based methods for action representation, segmentation, and recognition," Comput. Vis. Image Underst., vol. 115, no. 2, pp. 224-241, Feb. 2011.
22. P. Turaga, R. Chellappa, V. S. Subrahmanian, and O. Udrea, "Machine recognition of human activities: A survey," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 11, pp. 1473-1488, Nov. 2008.
23. W. Bian, D. Tao, and Y. Rui, "Cross-domain human action recognition," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 2, pp. 298-307, Apr. 2012.
24. R. Poppe, "A survey on vision-based human action recognition," Image Vis. Comput., vol. 28, no. 6, pp. 976-990, Jun. 2010.
25. L. Liu, L. Shao, X. Zhen, and X. Li, "Learning discriminative key poses for action recognition," IEEE Trans. Cybern., vol. 43, no. 6, pp. 1860-1870, Dec. 2013.
26. Y. Tian, L. Cao, Z. Liu, and Z. Zhang, "Hierarchical filtered motion for action recognition in crowded videos," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 42, no. 3, pp. 313-323, May 2012.
27. L. Shao, X. Zhen, D. Tao, and X. Li, "Spatio-temporal Laplacian pyramid coding for action recognition," IEEE Trans. Cybern., vol. 44, no. 6, pp. 817-827, Jun. 2014.
28. A. Veeraraghavan, A. Roy-Chowdhury, and R. Chellappa, "Matching shape sequences in video with applications in human movement analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 12, pp. 1896-1909, Dec. 2005.
29. M. T. Harandi, C. Sanderson, A. Wiliem, and B. C. Lovell, "Kernel analysis over Riemannian manifolds for visual recognition of actions, pedestrians and textures," in Proc. IEEE Workshop Appl. Comput. Vis., Breckenridge, CO, USA, 2012, pp. 433-439.
30. Y. M. Lui, "Tangent bundles on special manifolds for action recognition," IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 6, pp. 930-942, Jun. 2012.
31. S. Shirazi, M. T. Har, C. Sanderson, A. Alavi, and B. C. Lovell, "Clustering on Grassmann manifolds via kernel embedding with application to action analysis," in Proc. Int. Conf. Image Process., Orlando, FL, USA, 2012, pp. 781-784.
32. P. Turaga, A. Veeraraghavan, A. Srivastava, and R. Chellappa, "Statistical computations on Grassmann and Stiefel manifolds for image and video-based recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 11, pp. 2273-2286, Nov. 2011.
33. M. F. Abdelkader, W. Abd-Almageed, A. Srivastava, and R. Chellappa, "Silhouette-based gesture and action recognition via modeling trajectories on Riemannian shape manifolds," Comput. Vis. Image Underst., vol. 115, no. 3, pp. 439-455, 2011.
34. S. H. Joshi, E. Klassen, A. Srivastava, and I. Jermyn, "A novel representation for Riemannian analysis of elastic curves in Rn," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Minneapolis, MN, USA, Jun. 2007, pp. 1-7.
35. H. Drira, B. B. Amor, A. Srivastava, M. Daoudi, and R. Slama, "3D face recognition under expressions, occlusions, and pose variations," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 9, pp. 2270-2283, Sep. 2013.
36. R. Slama, H. Wannous, and M. Daoudi, "3D human motion analysis framework for shape similarity and retrieval," Image Vis. Comput., vol. 32, no. 2, pp. 131-154, 2014.
37. A. Srivastava, E. Klassen, S. H. Joshi, and I. Jermyn, "Shape analysis of elastic curves in Euclidean spaces," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 7, pp. 1415-1428, Jul. 2011.
38. H. Karcher, "Riemannian center of mass and mollifier smoothing," Commun. Pure Appl. Math., vol. 30, no. 5, pp. 509-541, 1977.
39. L. Seidenari, V. Varano, S. Berretti, A. Del Bimbo, and P. Pala, "Recognizing actions from depth cameras as weakly aligned multipart bag-of-poses," in Proc. Comput. Vis. Pattern Recognit. Workshop (CVPRW) Human Activity Underst. 3D Data, Portland, OR, USA, Jun. 2013, pp. 479-485.
40. I. Borg and P. Groenen, Modern Multidimensional Scaling: Theory and Applications, 2nd ed. New York, NY, USA: Springer-Verlag, 2005.
41. M. Devanne et al., "Space-time pose representation for 3D human action recognition," in Proc. Int. Workshop Soc. Behav. Anal. (SBA) Conjunction ICIAP, Naples, Italy, Sep. 2013, pp. 456-464.