A survey of human motion analysis using depth imagery

Pattern Recognition Letters

Volumn 34, Issue 15, 2013, Pages 1995-2006

  Chen, Lulu ^a   Wei, Hong ^a   Ferryman, James ^a  

^a UNIVERSITY OF READING   (United Kingdom)

Author keywords

3D body model; Depth sensor; Human action recognition; Human pose estimation; Range data; Survey

Indexed keywords

GESTURE RECOGNITION; IMAGE ANALYSIS; IMAGE RECOGNITION; MOTION ANALYSIS; SURVEYING; SURVEYS;

BODY MODELING; DEPTH SENSORS; HUMAN POSE ESTIMATIONS; HUMAN-ACTION RECOGNITION; RANGE DATA;

BEHAVIORAL RESEARCH;

EID: 84885063479     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2013.02.006     Document Type: Article

References (78)

1. Aggarwal, J., Ryoo, M., 2011. Human activity analysis: A review. ACM Comput. Surv. 43, 16:1-16:43.
2. Ahad, M.A.R., Tan, J.K., Kim, H., Ishikawa, S., 2012. Motion history image: Its variants and applications. Machine Vision Appl. 23, 255-281.
3. Allen, J., 1983. Maintaining knowledge about temporal intervals. Comm. ACM 26, 832-843.
4. Anguelov, D., Taskarf, B., Chatalbashev, V., Koller, D., Gupta, D., Heitz, G., Ng, A., 2005. Discriminative learning of Markov random fields for segmentation of 3D scan data. In: IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, CVPR 2005, pp. 169-176.
5. Batlle, J., Mouaddib, E., Salvi, J., 1998. Recent progress in coded structured light as a technique to solve the correspondence problem: A survey. Pattern Recognit. 31, 963-982. (Pubitemid 128428106)
6. Benko, H., Wilson, A.D., 2009. Depthtouch: Using depth-sensing camera to enable freehand interactions on and above the interactive surface. In: IEEE Workshop on Tabletops and Interactive Surfaces.
7. Besl, P., McKay, H., 1992. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal Machine Intell. 14, 239-256.
8. Bobick, A., Davis, J., 2001. The recognition of human movement using temporal templates. IEEE Trans. Pattern Anal Machine Intell. 23, 257-267. (Pubitemid 32286632)
9. Breuer, P., Eckes, C., Müller, S., 2007. Hand gesture recognition with a novel ir timeof- flight range camera: A pilot study. In: Proc. of the Third Internat. Conf. on Computer Vision/Computer Graphics Collaboration Techniques. Springer- Verlag, Berlin, Heidelberg, pp. 247-260. (Pubitemid 350259244)
10. Charles, J., Everingham, M., 2011. Learning shape models for monocular human pose estimation from the Microsoft Xbox Kinect. In: 2011 IEEE Internat. Conf. on Computer Vision Workshops (ICCVW), pp. 1202-1208.
11. Chen, C.S., Hung, Y.P., Chiang, C.C., Wu, J.L., 1997. Range data acquisition using color structured lighting and stereo vision. Image Vision Comput. 15, 445-456.
12. Chen, L., Wei, H., Ferryman, J., 2011. Recognition of everyday domestic activities using a depth sensor. In: BMVC 2011 Student, Workshop, pp. 27-37.
13. Demirdjian, D., Ko, T., Darrell, T., 2003. Constraining human body tracking. In: Proc. of Ninth IEEE Internat. Conf. on Computer Vision, vol. 2, pp. 1071-1078.
14. Dollár, P., Rabaud, V., Cottrell, G., Belongie, S., 2005. Behavior recognition via sparse spatio-temporal features. In: Visual Surveillance and Performance Evaluation of Tracking and Surveillance (VS-PETS) 0, 65-72. (Pubitemid 46178025)
15. Ganapathi, V., Plagemann, C., Koller, D., Thrun, S., 2010. Real time motion capture using a single time-of-flight camera. In: 2010 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 755-762.
16. Girshick, R., Shotton, J., Kohli, P., Criminisi, A., Fitzgibbon, A., 2011. Efficient regression of general-activity human poses from depth images. In: 2011 IEEE Internat. Conf. on Computer Vision (ICCV), pp. 415-422.
17. Grammalidis, N., Goussis, G., Troufakos, G., Strintzis, M., 2001. 3-D human body tracking from depth images using analysis by synthesis. In: Proc. of 2001 Internat. Conf. on Image Processing, vol. 2, pp. 185-188. (Pubitemid 33057671)
18. Grest, D., Woetzel, J., Koch, R., 2005. Nonlinear body pose estimation from depth images. In: Proc. of the 27th DAGM Conf. on Pattern Recognition. Springer- Verlag, Berlin, Heidelberg, pp. 285-292. (Pubitemid 41520666)
19. Guomundsson, S., Larsen, R., Aanaes, H., Pardas, M., Casas, J., 2008. TOF imaging in smart room environments towards improved people tracking. In: IEEE Comput. Society Conf. Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1-6.
20. Hartley, R.I., Zisserman, A., 2004. Multiple View Geometry in Computer Vision, Cambridge University Press, ISBN 0521540518.
21. Holte, M.B., Moeslund, T.B., 2007. Gesture recognition using a range camera. Technical Report, pp. 1-5.
22. Holt, B., Ong, E.J., Cooper, H., Bowden, R., 2011. Putting the pieces together: Connected poselets for human pose estimation. In: 2011 IEEE Internat. Conf. on Computer Vision Workshops (ICCV Workshops), pp. 1196-1201.
23. Hu, G., Stockman, G., 1989. 3-D surface solution using structured light and constraint propagation. IEEE Trans. Pattern Anal Machine Intell. 11, 390-402.
24. Iddan, G.J., Yahav, G., 2001. 3D imaging in the studio. In: SPIE, vol. 4298, pp. 48-55.
25. Jansen, B., Temmermans, F., Deklerck, R., 2007. 3D human pose recognition for home monitoring of elderly. In: Engineering in Medicine and Biology Society, 29th Annual Internat. Conf. of the IEEE, EMBS 2007, pp. 4049-4051.
26. Ji, X., Liu, H., 2010. Advances in view-invariant human motion analysis: A review. IEEE Trans. Systems Man Cybernet. 40, 13-24.
27. Johansson, G., 1973. Visual perception of biological motion and a model for its analysis. Attention Perception Psychophys. 14, 201-211.
28. Kalogerakis, E., Hertzmann, A., Singh, K., 2010. Learning 3D mesh segmentation and labeling. In: ACM SIGGRAPH 2010, ACM, New York, NY, 775 USA, pp. 102:1-102:12.
29. Knoop, S., Vacek, S., Dillmann, R., 2009. Fusion of 2D and 3D sensor data for articulated body tracking. Robot. Autonom. Systems 57, 321-329.
30. Kolb, A., Barth, E., Koch, R., 2008. Tof-sensors: New dimensions for realism and interactivity. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1518-1523.
31. Kollorz, E., Penne, J., Hornegger, J., Barke, A., 2008. Gesture recognition with a timeof- flight camera. Internat. J. Intell. Systems Technol. Appl. 5, 334.
32. Kurakin, A., Zhang, Z., Liu, Z., 2012. A real time system for dynamic hand gesture recognition with a depth sensor. In: Proc. of the 20th European Signal Processing Conf. (EUSIPCO), pp. 1975-1979.
33. Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B., 2008. Learning realistic human actions from movies. In: IEEE Conf. Computer Vision and Pattern Recognition, CVPR 2008, pp. 1-8.
34. Li, W., Zhang, Z., Liu, Z., 2010. Action recognition based on a bag of 3D points. In: 2010 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 9-14.
35. Lui, Y.M., 2012. A least squares regression framework on manifolds and its application to gesture recognition. In: 2012 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 13-18.
36. Malgireddy, M., Inwogu, I., Govindaraju, V., 2012. A temporal bayesian model for classifying, detecting and localizing activities in video sequences. In: 2012 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 43-48.
37. Marszalek, M., Laptev, I., Schmid, C., 2009. Actions in context. In: IEEE Conf. on Computer Vision and Pattern Recognition, CVPR 2009, pp. 2929-2936.
38. Moeslund, T., Hilton, A., Krüger, V., 2006. A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding 104, 90-126. (Pubitemid 44573873)
39. Ni, B., Wang, G., Moulin, P., 2011. RGBD-HuDaAct: A color-depth video database for human daily activity recognition. In: 2011 IEEE Internat. Conf. on Computer Vision Workshops (ICCV Workshops), pp. 1147-1153.
40. Oggier, T., Büttgen, B., Lustenberger, F., Becker, G., Rüegg, B., Hodac, A., 2005 Swissranger SR3000 and first experiences based on mniaturized 3D-ToF cameras. In: Proc. of the First Range Imaging Research Day, ETH Zurich.
41. Pellegrini, S., Iocchi, L., 2008. Human posture tracking and classification through stereo vision and 3D model matching. J. Image Video Process. 2008, 7:1-7:12.
42. Phillips, P., Flynn, P., Scruggs, T., Bowyer, K., Chang, J., Hoffman, K., Marques, J., Min, J., Worek, W., 2005. Overview of the face recognition grand challenge. In: IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 947-954. (Pubitemid 43897294)
43. Plagemann, C., Ganapathi, V., Koller, D., Thrun, S., 2010. Real-time identification and localization of body parts from depth images. In: IEEE Internat. Conf. on Robotics and Automation (ICRA).
44. Poppe, R., 2010. A survey on vision-based human action recognition. Image Vision Comput. 28, 976-990.
45. Reyes, M., Dominguez, G., Escalera, S., 2011. Featureweighting in dynamic timewarping for gesture recognition in depth data. In: 2011 IEEE Internat. Conference on Computer Vision Workshops (ICCVW), pp. 1182-1188.
46. Rodriguez, M., Ahmed, J., Shah, M., 2008. Action MACH a spatio-temporal maximum average correlation height filter for action recognition. In: IEEE Conf. on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1-8.
47. Roh, M.C., Shin, H.K., Lee, S.W., 2010. View-independent human action recognition with volume motion template on single stereo camera. Pattern Recognition Lett. 31, 639-647.
48. Rusu, R., Cousins, S., 2011. 3D is here: point cloud library (PCL). In: 2011 IEEE Internat. Conf. on Robotics and Automation (ICRA), pp. 1-4.
49. Scharstein, D., Szeliski, R., 2003. High-accuracy stereo depth maps using structured light. In: Proc. 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. I-195- I-202.
50. Schuldt, C., Laptev, I., Caputo, B., 2004. Recognizing human actions: A local svm approach. In: Proc. of the 17th Internat. Conf. on Pattern Recognition, ICPR 2004, vol. 3, pp. 32-36.
51. Schwarz, L.A., Mateus, D., Castaneda, V., Nava, N., 2010. Manifold learning for tofbased human body tracking and activity recognition. In: British Machine Vision Conf. (BMVC).
52. Schwarz, L.A., Mkhitaryan, A., Mateus, D., Navab, N., 2011. Estimating human 3D pose from time-of-flight images based on geodesic distances and optical flow. In: FG, pp. 700-706.
53. Schwarz, L.A., Mateus, D., Navab, N., 2012a. Recognizing multiple human activities and tracking full-body pose in unconstrained environments. Pattern Recognit. 45, 11-23.
54. Schwarz, L.A., Mkhitaryan, A., Mateus, D., Navab, N., 2012b. Human skeleton tracking from depth data using geodesic distances and optical flow. Image Vision Comput. 30, 217-226.
55. Sempena, S., Maulidevi, N., Aryan, P., 2011. Human action recognition using dynamic time warping. In: 2011 Internat. Conf. on Electrical Engineering and Informatics (ICEEI), pp. 1-5.
56. Shirai, Y., Suwa, M., 1971. Recognition of polyhedrons with a range finder. In: Proc. of the Second Internat. Joint Conf. on Artificial intelligence, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, pp. 80-87.
57. Shotton, J., Girshick, R., Fitzgibbon, A., Sharp, T., Cook, M., Finocchio, M., Moore, R., Kohli, P., Criminisi, A., Kipman, A., Blake, A., 2012. Efficient human pose estimation from single depth images. IEEE Trans. Pattern Anal. Machine Intell., 1.
58. Siddiqui, M., Medioni, G., 2010. Human pose estimation from a single view point, real-time range sensor. In: 2010 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1-8.
59. Singh, S., Velastin, S.A., Ragheb, H., 2010. Muhavi: A multicamera human action video dataset for the evaluation of action recognition methods. In: Proceedings of the 2010 7th IEEE Internat. Conf. on Advanced Video and Signal Based Surveillance, IEEE Computer Society, Washington, DC, USA, pp. 48-55.
60. Sung, J., Ponce, C., Selman, B., Saxena, A., 2012. Unstructured human activity detection from RGBD images. In: 2012 IEEE Internat. Conf. on Robotics and Automation.
61. Suryanarayan, P., Subramanian, A., Mandalapu, D., 2010. Dynamic hand pose recognition using depth data. In: Proc. of the 2010 20th Internat. Conf. on Pattern Recognition, IEEE Computer Society, Washington, DC, USA, pp. 3105-3108.
62. Trucco, E., Verri, A., 1998. Introductory Techniques for 3-D Computer Vision. Prentice Hall PTR, Upper Saddle River, NJ, USA.
63. Van den Bergh, M., Van Gool, L., 2011. Combining RGB and ToF cameras for realtime 3D hand gesture interaction. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV), pp. 66-72.
64. Vuylsteke, P., Oosterlinck, A., 1990. Range image acquisition with a single binaryencoded light pattern. IEEE Trans. Pattern Anal. Machine Intell. 12, 148-164. (Pubitemid 20666140)
65. Wang, J., Liu, Z., Chorowski, J., Chen, Z., Wu, Y., 2012a. Robust 3D action recognition with random occupancy patterns. In: Computer Vision - ECCV 2012, pp. 872-885.
66. Wang, J., Liu, Z., Wu, Y., Yuan, J., 2012b. Mining actionlet ensemble for action recognition with depth cameras. In: 2012 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 1290-1297.
67. Weinland, D., Ronfard, R., Boyer, E., 2010. A survey of vision-based methods for action representation, segmentation and recognition. Computer Vision and Image Understanding 115, 224-241.
68. Werghi, N., Xiao, Y., 2002. Recognition of human body posture from a cloud of 3D data points using wavelet transform coefficients. In: Proc. of the Fifth IEEE Internat. Conf. on Automatic Face and Gesture Recognition, pp. 70-75.
69. Wigdor, D., Wixon, D., 2011. Brave NUI World: Designing Natural User Interfaces for Touch and Gesture. Morgan Kaufmann.
70. Will, P.M., Pennington, K.S., 1971. Grid coding: A preprocessing technique for robot and machine vision. In: Proceedings of the Second Internat. Joint Conf. on Artificial Intelligence, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, pp. 66-70.
71. Wolf, C., Mille, J., Lombardi, L., Celiktutan, O., Jiu, M., Baccouche, M., Dellandrea, E., Bichot, C.E., Garcia, C., Sankur, B., 2012. The LIRIS Human activities dataset and the ICPR 2012 human activities recognition and localization competition. Technical Report, RR-LIRIS-2012-004, LIRIS Laboratory.
72. Wu, D., Zhu, F., Shao, L., 2012. One shot learning gesture recognition from RGBD images. In: 2012 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 7-12.
73. Xia, L., Chen, C.C., Aggarwal, J., 2012. View invariant human action recognition using histograms of 3D joints. In: IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 20-27.
74. Xu, Z., Schwarte, R., Heinol, H., Buxbaum, B., Ringbeck, T., Nachrichtenverarbeitung, I., Gmbh, S.t., Strae, K., 1998. Smart pixel n photonic mixer device (PMD) new system concept of a 3D-imaging camera-on-a-chip. In: Proc. Internat. Conf. on Mechatron Machine Vision, pp. 259-264.
75. Zhang, H., Parker, L.E., 2011. 4-Dimensional local spatio-temporal features for human activity recognition. In: 2011 IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems (IROS), pp. 2044-2049.
76. Zhu, Y., Fujimura, K., 2007. Constrained optimization for human pose estimation from depth sequences. In: Proc. of the Eighth Asian Conf. on Computer Vision, Part I, Springer-Verlag, Berlin, Heidelberg, pp. 408-418.
77. Zhu, Y., Fujimura, K., 2010. A bayesian framework for human body pose tracking from depth image sequences. Sensors 10, 5280-5293.
78. Zhu, Y., Dariush, B., Fujimura, K., 2008. Controlled human pose estimation from depth image streams. In: IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops, CVPRW 2008, pp. 1-8.