A survey on model based approaches for 2D and 3D visual human pose recovery

Sensors (Switzerland)

Volumn 14, Issue 3, 2014, Pages 4189-4210

  Perez Sala, Xavier ^a   Escalera, Sergio ^b   Angulo, Cecilio ^a   Gonzàlez, Jordi ^c  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

^b UNIVERSITY OF BARCELONA   (Spain)

^c UNIVERSITAT AUTÒNOMA DE BARCELONA   (Spain)

Author keywords

Behavior analysis; Computer vision; Human body modelling; Human pose recovery

Indexed keywords

COMPUTER VISION; TAXONOMIES;

BEHAVIOR ANALYSIS; DATA REPRESENTATIONS; GROUP MODELING; HUMAN BODY MODELLING; HUMAN POSE; METHODOLOGICAL COMPARISON; MODEL BASED APPROACH; SPATIAL RELATIONS;

RECOVERY;

EID: 84896841632     PISSN: 14248220     EISSN: None     Source Type: Journal    
DOI: 10.3390/s140304189     Document Type: Review

References (90)

1. Moeslund, T.; Hilton, A.; Krüger, V. A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 2006, 104, 90-126.
2. Marr, D.; Vaina, L. Representation and recognition of the movements of shapes. Proc. R. Soc. Lond. Ser. B. Biol. Sci. 1982, 214, 501-524.
3. Eichner, M.; Marin-Jimenez, M.; Zisserman, A.; Ferrari, V. Articulated Human Pose Estimation and Search in (Almost) Unconstrained Still Images; Technical Report No. 272; ETH Zurich: Zurich, Switzerland, September 2010.
4. Gowsikhaa, D.; Abirami, S.; Baskaran, R. Automated human behavior analysis from surveillance videos: A survey. Artif. Intell. Rev. 2012, doi:10.1007/s10462-012-9341-3.
5. Dollár, P.; Wojek, C.; Schiele, B.; Perona, P. Pedestrian detection: An evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 2011, 34, 743-761.
6. Singh, V.; Nevatia, R. Action recognition in cluttered dynamic scenes using Pose-Specific Part Models. In Proceedings of the 2011 IEEE International Conference on Computer Vision (ICCV), Barcelona, Brazil, 6-13 November 2011; pp. 113-120.
7. Seemann, E.; Nickel, K.; Stiefelhagen, R. Head pose estimation using stereo vision for human-robot interaction. In Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, Seoul, Korea, 17-19 May 2004; pp. 626-631.
8. Nickel, K.; Stiefelhagen, R. Visual recognition of pointing gestures for human-robot interaction. Image Vis. Comput. 2007, 25, 1875-1884.
9. Escalera, S. Human Behavior Analysis from Depth Maps. In Articulated Motion and Deformable Objects; Springer: Berlin, Germany, 2012; pp. 282-292.
10. Andriluka, M.; Roth, S.; Schiele, B. Monocular 3D pose estimation and tracking by detection. In Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, 13-18 June 2010; pp. 623-630.
11. Agarwal, A.; Triggs, B. Recovering 3D human pose from monocular images. IEEE Trans. Pattern Anal. Mach. Intell. 2006, 28, 44-58.
12. Rogez, G.; Orrite, C.; Martínez-del Rincón, J. A spatio-temporal 2D-models framework for human pose recovery in monocular sequences. Pattern Recognit. 2008, 41, 2926-2944.
13. Enzweiler, M.; Gavrila, D. Monocular pedestrian detection: Survey and experiments. IEEE Trans. Pattern Anal. Mach. Intell. 2009, 31, 2179-2195.
14. Gerónimo, D.; López, A.; Sappa, A. Computer vision approaches to pedestrian detection: Visible spectrum survey. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 4477, 547-554.
15. Ramanan, D. Part-based models for finding people and estimating their pose. In Visual Analysis of Humans; Springer: Berlin, Germany, 2011; pp. 199-223.
16. Poppe, R. Vision-based human motion analysis: An overview. Comput. Vis. Image Underst. 2007, 108, 4-18.
17. Perez-Sala, X.; Escalera, S.; Angulo, C. Survey on spatio-temporal view invariant human pose recovery. In Proceedings of the 15th International Conference of the Catalan Association of Artificial Intelligence (CCIA2012), Catalonia, Spain, 24-26 October 2012.
18. Gavrila, D. The visual analysis of human movement: A survey. Comput. Vis. Image Underst. 1999, 73, 82-98.
19. Shotton, J.; Fitzgibbon, A.; Cook, M.; Sharp, T.; Finocchio, M. Real-time human pose recognition in rarts from single depth images. Mach. Learn. Comput. Vis. Stud. Comput. Intell. 2011, 411, 119-135.
20. Hernández, A.; Reyes, M.; Escalera, S.; Radeva, P. Spatio-Temporal GrabCut human segmentation for face and pose recovery. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern RecognitionWorkshops (CVPRW), San Francisco, CA, USA, 13-18 June 2010; pp. 33-40.
21. Hernández-Vela, A.; Zlateva, N.; Marinov, A.; Reyes, M.; Radeva, P.; Dimov, D.; Escalera, S. Graph cuts optimization for multi-limb human segmentation in depth maps. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, 16-21 June 2012.
22. Ramanan, D. Learning to parse images of articulated bodies. In Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 4-7 December 2006; Volume 19, p. 1129.
23. Andriluka, M.; Roth, S.; Schiele, B. Pictorial structures revisited: People detection and articulated pose estimation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern, Miami, FL, USA, 20-25 June 2009; pp. 1014-1021.
24. Wang, Y.; Tran, D.; Liao, Z. Learning hierarchical poselets for human parsing. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 20-25 June 2011; pp. 1705-1712.
25. Pirsiavash, H.; Ramanan, D. Steerable part models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 16-21 June 2012.
26. Dalal, N.; Triggs, B. Histograms of oriented gradients for human detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 20-26 June 2005; Volume 1, pp. 886-893.
27. Bourdev, L.D.; Malik, J. Poselets: Body part detectors trained using 3D human pose annotations. In Proceedings of the IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 27 September-4 October, 2009; pp. 1365-1372.
28. Mittal, A.; Zhao, L.; Davis, L. Human body pose estimation using silhouette shape analysis. In Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, Miami, FL, USA, 21-22 July 2003; pp. 263-270.
29. Sande, K.; Gevers, T.; Snoek, C. Evaluating color descriptors for object and scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2010, 32, 1582-1596.
30. Navarathna, R.; Sridharan, S.; Lucey, S. Fourier active appearance models. In Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain, 6-13 November 2011; pp. 1919-1926.
31. Daugman, J.; others. Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. J. Opt. Soc. Am. A 1985, 2, 1160-1169.
32. Pugeault, N.; Bowden, R. Spelling it out: Real-time ASL fingerspelling recognition. In Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain, 6-13 November 2011.
33. Plagemann, C.; Ganapathi, V.; Koller, D.; Thrun, S. Real-time identification and localization of body parts from depth images. In Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain, 6-13 November 2011; pp. 3108-3113.
34. Barron, J.; Fleet, D.; Beauchemin, S. Performance of optical flow techniques. Int. J. Comput. Vis. 1994, 12, 43-77.
35. Laptev, I.; Marszalek, M.; Schmid, C.; Rozenfeld, B. Learning realistic human actions from movies. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008), Anchorage, AK, USA, 24-26 June 2008; pp. 1-8.
36. Chakraborty, B.; Holte, M.; Moeslund, T.; Gonzàlez, J. Selective spatio-temporal interest points. Comput. Vis. Image Underst. 2012, 116, 396-410.
37. Laptev, I. On space-time interest points. Int. J. Comput. Vis. 2005, 64, 107-123.
38. Yao, B.; Li, F.-F. Grouplet: A structured image representation for recognizing human and object interactions. In Proceedings of the the 23rd IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13-18 June 2010.
39. Felzenszwalb, P.; Girshick, R.;McAllester, D.; Ramanan, D. Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 2010, 32, 1627-1645.
40. Rother, C.; Kolmogorov, V.; Blake, A. GrabCut: Interactive Foreground Extraction Using Iterated Graph Cuts. ACM Trans. Graph. 2004, 23, 309-314.
41. Mikolajczyk, K.; Tuytelaars, T.; Schmid, C.; Zisserman, A.; Matas, J.; Schaffalitzky, F.; Kadir, T.; Gool, L.V. A comparison of affine region detectors. Int. J. Comput. Vis. 2005, 65, 43-72.
42. Karaulova, I.; Hall, P.; Marshall, A. A hierarchical model of dynamics for tracking people with a single video camera. In Proceedings of the British Machine Vision Conference 2000, Bristol, UK, 11-14 September 2000; Volume 1, pp. 352-361.
43. Savarese, S.; Li, F.-F. 3D generic object categorization, localization and pose estimation. In Proceedings of the IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Brazil, 14-20 October 2007; pp. 1-8.
44. Sun, M.; Su, H.; Savarese, S.; Li, F.-F. A multi-view probabilistic model for 3D object classes. In Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20-25 June 2009; pp. 1247-1254.
45. Su, H.; Sun, M.; Li, F.-F.; Savarese, S. Learning a dense multi-view representation for detection, viewpoint classification and synthesis of object categories. In Proceedings of the IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 27 September-4 October 2009; pp. 213-220.
46. Moreno-Noguer, F.; Lepetit, V.; Fua, P. Pose priors for simultaneously solving alignment and correspondence. In Proceedings of the 10th European Conference on Computer Vision, Marseille, France, 12-18 October 2008; pp. 405-418.
47. Salzmann, M.; Moreno-Noguer, F.; Lepetit, V.; Fua, P. Closed-form solution to non-rigid 3D surface registration. In Proceedings of the 10th European Conference on Computer Vision, Marseille, France, 12-18 October 2008; pp. 581-594.
48. Simo-Serra, E.; Ramisa, A.; Alenya, G.; Torras, C.; Moreno-Noguer, F. Single Image 3D Human Pose Estimation from Noisy Observations. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 16-21 June 2012.
49. Sánchez-Riera, J.; Ostlund, J.; Fua, P.; Moreno-Noguer, F. Simultaneous pose, correspondence and non-rigid shape. In Proceedings of the 23rd IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13-18 June 2010; pp. 1189-1196.
50. Eichner, M.; Marin-Jimenez, M.; Zisserman, A.; Ferrari, V. 2d articulated human pose estimation and retrieval in (almost) unconstrained still images. Int. J. Comput. Vis. 2012, 99, 190-214.
51. Sapp, B.; Weiss, D.; Taskar, B. Parsing human motion with stretchable models. In Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, 20-25 June 2011; pp. 1281-1288.
52. Ferrari, V.; Eichner, M.; Marin-Jimenez, M.; Zisserman, A. Buffy Stickmen Dataset. Available online: http://www.robots.ox.ac.uk/-vgg/data/stickmen/ (accessed on 23th July 2012).
53. Fischler, M.; Elschlager, R. The representation andmatching of pictorial structures. Comput. Trans. 1973, 100, 67-92.
54. Felzenszwalb, P.; Huttenlocher, D. Pictorial structures for object recognition. Int. J. Comput. Vis. 2005, 61, 55-79.
55. Sigal, L.; Bhatia, S.; Roth, S.; Black, M.; Isard, M. Tracking loose-limbed people. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, 27 June-2 July 2004; Volume 1, pp. 421-428.
56. Yang, Y.; Ramanan, D. Articulated pose estimation with flexible mixtures-of-parts. In Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, 20-25 June 2011; pp. 1385-1392.
57. Sminchisescu, C.; Triggs, B. Kinematic jump processes for monocular 3D human tracking. In Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI, USA, 16-22 June 2003; Volume 1, pp. 1-69.
58. Felzenszwalb, P.; McAllester, D. Object Detection Grammars; Technical Report: Computer Science TR; University of Chicago: Chicago, IL, USA, 2010.
59. Girshick, R.; Felzenszwalb, P.; McAllester, D. Object detection with grammar models. IEEE Trans. Pattern Anal. Mach. Intell. 2011, 33, 6.
60. Pedersoli, M.; Gonzàlez, J.; Hu, X.; Roca, F. Toward real-time pedestrian detection based on a deformable template model. Trans. Intell. Transp. Syst. 2013, 15, 355-364.
61. Sigal, L.; Isard, M.; Haussecker, H.; Black, M. Loose-limbed people: Estimating 3d human pose and motion using non-parametric belief propagation. Int. J. Comput. Vis. 2012, 98, 15-48.
62. Zhu, L.; Chen, Y.; Lu, Y.; Lin, C.; Yuille, A. Max margin and/or graph learning for parsing the human body. In Proceedings of the 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA, 24-26 June 2008; pp. 1-8.
63. Chen, Y.; Zhu, L.; Lin, C.; Yuille, A.; Zhang, H. Rapid inference on a novel and/or graph for object detection, segmentation and parsing. NIPS 2007, 20, 289-296.
64. Lan, X.; Huttenlocher, D. Beyond trees: Common-factor models for 2d human pose recovery. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV 2005), Beijing, China, 17-20 October 2005; Volume 1, pp. 470-477.
65. Singh, V.; Nevatia, R.; Huang, C. Efficient inference with multiple heterogeneous part detectors for human pose estimation. ECCV 2010, 6313, 314-327.
66. Agarwal, A.; Triggs, B. Tracking articulated motion with piecewise learned dynamical models. In Proceedings of the 8th European Conference on Computer Vision, Prague, Czech Republic, 11-14 May 2004; Volume 3, pp. 54-65.
67. Wei, X.; Chai, J. Modeling 3d human poses from uncalibrated monocular images. In Proceedings of the IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 27 September-4 October 2009; pp. 1873-1880.
68. Valmadre, J.; Lucey, S. Deterministic 3D human pose estimation using rigid structure. In Proceedings of the 11th European Conference on Computer Vision, Heraklion, Crete, Greece, 5-11 September 2010; pp. 467-480.
69. Bregler, C.; Malik, J.; Pullen, K. Twist based acquisition and tracking of animal and human kinematics. Int. J. Comput. Vis. 2004, 56, 179-194.
70. Howe, N.; Leventon, M.; Freeman, W. Bayesian Reconstruction of 3D Human Motion from Single-Camera Video; NIPS: Cambridge, MA, USA, 1999; Volume 1999, p. 1.
71. Gall, J.; Stoll, C.; de Aguiar, E.; Theobalt, C.; Rosenhahn, B.; Seidel, H. Motion capture using joint skeleton tracking and surface estimation. In Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami, FL, USA, 20-25 June 2009; pp. 1746-1753.
72. Rius, I.; Gonzàlez, J.; Varona, J.; Roca, F. Action-specific motion prior for efficient bayesian 3D human body tracking. Pattern Recogn. 2009, 42, 2907-2921.
73. Cootes, T.; Taylor, C.; Cooper, D.; Graham, J.; others. Active shape models-their training and application. Comput. Vis. Image Underst. 1995, 61, 38-59.
74. Cootes, T.; Edwards, G.; Taylor, C. Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 2001, 23, 681-685.
75. Murphy-Chutorian, E.; Trivedi, M. Head Pose Estimation in Computer Vision: A Survey. IEEE Trans. Pattern Anal. Mach. Intell. 2009, 31, 607-626.
76. Kim, D.; Paik, J. Gait recognition using active shape model and motion prediction. Comput. Vis. IET 2010, 4, 25-36.
77. Urtasun, R.; Fleet, D.; Fua, P. Temporal motion models for monocular and multiview 3D human body tracking. Comput. Vis. Image Underst. 2006, 104, 157-177.
78. Akhter, I.; Sheikh, Y.; Khan, S.; Kanade, T. Trajectory space: A dual representation for nonrigid structure from motion. IEEE Trans. Pattern Anal. Mach. Intell. 2011, 33, 1442-1456.
79. Moreno-Noguer, F.; Porta, J. Probabilistic simultaneous pose and non-rigid shape recovery. In Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, 20-25 June 2011; pp. 1289-1296.
80. Urtasun, R.; Fua, P. 3D human body tracking using deterministic temporal motion models. In Proceedings of the 8th European Conference on Computer Vision, Prague, Czech Republic, 11-14 May 2004; pp. 92-106.
81. Urtasun, R.; Fleet, D.; Fua, P. Monocular 3D tracking of the golf swing. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), San Diego, CA, USA, 20-26 June 2005; Volume 2, pp. 932-938.
82. Urtasun, R.; Fleet, D.; Hertzmann, A.; Fua, P. Priors for people tracking from small training sets. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV 2005), Beijing, China, 17-20 October 2005; Volume 1, pp. 403-410.
83. Fossati, A.; Salzmann, M.; Fua, P. Observable subspaces for 3D human motion recovery. In Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami, FL, USA, 20-25 June 2009; pp. 1137-1144.
84. Akhter, I.; Sheikh, Y.; Khan, S.; Kanade, T. Nonrigid structure from motion in trajectory space. In Proceedings of the 22nd Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 8-11 December 2008; pp. 41-48.
85. Park, H.; Shiratori, T.; Matthews, I.; Sheikh, Y. 3D Reconstruction of a Moving Point from a Series of 2D Projections. In Proceedings of the 11th European Conference on Computer Vision, Heraklion, Crete, Greece, 5-11 September 2010; pp. 158-171.
86. Park, H.; Sheikh, Y. 3D reconstruction of a smooth articulated trajectory from a monocular image sequence. In Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain, 6-13 November 2011; pp. 201-208.
87. Shapovalova, N.; Fernández, C.; Roca, F.; Gonzàlez, J. Semantics of Human Behavior in Image Sequences. Computer Analysis of Human Behavior; Springer: Berlin, Germany, 2011; pp. 151-182.
88. Sigal, L.; Black, M. Humaneva: Synchronized Video and Motion Capture Dataset for Evaluation of Articulated Human Motion; Technical Report; Brown Univertsity: Providence, RI, USA, 2006.
89. Yao, B.; Fei-Fei, L. Modeling mutual context of object and human pose in human-object interaction activities. In Proceedings of the 23rd IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13-18 June 2010; pp. 17-24.
90. Andriluka, M.; Sigal, L. Human Context: Modeling human-human interactions for monocular 3D pose estimation. Articul. Motion Deform. Objects 2012, 7378, 260-272.