Loose-limbed people: Estimating 3D human pose and motion using non-parametric belief propagation

International Journal of Computer Vision

Volumn 98, Issue 1, 2012, Pages 15-48

  Sigal, Leonid ^a   Isard, Michael ^b   Haussecker, Horst ^c   Black, Michael J ^d  

^a DISNEY RESEARCH   (United States)

^b MICROSOFT RESEARCH   (United States)

^c INTEL CORPORATION   (United States)

^d MAX PLANCK INSTITUTE FOR INTELLIGENT SYSTEMS   (Germany)

Author keywords

Articulated pose estimation; Articulated tracking; Human motion tracking; Human pose estimation; Non parametric belief propagation

Indexed keywords

ARTICULATED TRACKING; HUMAN MOTION TRACKING; HUMAN POSE ESTIMATION; NON-PARAMETRIC BELIEF PROPAGATION; POSE ESTIMATION;

GESTURE RECOGNITION; GRAPHIC METHODS; MOTION ESTIMATION; THREE DIMENSIONAL;

TARGET TRACKING;

EID: 84859270367     PISSN: 09205691     EISSN: 15731405     Source Type: Journal    
DOI: 10.1007/s11263-011-0493-4     Document Type: Article

References (103)

1. A. Agarwal B. Triggs 2006 Recovering 3D human pose from monocular images IEEE Transactions on Pattern Analysis and Machine Intelligence 28 1 44 58 10.1109/TPAMI.2006.21 (Pubitemid 46395303)
2. Andriluka, M., Roth, S., & Schiele, B. (2009). Pictorial structures revisited: people detection and articulated pose estimation. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).
3. Balan, A., Sigal, L., & Black, M. J. (2005). A quantitative evaluation of video-based 3D person tracking. In IEEE workshop on visual surveillance and performance evaluation of tracking and surveillance (pp. 349-356). October 2005. (Pubitemid 46170648)
4. A. Banerjee I. S. Dhillon J. Ghosh S. Sra 2005 Clustering on the unit hypersphere using von Mises-Fisher distributions Journal of Machine Learning Research 6 1345 1382 2249858 1190.62116
5. M. Bergtholdt J. Kappes S. Schmidt C. Schnorr 2010 A study of parts-based object class detection using complete graphs International Journal of Computer Vision 87 1-2 93 117 2718743 10.1007/s11263-009-0209-1
6. Bhatia, S., Sigal, L., Isard, M., & Black, M. J. (2004). 3D human limb detection using space carving and multi-view eigen models. In IEEE Workshop on articulated and nonrigid motion, CVPR'04 CDROM proceedings.
7. Bregler, C., & Malik, J. (1998). Tracking people with twists and exponential maps. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (pp. 8-15).
8. Bo, L., Sminchisescu, C., Kanaujia, A., & Metaxas, D. (2008). Fast algorithms for large scale conditional 3D prediction. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).
9. J. Canny 1986 A computational approach to edge detection IEEE Transactions on Pattern Analysis and Machine Intelligence 8 679 714 10.1109/TPAMI.1986.4767851
10. Cham, T.-J., & Rehg, J. (1999). A multiple hypothesis approach to figure tracking. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 239-245).
11. Cheung, G. K. M., Baker, S., & Kanade, T. (2003). Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 77-84).
12. Choo, K., & Fleet, D. J. (2001). People tracking with hybrid Monte Carlo. In IEEE international conference on computer vision (ICCV) (Vol. 2, pp. 321-328). (Pubitemid 32795077)
13. Cochran, W. G. (1977). Sampling techniques. New York: Wiley.
14. D. Comaniciu P. Meer 2002 Mean shift: A robust approach toward feature space analysis IEEE Transactions on Pattern Analysis and Machine Intelligence 24 5 603 619 10.1109/34.1000236 (Pubitemid 34550429)
15. G. Cooper 1990 The computational complexity of probabilistic inference using Bayesian belief networks Artificial Intelligence 42 393 405 1045483 0717.68080 10.1016/0004-3702(90)90060-D (Pubitemid 20673788)
16. S. Corazza L. Muendermann A. Chaudhari T. Demattio C. Cobelli T. Andriacchi 2006 A markerless motion capture system to study musculoskeletal biomechanics: visual hull and simulated annealing approach Annals of Biomedical Engineering 34 6 1019 1029 10.1007/s10439-006-9122-8 (Pubitemid 43954334)
17. J. Deutscher I. D. Reid 2005 Articulated body motion capture by stochastic search International Journal of Computer Vision 61 2 185 205 10.1023/B:VISI.0000043757.18370.9c
18. Deutscher, J., Blake, A., & Reid, I. (2000). Articulated body motion capture by annealed particle filtering. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 126-133).
19. Deutscher, J., Isard, M., & MacCormick, J. (2002). Automatic camera calibration from a single Manhattan image. In European conference on computer vision (ECCV) (Vol. 4, pp. 175-188).
20. A. Doucet S. J. Godsill C. Andrieu 2000 On sequential Monte Carlo sampling methods for Bayesian filtering Statistics and Computing 10 3 197 208 10.1023/A:1008935410038
21. Doucet, A., de Freitas, N., & Gordon, N. (2001). Sequential Monte Carlo methods in practice. In Statistics for engineering and information sciences. Berlin: Springer.
22. Eichner, M., & Ferrari, V. (2009). Better appearance models for pictorial structures. In British machine vision conference (BMVC).
23. Elgammal, A., & Lee, C. (2004). Inferring 3D body pose from silhouettes using activity manifold learning. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 681-688).
24. Elidan, G., McGraw, I., & Koller, D. (2006). Residual belief propagation: Informed scheduling for asynchronous message passing. In Proceedings of the twenty-second conference on uncertainty in AI (UAI), July 2006.
25. P. Felzenszwalb D. Huttenlocher 2005 Pictorial structures for object recognition International Journal of Computer Vision 61 1 55 79 10.1023/B:VISI.0000042934.15159.49
26. Fergus, R., Perona, P., & Zisserman, A. (2003). Object class recognition by unsupervised scale-invariant learning. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 264-271).
27. M. Fischler R. Elschlager 1973 The representation and matching of pictorial structures IEEE Transactions on Computers 22 1 67 92 10.1109/T-C.1973.223602
28. Foley, J., van Dam, A., Feiner, S., & Hughes, J. (1990). Computer graphics: Principles and practice. Reading: Addison Wesley. ISBN:0-201-12110-7
29. Forsyth, D. A., Arikan, O., Ikemoto, L., O'Brien, J., & Ramanan, D. (2006). Computational studies of human motion: Part 1, tracking and motion synthesis. ISBN:1-933019-30-1, 178 pp.
30. Gall, J., Potthoff, J., Schnoerr, C., Rosenhahn, B., & Seidel, H.-P. (2006). Interacting annealing particle filters: Mathematics and a recipe for applications (Technical Report MPI-I-2006-4-009). Saarbruecken, Germany, September 2006.
31. Gall, J., Rosenhahn, B., & Seidel, H.-P. (2007). Clustered stochastic optimization for object recognition and pose estimation. In LNCS: Vol. 4713. Annual symposium of the German association for pattern recognition (DAGM) (pp. 32-41).
32. J. Gall B. Rosenhahn T. Brox H.-P. Seidel 2010 Optimization and filtering for human motion capture-A multi-layer framework International Journal of Computer Vision 87 1 75 92 10.1007/s11263-008-0173-1
33. D. Gavrila 1999 The visual analysis of human movement: A survey Computer Vision and Image Understanding 73 1 82 98 0924.68174 10.1006/cviu.1998.0716
34. Gavrila, D., & Davis, L. (1996). 3-D model-based tracking of humans in action: A multi-view approach. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (pp. 73-80).
35. Grauman, K., Shakhnarovich, G., & Darrell, T. (2003). Inferring 3D structure with a statistical image-based shape model. In IEEE International conference on computer vision (ICCV) (pp. 641-648).
36. Guan, P., Weiss, A., Balan, A., & Black, M. J. (2009). Estimating human shape and pose from a single image. In IEEE International Conference on computer vision (ICCV).
37. Hinton, G. E. (1976). Using relaxation to find a puppet. In Proceeding of the A.I.S.B. Summer conference (pp. 148-157).
38. D. C. Hogg 1983 Model-based vision: A program to see a walking person Image and Vision Computing 1 5 20 10.1016/0262-8856(83)90003-3 (Pubitemid 13572232)
39. Horaud, R., Niskanen, M., Dewaele, G., & Boyer, E. (2008). Human motion tracking by registering an articulated surface to 3-D points and normals. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI).
40. Hua, G., Yang, M.-H., & Wu, Y. (2005). Learning to estimate human pose with data driven belief propagation. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp, 747-754). (Pubitemid 43897432)
41. A. T. Ihler E. B. Sudderth W. T. Freeman A. S. Willsky 2003 Efficient multiscale sampling from products of Gaussian mixtures Advances in Neural Information Processing Systems 16 1 8
42. Intel Open Source Computer Vision Library. Available at http://www.intel.com/research/mrl/research/opencv/.
43. Ioffe, S., & Forsyth, D. (2001a). Human tracking with mixtures of trees. In IEEE international conference on computer vision (ICCV) (Vol. 1, pp. 690-695). (Pubitemid 32804835)
44. S. Ioffe D. Forsyth 2001 Probabilistic methods for finding people International Journal of Computer Vision 43 1 45 68 0972.68605 10.1023/A:1011179004708 (Pubitemid 32723477)
45. Isard, M. (2003). Pampas: Real-valued graphical models for computer vision. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 613-620).
46. John, V., Ivekovic, S., & Trucco, E. (2009). Articulated human motion tracking with HPSO. In International conference on computer vision theory and applications (VISSAPP) (pp. 531-538).
47. Jordan, M. I., Sejnowski, T. J., & Poggio, T. (2001). Graphical models: Foundations of neural computation. Cambridge: MIT Press.
48. Ju, S., Black, M. J., & Yacoob, Y. (1996). Cardboard people: A parameterized model of articulated motion. In International conference on automatic face and gesture recognition (pp. 38-44).
49. Kakadiaris, I. A., & Metaxas, D. (1996). Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (pp. 81-87).
50. Kehl, R., Bray, M., & Gool, L. V. (2005). Full body tracking from multiple views using stochastic sampling. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 129-136). (Pubitemid 43897347)
51. Kinoshita, K., Ma, Y., Lao, S., & Kawade, M. (2006). A fast and robust 3D head pose and gaze estimation system. In International conference on multimodal interfaces (ICMI) (pp. 137-138). (Pubitemid 47113443)
52. Kirkpatrick, S., Gellatt, C., & Vecchi, M. (1982). Optimisation by simulated annealing (Technical report). IBM Thomas J. Watson Research Centre, Yorktown Heights, NY, USA.
53. D. Knossow R. Ronfard R. Horaud 2008 Human motion tracking with a kinematic parameterization of extremal contours International Journal of Computer Vision 79 2 247 269 10.1007/s11263-007-0116-2
54. Koller, D., Lerner, U., & Angelov, D. (1999). A general algorithm for approximate inference and its application to hybrid Bayes nets. In Proceedings of the 15th annual conference on uncertainty in artificial intelligence (pp. 324-333).
55. Lan, X., & Huttenlocher, D. (2005). Beyond trees: Common factor models for 2D human pose recovery. In IEEE international conference on computer vision (ICCV) (pp. 470-477). (Pubitemid 44054994)
56. Lee, C.-S., & Elgammal, A. (2007). Modeling view and posture manifold for tracking. In IEEE international conference on computer vision (ICCV).
57. Li, R., Yang, M.-H., Sclaroff, S., & Tian, T.-P. (2006). Monocular tracking of 3D human motion with a coordinated mixture of factor analyzers. In European conference on computer vision (ECCV) (Vol. 2, pp. 137-150). (Pubitemid 44029723)
58. Li, R., Tian, T.-P., & Sclaroff, S. (2007). Simultaneous learning of nonlinear manifold and dynamical models for high-dimensional time series. In IEEE international conference on computer vision (ICCV).
59. Lu, Z., Perpinan, M. C., & Sminchisescu, C. (2007). People tracking with the Laplacian eigenmaps latent variable model. Advances in Neural Information Processing Systems (NIPS).
60. MacCormick, J., & Isard, M. (2000). Partitioned sampling, articulated objects, and interface-quality hand tracking. In European conference on computer vision (ECCV) (Vol. 2, pp. 3-19).
61. D. Marr H. K. Nishihara 1978 Representation and recognition of the spatial organization of three dimensional structure Proceedings of the Royal Society of London. Series B, Biological Sciences 200 269 294 10.1098/rspb.1978.0020
62. T. Moeslund E. Granum 2001 A survey of computer vision-based human motion capture Computer Vision and Image Understanding 81 3 231 268 1011.68548 10.1006/cviu.2000.0897 (Pubitemid 32415976)
63. Mori, G., Ren, X., Efros, A., & Malik, J. (2004). Recovering human body configurations: Combining segmentation and recognition. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 326-333).
64. Navaratnam, R., Fitzgibbon, A., & Cipolla, R. (2007). Semi-supervised joint manifold learning for multi-valued regression. In IEEE international conference on computer vision (ICCV).
65. Nevatia, R., & Binford, T. O. (1973). Structured descriptions of complex objects. In Proc. 3rd international joint conference on artificial intelligence (pp. 641-647).
66. Opelt, A., Pinz, A., & Zisserman, A. (2006). A boundary-fragment- model for object detection. In European conference on computer vision (ECCV) (Vol. 2, pp. 575-588). (Pubitemid 44029756)
67. R. W. Poppe 2007 Vision-based human motion analysis: An overview Computer Vision and Image Understanding 108 1-2 4 18 10.1016/j.cviu.2006.10.016 (Pubitemid 47333391)
68. Poppe, R. (2007b). Evaluating example-based pose estimation: experiments on the HumanEva sets. In Workshop on evaluation of articulated human motion and pose estimation (EHuM2).
69. Ramanan, D., Forsyth, D., & Zisserman, A. (2005). Strike a pose: Tracking people by finding stylized poses. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 271-278). (Pubitemid 43897204)
70. Ramanan, D., & Forsyth, D. (2003). Finding and tracking people from the bottom up. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 467-474).
71. Rodgers, J., Anguelov, D., Pang, H.-C., & Koller, D. (2006). Object pose detection in range scan data. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 2445-2452). (Pubitemid 44931616)
72. Rosales, R., & Sclaroff, S. (2000). Inferring body pose without tracking body parts. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 721-727).
73. R. Rosales S. Sclaroff 2002 Learning body pose via specialized maps Advances in Neural Information Processing Systems 15 1263 1270
74. Rosenhahn, B., Schmaltz, C., Brox, T., Weickert, J., Cremers, D., & Seidel, H.-P. (2008). Markerless motion capture of man-machine interaction. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).
75. Shakhnarovich, G., Viola, P., & Darrell, T. (2003). Fast pose estimation with parameter sensitive hashing. In IEEE international conference on computer vision (ICCV) (Vol. 2, pp. 750-757).
76. Siddiqui, M., & Medioni, G. (2006). Robust real-time upper body limb detection and tracking. In ACM international workshop on video surveillance & sensor networks (VSSN).
77. H. Sidenbladh M. J. Black 2003 Learning the statistics of people in images and video International Journal of Computer Vision 54 1-3 183 209 1077.68881
78. Sidenbladh, H., Black, M. J., & Fleet, D. (2000). Stochastic tracking of 3D human figures using 2D image motion. In European conference on computer vision (ECCV) (Vol. 2, pp. 702-718).
79. Sigal, L., & Black, M. J. (2006a). Predicting 3D people from 2D pictures. In LNCS: Vol. 4069. AMDO 2006-IV conference on articulated motion and deformable objects, Mallorca, Spain, July (pp. 185-195). (Pubitemid 44153446)
80. Sigal, L., & Black, M. J. (2006b). Measure locally, reason globally: Occlusion-sensitive articulated pose estimation. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 2041-2048). (Pubitemid 44931566)
81. Sigal, L., Zhu, Y., Comaniciu, D., & Black, M. J. (2004a). Tracking complex objects using graphical object models. In LNCS: Vol. 3417. 1st international workshop on complex motion (pp. 227-238). Berlin: Springer.
82. Sigal, L., Bhatia, S., Roth, S., Black, M. J., & Isard, M. (2004b). Tracking loose-limbed people. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 421-428).
83. Sigal, L., Balan, A., & Black, M. J. (2007). Combined discriminative and generative articulated pose and non-rigid shape estimation. Advances in Neural Information Processing Systems (NIPS).
84. L. Sigal A. Balan M. J. Black 2010 HumanEva synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion International Journal of Computer Vision 87 1/2 4 27 10.1007/s11263-009- 0273-6
85. C. Sminchisescu B. Triggs 2003 Estimating articulated human motion with covariance scaled sampling The International Journal of Robotics Research 22 6 371 393 10.1177/0278364903022006003
86. Sminchisescu, C., Kanaujia, A., Li, Z., & Metaxas, D. (2005). Discriminative density propagation for 3D human motion estimation. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 390-397). (Pubitemid 43897220)
87. Sminchisescu, C., Kanajujia, A., & Metaxas, D. (2006). Learning joint top-down and bottom-up processes for 3D visual inference. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 1743-1752).
88. Sudderth, E., Ihler, A., Freeman, W., & Willsky, A. (2003). Nonparametric belief propagation. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 605-612).
89. E. Sudderth M. Mandel W. Freeman A. Willsky 2004 Distributed occlusion reasoning for tracking with nonparametric belief propagation Advances in Neural Information Processing Systems 17 1369 1376
90. Sun, J., Shum, H., & Zheng, N. (2002). Stereo matching using belief propagation. In European conference on computer vision (ECCV) (pp. 510-524).
91. Tian, T.-P., & Sclaroff, S. (2010). Fast globally optimal 2D human detection with loopy graph models. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).
92. Urtasun, R., & Darrell, T. (2008). Local probabilistic regression for activity-independent human pose inference. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).
93. Urtasun, R., Fleet, D. J., & Fua, P. (2006). Gaussian process dynamical models for 3D people tracking. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 238-245). (Pubitemid 44931336)
94. Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 511-518).
95. S. Wachter H. Nagel 1999 Tracking of persons in monocular image sequences Computer Vision and Image Understanding 74 3 174 192 10.1006/cviu.1999.0758
96. Wainwright, M., Jaakkola, T., & Willsky, A. (2001). Tree-based reparameterization for approximate estimation on loopy graphs. Advances in Neural Information Processing Systems (NIPS), 1001-1008.
97. Wang, P., & Rehg, J. M. (2006). A modular approach to the analysis and evaluation of particle filters for figure tracking. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 790-797). (Pubitemid 44931405)
98. Weber, M., Welling, M., & Perona, P. (2000). Unsupervised learning of models for recognition. In European conference on computer vision (ECCV) (pp. 18-32).
99. Y. Weiss W. T. Freeman 2001 Correctness of belief propagation in Gaussian graphical models of arbitrary topology Neural Computation 13 2173 2200 0992.68055 10.1162/089976601750541769 (Pubitemid 33676880)
100. Wu, Y., Hua, G., & Yu, T. (2003). Tracking articulated body by dynamic Markov network. In IEEE international conference on computer vision (ICCV) (pp. 1094-1101).
101. G. Wywill T. L. Kunii 1985 A functional model for constructive solid geometry The Visual Computer 1 1 3 14 10.1007/BF01901265
102. Xu, X., & Li, B. (2007). Learning motion correlation for tracking articulated human body with a rao-blackwellised particle filter. In IEEE international conference on computer vision (ICCV).
103. Yonemoto, S., Arita, D., & Taniguchi, R. (2000). Real-time human motion analysis and IK-based human figure control. In Proceedings of the workshop on human motion (HUMO).