Spatio-Temporal Matching for Human Pose Estimation in Video

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 38, Issue 8, 2016, Pages 1492-1504

  Zhou, Feng ^a   De La Torre, Fernando ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

dense trajectories; Human pose estimation; spatio temporal bilinear model; trajectory matching

Indexed keywords

CAMERAS; COMPUTER VISION; IMAGE MATCHING; LINEAR PROGRAMMING; MOTION ESTIMATION; TRAJECTORIES;

BILINEAR MODELS; DEFORMABLE PARTS MODELS; DETECTION AND TRACKING; DISCRIMINATIVE MODELS; HUMAN POSE ESTIMATIONS; STANDING PROBLEMS; STATE-OF-THE-ART APPROACH; TRAJECTORY MATCHING;

THREE DIMENSIONAL COMPUTER GRAPHICS;

ALGORITHM; BODY POSITION; FACTUAL DATABASE; HUMAN; MOTION; REPRODUCIBILITY; VIDEORECORDING;

ALGORITHMS; DATABASES, FACTUAL; HUMANS; MOTION; POSTURE; REPRODUCIBILITY OF RESULTS; VIDEO RECORDING;

EID: 84978426230     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2016.2526002     Document Type: Article

References (52)

1. J. Shotton, R. B. Girshick, A. W. Fitzgibbon, T. Sharp, M. Cook, M. Finocchio, R. Moore, P. Kohli, A. Criminisi, A. Kipman, and A. Blake, "Efficient human pose estimation from single depth images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 12, pp. 2821-2840, Dec. 2013.
2. O. Boiman and M. Irani, "Detecting irregularities in images and in video," Int. J. Comput. Vis., vol. 74, no. 1, pp. 17-31, 2007.
3. X. K. Wei and J. Chai, "VideoMocap: Modeling physically realistic human motion from monocular video sequences," ACM Trans. Graph., vol. 29, no. 4, pp. 1-10, 2010.
4. P. F. Felzenszwalb, R. B. Girshick, D. A. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1627-1645, Sep. 2010.
5. Y. Yang and D. Ramanan, "Articulated human detection with flexible mixtures of parts," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 12, pp. 2878-2890, Dec. 2013.
6. M. Andriluka, S. Roth, and B. Schiele, "Discriminative appearance models for pictorial structures," Int. J. Comput. Vis., vol. 99, no. 3, pp. 259-280, 2012.
7. C. Ionescu, F. Li, and C. Sminchisescu, "Latent structured models for human pose estimation," in Proc. IEEE Int. Conf. Comput. Vision, 2011, pp. 2220-2227.
8. M. Eichner, M. Jesus, A. Zisserman, and V. Ferrari, "2d articulated human pose estimation and retrieval in (almost) unconstrained still images," Int. J. Comput. Vis., vol. 99, no. 2, pp. 190-214, 2012.
9. F. Ofli, R. Chaudhry, G. Kurillo, R. Vidal, and R. Bajcsy, "Berkeley MHAD: A comprehensive multimodal human action database," in Proc. IEEE Workshop Appl. Comput. Vis., 2013, pp. 53-60.
10. C. Ionescu, D. Papava, V. Olaru, and C. Sminchisescu, "Human3.6M: Large scale datasets and predictive methods for 3d human sensing in natural environments," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 7, pp. 1325-1339, Jul. 2014.
11. D. Huang, Y. Wang, S. Yao, and F. De la Torre, "Sequential maxmargin event detectors," in Proc. 13th Eur. Conf. Comput. Vis., 2014, pp. 410-424.
12. S. X. Ju, M. J. Black, and Y. Yacoob, "Cardboard people: A parameterized model of articulated image motion," in Proc. 2nd Int. Conf. Automat. Face Gesture Recog., 1996, pp. 38-44.
13. J. Deutscher and I. D. Reid, "Articulated body motion capture by stochastic search," Int. J. Comput. Vis., vol. 61, no. 2, pp. 185-205, 2005.
14. C. Bregler and J. Malik, "Tracking people with twists and exponential maps," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., 1998, pp. 8-15.
15. K. Rohr, "Incremental recognition of pedestrians from image sequences," in Proc. IEEE Conf. Comput. Vision Pattern Recog., 1993, pp. 8-13.
16. H. Sidenbladh, M. J. Black, and D. J. Fleet, "Stochastic tracking of 3d human figures using 2d image motion," in Proc. 6th Eur. Conf. Comput. Vis., 2000, pp. 702-718.
17. J. Deutscher, A. Blake, and I. D. Reid, "Articulated body motion capture by annealed particle filtering," in Proc. IEEE Conf. Comput. Vision Pattern Recog., 2000, pp. 126-133.
18. Y. Song, L. Goncalves, and P. Perona, "Unsupervised learning of human motion," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 7, pp. 814-827, Jul. 2003.
19. B. Sapp, A. Toshev, and B. Taskar, "Cascaded models for articulated pose estimation," in Proc. 11th Eur. Conf. Comput. Vis.. 2010, pp. 406-420.
20. M. Andriluka, S. Roth, and B. Schiele, "People-tracking-by-detection and people-detection-by-tracking," in Proc. IEEE Conf. Comput. Vision Pattern Recog., 2008, pp. 1-8.
21. M. Andriluka, "Monocular 3d pose estimation and tracking by detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 623-630.
22. B. Sapp, D. Weiss, and B. Taskar, "Parsing human motion with stretchable models," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2011, pp. 1281-1288.
23. B. Wu and R. Nevatia, "Detection and tracking of multiple, partially occluded humans by Bayesian combination of edgelet based part detectors," Int. J. Comput. Vis., vol. 75, no. 2, pp. 247-266, 2007.
24. X. Burgos, D. Hall, P. Perona, and P. Dollár, "Merging pose estimates across space and time," in Proc. Brit. Mach. Vision Conf., 2013, pp. 58.1-58.11.
25. D. Park and D. Ramanan, "N-best maximal decoders for part models," in Proc. IEEE Int. Conf. Comput. Vision, 2011, pp. 2627-2634.
26. A. Toshev and C. Szegedy, "DeepPose: Human pose estimation via deep neural networks," in Proc. IEEE Conf. Comput. Vision Pattern Recog., 2014, pp. 1653-1660.
27. A. Jain, J. Tompson, Y. LeCun, and C. Bregler, "Modeep: A deep learning framework using motion features for human pose estimation," in ACCV, 2014.
28. A. Agarwal and B. Triggs, "Recovering 3D human pose from monocular images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 1, pp. 44-58, Jan. 2006.
29. A. M. Elgammal and C.-S. Lee, "Inferring 3d body pose from silhouettes using activity manifold learning," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., 2004, pp. II-681-II-688.
30. R. Urtasun, D. J. Fleet, and P. Fua, "3d people tracking with Gaussian process dynamical models," in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recog., 2006, pp. 238-245.
31. L. Sigal and M. J. Black, "Predicting 3D people from 2D pictures," in Proc. 4th Int. Conf. Articulated Motion Deformable Objects, 2006, pp. 185-195.
32. E. Simo-Serra, A. Ramisa, G. Alenyá, C. Torras, and F. Moreno- Noguer, "Single image 3D human pose estimation from noisy observations," in Proc. IEEE Conf. Comput. Vision Pattern Recog., 2012, pp. 2673-2680.
33. A. Yao, J. Gall, and L. J. V. Gool, "Coupled action recognition and pose estimation from multiple views," Int. J. Comput. Vis., vol. 100, no. 1, pp. 16-37, 2012.
34. T.-H. Yu, T.-K. Kim, and R. Cipolla, "Unconstrained monocular 3D human pose estimation by action detection and cross-modality regression forest," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 3642-3649.
35. H. Wang, A. Kläser, C. Schmid, and C.-L. Liu, "Dense trajectories and motion boundary descriptors for action recognition," Int. J. Comput. Vis., vol. 103, no. 1, pp. 60-79, 2013.
36. G. Farnebäck, "Two-frame motion estimation based on polynomial expansion," in Proc. 13th Scandinavian Conf. Image Anal., 2003, pp. 363-370.
37. R. Messing, C. J. Pal, and H. A. Kautz, "Activity recognition using the velocity histories of tracked keypoints," in Proc. IEEE 12th Int. Conf. Comput. Vision, 2009, pp. 104-111.
38. P. Matikainen, M. Hebert, and R. Sukthankar, "Trajectons: Action recognition through the motion analysis of tracked features," in Proc. IEEE 12th Int. Conf. Comput. Vis. Workshops, 2009, pp. 514-521.
39. I. Laptev, "On space-time interest points," Int. J. Comput. Vis., vol. 64, no. 2-3, pp. 107-123, 2005.
40. P. Dollár, V. Rabaud, G. Cottrell, and S. Belongie, "Behavior recognition via sparse spatio-temporal features," in Proc. 14th Int. Conf. Comput. Commun. Netw., 2005, pp. 65-72.
41. H. Wang, M. Ullah, A. Kläser, I. Laptev, and C. Schmid, "Evaluation of local spatio-temporal features for action recognition," in Proc. Brit. Mach. Vis. Conf., 2009, pp. 1-11.
42. (2015). Carnegie Mellon University Motion Capture Database [Online]. Available: http://mocap.cs.cmu.edu
43. A. M. Bronstein, M. M. Bronstein, and R. Kimmel, Numerical Geometry of Non-Rigid Shapes. Berlin, Germany: Springer, 2008.
44. C. Bregler, A. Hertzmann, and H. Biermann, "Recovering nonrigid 3d shape from image streams," in Proc. IEEE Conf. Comput. Vision Pattern Recog., 2000, pp. 690-696.
45. I. Akhter, Y. Sheikh, S. Khan, and T. Kanade, "Trajectory space: A dual representation for nonrigid structure from motion," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 7, pp. 1442-1456, Jul. 2011.
46. I. Akhter, T. Simon, S. Khan, I. Matthews, and Y. Sheikh, "Bilinear spatiotemporal basis models," ACM Trans. Graph., vol. 31, no. 2, p. 17, 2012.
47. J. C. Gower and G. B. Dijksterhuis, Procrustes Problems. London, U.K.: Oxford Univ. Press, 2004.
48. A. Y. Ng, M. I. Jordan, and Y. Weiss, "On spectral clustering: Analysis and an algorithm," in Proc. Adv. Neural Inf. Process. Syst., 2001, pp. 849-856.
49. H. Jiang, M. S. Drew, and Z.-N. Li, "Matching by linear programming and successive convexification," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 959-975, Jun. 2007.
50. N. Trendafilov, "On the l1 Procrustes problem," Future Gener. Comput. Syst., vol. 19, no. 7, pp. 1177-1186, 2004.
51. Z. Lin, M. Chen, and Y. Ma, "The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices," arXiv preprint arXiv:1009.5055, 2010.
52. (2015). Mosek [Online]. Available: http://www.mosek.com/