Detailed, accurate, human shape estimation from clothed 3D scan sequences

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 5484-5493

  Zhang, Chao ^a,b   Pujades, Sergi ^a   Black, Michael ^a   Pons Moll, Gerard ^a  

^a MAX PLANCK INSTITUTE FOR INTELLIGENT SYSTEMS   (Germany)

^b UNIVERSITY OF YORK   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; VIRTUAL REALITY;

HEALTH MONITORING; MULTI-VIEW IMAGE; PARAMETRIC MODELING; POSE ESTIMATION; QUANTITATIVE EVALUATION; SHAPE ESTIMATION; STATE OF THE ART; VIRTUAL TRY-ON;

THREE DIMENSIONAL COMPUTER GRAPHICS;

EID: 85041430577     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.582     Document Type: Conference Paper

References (49)

1. D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, and J. Davis. Scape: shape completion and animation of people. In ACM Transactions on Graphics (TOG), Volume 24, pages 408-416. ACM, 2005.
2. A. O. Bǎlan and M. J. Black. The naked truth: Estimating body shape under clothing. In European Conference on Computer Vision, pages 15-29. Springer, 2008.
3. C. Barron and I. A. Kakadiaris. Estimating anthropometry and pose from a single image. In Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on, Volume 1, pages 669-676. IEEE, 2000.
4. F. Bogo, M. J. Black, M. Loper, and J. Romero. Detailed full-body reconstructions of moving people from monocular RGB-D sequences. In International Conference on Computer Vision (ICCV), pages 2300-2308, Dec. 2015.
5. F. Bogo, A. Kanazawa, C. Lassner, P. Gehler, J. Romero, and M. J. Black. Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image. In Computer Vision - ECCV 2016, Lecture Notes in Computer Science. Springer International Publishing, Oct. 2016.
6. A. O. Bǎlan, L. Sigal, M. J. Black, J. E. Davis, and H. W. Haussecker. Detailed human shape and pose from images. In 2007 IEEE Conference on Computer Vision and Pattern Recognition, pages 1-8. IEEE, 2007.
7. Y. Chen, Z. Liu, and Z. Zhang. Tensor-based human body modeling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 105-112, 2013.
8. E. De Aguiar, C. Stoll, C. Theobalt, N. Ahmed, H.-P. Seidel, and S. Thrun. Performance capture from sparse multi-view video. 27(3):98, 2008.
9. E. Dibra, C. Öztireli, R. Ziegler, and M. Gross. Shape from selfies: Human body shape estimation using cca regression forests. In European Conference on Computer Vision, pages 88-104. Springer, 2016.
10. M. Dou, S. Khamis, Y. Degtyarev, P. Davidson, S. R. Fanello, A. Kowdle, S. O. Escolano, C. Rhemann, D. Kim, J. Taylor, et al. Fusion4d: Real-time performance capture of challenging scenes. ACM Transactions on Graphics (TOG), 35(4):114, 2016.
11. J. Gall, B. Rosenhahn, T. Brox, and H.-P. Seidel. Optimization and filtering for human motion capture. International journal of computer vision, 87(1-2):75-92, 2010.
12. D. M. Gavrila and L. S. Davis. 3-d model-based tracking of humans in action: a multi-view approach. In Computer Vision and Pattern Recognition, 1996. Proceedings CVPR'96, 1996 IEEE Computer Society Conference on, pages 73-80. IEEE, 1996.
13. P. Guan, O. Freifeld, and M. J. Black. A 2d human body model dressed in eigen clothing. In European conference on computer vision, pages 285-298. Springer, 2010.
14. N. Hasler, C. Stoll, B. Rosenhahn, T. Thormählen, and H.-P. Seidel. Estimating body shape of dressed humans. Computers & Graphics, 33(3):211-216, 2009.
15. N. Hasler, C. Stoll, M. Sunkel, B. Rosenhahn, and H.-P. Seidel. A statistical model of human pose and body shape. In Computer Graphics Forum, Volume 28, pages 337-346. Wiley Online Library, 2009.
16. T. Helten, A. Baak, G. Bharaj, M. Müller, H.-P. Seidel, and C. Theobalt. Personalization and evaluation of a real-time depth-based full body tracker. In 2013 International Conference on 3D Vision-3DV 2013, pages 279-286. IEEE, 2013.
17. A. Jain, T. Thormählen, H.-P. Seidel, and C. Theobalt. Moviereshape: Tracking and reshaping of humans in videos. 29(6):148, 2010.
18. C. Lassner, J. Romero, M. Kiefel, F. Bogo, M. J. Black, and P. V. Gehler. Unite the people: Closing the loop between 3d and 2d human representations. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition CVPR, 2017.
19. M. Loper. Chumpy is a python-based framework designed to handle the auto-differentiation problem. htttps://pypi.python.org/pypi/chumpy, 2015.
20. M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black. SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6):248:1-248:16, Oct. 2015.
21. A. Neophytou and A. Hilton. A layered model of human body and garment deformation. In 2014 2nd International Conference on 3D Vision, Volume 1, pages 171-178. IEEE, 2014.
22. R. A. Newcombe, D. Fox, and S. M. Seitz. Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 343-352, 2015.
23. A. Newell, K. Yang, and J. Deng. Stacked hourglass networks for human pose estimation. arXiv preprint arXiv:1603.06937 2016.
24. J. Nocedal and S. J. Wright. Numerical Optimization. Springer, New York, 2nd edition, 2006.
25. S. Orts-Escolano, C. Rhemann, S. Fanello, W. Chang, A. Kowdle, Y. Degtyarev, D. Kim, P. L. Davidson, S. Khamis, M. Dou, et al. Holoportation: Virtual 3d teleportation in real-time. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, pages 741-754. ACM, 2016.
26. S. I. Park and J. K. Hodgins. Data-driven modeling of skin and muscle deformation. ACM Trans. Graph., 27(3):96:1-96:6, Aug. 2008.
27. L. Pishchulin, E. Insafutdinov, S. Tang, B. Andres, M. Andriluka, P. Gehler, and B. Schiele. Deepcut: Joint subset partition and labeling for multi person pose estimation.
28. L. Pishchulin, S. Wuhrer, T. Helten, C. Theobalt, and B. Schiele. Building statistical shape spaces for 3d human modeling. [arXiv], March 2015.
29. R. Plankers and P. Fua. Articulated soft objects for video-based body modeling. In International Conference on Computer Vision, Vancouver, Canada, number CVLAB-CONF-2001-005, pages 394-401, 2001.
30. G. Pons-Moll, S. Pujades, S. Hu, and M. Black. Clothcap: Seamless 4d clothing capture and retargeting. ACM Transactions on Graphics, (Proc. SIGGRAPH) [to appear], 2017.
31. G. Pons-Moll, J. Romero, N. Mahmood, and M. J. Black. Dyna: a model of dynamic human shape in motion. ACM Transactions on Graphics (TOG), 34:120, 2015.
32. G. Pons-Moll and B. Rosenhahn. Model-Based Pose Estimation, chapter 9, pages 139-170. Springer, 2011.
33. G. Pons-Moll, J. Taylor, J. Shotton, A. Hertzmann, and A. Fitzgibbon. Metric regression forests for correspondence estimation. International Journal of Computer Vision, pages 1-13, 2015.
34. H. Rhodin, N. Robertini, D. Casas, C. Richardt, H.-P. Seidel, and C. Theobalt. General automatic human shape and motion capture using volumetric contour cues. In European Conference on Computer Vision, pages 509-526. Springer, 2016.
35. B. Rosenhahn, U. Kersting, K. Powell, R. Klette, G. Klette, and H.-P. Seidel. A system for articulated tracking incorporating a clothing model. Machine Vision and Applications, 18(1):25-40, 2007.
36. L. Sigal, S. Bhatia, S. Roth, M. J. Black, and M. Isard. Tracking loose-limbed people. In Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, Volume 1, pages I-421. IEEE, 2004.
37. D. Song, R. Tong, J. Chang, X. Yang, M. Tang, and J. J. Zhang. 3d body shapes estimation from dressed-human silhouettes. In Computer Graphics Forum, Volume 35, pages 147-156. Wiley Online Library, 2016.
38. C. Stoll, J. Gall, E. De Aguiar, S. Thrun, and C. Theobalt. Video-based reconstruction of animatable human characters. ACM Transactions on Graphics (TOG), 29(6):139, 2010.
39. C. Stoll, N. Hasler, J. Gall, H.-P. Seidel, and C. Theobalt. Fast articulated motion tracking using a sums of gaussians body model. In 2011 International Conference on Computer Vision, pages 951-958. IEEE, 2011.
40. T. von Marcard, G. Pons-Moll, and B. Rosenhahn. Human pose estimation from video and imus. Transactions on Pattern Analysis and Machine Intelligence PAMI, Jan. 2016.
41. T. von Marcard, B. Rosenhahn, M. Black, and G. Pons-Moll. Sparse inertial poser: Automatic 3d human pose estimation from sparse imus. Computer Graphics Forum 36(2), Proceedings of the 38th Annual Conference of the European Association for Computer Graphics (Eurographics), 2017.
42. S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh. Convolutional pose machines. 2016.
43. A. Weiss, D. Hirshberg, and M. J. Black. Home 3d body scans from noisy image and range data. In 2011 International Conference on Computer Vision, pages 1951-1958. IEEE, 2011.
44. S. Wuhrer, L. Pishchulin, A. Brunton, C. Shu, and J. Lang. Estimation of human body shape and posture under clothing. Computer Vision and Image Understanding, 127:31-42, 2014.
45. J. Yang, J.-S. Franco, F. Hétroy-Wheeler, and S. Wuhrer. Estimation of Human Body Shape in Motion with Wide Clothing. In European Conference on Computer Vision 2016, Amsterdam, Netherlands, Oct. 2016.
46. M. Ye and R. Yang. Real-time simultaneous pose and shape estimation for articulated objects using a single depth camera. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2345-2352, 2014.
47. Q. Zhang, B. Fu, M. Ye, and R. Yang. Quality dynamic human body modeling using a single low-cost depth camera. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 676-683. IEEE, 2014.
48. M. Zollhöfer, M. Nießner, S. Izadi, C. Rehmann, C. Zach, M. Fisher, C. Wu, A. Fitzgibbon, C. Loop, C. Theobalt, et al. Real-time non-rigid reconstruction using an rgb-d camera. ACM Transactions on Graphics (TOG), 33(4):156, 2014.
49. S. Zuffi and M. J. Black. The stitched puppet: A graphical model of 3d human shape and pose. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3537-3546. IEEE, 2015.