Dynamic FAUST: Registering human bodies in motion

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 5573-5582

  Bogo, Federica ^a   Romero, Javier ^b   Pons Moll, Gerard ^c   Black, Michael J ^c  

^a MICROSOFT   (United States)

^b Body Labs Inc   (United States)

^c MAX PLANCK INSTITUTE FOR INTELLIGENT SYSTEMS   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER GRAPHICS; MESH GENERATION; TOPOLOGY;

HIGH RESOLUTION; NON-RIGID OBJECTS; QUANTITATIVE EVALUATION; REGISTRATION METHODS; RESEARCH PURPOSE; SHAPE AND TEXTURES; TEXTURE INFORMATION; VISION RESEARCH;

COMPUTER VISION;

EID: 85041407734     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.591     Document Type: Conference Paper

References (55)

1. http://dfaust.is.tue.mpg.de. 2, 8
2. N. Ahmed, C. Theobalt, C. Roessl, S. Thrun, and H.-P. Seidel. Dense correspondences finding for parameterization-free animation reconstruction from video. In CVPR, pages 1-8, 2008. 2
3. B. Allain, J. Franco, and E. Boyer. An efficient volumetric framework for shape tracking. In CVPR, pages 268-276, 2015. 2
4. D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, and J. Davis. SCAPE: Shape Completion and Animation of PEople. ACM Transactions on Graphics, 24(3):408-416, 2005. 3
5. T. Beeler, F. Hahn, D. Bradley, B. Bickel, P. Bearsdley, C. Gotsman, R. Sumner, and M. Gross. High-quality passive facial performance capture using anchor frames. ACM Transactions on Graphics, (Proc. SIGGRAPH), 30(4):75:1-75:10, 2011. 2
6. V. Blanz and T. Vetter. A morphable model for the synthesis of 3D faces. In ACM SIGGRAPH, pages 187-194, 1999. 2
7. F. Bogo, M. J. Black, M. Loper, and J. Romero. Detailed full-body reconstructions of moving people from monocular RGB-D sequences. In ICCV, pages 2300-2308, 2015. 2
8. F. Bogo, J. Romero, M. Loper, and M. J. Black. FAUST: Dataset and evaluation for 3D mesh registration. In CVPR, pages 3794-3801, 2014. 2, 3, 4, 5, 6, 7
9. D. Boscaini, D. Eynard, D. Kourounis, and M. Bronstein. Shape-From-Operator: Recovering shapes from intrinsic operators. Computer Graphics Forum, 34(3):265-274, 2015. 3
10. D. Boscaini, J. Masci, S. Melzi, M. Bronstein, U. Castellani, and P. Vandergheynst. Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks. Computer Graphics Forum, 34(5):12-23, 2015. 2, 3
11. D. Boscaini, J. Masci, E. Rodolà, and M. Bronstein. Learning shape correspondence with anisotropic convolutional neural networks. In NIPS, pages 3189-3197, 2016. 2, 3
12. A. Boukhayma, V. Tsiminaki, J. Franco, and E. Boyer. Eigen appearance maps of dynamic shapes. In ECCV, pages 230-245, 2016. 2, 3
13. A. Bronstein, M. Bronstein, U. Castellani, A. Dubrovina, L. Guibas, R. Horaud, R. Kimmel, D. Knossow, E. von La-vante, D. Mateus, M. Ovsjanikov, and A. Sharma. SHREC 2010: Robust correspondence benchmark. In 3DOR, 2010. 3
14. A. Bronstein, M. Bronstein, and R. Kimmel. Numerical geometry of non-rigid shapes. Springer, 2008. 3
15. C. Budd, P. Huang, M. Klaudiny, and A. Hilton. Global nonrigid alignment of surface sequences. International Journal of Computer Vision, 102(1-3):256-270, 2013. 2
16. C. Cagniart, E. Boyer, and S. Ilic. Probabilistic deformable surface tracking from multiple videos. In ECCV, pages 326-339, 2010. 2
17. C. Cao, D. Bradley, K. Zhou, and T. Beeler. Real-time high-fidelity facial performance capture. ACM Transactions on Graphics, (Proc. SIGGRAPH), 34(4):46:1-46:9, 2015. 2
18. D. Casas, C. Richardt, J. Collomosse, C. Theobalt, and A. Hilton. 4D model flow: Precomputed appearance alignment for real-time 4D video interpolation. Computer Graphics Forum, 34(7):173-182, 2015. 3
19. D. Casas, M. Tejera, J. Guillemaut, and A. Hilton. 4D parametric motion graphs for interactive animation. In Symp. on Interactive 3D Graphics and Games, pages 102-110, 2012. 2
20. Q. Chen and V. Koltun. Robust nonrigid registration by convex optimization. In ICCV, pages 2039-2047, 2015. 2, 3
21. A. Collet, M. Chuang, P. Sweeney, D. Gillet, D. Evseev, D. Calabrese, H. Hoppe, A. Kirk, and S. Sullivan. High-quality streamable free-viewpoint video. ACM Transactions on Graphics, (Proc. SIGGRAPH), 34(4):69:1-69:13, 2015. 2, 3
22. E. De Aguiar, C. Stoll, C. Theobalt, N. Ahmed, H.-P. Seidel, and S. Thrun. Performance capture from sparse multi-view video. ACM Transactions on Graphics, (Proc. SIGGRAPH), 27(3):98:1-75:10, 2008. 2
23. M. de la Gorce, D. Fleet, and N. Paragios. Model-based 3D hand pose estimation from monocular video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(9):1793-1805, 2011. 2
24. M. Dou, J. Taylor, H. Fuchs, A. Fitzgibbon, and S. Izadi. 3D scanning deformable objects with a single RGBD sensor. In CVPR, pages 493-501, 2015. 2
25. A. Elhayek, C. Stoll, N. Hasler, K. I. Kim, H. P. Seidel, and C. Theobalt. Spatio-temporal motion tracking with unsyn-chronized cameras. In CVPR, pages 1870-1877, 2012. 2
26. J. Gall, B. Rosenhahn, and H.-P. Seidel. Drift-free tracking of rigid and articulated objects. In CVPR, pages 1-8, 2008. 3
27. J. Gall, C. Stoll, E. De Aguiar, C. Theobalt, B. Rosenhahn, and H.-P. Seidel. Motion capture using joint skeleton tracking and surface estimation. In CVPR, pages 1746-1753, 2009. 2
28. N. Hasler, C. Stoll, M. Sunkel, B. Rosenhahn, and H. P. Seidel. A statistical model of human pose and body shape. Computer Graphics Forum, 28(2):337-346, 2009. 3
29. D. A. Hirshberg, M. Loper, E. Rachlin, and M. J. Black. Coregistration: Simultaneous alignment and modeling of articulated 3D shape. In ECCV, pages 242-255, 2012. 4
30. P. Huang, C. Budd, and A. Hilton. Global temporal registration of multiple non-rigid surface sequences. In CVPR, pages 3473-3480, 2011. 2
31. P. Huang, A. Hilton, and J. Starck. Human motion synthesis from 3D video. In CVPR, pages 1478-1485, 2009. 2
32. Z. Lähner, E. Rodolà, M. Bronstein, D. Cremers, O. Burghard, L. Cosmo, A. Dieckmann, R. Klein, and Y. Sahillioglu. SHREC16: Matching of deformable shapes with topological noise. In 3DOR, 2016. 3
33. Z. Lähner, E. Rodolà, F. R. Schmidt, M. Bronstein, and D. Cremers. Efficient globally optimal 2D-to-3D deformable shape matching. In CVPR, pages 2185-2193, 2016. 3
34. H. Li, B. Adams, L. J. Guibas, and M. Pauly. Robust single-view geometry and motion reconstruction. ACM Transactions on Graphics, 28(5):175:1-175:10, 2009. 2
35. M. Loper and M. J. Black. OpenDR: An approximate differentiable renderer. In ECCV, pages 154-169, 2014. 7
36. M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black. SMPL: A skinned multi-person linear model. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 34(6):248:1-248:16, 2015. 4, 5, 6
37. R. A. Newcombe, D. Fox, and S. M. Seitz. DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time. In CVPR, pages 343-352, 2015. 2
38. J. Nocedal and S. J. Wright. Numerical Optimization. Springer, 2006. 7
39. G. Pons-Moll, J. Romero, N. Mahmood, and M. J. Black. Dyna: A model of dynamic human shape in motion. ACM Transactions on Graphics, (Proc. SIGGRAPH), 34(4):120:1-120:14, 2015. 2, 3, 4, 6
40. G. Pons-Moll, J. Taylor, J. Shotton, A. Hertzmann, and A. Fitzgibbon. Metric regression forests for correspondence estimation. International Journal of Computer Vision, pages 1-13, 2015. 2
41. F. Prada, M. Kazhdan, M. Chaung, A. Collet, and H. Hoppe. Motion graphs for unstructured textured meshes. ACM Transactions on Graphics, (Proc. SIGGRAPH), 35(4):108:1-108:14, 2016. 2
42. K. Robinette, H. Dannen, and E. Paquet. The CAESAR project: A 3-D surface anthropometry survey. In Conference on 3D Digital Imaging and Modeling, pages 380-386, 1999. 3
43. S. Saito, T. Li, and H. Li. Real-time facial segmentation and performance capture from RGB input. In ECCV, pages 244-261, 2016. 2
44. J. Starck and A. Hilton. Model-based multiple view reconstruction of people. In ICCV, pages 915-922, 2003. 2
45. J. Starck and A. Hilton. Surface capture for performance-based animation. Computer Graphics and Applications, 27:21-31, 2007. 3
46. C. Theobalt, J. Carranza, and M. A. Magnor. Enhancing silhouette-based human motion capture with 3D motion fields. In Computer Graphics and Applications, pages 185-193, 2003. 3
47. V. Tsiminaki, J. Franco, and E. Boyer. High resolution 3D shape texture from multiple videos. In CVPR, pages 1502-1509, 2014. 3
48. T. Tung and T. Matsuyama. Dynamic surface matching by geodesic mapping for animation transfer. In CVPR, pages 1402-1409, 2010. 2
49. D. Vlasic, I. Baran, W. Matusik, and J. Popovic. Articulated mesh animation from multi-view silhouettes. ACM Transactions on Graphics, (Proc. SIGGRAPH), 27(3):97:1-97:9, 2008. 2
50. R. Wang, L. Wei, E. Vouga, Q. Huang, D. Ceylan, G. Medioni, and H. Li. Capturing dynamic textured surfaces of moving targets. In ECCV, pages 271-288, 2016. 2
51. L. Wei, Q. Huang, D. Ceylan, E. Vouga, and H. Li. Dense human body correspondences using convolutional networks. In CVPR, pages 1544-1553, 2016. 2, 3
52. P. Weinzaepfel, J. Revaud, Z. Harchaoui, and C. Schmid. DeepFlow: Large displacement optical flow with deep matching. In ICCV, pages 1385-1392, 2013. 4, 5, 7, 8
53. C. Zhang, B. Heeren, M. Rumpf, and W. Smith. Shell PCA: Statistical shape modelling in shell space. In ICCV, pages 1671-1679, 2015. 3
54. M. Zollhöfer, M. Nießner, S. Izadi, C. Rehmann, C. Zach, M. Fisher, C. Wu, A. Fitzgibbon, C. Loop, C. Theobalt, and M. Stamminger. Real-time non-rigid reconstruction using an RGB-D camera. ACM Transactions on Graphics, 33(4):156:1-156:12, 2014. 2
55. S. Zuffi and M. J. Black. The stitched puppet: A graphical model of 3D human shape and pose. In CVPR, pages 3537-3546, 2015. 3