A Generative Model of People in Clothing

Proceedings of the IEEE International Conference on Computer Vision

Volumn 2017-October, Issue , 2017, Pages 853-862

  Lassner, Christoph ^a,b   Pons Moll, Gerard ^b   Gehler, Peter V ^c  

^a BCCN   (Germany)

^b MAX PLANCK INSTITUTE FOR INTELLIGENT SYSTEMS   (Germany)

^c UNIVERSITY OF WÜRZBURG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER GRAPHICS; IMAGE RETRIEVAL; SEMANTICS; THREE DIMENSIONAL COMPUTER GRAPHICS;

CONDITIONAL MODELS; DATA-DRIVEN APPROACH; GENERATIVE MODEL; GRAPHICS RENDERING; HIGH QUALITY; LARGE IMAGE DATABASE; REALISTIC IMAGES; SEMANTIC SEGMENTATION;

COMPUTER VISION;

EID: 85041387116     PISSN: 15505499     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCV.2017.98     Document Type: Conference Paper

References (56)

1. M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele. 2D human pose estimation: New benchmark and state of the art analysis. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), June 2014
2. D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, and J. Davis. SCAPE: shape completion and animation of people. ACM Trans. Graphics (Proc. SIGGRAPH), 24(3):408-416, 2005
3. F. Bogo, A. Kanazawa, C. Lassner, P. Gehler, J. Romero, and M. J. Black. Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image. In Proc. European Conf. on Computer Vision (ECCV), Oct. 2016
4. L.-C. Chen, Y. Yang, J. Wang, W. Xu, and A. L. Yuille. Attention to scale: Scale-aware semantic image segmentation. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 3640-3649, 2016
5. W. Chen, H.Wang, Y. Li, H. Su, Z.Wang, C. Tu, D. Lischinski, D. Cohen-Or, and B. Chen. Synthesizing training images for boosting human 3d pose estimation. In 3D Vision (3DV), 2016
6. A. Dai, C. R. Qi, and M. Nießner. Shape completion using 3D-encoder-predictor CNNs and shape synthesis. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2017
7. E. de Aguiar, L. Sigal, A. Treuille, and J. K. Hodgins. Stable spaces for real-time clothing. ACM Trans. Graphics (Proc. SIGGRAPH), 29(4):106:1-106:9, July 2010
8. E. L. Denton, S. Chintala, R. Fergus, et al. Deep generative image models using a laplacian pyramid of adversarial networks. In Advances in Neural Information Processing Systems (NIPS), pages 1486-1494, 2015
9. R. Goldenthal, D. Harmon, R. Fattal, M. Bercovier, and E. Grinspun. Efficient simulation of inextensible cloth. ACM Trans. Graphics (Proc. SIGGRAPH), 26(3):to appear, 2007
10. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D.Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS), pages 2672-2680, 2014
11. K. Gregor, I. Danihelka, A. Graves, D. J. Rezende, and D. Wierstra. Draw: A recurrent neural network for image generation. ArXiv preprint arXiv:1502.04623, 2015
12. P. Guan, O. Freifeld, and M. J. Black. A 2d human body model dressed in eigen clothing. In Proc. European Conf. on Computer Vision, pages 285-298. Springer, 2010
13. P. Guan, L. Reiss, D. Hirshberg, A. Weiss, and M. J. Black. DRAPE: DRessing Any PErson. ACM Trans. Graphics (Proc. SIGGRAPH), 31(4):35:1-35:10, July 2012
14. E. Insafutdinov, L. Pishchulin, B. Andres, M. Andriluka, and B. Schiele. Deepercut: A deeper, stronger, and faster multiperson pose estimation model. In Proc. European Conf. on Computer Vision (ECCV), pages 34-50. Springer, 2016
15. S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ArXiv preprint arXiv:1502.03167, 2015
16. C. Ionescu, D. Papava, V. Olaru, and C. Sminchisescu. Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI), 2014
17. P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros. Image-toimage translation with conditional adversarial networks. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2016
18. A. Jain, T. Thormählen, H.-P. Seidel, and C. Theobalt. Moviereshape: Tracking and reshaping of humans in videos. ACM Trans. Graphics (Proc. SIGGRAPH), 29(6):148:1-148:10, Dec. 2010
19. S. Johnson and M. Everingham. Clustered pose and nonlinear appearance models for human pose estimation. In Proc. British Machine Vision Conf. (BMVC), 2010. doi:10.5244/C.24.12
20. L. Kavan, D. Gerszewski, A. W. Bargteil, and P.-P. Sloan. Physics-inspired upsampling for cloth simulation in games. ACM Trans. Graphics (Proc. SIGGRAPH), 30(4):93:1-93:10, July 2011
21. V. Kazemi and J. Sullivan. One millisecond face alignment with an ensemble of regression trees. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 1867-1874, 2014
22. D. Kim, W. Koh, R. Narain, K. Fatahalian, A. Treuille, and J. F. O'Brien. Near-exhaustive precomputation of secondary cloth effects. ACM Trans. Graphics (Proc. SIGGRAPH), 32(4):87:1-7, July 2013
23. D. E. King. Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research (JMLR), 10:1755-1758, 2009
24. D. P. Kingma, S. Mohamed, D. J. Rezende, and M. Welling. Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems (NIPS), pages 3581-3589, 2014
25. D. P. Kingma and M. Welling. Auto-encoding variational bayes. ArXiv preprint arXiv:1312.6114, 2013
26. T. D. Kulkarni, W. F. Whitney, P. Kohli, and J. Tenenbaum. Deep convolutional inverse graphics network. In Advances in Neural Information Processing Systems (NIPS), pages 2539-2547, 2015
27. X. Liang, C. Xu, X. Shen, J. Yang, S. Liu, J. Tang, L. Lin, and S. Yan. Human parsing with contextualized convolutional neural network. In Proc. IEEE International Conf. on Computer Vision (ICCV), pages 1386-1394, 2015
28. M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black. Smpl: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6):248:1-248:16, Oct. 2015
29. A. L. Maas, A. Y. Hannun, and A. Y. Ng. Rectifier nonlinearities improve neural network acoustic models. In Proc. International Conf. on Machine Learning (ICML), volume 30, 2013
30. R. Narain, A. Samii, and J. F. O'Brien. Adaptive anisotropic remeshing for cloth simulation. ACM Trans. Graphics (Proc. SIGGRAPH), 31(6):147:1-10, Nov. 2012
31. M. Oberweger, P. Wohlhart, and V. Lepetit. Training a feedback loop for hand pose estimation. In Proc. IEEE International Conf. on Computer Vision (ICCV), pages 3316-3324, 2015
32. A. v. d. Oord, N. Kalchbrenner, and K. Kavukcuoglu. Pixel recurrent neural networks. In Proc. International Conf. on Machine Learning (ICML), 2016
33. D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros. Context encoders: Feature learning by inpainting. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 2536-2544, 2016
34. L. Pishchulin, A. Jain, M. Andriluka, T. Thormählen, and B. Schiele. Articulated people detection and pose estimation: Reshaping the future. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 3178-3185. IEEE Press, 2012
35. L. Pishchulin, A. Jain, C. Wojek, M. Andriluka, T. Thormählen, and B. Schiele. Learning people detection models from few training samples. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2011
36. G. Pons-Moll, S. Pujades, S. Hu, and M. Black. ClothCap: Seamless 4D clothing capture and retargeting. ACM Trans. Graphics (Proc. SIGGRAPH), 36(4), 2017
37. G. Pons-Moll, J. Taylor, J. Shotton, A. Hertzmann, and A. Fitzgibbon. Metric regression forests for human pose estimation. In Proc. British Machine Vision Conf. (BMVC). BMVA Press, Sept. 2013
38. G. Rogez and C. Schmid. Mocap-guided data augmentation for 3d pose estimation in the wild. In Advances in Neural Information Processing Systems (NIPS), pages 3108-3116, 2016
39. L. Rogge, F. Klose, M. Stengel, M. Eisemann, and M. Magnor. Garment replacement in monocular video sequences. ACM Trans. Graphics (Proc. SIGGRAPH), 34(1):6:1-6:10, Nov. 2014
40. O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In Proc. International Conf. on Medical Image Computing and Computer-Assisted Intervention, pages 234-241. Springer, 2015
41. S. Saito, L. Wei, L. Hu, K. Nagano, and H. Li. Photorealistic facial texture inference using deep neural networks. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2017
42. J. Shotton, T. Sharp, A. Kipman, A. Fitzgibbon, M. Finocchio, A. Blake, M. Cook, and R. Moore. Real-time human pose recognition in parts from single depth images. Communications of the ACM, 56(1):116-124, 2013
43. K. Sohn, H. Lee, and X. Yan. Learning structured output representation using deep conditional generative models. In Advances in Neural Information Processing Systems (NIPS), pages 3483-3491, 2015
44. C. Stoll, J. Gall, E. de Aguiar, S. Thrun, and C. Theobalt. Video-based reconstruction of animatable human characters. ACM Trans. Graphics (Proc. SIGGRAPH), 29(6):139:1-139:10, Dec. 2010
45. J. Taylor, J. Shotton, T. Sharp, and A. Fitzgibbon. The vitruvian manifold: Inferring dense correspondences for oneshot human pose estimation. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 103-110. IEEE, 2012
46. A. van den Oord, N. Kalchbrenner, L. Espeholt, O. Vinyals, A. Graves, et al. Conditional image generation with pixelcnn decoders. In Advances in Neural Information Processing Systems (NIPS), pages 4790-4798, 2016
47. G. Varol, J. Romero, X. Martin, N. Mahmood, M. Black, I. Laptev, and C. Schmid. Learning from synthetic humans. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2017
48. D. Vázquez, A. M. Lopez, J. Marin, D. Ponsa, and D. Geronimo. Virtual and real world adaptation for pedestrian detection. IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI), 36(4):797-809, 2014
49. H. Wang, J. F. O'Brien, and R. Ramamoorthi. Data-driven elastic models for cloth: Modeling and measurement. ACM Trans. Graphics (Proc. SIGGRAPH), 30(4):71:1-11, July 2011
50. F. Xu, Y. Liu, C. Stoll, J. Tompkin, G. Bharaj, Q. Dai, H.-P. Seidel, J. Kautz, and C. Theobalt. Video-based characters: Creating new human performances from a multi-view video database. ACM Trans. Graphics (Proc. SIGGRAPH, 30(4):32:1-32:10, July 2011
51. X. Yan, J. Yang, K. Sohn, and H. Lee. Attribute2image: Conditional image generation from visual attributes. In Proc. European Conf. on Computer Vision (ECCV), pages 776-791. Springer, 2016
52. C. Yang, X. Lu, Z. Lin, E. Shechtman, O. Wang, and H. Li. High-resolution image inpainting using multi-scale neural patch synthesis. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2017
53. F. Yu, Y. Zhang, S. Song, A. Seff, and J. Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. ArXiv preprint arXiv:1506.03365, 2015
54. M. D. Zeiler, G. W. Taylor, and R. Fergus. Adaptive deconvolutional networks for mid and high level feature learning. In Proc. IEEE International Conf. on Computer Vision (ICCV), pages 2018-2025. IEEE, 2011
55. C. Zhang, S. Pujades, M. Black, and G. Pons-Moll. Detailed, accurate, human shape estimation from clothed 3D scan sequences. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2017
56. S. Zhou, H. Fu, L. Liu, D. Cohen-Or, and X. Han. Parametric reshaping of human bodies in images. ACM Trans. Graphics (Proc. SIGGRAPH), 29(4):126, 2010.