DeepGarment : 3D Garment Shape Estimation from a Single Image

Computer Graphics Forum

Volumn 36, Issue 2, 2017, Pages 269-280

  Danerek R ^a,b   Dibra, E ^a,b   Oztireli C ^a   Ziegler, R ^b   Gross, M ^a  

^a ETH ZURICH   (Switzerland)

^b Vizrt Switzerland   (Switzerland)

Author keywords

Categories and Subject Descriptors (according to ACM CCS); I.3.5 Gomputer Graphics : Computational Geometry and Object Modeling ; I.3.7 Computer Graphics : Three Dimensional Graphics and Realism

Indexed keywords

CAMERAS; COMPLEX NETWORKS; COMPUTATIONAL GEOMETRY; COMPUTER GRAPHICS; LIGHTING; MESH NETWORKING; NEURAL NETWORKS; RENDERING (COMPUTER GRAPHICS);

COMPUTATIONAL GEOMETRY AND OBJECT MODELING; CONVOLUTIONAL NEURAL NETWORK; DESCRIPTORS; FREE-VIEWPOINT VIDEO; I.3.7 [COMPUTER GRAPHICS]: THREE-DIMENSIONAL GRAPHICS AND REALISM; LIGHTING CONDITIONS; MANUAL INTERACTION; PHYSICALLY-BASED SIMULATION;

THREE DIMENSIONAL COMPUTER GRAPHICS;

EID: 85019746731     PISSN: 01677055     EISSN: 14678659     Source Type: Journal    
DOI: 10.1111/cgf.13125     Document Type: Article

References (52)

1. Alexandros Neophytou A.H.: A layered model of human body and garment deformation. 3
2. Anguelov D., Srinivasan P., Koller D., Thrun S., Rodgers J., Davis J.: Scape: Shape completion and animation of people. In ACM SIGGRAPH 2005 Papers (New York, NY, USA, 2005), SIGGRAPH '05, ACM, pp. 408–416. 2, 3
3. Bălan A.O., Black M.J.: The naked truth: Estimating body shape under clothing. In Proceedings of the 10th European Conference on Computer Vision: Part II (Berlin, Heidelberg, 2008), ECCV'08, Springer-Verlag, pp. 15–29. 2
4. Bérard P., Bradley D., Gross M., Beeler T.: Lightweight eye capture using a parametric model. ACM Trans. Graph. 35, 4 (2016). 1
5. Boykov Y., Funka-Lea G.: Graph cuts and efficient n-d image segmentation. Int. J. Comput. Vision 70, 2 (Nov. 2006), 109–131. 4
6. Badrinarayanan V., Kendall A., Cipolla R.: Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv preprint arXiv:1511.00561 (2015). 4
7. Baran I., Popović J.: Automatic rigging and animation of 3d characters. In ACM SIGGRAPH 2007 Papers (New York, NY, USA, 2007), SIGGRAPH '07, SACM. 3
8. Bradley D., Popa T., Sheffer A., Heidrich W., Boubekeur T.: Markerless garment capture. ACM Trans. Graphics (Proc. SIGGRAPH) 27, 3 (2008), 99. 2
9. Cao C., Bradley D., Zhou K., Beeler T.: Real-time high-fidelity facial performance capture. ACM Trans. Graph. 34, 4 (July 2015), 46:1–46:9. 1
10. Chen Y., Kim T.-K., Cipolla R.: Inferring 3d shapes and deformations from single views. In ECCV (3) (2010), Daniilidis K., Maragos P., Paragios N., (Eds.), vol. 6313 of Lecture Notes in Computer Science, Springer, pp. 300–313. 3
11. Cmu: Carnegie-Mellon Mocap Database. 3
12. Chen X., Zhou B., Lu F., Wang L., Bi L., Tan P.: Garment modeling with a depth camera. ACM Trans. Graph. 34, 6 (oct. 2015), 203:1–203:12. 2, 3
13. Dibra E., Jain H., Oztireli C., Ziegler R., Gross M.: Hs-nets : Estimating human body shape from silhouettes with convolutional neural networks. In Int. Conf. on 3D Vision (October 2016). 1, 2, 3, 6
14. Dibra E., Oztireli C., Ziegler R., Gross M.: Shape from selfies: Human body shape estimation using cca regression forests. In Proceedings of the 14th European Conference on Computer Vision: Part IV (2016), ECCV '16. 1, 3
15. Fischer P., Dosovitskiy A., Ilg E., Häusser P., Hazirbas C., Golkov V., van der Smagt P., Cremers D., Brox T.: Flownet: Learning optical flow with convolutional networks. CoRR abs/1504.06852 (2015). 3
16. Guan P., Freifeld O., Black M.J.: A 2d human body model dressed in eigen clothing. In Proceedings of the 11th European Conference on Computer Vision: Part I (Berlin, Heidelberg, 2010), ECCV'10, Springer-Verlag, pp. 285–298. 3
17. Guskov I., Klibanov S., Bryant B.: Trackable surfaces. In SCA (2003). 2
18. Guan P., Reiss L., Hirshberg D.A., Weiss A., Black M.J.: Drape: Dressing any person. ACM Trans. Graph. 31, 4 (July 2012), 35:1–35:10. 2, 3, 6
19. Hu L., Ma C., Luo L., Li H.: Single-view hair modeling using a hairstyle database. ACM Transactions on Graphics (Proceedings SIGGRAPH 2015) 34, 4 (July 2015). 1
20. Hahn F., Thomaszewski B., Coros S., Sumner R.W., Cole F., Meyer M., DeRose T., Gross M.: Subspace clothing simulation using adaptive bases. ACM Trans. Graph. 33, 4 (July 2014), 105:1–105:9. 2, 3
21. HE K., Zhang X., Ren S., Sun J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015). 3
22. Iandola F.N., Moskewicz M.W., Ashraf K., Han S., Dally W.J., Keutzer K.: Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <1mb model size. CoRR abs/1602.07360 (2016). 3, 5
23. Jakob W.: Mitsuba renderer, 2010. http://www.mitsubarenderer.org. 4
24. Jeong M.-H., Han D.-H., Ko H.-S.: Garment capture from a photograph. Comput. Animat. Virtual Worlds 26, 3–4 (May 2015), 291–300. 2, 3
25. Kavan L., Collins S., Žára J., O'Sullivan C.: Geometric skinning with approximate dual quaternion blending. ACM Trans. Graph. 27, 4 (Nov. 2008), 105:1–105:23. 3
26. Krizhevsky A., Sutskever I., Hinton G.E.: Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held December 3–6, 2012, Lake Tahoe, Nevada, United States. (2012), pp. 1106–1114. 3, 5
27. Liu Z., Luo P., Qiu S., Wang X., Tang X.: Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016). 3
28. MD: Marvelous designer. 3
29. Narain R., Pfaff T., O'Brien J.F.: Folding and crumpling adaptive sheets. ACM Transactions on Graphics 32, 4 (July 2013), 51:1–8. Proceedings of ACM SIGGRAPH 2013, Anaheim. 3
30. Narain R., Samii A., O'Brien J.F.: Adaptive anisotropic remeshing for cloth simulation. ACM Transactions on Graphics 31, 6 (Nov. 2012), 147:1–10. Proceedings of ACM SIGGRAPH Asia 2012, Singapore. 3
31. Pritchard D., Heidrich W.: Cloth motion capture, 2003. 2
32. Pishchulin L., Wuhrer S., Helten T., Theobalt C., Schiele B.: Building statistical shape spaces for 3d human modeling. CoRR abs/1503.05860 (2015). 3
33. Popa T., Zhou Q., Bradley D., Kraevoy V., Fu H., Sheffer A., Heidrich W.: Wrinkling captured garments using space-time data-driven deformation. Computer Graphics Forum (Proc. Eurographics) 28, 2 (2009), 427–435. 2
34. Robson C., Maharik R., Sheffer A., Carr N.: Context-aware garment modeling from sketches. Computers and Graphics (Proc. SMI 2011) (2011), 604–613. 2
35. Sigal L., Balan A.O., Black M.J.: Combined discriminative and generative articulated pose and non-rigid shape estimation. In NIPS (2007), Platt J.C., Koller D., Singer Y., Roweis S.T., (Eds.), Curran Associates, Inc. 3
36. Sermanet P., Eigen D., Zhang X., Mathieu M., Fergus R., LeCun Y.: Overfeat: Integrated recognition, localization and detection using convolutional networks. CoRR abs/1312.6229 (2013). 3
37. Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V., Rabinovich A.: Going deeper with convolutions. In Computer Vision and Pattern Recognition (CVPR) (2015). 3
38. Scholz V., Magnor M.A.: Cloth motion from optical flow. In VMV (2004). 2
39. Su H., Maji S., Kalogerakis E., Learned-Miller E.G.: Multi-view convolutional neural networks for 3d shape recognition. In Proc. ICCV (2015). 3
40. Scholz V., Stich T., Keckeisen M., Wacker M., Magnor M.: Garment motion capture using color-coded patterns. Computer Graphics forum 24, 3 (September 2005), 439–448. Conference Issue: 26th annual Conference Eurographics 2005, Dublin, Ireland, August 29th - September 2nd, 2005. 2
41. Sekine M., Sugita K., Perbet F., Stenger B., Nishiyama M.: Virtual fitting by single-shot body shape estimation. In Int. Conf. on 3D Body Scanning Technologies (October 2014), pp. 406–413. 3
42. Simonyan K., Zisserman A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014). 3
43. Tatarchenko M., Dosovitskiy A., Brox T.: Multi-view 3d models from single images with a convolutional network. In Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII (2016), pp. 322–337. 2
44. Tkach A., Pauly M., Tagliasacchi A.: Sphere-meshes for real-time hand modeling and tracking. ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia) (2016). 1
45. Toshev A., Szegedy C.: Deeppose: Human pose estimation via deep neural networks. CoRR abs/1312.4659 (2013). 3
46. White R., Crane K., Forsyth D.: Capturing and animating occluded cloth. In ACM Transactions on Graphics (SIGGRAPH) (2007). 2
47. Wang F., Kang L., Li Y.: Sketch-based 3d shape retrieval using convolutional neural networks. CoRR abs/1504.03504 (2015). 3
48. Wu Z., Song S., Khosla A., Yu F., Zhang L., Tang X., Xiao J.: 3d shapenets: A deep representation for volumetric shapes. In CVPR (2015), IEEE Computer Society, pp. 1912–1920. 2, 3
49. Yang S., Ambert T., Pan Z., Wang K., Yu L., Berg T.L., Lin M.C.: Detailed garment recovery from a single-view image. CoRR abs/1608.01250 (2016). 2, 3, 4
50. Yamaguchi K., Hadi Kiapour M., Berg T.L.: Paper doll parsing: Retrieving similar styles to parse clothing items. In The IEEE International Conference on Computer Vision (ICCV) (December 2013). 4
51. Yi K.M., Trulls E., Lepetit V., Fua P.: LIFT: learned invariant feature transform. CoRR abs/1603.09114 (2016). 3
52. Zhou B., Chen X., Fu Q., Guo K., Tan P.: Garment modeling from a single image. Pacific Graphics (2013). 2