Single image 3D interpreter network

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 9910 LNCS, Issue , 2016, Pages 365-382

  Wu, Jiajun ^a   Xue, Tianfan ^a   Lim, Joseph J ^a,b   Tian, Yuandong ^c   Tenenbaum, Joshua B ^a   Torralba, Antonio ^a   Freeman, William T ^a,d  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b STANFORD UNIVERSITY   (United States)

^c FACEBOOK AI RESEARCH   (United States)

^d GOOGLE INC   (United States)

Author keywords

3D structure; Keypoint estimation; Neural network; Single image 3D reconstruction; Synthetic data

Indexed keywords

COMPUTER VISION; IMAGE RETRIEVAL; NEURAL NETWORKS;

3D RECONSTRUCTION; 3D STRUCTURE; INTERMEDIATE REPRESENTATIONS; KEYPOINT; STATE-OF-THE-ART PERFORMANCE; STRUCTURAL PARAMETER; SYNTHETIC DATA; VISION APPLICATIONS;

THREE DIMENSIONAL COMPUTER GRAPHICS;

EID: 84990046297     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-319-46466-4_22     Document Type: Conference Paper

References (62)

1. Akhter, I., Black, M.J.: Pose-conditioned joint angle limits for 3d human pose reconstruction. In: CVPR (2015)
2. Aubry, M., Maturana, D., Efros, A., Russell, B., Sivic, J.: Seeing 3d chairs: exemplar part-based 2d–3d alignment using a large dataset of cad models. In: CVPR (2014)
3. Bansal, A., Russell, B.: Marr revisited: 2d–3d alignment via surface normal prediction. In: CVPR (2016)
4. Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. IEEE TPAMI 35(12), 2930–2940 (2013)
5. Bever, T.G., Poeppel, D.: Analysis by synthesis: a (re-) emerging program of research for language and vision. Biolinguistics 4(23), 174–200 (2010)
6. Bourdev, L., Maji, S., Brox, T., Malik, J.: Detecting people using mutually consistent poselet activations. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 168–181. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15567-3_13
7. Carreira, J., Agrawal, P., Fragkiadaki, K., Malik, J.: Human pose estimation with iterative error feedback. In: CVPR (2016)
8. Choy, C.B., Xu, D., Gwak, J., Chen, K., Savarese, S.: 3d–r2n2: a unified approach for single and multi-view 3D object reconstruction. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2006, Part VIII. LNCS, vol. 9912, pp. 1–17. Springer, Heidelberg (2016)
9. Dosovitskiy, A., Tobias Springenberg, J., Brox, T.: Learning to generate chairs with convolutional neural networks. In: CVPR (2015)
10. Fidler, S., Dickinson, S.J., Urtasun, R.: 3d object detection and viewpoint estimation with a deformable 3d cuboid model. In: NIPS (2012)
11. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR (2014)
12. Hejrati, M., Ramanan, D.: Analysis by synthesis: 3d object recognition by object reconstruction. In: CVPR (2014)
13. Hejrati, M., Ramanan, D.: Analyzing 3d objects in cluttered images. In: NIPS (2012)
14. Hinton, G.E., Ghahramani, Z.: Generative models for discovering sparse distributed representations. Philos. Trans. R. Soc. London B: Biol. Sci. 352(1358), 1177–1190 (1997)
15. Hu, W., Zhu, S.C.: Learning 3d object templates by quantizing geometry and appearance spaces. IEEE TPAMI 37(6), 1190–1205 (2015)
16. Huang, Q., Wang, H., Koltun, V.: Single-view reconstruction via joint analysis of image and shape collections. ACM SIGGRAPH 34(4), 87 (2015)
17. Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. In: NIPS (2015)
18. Kar, A., Tulsiani, S., Carreira, J., Malik, J.: Category-specific object reconstruction from a single image. In: CVPR (2015)
19. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)
20. Kulkarni, T.D., Kohli, P., Tenenbaum, J.B., Mansinghka, V.: Picture: a probabilistic programming language for scene perception. In: CVPR (2015)
21. Kulkarni, T.D., Whitney, W.F., Kohli, P., Tenenbaum, J.B.: Deep convolutional inverse graphics network. In: NIPS (2015)
22. Leclerc, Y.G., Fischler, M.A.: An optimization-based approach to the interpretation of single line drawings as 3d wire frames. IJCV 9(2), 113–136 (1992)
23. Li, Y., Su, H., Qi, C.R., Fish, N., Cohen-Or, D., Guibas, L.J.: Joint embeddings of shapes and images via cnn image purification. ACM SIGGRAPH Asia 34(6), 234 (2015)
24. Lim, J.J., Khosla, A., Torralba, A.: FPM: fine pose parts-based model with 3D CAD models. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 478–493. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10599-4_31
25. Lim, J.J., Pirsiavash, H., Torralba, A.: Parsing ikea objects: fine pose estimation. In: ICCV (2013)
26. Liu, J., Belhumeur, P.N.: Bird part localization using exemplar-based models with enforced pose and subcategory consistency. In: ICCV (2013)
27. Lowe, D.G.: Three-dimensional object recognition from single two-dimensional images. Artif. Intell. 31(3), 355–395 (1987). Elsevier
28. Van der Maaten, L., Hinton, G.: Visualizing data using t-sne. JMLR 9(11), 2579–2605 (2008)
29. Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. arXiv preprint arXiv:1603.06937 (2016)
30. Peng, X., Sun, B., Ali, K., Saenko, K.: Exploring invariances in deep convolutional neural networks using synthetic images. CoRR, abs/1412.7122 2 (2014)
31. Pepik, B., Stark, M., Gehler, P., Schiele, B.: Teaching 3d geometry to deformable part models. In: CVPR (2012)
32. Prasad, M., Fitzgibbon, A., Zisserman, A., Van Gool, L.: Finding nemo: deformable object class modelling using curve matching. In: CVPR (2010)
33. Ramakrishna, V., Kanade, T., Sheikh, Y.: Reconstructing 3D human pose from 2D image landmarks. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7575, pp. 573–586. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33765-9_41
34. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: NIPS (2015)
35. Sapp, B., Taskar, B.: Modec: multimodal decomposable models for human pose estimation. In: CVPR (2013)
36. Satkin, S., Lin, J., Hebert, M.: Data-driven scene understanding from 3D models. In: BMVC (2012)
37. Shakhnarovich, G., Viola, P., Darrell, T.: Fast pose estimation with parametersensitive hashing. In: ICCV (2003)
38. Shih, K.J., Mallya, A., Singh, S., Hoiem, D.: Part localization using multi-proposal consensus for fine-grained categorization. In: BMVC (2015)
39. Shrivastava, A., Gupta, A.: Building part-based object detectors via 3d geometry. In: ICCV, pp. 1745–1752 (2013)
40. Su, H., Huang, Q., Mitra, N.J., Li, Y., Guibas, L.: Estimating image depth using shape collections. ACM TOG 33(4), 37 (2014)
41. Su, H., Qi, C.R., Li, Y., Guibas, L.: Render for cnn: viewpoint estimation in images using cnns trained with rendered 3d model views. In: ICCV (2015)
42. Sun, B., Saenko, K.: From virtual to reality: fast adaptation of virtual object detectors to real domains. In: BMVC (2014)
43. Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Web-scale training for face identification. In: CVPR (2015)
44. Tompson, J., Goroshin, R., Jain, A., LeCun, Y., Bregler, C.: Efficient object localization using convolutional networks. In: CVPR (2015)
45. Tompson, J.J., Jain, A., LeCun, Y., Bregler, C.: Joint training of a convolutional network and a graphical model for human pose estimation. In: NIPS (2014)
46. Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: CVPR (2011)
47. Torresani, L., Hertzmann, A., Bregler, C.: Learning non-rigid 3d shape from 2d motion. In: NIPS (2003)
48. Toshev, A., Szegedy, C.: Deeppose: human pose estimation via deep neural networks. In: CVPR, pp. 1653–1660 (2014)
49. Tulsiani, S., Malik, J.: Viewpoints and keypoints. In: CVPR (2015)
50. Vicente, S., Carreira, J., Agapito, L., Batista, J.: Reconstructing pascal voc. In: CVPR (2014)
51. Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 Dataset. Technical report. CNS-TR-2011-001, California Institute of Technology (2011)
52. Wu, J., Yildirim, I., Lim, J.J., Freeman, B., Tenenbaum, J.: Galileo: perceiving physical object properties by integrating a physics engine with deep learning. In: NIPS (2015)
53. Xiang, Y., Mottaghi, R., Savarese, S.: Beyond pascal: a benchmark for 3d object detection in the wild. In: WACV (2014)
54. Xiao, J., Hays, J., Ehinger, K., Oliva, A., Torralba, A.: Sun database: large-scale scene recognition from abbey to zoo. In: CVPR (2010)
55. Xue, T., Liu, J., Tang, X.: Example-based 3d object reconstruction from line drawings. In: CVPR (2012)
56. Yang, Y., Ramanan, D.: Articulated pose estimation with flexible mixtures-ofparts. In: CVPR (2011)
57. Yasin, H., Iqbal, U., Krüger, B., Weber, A., Gall, J.: A dual-source approach for 3d pose estimation from a single image. In: CVPR (2016)
58. Yuille, A., Kersten, D.: Vision as bayesian inference: analysis by synthesis? Trends Cogn. Sci. 10(7), 301–308 (2006)
59. Zeng, A., Song, S., Nießner, M., Fisher, M., Xiao, J.: 3dmatch: learning the matching of local 3d geometry in range scans. arXiv preprint arXiv:1603.08182 (2016)
60. Zhou, T., Krähenbühl, P., Aubry, M., Huang, Q., Efros, A.A.: Learning dense correspondence via 3d-guided cycle consistency. In: CVPR (2016)
61. Zhou, X., Leonardos, S., Hu, X., Daniilidis, K.: 3d shape reconstruction from 2d landmarks: a convex formulation. In: CVPR (2015)
62. Zia, M.Z., Stark, M., Schiele, B., Schindler, K.: Detailed 3d representations for object recognition and modeling. IEEE TPAMI 35(11), 2608–2623 (2013)