Transformation-grounded image generation network for novel 3D view synthesis

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 702-711

  Park, Eunbyung ^a   Yang, Jimei ^b   Yumer, Ersin ^b   Ceylan, Duygu ^b   Berg, Alexander C ^a  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

^b ADOBE RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; PATTERN RECOGNITION; THREE DIMENSIONAL COMPUTER GRAPHICS;

HIGH FREQUENCY HF; IMAGE COMPLETION; IMAGE GENERATIONS; INTERMEDIATE RESULTS; NETWORK STRUCTURES; NOVEL VIEW SYNTHESIS; QUANTITATIVE RESULT; SYNTHESIS PROBLEMS;

IMAGE PROCESSING;

EID: 85041911383     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.82     Document Type: Conference Paper

References (51)

1. openmvs: open multi-view stereo reconstruction library. https://github.com/cdcseacave/openMVS. Accessed: 2016-11-14. 7
2. C. Barnes, E. Shechtman, A. Finkelstein, and D. B. Goldman. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Computer Graphics (TOG), 2009. 5, 8
3. B. D. Brabandere, X. Jia, T. Tuytelaars, and L. V. Gool. Dynamic filter networks. In NIPS, 2016. 8
4. A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu. ShapeNet: An Information-Rich 3D Model Repository. arXiv:1512.03012, 2015. 2, 6
5. C. B. Choy, D. Xu, J. Gwak, K. Chen, and S. Savarese. 3dr2n2: A unified approach for single and multi-view 3d object reconstruction. In ECCV, 2016. 7
6. A. Dosovitskiy and T. Brox. Generating images with perceptual similarity metrics based on deep networks. In NIPS, 2016. 2, 5
7. A. Dosovitskiy, P. Fischery, E. Ilg, C. Hazirbas, V. Golkov, P. van der Smagt, D. Cremers, T. Brox, et al. Flownet: Learning optical flow with convolutional networks. In ICCV, 2015. 1
8. A. Dosovitskiy, J. T. Springenberg, and T. Brox. Learning to generate chairs with convolutional neural networks. In CVPR, 2015. 3
9. A. A. Efros and W. T. Freeman. Image quilting for texture synthesis and transfer. 28th Annual Conference on Computer Graphics and Interactive Techniques, 2001. 5
10. C. Finn, I. Goodfellow, and S. Levine. Unsupervised learning for physical interaction through video prediction. In NIPS, 2016. 8
11. J. Flynn, I. Neulander, J. Philbin, and N. Snavely. Deepstereo: Learning to predict new views from the world's imagery. In CVPR, 2016. 2
12. Y. Furukawa. Multi-view stereo: A tutorial. Foundations and Trends in Computer Graphics and Vision, 2015. 2, 7
13. R. Garg, V. K. BG, G. Carneiro, and I. Reid. Unsupervised cnn for single view depth estimation: Geometry to the rescue. In ECCV, 2016. 1, 2
14. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D.Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, 2014. 2, 3, 5
15. K. He, Y. Wang, and J. Hopcroft. A powerful generative model using random weights for the deep image representation. In NIPS, 2016. 5
16. G. E. Hinton, A. Krizhevsky, and S. D. Wang. Transforming auto-encoders. In International Conference on Artificial Neural Networks, 2011. 3
17. D. Hoiem, A. A. Efros, and M. Hebert. Automatic photo pop-up. ACM Transactions on Computer Graphics (TOG), 2005. 1
18. M. Jaderberg, K. Simonyan, A. Zisserman, and K. Kavukcuoglu. Spatial transformer networks. In NIPS, 2015. 3, 4
19. D. Ji, J. Kwon, M. McFarland, and S. Savarese. Deep view morphing. In CVPR, 2017. 2
20. J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In ECCV, 2016. 1, 2, 3, 5
21. S. B. Kang and H.-Y. Shum. A review of image-based rendering techniques. 2000. 1
22. N. Kholgade, T. Simon, A. Efros, and Y. Sheikh. 3d object manipulation in a single photograph using stock 3d models. ACM Transactions on Computer Graphics (TOG), 2014. 1, 3
23. T. D. Kulkarni, W. F. Whitney, P. Kohli, and J. B. Tenenbaum. Deep convolutional inverse graphics network. In NIPS, 2015. 3
24. H. Kwak and B.-T. Zhang. Generating images part by part with composite generative adversarial networks. arXiv:1607.05387, 2016. 3
25. A. Lamb, V. Dumoulin, and A. Courville. Discriminative regularization for generative models. arXiv:1602.03220, 2016. 2, 3
26. A. B. L. Larsen, S. K. Snderby, H. Larochelle, and Ole Winther. Autoencoding beyond pixels using a learned similarity metric. In ICML, 2016. 2, 3, 5
27. S. Lee, S. Purushwalkam, M. Cogswell, V. Ranjan, D. Crandall, and D. Batra. Stochastic multiple choice learning for training diverse deep ensembles. In NIPS, 2016. 8
28. M. Mathieu, C. Couprie, and Y. LeCun. Deep multi-scale video prediction beyond mean square error. In ICLR, 2016. 6, 8
29. M. Mirza and S. Osindero. Conditional generative adversarial nets. arXiv:1411.1784, 2016. 8
30. A. Newell, K. Yang, and J. Deng. Stacked hourglass networks for human pose estimation. In ECCV, 2016. 5
31. H. Noh, S. Hong, and B. Han. Learning deconvolution network for semantic segmentation. 2015. 1
32. D. Pathak, P. Krähenbühl, J. Donahue, T. Darrell, and A. A. Efros. Context encoders: Feature learning by inpainting deepak. In CVPR, 2016. 3
33. A. Radford, L. Metz, and S. Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In ICLR, 2016. 2, 3, 5
34. K. Rematas, C. Nguyen, T. Ritschel, M. Fritz, and T. Tuytelaars. Novel views of objects from a single image. arXiv:1602.00328, 2016. 1, 3
35. T. Salimans, I. Goodfellow,W. Zaremba, V. Cheung, A. Radford, and X. chen. Improved techniques for training gans. In NIPS, 2016. 2, 5
36. H. Su, C. R. Qi, Y. Li, and L. J. Guibas. Render for cnn: Viewpoint estimation in images using cnns trained with rendered 3d model views. In ICCV, 2015. 7
37. M. Tatarchenko, A. Dosovitskiy, and T. Brox. Multi-view 3d models from single images with a convolutional network. In ECCV, 2016. 1, 2, 3, 6, 7
38. D. Ulyanov, V. Lebedev, A. Vedaldi, and V. Lempitsky. Texture networks: Feed-forward synthesis of textures and stylized images. In ICML, 2016. 2, 3, 5
39. I. Ustyuzhaninov, W. Brendel, L. Gatys, and M. Bethge. Texture synthesis using shallow convolutional networks with random filters. arXiv:1606.00021, 2016. 5
40. A. van den Oord, N. Kalchbrenner, and K. Kavukcuoglu. Pixel recurrent neural networks. In ICML, 2016. 8
41. J. Varley, C. DeChant, A. Richardson, A. Nair, J. Ruales, and P. Allen. Shape completion enabled robotic grasping. arXiv:1609.08546, 2016. 1
42. C. Vondrick, H. Pirsiavash, and A. Torralba. Generating videos with scene dynamics. In NIPS, 2016. 8
43. J. Walker, C. Doersch, A. Gupta, and M. Hebert. An uncertain future: Forecasting from static images using variational autoencoders. In ECCV, 2016. 8
44. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600-612, 2004. 6
45. Y. Wexler, E. Shechtman, and M. Irani. Space-time completion of video. TPAMI, 2007. 5
46. J. Wu, C. Zhang, T. Xue, W. T. Freeman, and J. B. Tenenbaum. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. In NIPS, 2016. 7
47. T. Xue, J. Wu, K. L. Bouman, and W. T. Freeman. Visual dynamics: Probabilistic future frame synthesis via cross convolutional networks. In NIPS, 2016. 8
48. X. Yan, J. Y. K. Sohn, and H. Lee. Attribute2image: Conditional image generation from visual attributes. In ECCV, 2016. 8
49. J. Yang, S. Reed, M.-H. Yang, and H. Lee. Weaklysupervised disentangling with recurrent transformations for 3d view synthesis. In NIPS, 2015. 1, 3
50. M. E. Yumer and N. J. Mitra. Learning semantic deformation flows with 3d convolutional networks. In ECCV, 2016. 3
51. T. Zhou, S. Tulsiani,W. Sun, J. Malik, and A. A. Efros. View synthesis by appearance flow. In ECCV, 2016. 2, 3, 4, 6