What is it like down there? Generating dense ground-level views and image features from overhead imagery using conditional generative adversarial networks

GIS: Proceedings of the ACM International Symposium on Advances in Geographic Information Systems

Volumn , Issue , 2018, Pages 43-52

  Deng, Xueqing ^a   Zhu, Yi ^a   Newsam, Shawn ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Computer vision; Generative adversarial networks; Geotagged social media; Land cover classification

Indexed keywords

COMPUTER VISION; INFORMATION SYSTEMS; INFORMATION USE;

ADVERSARIAL NETWORKS; FEATURE MAP; FUNDAMENTAL LIMITATIONS; GROUND LEVEL; IMAGE FEATURES; LAND COVER CLASSIFICATION; REAL IMAGES; SOCIAL MEDIA;

GEOGRAPHIC INFORMATION SYSTEMS;

EID: 85058646108     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/3274895.3274969     Document Type: Conference Paper

References (41)

1. M. Arjovsky, S. Chintala, and L. Bottou. 2017. Wasserstein Generative Adversarial Networks. In Proceedings of the 34th International Conference on Machine Learning, Doina Precup and Yee Whye Teh (Eds.), Vol. 70. PMLR, International Convention Centre, Sydney, Australia, 214–223.
2. D. Berthelot, T. Schumm, and L. Metz. 2017. BEGAN: Boundary Equilibrium Generative Adversarial Networks. ArXiv e-prints (March 2017). arXiv:cs.LG/1703.10717
3. K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, and D. Krishnan Krishnan. 2017. Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 95–104.
4. Y. Chen, C. Shen, X.-S. Wei, L. Liu, and J. Yang. 2017. Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation. ArXiv e-prints (April 2017). arXiv:cs.CV/1705.00389
5. X. Deng, Y. Zhu, and S. Newsam. 2018. Spatial Morphing Kernel Regression for Feature Interpolation. In 25th IEEE International Conference on Image Processing (ICIP). 2182–2186.
6. E. Denton, S. Chintala, A. Szlam, and R. Fergus. 2015. Deep Generative Image Models Using a Laplacian Pyramid of Adversarial Networks. In Proceedings of 28th International Conference on Neural Information Processing Systems. 1486–1494.
7. H. Dong, S. Yu, C. Wu, and Y. Guo. 2017. Semantic Image Synthesis via Adversarial Learning. In Proceedings of the IEEE International Conference on Computer Vision. 5707–5715.
8. A. Dosovitskiy, P. Fischer, J. Springenberg, M. Riedmiller, and T. Brox. 2016. Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 9 (2016), 1734–1747.
9. I. Goodfellow. 2017. NIPS 2016 Tutorial: Generative Adversarial Networks. ArXiv e-prints (Dec. 2017). arXiv:cs.LG/1701.00160
10. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems. 2672–2680.
11. K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
12. T. Hu, J. Yang, X. Li, and P. Gong. 2016. Mapping Urban Land Use by Using Landsat Images and Open Social Data. Remote Sensing 8 (2016), 151.
13. R. Huang, S. Zhang, T. Li, and R. He. 2017. Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis. In Proceedings of the IEEE International Conference on Computer Vision. 2458–2467.
14. S. Ioffe and C. Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research), Francis Bach and David Blei (Eds.), Vol. 37. PMLR, Lille, France, 448–456.
15. P. Isola, J. Zhu, T. Zhou, and A. A Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. In Proceedings of the IEEE International Conference on Computer Vision.
16. D. P. Kingma and J. Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014).
17. C. Ledig, L. Theis, F. HuszÃąr, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi. 2017. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
18. D. Leung and S. Newsam. 2010. Proximate Sensing: Inferring What-Is-Where From Georeferenced Photo Collections. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2955–2962.
19. Y. Li, J. Huang, and J. Luo. 2015. Using User Generated Online Photos to Estimate and Monitor Air Pollution in Major Cities. In Proceedings of the 7th International Conference on Internet Multimedia Computing and Service. 79.
20. Y. Li, S. Liu, J. Yang, and M. Yang. 2017. Generative Face Completion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
21. X. Liu, J. He, Y. Yao, J. Zhang, H. Liang, H. Wang, and Y. Hong. 2017. Classifying Urban Land Use by Integrating Remote Sensing and Social Media Data. International Journal of Geographical Information Science 31, 8 (2017), 1675–1696.
22. L. Ma, X. Jia, Q. Sun, B. Schiele, T. Tuytelaars, and L. Van Gool. 2017. Pose Guided Person Image Generation. In Advances in Neural Information Processing Systems (NIPS). 405–415.
23. X. Mao, Q. Li, H. Xie, R. Y. K. Lau, Z. Wang, and S. P. Smolley. 2016. Least Squares Generative Adversarial Networks. ArXiv e-prints (Nov. 2016). arXiv:cs.CV/1611.04076
24. M. Mirza and S. Osindero. 2014. Conditional Generative Adversarial Nets. arXiv preprint arXiv:1411.1784 (2014).
25. S. Mohamed and B. Lakshminarayanan. 2016. Learning in Implicit Generative Models. ArXiv e-prints (Oct. 2016). arXiv:stat.ML/1610.03483
26. A. Radford, L. Metz, and S. Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. In International Conference on Representation Learning (ICRL).
27. S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee. 2016. Generative Adversarial Text to Image Synthesis. In Proceedings of the 33rd International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research), Maria Florina Balcan and Kilian Q. Weinberger (Eds.), Vol. 48. PMLR, New York, New York, USA, 1060–1069.
28. K. Regmi and A. Borji. 2018. Cross-View Image Synthesis Using Conditional GANs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
29. K. Regmi and A. Borji. 2018. Cross-view image synthesis using geometry-guided conditional GANs. ArXiv e-prints (Aug. 2018). arXiv:cs.CV/1808.05469
30. T. Salimans, I. Goodfellow, W. Zaremba, V.i Cheung, A. Radford, and X. Chen. 2016. Improved Techniques for Training GANs. In Advances in Neural Information Processing Systems (NIPS). 2234–2242.
31. L. Tran, X. Yin, and X. Liu. 2017. Representation Learning by Rotating Your Faces. ArXiv e-prints (May 2017). arXiv:cs.CV/1705.11136
32. J. Wang, M. Korayem, S. Blanco, and D. Crandall. 2016. Tracking Natural Events Through Social Media and Computer Vision. In 2016 ACM on Multimedia Conference. 1097–1101.
33. S. Workman, M. Zhai, D. J. Crandall, and N. Jacobs. 2017. A Unified Model for Near and Remote Sensing. In Proceedings of the IEEE International Conference on Computer Vision. 2707–2716.
34. R. A. Yeh, C. Chen, T. Lim, A. G. Schwing, M. Hasegawa-Johnson, and M. N. Do. 2017. Semantic Image Inpainting with Deep Generative Models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. equal contribution.
35. M. Zhai, Z. Bessinger, S. Workman, and N. Jacobs. 2017. Predicting Ground-Level Scene Layout from Aerial Imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
36. H. Zhang, T. Xu, H. Li, S. Zhang, X. Huang, X. Wang, and D. Metaxas. 2017. Stack-gan: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks. In Proceedings of the IEEE International Conference on Computer Vision. 5907–5915.
37. J. Zhu, T. Park, P. Isola, and A. A Efros. 2017. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. In Proceedings of the IEEE International Conference on Computer Vision.
38. Y. Zhu, X. Deng, and S. Newsam. 2018. Fine-Grained Land Use Classification at the City Scale Using Ground-Level Images. ArXiv e-prints (Feb. 2018). arXiv:cs.CV/1802.02668
39. Y. Zhu, S. Liu, and S. Newsam. 2017. Large-Scale Mapping of Human Activity Using Geo-Tagged Videos. In Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 68:1–68:4.
40. Y. Zhu and S. Newsam. 2015. Land Use Classification Using Convolutional Neural Networks Applied to Ground-Level Images. In Proceedings of the 23rd SIGSPATIAL International Conference on Advances in Geographic Information Systems. 61.
41. Y. Zhu and S. Newsam. 2016. Spatio-Temporal Sentiment Hotspot Detection Using Geotagged Photos. In Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 76.