Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn , Issue , 2020, Pages 14519-14527

  Daras, Giannis ^a   Odena, Augustus ^b   Zhang, Han ^b   Dimakis, Alexandros G ^c  

^a NATIONAL TECHNICAL UNIVERSITY OF ATHENS   (Greece)

^b GOOGLE INC   (United States)

^c UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FLOW GRAPHS; INFORMATION THEORY; PATTERN RECOGNITION;

ADVERSARIAL NETWORKS; ATTENTION MECHANISMS; GENERATIVE MODEL; INFORMATION FLOW GRAPHS; LOSS FUNCTIONS; NOVEL INFORMATION; TWO-DIMENSIONAL GEOMETRY; VISUAL IMPROVEMENTS;

IMAGE ENHANCEMENT;

EID: 85090822905     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR42600.2020.01454     Document Type: Conference Paper

References (26)

1. Rudolf Ahlswede, Ning Cai, S-YR Li, and Raymond W Yeung. Network information flow. IEEE Transactions on information theory, 46(4):1204-1216, 2000. 3
2. Ashish Bora, Ajil Jalal, Eric Price, and Alexandros G Dimakis. Compressed sensing using generative models. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 537-546. JMLR. org, 2017. 2, 6, 7, 8
3. Andrew Brock, Jeff Donahue, and Karen Simonyan. Large Scale GAN Training for High Fidelity Natural Image Synthesis. arXiv e-prints, page arXiv:1809.11096, Sep 2018. 1, 3
4. Dan A. Calian, Peter Roelants, Jacques Cali, Ben Carr, Krishna Dubba, John E. Reid, and Dell Zhang. SCRAM: Spatially Coherent Randomized Attention Maps. arXiv e-prints, page arXiv:1905.10308, May 2019. 7
5. Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509, 2019. 1, 2, 3, 4, 6
6. Goncalo M. Correia, Vlad Niculae, and Andre F. T. Martins. Adaptively sparse transformers. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019. 7
7. Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. arXiv e-prints, page arXiv:1901.02860, Jan 2019. 7
8. Alexandros G Dimakis, P Brighten Godfrey, Yunnan Wu, Martin J Wainwright, and Kannan Ramchandran. Network coding for distributed storage systems. IEEE transactions on information theory, 56(9):4539-4551, 2010. 1, 3
9. Jeff Donahue, Philipp Krähenbühl, and Trevor Darrell. Adversarial feature learning. arXiv preprint arXiv:1605.09782, 2016. 7
10. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, DavidWarde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Networks. arXiv eprints, page arXiv:1406.2661, Jun 2014. 1
11. Scott Gray, Alec Radford, and Diederik P Kingma. Gpu kernels for block-sparse weights. arXiv preprint arXiv:1711.09224, 2017. 3
12. Qipeng Guo, Xipeng Qiu, Pengfei Liu, Yunfan Shao, Xiangyang Xue, and Zheng Zhang. Star-Transformer. arXiv e-prints, page arXiv:1902.09113, Feb 2019. 3, 7
13. Paul Hand and Vladislav Voroninski. Global guarantees for enforcing deep generative priors by empirical risk. IEEE Transactions on Information Theory, 2019. 6
14. Maya Kabkab, Pouya Samangouei, and Rama Chellappa. Task-Aware compressed sensing with generative adversarial networks. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018. 2, 6
15. Tero Karras, Samuli Laine, and Timo Aila. A Style-Based Generator Architecture for Generative Adversarial Networks. arXiv e-prints, page arXiv:1812.04948, Dec 2018. 1
16. Zachary C Lipton and Subarna Tripathi. Precise recovery of latent vectors from generative adversarial networks. arXiv preprint arXiv:1702.04782, 2017. 6
17. Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, and Dustin Tran. Image transformer. arXiv preprint arXiv:1802.05751, 2018. 8
18. Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arXiv e-prints, page arXiv:1511.06434, Nov 2015. 1
19. Ankit Raj, Yuqi Li, and Yoram Bresler. Gan-based projector for faster recovery with convergence guarantees in linear inverse problems. In Proceedings of the IEEE International Conference on Computer Vision, pages 5602-5611, 2019. 2, 6
20. JH Rick Chang, Chun-Liang Li, Barnabas Poczos, BVK Vijaya Kumar, and Aswin C Sankaranarayanan. One network to solve them all-solving linear inverse problems using deep projection models. In Proceedings of the IEEE International Conference on Computer Vision, pages 5888-5897, 2017. 2, 6
21. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211-252, 2015. 3
22. Ganlin Song, Zhou Fan, and John Lafferty. Surfing: Iterative optimization over incrementally trained deep networks. arXiv preprint arXiv:1907.08653, 2019. 6
23. Sainbayar Sukhbaatar, Edouard Grave, Piotr Bojanowski, and Armand Joulin. Adaptive Attention Span in Transformers. arXiv e-prints, page arXiv:1905.07799, May 2019. 1, 7
24. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention Is All You Need. arXiv e-prints, page arXiv:1706.03762, Jun 2017. 1, 2, 5
25. Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. Self-Attention generative adversarial networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 7354-7363, Long Beach, California, USA, 09-15 Jun 2019. PMLR. 1, 2, 3, 8
26. Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris Metaxas. Stack-GAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks. arXiv e-prints, page arXiv:1612.03242, Dec 2016.