StyleBank: An explicit representation for neural image style transfer

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 2770-2779

  Chen, Dongdong ^a   Yuan, Lu ^b   Liao, Jing ^b   Yu, Nenghai ^a   Hua, Gang ^b  

^a UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; FILTER BANKS; LEARNING SYSTEMS; SIGNAL ENCODING;

AUTO ENCODERS; EXPLICIT REPRESENTATION; FEATURE EMBEDDING; FLEXIBLE NETWORK DESIGNS; INCREMENTAL LEARNING; MAPPING METHOD; MULTIPLE CONVOLUTION; TRANSFER NETWORK;

PATTERN RECOGNITION;

EID: 85041923859     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.296     Document Type: Conference Paper

References (42)

1. M. T. Alexander Mordvintsev, Christopher Olah. Inceptionism: Going deeper into neural networks, 2015.
2. H. A. Aly and E. Dubois. Image up-sampling using total-variation regularization with a new observation model. IEEE Transactions on Image Processing, 14(10): 1647-1659, 2005.
3. Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8): 1798-1828, 2013.
4. Y. Bengio, R. Ducharme, P. Vincent, and C. Jauvin. A neural probabilistic language model. Journal of Machine Learning Research, 3(Feb): 1137-1155, 2003.
5. Y. Y. Boykov and M.-P. Jolly. Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In Proceedings of 8th IEEE International Conference on Computer Vision, Volume 1, pages 105-112. IEEE, 2001.
6. V. Dumoulin, J. Shlens, and M. Kudlur. A learned representation for artistic style. arXiv preprint arXiv: 1610.07629, 2016.
7. A. A. Efros and W. T. Freeman. Image quilting for texture synthesis and transfer. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pages 341-346. ACM, 2001.
8. A. A. Efros and T. K. Leung. Texture synthesis by non-parametric sampling. In Proceedings of the 7th IEEE International Conference on Computer Vision, Volume 2, pages 1033-1038. IEEE, 1999.
9. M. Elad and P. Milanfar. Style-transfer via texture-synthesis. arXiv preprint arXiv: 1609.03057, 2016.
10. O. Frigo, N. Sabater, J. Delon, and P. Hellier. Split and match: Example-based adaptive patch sampling for unsupervised style transfer. In Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition, 2016.
11. L. Gatys, A. S. Ecker, and M. Bethge. Texture synthesis using convolutional neural networks. In Advances in Neural Information Processing Systems, pages 262-270, 2015.
12. L. A. Gatys, A. S. Ecker, and M. Bethge. A neural algorithm of artistic style. arXiv preprint arXiv: 1508.06576, 2015.
13. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, pages 2672-2680, 2014.
14. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. arXiv preprint arXiv: 1512.03385, 2015.
15. D. J. Heeger and J. R. Bergen. Pyramid-based texture analysis/synthesis. In Proceedings of the 22nd a Annual Conference on Computer Graphics and Interactive Techniques, pages 229-238. ACM, 1995.
16. A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin. Image analogies. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pages 327-340. ACM, 2001.
17. G. Hinton and R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786): 504-507, 2006.
18. S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv: 1502.03167, 2015.
19. J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. arXiv preprint arXiv: 1603.08155, 2016.
20. D. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
21. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, pages 1097-1105, 2012.
22. C. Li and M. Wand. Combining markov random fields and convolutional neural networks for image synthesis. arXiv preprint arXiv: 1601.04589, 2016.
23. C. Li and M. Wand. Precomputed real-time texture synthesis with markovian generative adversarial networks. arXiv preprint arXiv: 1604.04382, 2016.
24. Y. Li, T. Wang, and H.-Y. Shum. Motion texture: a two-level statistical model for character motion synthesis. In ACM Transactions on Graphics (ToG), Volume 21, pages 465-472. ACM, 2002.
25. L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum. Realtime texture synthesis by patch-based sampling. ACM Transactions on Graphics (ToG), 20(3): 127-150, 2001.
26. D. Lin, J. Dai, J. Jia, K. He, and J. Sun. Scribblesup: Scribble-supervised convolutional networks for semantic segmentation. arXiv preprint arXiv: 1604.05144, 2016.
27. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision, pages 740-755. Springer, 2014.
28. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the 25th IEEE Conference on Computer Vision and Pattern Recognition, pages 3431-3440, 2015.
29. Y. Lu, S.-C. Zhu, and Y. N. Wu. Learning frame models using cnn filters. arXiv preprint arXiv: 1509.08379, 2015.
30. J. Malik, S. Belongie, J. Shi, and T. Leung. Textons, contours and regions: Cue integration in image segmentation. In Proceedings of the 7th IEEE International Conference on Computer Vision, Volume 2, pages 918-925. IEEE, 1999.
31. J. Portilla and E. P. Simoncelli. A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision, 40(1): 49-70, 2000.
32. S. E. Reed, Y. Zhang, Y. Zhang, and H. Lee. Deep visual analogy-making. In Advances in Neural Information Processing Systems 28, pages 1252-1260. 2015.
33. C. Rother, V. Kolmogorov, and A. Blake. Grabcut: Interactive foreground extraction using iterated graph cuts. In ACM transactions on graphics (ToG), Volume 23, pages 309-314. ACM, 2004.
34. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3): 211-252, 2015.
35. A. Selim, M. Elgharib, and L. Doyle. Painting style transfer for head portraits using convolutional neural networks. ACM Transactions on Graphics (ToG), 35(4): 129, 2016.
36. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409.1556, 2014.
37. D. Ulyanov, V. Lebedev, A. Vedaldi, and V. Lempitsky. Texture networks: Feed-forward synthesis of textures and stylized images. arXiv preprint arXiv: 1603.03417, 2016.
38. D. Ulyanov, A. Vedaldi, and V. Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv: 1607.08022, 2016.
39. L.-Y. Wei and M. Levoy. Fast texture synthesis using tree-structured vector quantization. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pages 479-488. ACM Press/Addison-Wesley Publishing Co., 2000.
40. T. Xue, J. Wu, K. L. Bouman, and W. T. Freeman. Visual dynamics: Probabilistic future frame synthesis via cross convolutional networks. arXiv preprint arXiv: 1607.02586, 2016.
41. S.-C. Zhu, C.-E. Guo, Y. Wang, and Z. Xu. What are textons? International Journal of Computer Vision, 62(1-2): 121-143, 2005.
42. S. C. Zhu, Y. Wu, and D. Mumford. Filters, random fields and maximum entropy (frame): Towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2): 107-126, 1998.