DeepWarp: Photorealistic image resynthesis for gaze manipulation

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 9906 LNCS, Issue , 2016, Pages 311-326

  Ganin, Yaroslav ^a   Kononenko, Daniil ^a   Sungatullina, Diana ^a   Lempitsky, Victor ^a  

^a SKOLKOVO INSTITUTE OF SCIENCE AND TECHNOLOGY   (Russian Federation)

Author keywords

Deep learning; Gaze correction; Spatial transformers; Supervised learning; Warping

Indexed keywords

NETWORK ARCHITECTURE; SUPERVISED LEARNING;

COARSE TO FINE; DEEP ARCHITECTURES; DEEP LEARNING; GAZE MANIPULATIONS; IMAGE GENERATIONS; NUMERICAL COMPARISON; PHOTOREALISTIC IMAGES; WARPING;

COMPUTER VISION;

EID: 84990855318     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-319-46475-6_20     Document Type: Conference Paper

References (28)

1. Criminisi, A., Shotton, J., Blake, A., Torr, P.H.: Gaze manipulation for one-to-one teleconferencing. In: ICCV (2003)
2. Denton, E.L., Chintala, S., Fergus, R., et al.: Deep generative image models using a laplacian pyramid of adversarial networks. In: NIPS (2015)
3. Dosovitskiy, A., Tobias Springenberg, J., Brox, T.: Learning to generate chairs with convolutional neural networks. In: CVPR (2015)
4. Ganin, Y., Kononenko, D., Sungatullina, D., Lempitsky, V.: Project website (2016). http://sites.skoltech.ru/compvision/projects/deepwarp/. Accessed 22 July 2016
5. Gatys, L., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: NIPS (2015)
6. Ghodrati, A., Jia, X., Pedersoli, M., Tuytelaars, T.: Towards automatic image editing: learning to see another you. arXiv preprint arXiv: 1511.08446 (2015)
7. Giger, D., Bazin, J.C., Kuster, C., Popa, T., Gross, M.: Gaze correction with a single webcam. In: IEEE International Conference on Multimedia and Expo (2014)
8. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: NIPS (2014)
9. Gregor, K., Danihelka, I., Graves, A., Rezende, D., Wierstra, D.: Draw: a recurrent neural network for image generation. In: ICML (2015)
10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. arXiv preprint arXiv: 1512.03385 (2015)
11. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML (2015)
12. Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: NIPS (2015)
13. Jones, A., Lang, M., Fyffe, G., Yu, X., Busch, J., McDowall, I., Bolas, M.T., Debevec, P.E.: Achieving eye contact in a one-to-many 3D video teleconferencing system. ACM Trans. Graph. 28(3), 64 (2009)
14. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv: 1412.6980 (2014)
15. Kleinke, C.L.: Gaze and eye contact: a research review. Psychol. Bull. 100(1), 78 (1986)
16. Kononenko, D., Lempitsky, V.: Learning to look up: realtime monocular gaze correction using machine learning. In: CVPR (2015)
17. Kulkarni, T.D., Whitney, W.F., Kohli, P., Tenenbaum, J.: Deep convolutional inverse graphics network. In: NIPS (2015)
18. Kuster, C., Popa, T., Bazin, J.C., Gotsman, C., Gross, M.: Gaze correction for home video conferencing. In: SIGGRAPH Asia (2012)
19. Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: CVPR (2015)
20. Okada, K.I., Maeda, F., Ichikawaa, Y., Matsushita, Y.: Multiparty videoconferencing at virtual social distance: MAJIC design. In: Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work, CSCW 1994, pp. 385-393 (1994)
21. Oquab, M.: Torch7 modules for spatial transformer networks (2015). https:// github.com/qassemoquab/stnbhwd
22. Reed, S.E., Zhang, Y., Zhang, Y., Lee, H.: Deep visual analogy-making. In: NIPS (2015)
23. Wallis, L.J., Range, F., Müller, C.A., Serisier, S., Huber, L., Virányi, Z.: Training for eye contact modulates gaze following in dogs. Anim. Behav. 106, 27-35 (2015)
24. Wolf, L., Freund, Z., Avidan, S.: An eye for an eye: a single camera gazereplacement method. In: CVPR (2010)
25. Xiong, X., Torre, F.: Supervised descent method and its applications to face alignment. In: CVPR (2013)
26. Yang, R., Zhang, Z.: Eye gaze correction with stereovision for video-teleconferencing. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 479-494. Springer, Heidelberg (2002). doi: 10.1007/3-540-47967-8_32
27. Yip, B., Jin, J.S.: Face re-orientation using ellipsoid model in video conference. In: Proceedings of the 7th IASTED International Conference on Internet and Multimedia Systems and Applications, pp. 245-250 (2003)
28. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818-833. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10590-1_53