Let there be color!: Joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification

ACM Transactions on Graphics

Volumn 35, Issue 4, 2016, Pages

  Iizuka, Satoshi ^a   Simo Serra, Edgar ^a   Ishikawa, Hiroshi ^a  

^a WASEDA UNIVERSITY   (Japan)

Author keywords

Colorization; Convolutional neural network

Indexed keywords

CONVOLUTION; INTERACTIVE COMPUTER GRAPHICS; NEURAL NETWORKS; PHOTOGRAPHY;

BLACK AND WHITE PHOTOGRAPHY; COLORIZATION; CONVOLUTIONAL NEURAL NETWORK; GRAY-SCALE IMAGES; IMAGE COLORIZATIONS; LOCAL IMAGE FEATURES; LOCAL INFORMATION; STATE OF THE ART;

CLASSIFICATION (OF INFORMATION);

EID: 84980049328     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/2897824.2925974     Document Type: Conference Paper

References (41)

1. AN, X., AND PELLACINI, F. 2008. AppProp: All-pairs appearance-space edit propagation. ACM Trans. Graph. 27, 3 (Aug.), 40:1-40:9.
2. BELL, S., AND BALA, K. 2015. Learning visual similarity for product design with convolutional neural networks. ACM Trans. on Graphics (SIGGRAPH) 34, 4.
3. BROMLEY, J., GUYON, I., LECUN, Y., SÄCKINGER, E., AND SHAH, R. 1994. Signature verification using a "siamese" time delay neural network. In NIPS.
4. CHARPIAT, G., HOFMANN, M., AND SCHÖLKOPF, B. 2008. Automatic image colorization via multimodal predictions. In ECCV.
5. CHEN, X., ZOU, D., ZHAO, Q., AND TAN, P. 2012. Manifold preserving edit propagation. ACM Trans. Graph. 31, 6 (Nov.), 132:1-132:7.
6. CHENG, Z., YANG, Q., AND SHENG, B. 2015. Deep colorization. In Proceedings of ICCV 2015, 29-43.
7. CHIA, A. Y.-S., ZHUO, S., GUPTA, R. K., TAI, Y.-W., CHO, S.-Y., TAN, P., AND LIN, S. 2011. Semantic colorization with internet images. ACM Trans. Graph. 30, 6, 156:1-156:8.
8. DONG, C., LOY, C. C., HE, K., AND TANG, X. 2016. Image super-resolution using deep convolutional networks. PAMI 38, 2, 295-307.
9. EIGEN, D., AND FERGUS, R. 2015. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In ICCV.
10. FISCHER, P., DOSOVITSKIY, A., ILG, E., HÄUSSER, P., HAZIRBAS, C., GOLKOV, V., VAN DER SMAGT, P., CREMERS, D., AND BROX, T. 2015. Flownet: Learning optical flow with convolutional networks.
11. FUKUSHIMA, K. 1988. Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural networks 1, 2, 119-130.
12. GUPTA, R. K., CHIA, A. Y.-S., RAJAN, D., NG, E. S., AND ZHIYONG, H. 2012. Image colorization using similar images. In ACM International Conference on Multimedia, 369-378.
13. HUANG, Y.-C., TUNG, Y.-S., CHEN, J.-C., WANG, S.-W., AND WU, J.-L. 2005. An adaptive edge detection based colorization algorithm and its applications. In ACM International Conference on Multimedia, 351-354.
14. IOFFE, S., AND SZEGEDY, C. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML.
15. IRONY, R., COHEN-OR, D., AND LISCHINSKI, D. 2005. Colorization by example. In Eurographics Conference on Rendering Techniques, 201-210.
16. KRIZHEVSKY, A., SUTSKEVER, I., AND HINTON, G. E. 2012. Imagenet classification with deep convolutional neural networks. In NIPS.
17. LECUN, Y., BOTTOU, L., BENGIO, Y., AND HAFFNER, P. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 11, 2278-2324.
18. LEVIN, A., LISCHINSKI, D., AND WEISS, Y. 2004. Colorization using optimization. ACM Transactions on Graphics 23, 689-694.
19. LI, Y., JU, T., AND HU, S.-M. 2010.Instant propagation of sparse edits on images and videos. Computer Graphics Forum 29, 7, 2049-2054.
20. LIU, X., WAN, L., QU, Y., WONG, T.-T., LIN, S., LEUNG, C.- S., AND HENG, P.-A. 2008. Intrinsic colorization. ACM Transactions on Graphics (SIGGRAPH Asia 2008 issue) 27, 5 (December), 152:1-152:9.
21. LONG, J., SHELHAMER, E., AND DARRELL, T. 2015. Fully convolutional networks for semantic segmentation. In CVPR.
22. LUAN, Q., WEN, F., COHEN-OR, D., LIANG, L., XU, Y.-Q., AND SHUM, H.-Y. 2007. Natural image colorization. In Eurographics Conference on Rendering Techniques, 309-320.
23. PITIÉ, F., KOKARAM, A. C., AND DAHYOT, R. 2007. Automated colour grading using colour distribution transfer. Comput. Vis. Image Underst. 107, 1-2 (July), 123-137.
24. QU, Y., WONG, T.-T., AND HENG, P.-A. 2006. Manga colorization. ACM Trans. Graph. 25, 3 (July), 1214-1220.
25. REINHARD, E., ASHIKHMIN, M., GOOCH, B., AND SHIRLEY, P. 2001. Color transfer between images. IEEE Computer Graphics and Applications 21, 5 (sep), 34-41.
26. RUMELHART, D., HINTON, G., AND WILLIAMS, R. 1986. Learning representations by back-propagating errors. In Nature.
27. SHEN, W., WANG, X., WANG, Y., BAI, X., AND ZHANG, Z. 2015. Deepcontour: A deep convolutional feature learned by positive-sharing loss for contour detection. In CVPR.
28. SIMONYAN, K., AND ZISSERMAN, A. 2015. Very deep convolutional networks for large-scale image recognition. In ICLR.
29. SPRINGENBERG, J. T., DOSOVITSKIY, A., BROX, T., AND RIEDMILLER, M. A. 2015. Striving for simplicity: The all convolutional net. In ICLR Workshop Track.
30. SÝKORA, D., DINGLIANA, J., AND COLLINS, S. 2009. Lazy- Brush: Flexible painting tool for hand-drawn cartoons. Computer Graphics Forum 28, 2, 599-608.
31. TAI, Y., JIA, J., AND TANG, C. 2005. Local color transfer via probabilistic segmentation by expectation-maximization. In CVPR, 747-754.
32. WANG, L., GUO, S., HUANG, W., AND QIAO, Y. 2015. Places205-vggnet models for scene recognition. CoRR abs/1508.01667.
33. WANG, L., LEE, C., TU, Z., AND LAZEBNIK, S. 2015. Training deeper convolutional networks with deep supervision. CoRR abs/1505.02496.
34. WANG, X., FOUHEY, D. F., AND GUPTA, A. 2015. Designing deep networks for surface normal estimation. In CVPR.
35. WELSH, T., ASHIKHMIN, M., AND MUELLER, K. 2002. Transferring color to greyscale images. ACM Trans. Graph. 21, 3 (July), 277-280.
36. WU, F., DONG, W., KONG, Y., MEI, X., PAUL, J.-C., AND ZHANG, X. 2013. Content-based colour transfer. Computer Graphics Forum 32, 1, 190-203.
37. XU, K., LI, Y., JU, T., HU, S.-M., AND LIU, T.-Q. 2009. Efficient affinity-based edit propagation using k-d tree. ACM Trans. Graph. 28, 5 (Dec.), 118:1-118:6.
38. XU, L., YAN, Q., AND JIA, J. 2013. A sparse control model for image and video editing. ACM Trans. Graph. 32, 6 (Nov.), 197:1-197:10.
39. YATZIV, L., YATZIV, L., SAPIRO, G., AND SAPIRO, G. 2004. Fast image and video colorization using chrominance blending. IEEE Transaction on Image Processing 15, 2006.
40. ZEILER, M. D. 2012. ADADELTA: an adaptive learning rate method. CoRR abs/1212.5701.
41. ZHOU, B., LAPEDRIZA, A., XIAO, J., TORRALBA, A., AND OLIVA, A. 2014. Learning deep features for scene recognition using places database. In NIPS.