Learning to simplify: Fully convolutional networks for rough sketch cleanup

ACM Transactions on Graphics

Volumn 35, Issue 4, 2016, Pages

  Simo Serra, Edgar ^a   Iizuka, Satoshi ^a   Sasaki, Kazuma ^a   Ishikawa, Hiroshi ^a  

^a WASEDA UNIVERSITY   (Japan)

Author keywords

Convolutional neural network; Sketch simplification

Indexed keywords

ASPECT RATIO; CONVOLUTION; INTERACTIVE COMPUTER GRAPHICS; NEURAL NETWORKS;

CONVOLUTION OPERATORS; CONVOLUTIONAL NETWORKS; CONVOLUTIONAL NEURAL NETWORK; NOVEL TECHNIQUES; SIMPLIFICATION MODELS; SKETCH SIMPLIFICATION; STATE OF THE ART; USER INTERVENTION;

RASTERIZATION;

EID: 84979964959     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/2897824.2925972     Document Type: Conference Paper

References (42)

1. BAE, S.-H., BALAKRISHNAN, R., AND SINGH, K. 2008. Ilovesketch: As-natural-as-possible sketching system for creating 3d curve models. In ACM Symposium on User Interface Software and Technology, 151-160.
2. BARLA, P., THOLLOT, J., AND SILLION, F. X. 2005. Geometric clustering for line drawing simplification. In ACM SIGGRAPH 2005 Sketches.
3. BARTOLO, A., CAMILLERI, K. P., FABRI, S. G., BORG, J. C., AND FARRUGIA, P. J. 2007. Scribbles to vectors: Preparation of scribble drawings for cad interpretation. In Eurographics Workshop on Sketch-based Interfaces and Modeling, 123-130.
4. BAUDEL, T. 1994. A mark-based interaction paradigm for freehand drawing. In ACM Symposium on User Interface Software and Technology, 185-192.
5. CHANG, H.-H., AND YAN, H. 1998. Vectorization of handdrawn image using piecewise cubic bézier curves fitting. Pattern Recognition 31, 11, 1747-1755.
6. CHEN, J., GUENNEBAUD, G., BARLA, P., AND GRANIER, X. 2013. Non-oriented mls gradient fields. Computer Graphics Forum 32, 8, 98-109.
7. COLE, F., DECARLO, D., FINKELSTEIN, A., KIN, K., MORLEY,K., AND SANTELLA, A. 2006. Directing gaze in 3d models with stylized focus. In Eurographics Conference on Rendering Techniques, 377-387.
8. DEUSSEN, O., AND STROTHOTTE, T. 2000. Computer-generated pen-and-ink illustration of trees. In Conference on Computer Graphics and Interactive Techniques, 13-18.
9. DONG, C., LOY, C. C., HE, K., AND TANG, X. 2016. Image super-resolution using deep convolutional networks. PAMI 38, 2, 295-307.
10. DOSOVITSKIY, A., SPRINGENBERG, J. T., AND BROX, T. 2015. Learning to generate chairs with convolutional neural networks. In CVPR.
11. FISCHER, P., DOSOVITSKIY, A., ILG, E., HÄUSSER, P., HAZIRBAS, C., GOLKOV, V., VAN DER SMAGT, P., CREMERS, D., AND BROX, T. 2015. Flownet: Learning optical flow with convolutional networks.
12. FIŠER, J., ASENTE, P., AND SÝKORA, D. 2015. Shipshape: A drawing beautification assistant. In Workshop on Sketch-Based Interfaces and Modeling, 49-57.
13. FREEMAN, H. 1974. Computer processing of line-drawing images. ACM Comput. Surv. 6, 1, 57-97.
14. FUKUSHIMA, K. 1988. Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural networks 1, 2, 119-130.
15. GRABLI, S., DURAND, F., AND SILLION, F. 2004. Density measure for line-drawing simplification. In Pacific Conference on Computer Graphics and Applications, 2004, 309-318.
16. GRIMM, C., AND JOSHI, P. 2012. Just drawit: A 3d sketching system. In International Symposium on Sketch-Based Interfaces and Modeling, 121-130.
17. HILAIRE, X., AND TOMBRE, K. 2006. Robust and accurate vectorization of line drawings. PAMI 28, 6, 890-904.
18. IGARASHI, T., MATSUOKA, S., KAWACHIYA, S., AND TANAKA, H. 1997. Interactive beautification: A technique for rapid geometric design. In ACM Symposium on User Interface Software and Technology, 105-114.
19. IOFFE, S., AND SZEGEDY, C. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML.
20. JANSSEN, R. D., AND VOSSEPOEL, A. M. 1997. Adaptive vectorization of line drawing images. Computer Vision and Image Understanding 65, 1, 38-56.
21. KRIZHEVSKY, A., SUTSKEVER, I., AND HINTON, G. E. 2012. Imagenet classification with deep convolutional neural networks. In NIPS.
22. LECUN, Y., BOTTOU, L., BENGIO, Y., AND HAFFNER, P. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 11, 2278-2324.
23. LINDLBAUER, D., HALLER, M., HANCOCK, M. S., SCOTT, S. D., AND STUERZLINGER, W. 2013. Perceptual grouping: selection assistance for digital sketching. In International Conference on Interactive Tabletops and Surfaces, 51-60.
24. LIU, X., WONG, T.-T., AND HENG, P.-A. 2015. Closure-aware sketch simplification. ACM Trans. Graph. 34, 6, 168:1-168:10.
25. LONG, J., SHELHAMER, E., AND DARRELL, T. 2015. Fully convolutional networks for semantic segmentation. In CVPR.
26. NAIR, V., AND HINTON, G. E. 2010. Rectified linear units improve restricted boltzmann machines. In ICML, 807-814.
27. NOH, H., HONG, S., AND HAN, B. 2015. Learning deconvolution network for semantic segmentation. In ICCV.
28. NORIS, G., HORNUNG, A., SUMNER, R. W., SIMMONS, M., AND GROSS, M. 2013. Topology-driven vectorization of clean line drawings. ACM Trans. Graph. 32, 1, 4:1-4:11.
29. ORBAY, G., AND KARA, L. 2011. Beautification of design sketches using trainable stroke clustering and curve fitting. IEEE Trans. on Visualization and Computer Graphics 17, 5, 694-708.
30. PREIM, B., AND STROTHOTTE, T. 1995. Tuning rendered linedrawings. In Winter School in Computer Graphics, 227-237.
31. PUSCH, R., SAMAVATI, F., NASRI, A., AND WYVILL, B. 2007. Improving the sketch-based interface. The Visual Computer 23, 9-11, 955-962.
32. ROSIN, P. L. 1994. Grouping curved lines. In Machine Graphics and Vision 7, 625-644.
33. RUMELHART, D., HINTON, G., AND WILLIAMS, R. 1986. Learning representations by back-propagating errors. In Nature.
34. SELINGER, P. 2003. Potrace: a polygon-based tracing algorithm. Potrace (online), http://potrace.sourceforge.net/potrace.pdf (2009-07-01).
35. SHEN, W., WANG, X., WANG, Y., BAI, X., AND ZHANG, Z. 2015. Deepcontour: A deep convolutional feature learned by positive-sharing loss for contour detection. In CVPR.
36. SHESH, A., AND CHEN, B. 2008. Efficient and dynamic simplification of line drawings. Computer Graphics Forum 27, 2, 537-545.
37. SIMONYAN, K., AND ZISSERMAN, A. 2015. Very deep convolutional networks for large-scale image recognition. In ICLR.
38. SPRINGENBERG, J. T., DOSOVITSKIY, A., BROX, T., AND RIEDMILLER, M. A. 2015. Striving for simplicity: The all convolutional net. In ICLR Workshop Track.
39. WILSON, B., AND MA, K.-L. 2004. Rendering complexity in computer-generated pen-and-ink illustrations. In International Symposium on Non-photorealistic Animation and Rendering, 129-137.
40. ZEILER, M. D., AND FERGUS, R. 2014. Visualizing and understanding convolutional networks. In ECCV.
41. ZEILER, M. D. 2012. ADADELTA: an adaptive learning rate method. CoRR abs/1212.5701.
42. ZHANG, T. Y., AND SUEN, C. Y. 1984. A fast parallel algorithm for thinning digital patterns. Commun. ACM 27, 3, 236-239.