Clearing the skies: A deep network architecture for single-image rain removal

IEEE Transactions on Image Processing

Volumn 26, Issue 6, 2017, Pages 2944-2956

  Fu, Xueyang ^a   Huang, Jiabin ^a   Ding, Xinghao ^a   Liao, Yinghao ^a   Paisley, John ^b  

^a XIAMEN UNIVERSITY   (China)

^b Department of Earth and Environmental Sciences   (United States)

Author keywords

convolutional neural networks; deep learning; image enhancement; Rain removal

Indexed keywords

CONVOLUTION; DEEP LEARNING; DEEP NEURAL NETWORKS; IMAGE ENHANCEMENT; IMAGE PROCESSING; NEURAL NETWORKS; RAIN;

COMPUTATION TIME; CONVOLUTIONAL NEURAL NETWORK; DOMAIN KNOWLEDGE; MAPPING RELATIONSHIPS; NETWORK TRAINING; OBJECTIVE FUNCTIONS; RAIN REMOVALS; REAL-WORLD IMAGE;

NETWORK ARCHITECTURE;

EID: 85018895520     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2017.2691802     Document Type: Article

References (34)

1. K. Garg, and S. K. Nayar, "Detection, and removal of rain from videos," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun./Jul. 2004, pp. I-528-I-535
2. P. C. Barnum, S. Narasimhan, and T. Kanade, "Analysis of rain, and snow in frequency space," Int. J. Comput. Vis., vol. 86, nos. 2-3, pp. 256-274, Jan. 2010
3. J. Bossu, N. Hautiere, and J. P. Tarel, "Rain or snow detection in image sequences through use of a histogram of orientation of streaks," Int. J. Comput. Vis., vol. 93, no. 3, pp. 348-367, Jul. 2011
4. V. Santhaseelan, and V. K. Asari, "Utilizing local phase information to remove rain from video," Int. J. Comput. Vis., vol. 112, no. 1, pp. 71-89, Mar. 2015
5. J.-H. Kim, C. Lee, J.-Y. Sim, and C.-S. Kim, "Single-image deraining using an adaptive nonlocal means filter," in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2013, pp. 914-917
6. D. Eigen, D. Krishnan, and R. Fergus, "Restoring an image taken through a window covered with dirt or rain," in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2013, pp. 633-640
7. Y. L. Chen, and C.-T. Hsu, "A generalized low-rank appearance model for spatio-temporally correlated rain streaks," in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2013, pp. 1968-1975
8. D.-A. Huang, L.-W. Kang, M.-C. Yang, C.-W. Lin, and Y.-C. F. Wang, "Context-aware single image rain removal," in Proc. IEEE Int. Conf. Multimedia Expo (ICME), Jul. 2012, pp. 164-169
9. L. W. Kang, C. W. Lin, and Y. H. Fu, "Automatic single-image-based rain streaks removal via image decomposition," IEEE Trans. Image Process., vol. 21, no. 4, pp. 1742-1755, Apr. 2012
10. D.-A. Huang, L.-W. Kang, Y.-C. F. Wang, and C.-W. Lin, "Self-learning based image decomposition with applications to single image denoising," IEEE Trans. Multimedia, vol. 16, no. 1, pp. 83-93, Jan. 2014
11. Y. Luo, Y. Xu, and H. Ji, "Removing rain from a single image via discriminative sparse coding," in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2015, pp. 3397-3405
12. D.-Y. Chen, C.-C. Chen, and L.-W. Kang, "Visual depth guided color image rain streaks removal using sparse coding," IEEE Trans. Circuits Syst. Video Technol., vol. 24, no. 8, pp. 1430-1455, Aug. 2014
13. Y. Li, R. T. Tan, X. Guo, J. Lu, and M. S. Brown, "Rain streak removal using layer priors," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 2736-2744
14. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2012, pp. 1097-1105
15. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998
16. J. Xie, L. Xu, E. Chen, J. Xie, and L. Xu, "Image denoising, and inpainting with deep neural networks," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2012, pp. 341-349
17. C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 2, pp. 295-307, Feb. 2016
18. J. Kim, J. K. Lee, and K. M. Lee, "Accurate image super-resolution using very deep convolutional networks," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 1646-1654
19. L. Xu, J. S. J. Ren, C. Liu, and J. Jia, "Deep convolutional neural network for image deconvolution," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2014, pp. 1790-1798
20. J. S. J. Ren, L. Xu, Q. Yan, and W. Sun, "Shepard convolutional neural networks," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2015, pp. 901-909
21. L. Xu, J. Ren, Q. Yan, R. Liao, and J. Jia, "Deep edge-aware filters," in Proc. Int. Conf. Mach. Learn. (ICML), 2015, pp. 1669-1678
22. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 770-778
23. J. Schmidhuber, "Deep learning in neural networks: An overview," Neural Netw., vol. 61, pp. 85-117, Jan. 2015
24. K. He, J. Sun, and X. Tang, "Guided image filtering," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 6, pp. 1397-1409, Jun. 2013
25. C. Tomasi, and R. Manduchi, "Bilateral filtering for gray, and color images," in Proc. 6th Int. Conf. Comput. Vis. (ICCV), Jan. 1998, pp. 839-846
26. Q. Zhang, X. Shen, L. Xu, and J. Jia, "Rolling guidance filter," in Proc. Eur. Conf. Comput. Vis. (ECCV), Sep. 2014, pp. 815-830
27. B. Gu, W. Li, M. Zhu, and M. Wang, "Local edge-preserving multiscale decomposition for high dynamic range image tone mapping," IEEE Trans. Image Process., vol. 22, no. 1, pp. 70-79, Jan. 2013
28. T. Qiu, A. Wang, N. Yu, and A. Song, "LLSURE: Local linear surebased edge-preserving image filtering," IEEE Trans. Image Process., vol. 22, no. 1, pp. 80-90, Jan. 2013
29. G. Schaefer, and M. Stich, "UCID: An uncompressed color image database," Proc. SPIE, vol. 5307, pp. 472-480, Dec. 2003
30. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "Contour detection, and hierarchical image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 898-916, May 2011
31. Y. Li, F. Guo, R. T. Tan, and M. S. Brown, "A contrast enhancement framework with JPEG artifacts suppression," in Proc. Eur. Conf. Comput. Vis. (ECCV), Sep. 2014, pp. 174-188
32. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004
33. K. Garg, and S. K. Nayar, "Photorealistic rendering of rain streaks," ACM Trans. Graph., vol. 25, no. 3, pp. 996-1002, Jul. 2006
34. A. K. Moorthy, and A. C. Bovik, "A two-step framework for constructing blind image quality indices," IEEE Signal Process. Lett., vol. 17, no. 5, pp. 513-516, May 2010