Automatic single-image-based rain streaks removal via image decomposition

IEEE Transactions on Image Processing

Volumn 21, Issue 4, 2012, Pages 1742-1755

  Kang, Li Wei ^a   Lin, Chia Wen ^b   Fu, Yu Hsiang ^c  

^a INSTITUTE OF INFORMATION SCIENCE   (Taiwan)

^b NATIONAL TSING HUA UNIVERSITY   (Taiwan)

^c MSTAR SEMICONDUCTOR INC   (Taiwan)

Author keywords

Dictionary learning; image decomposition; morphological component analysis (MCA); rain removal; sparse representation

Indexed keywords

BILATERAL FILTERS; COMPONENT ANALYSIS; DICTIONARY LEARNING; HIGH FREQUENCY HF; IMAGE DECOMPOSITION; ORIGINAL IMAGES; SINGLE IMAGES; SPARSE CODING; SPARSE REPRESENTATION; TEMPORAL INFORMATION;

ALGORITHMS;

RAIN;

RAIN;

ALGORITHM; ARTICLE; ARTIFACT; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; INFORMATION RETRIEVAL; METHODOLOGY; PHOTOGRAPHY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY;

ALGORITHMS; ARTIFACTS; ARTIFICIAL INTELLIGENCE; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAPHY; RAIN; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SUBTRACTION TECHNIQUE;

EID: 84859079535     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2011.2179057     Document Type: Article

References (45)

1. P. C. Barnum, S. Narasimhan, and T. Kanade, "Analysis of rain and snow in frequency space," Int. J. Comput. Vis., vol. 86, no. 2/3, pp. 256-274, Jan. 2010.
2. K. Garg and S. K. Nayar, "Detection and removal of rain from videos," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2004, vol. 1, pp. 528-535.
3. K. Garg and S. K. Nayar, "When does a camera see rain?," in Proc. IEEE Int. Conf. Comput. Vis., Oct. 2005, vol. 2, pp. 1067-1074. (Pubitemid 44042299)
4. K. Garg and S. K. Nayar, "Vision and rain," Int. J. Comput. Vis., vol. 75, no. 1, pp. 3-27, Oct. 2007. (Pubitemid 47120244)
5. K. Garg and S. K. Nayar, "Photorealistic rendering of rain streaks," ACM Trans. Graph., vol. 25, no. 3, pp. 996-1002, Jul. 2006. (Pubitemid 44481061)
6. X. Zhang, H. Li, Y. Qi, W. K. Leow, and T. K. Ng, "Rain removal in video by combining temporal and chromatic properties," in Proc. IEEE Int. Conf. Multimedia Expo., Toronto, ON, Canada, Jul. 2006, pp. 461-464.
7. N. Brewer and N. Liu, "Using the shape characteristics of rain to identify and remove rain from video," Lecture Notes Comput. Sci., vol. 5342/2008, pp. 451-458, 2008.
8. J. Bossu, N. Hautière, and J. P. Tarel, "Rain or snow detection in image sequences through use of a histogram of orientation of streaks," Int. J. Comput. Vis., vol. 93, no. 3, pp. 348-367, Jul. 2011.
9. M. S. Shehata, J. Cai, W. M. Badawy, T. W. Burr, M. S. Pervez, R. J. Johannesson, and A. Radmanesh, "Video-based automatic incident detection for smart roads: The outdoor environmental challenges regarding false alarms," IEEE Trans. Intell. Transp. Syst., vol. 9, no. 2, pp. 349-360, Jun. 2008.
10. M. Roser and A. Geiger, "Video-based raindrop detection for improved image registration," in IEEE Int. Conf. Comput. Vis.Workshops, Kyoto, Sep. 2009, pp. 570-577.
11. J. C. Halimeh and M. Roser, "Raindrop detection on car windshields using geometric-photometric environment construction and intensitybased correlation," in Proc. IEEE Intell. Veh. Symp., Xi'an, China, Jun. 2009, pp. 610-615.
12. Google Goggles [Online]. Available: http://www.google.com/mobile/goggles/
13. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.
14. H. Baya, A. Essa, T. Tuytelaarsb, and L. V. Gool, "Speeded-up robust features (SURF)," Comput. Vis. Image Understand., vol. 110, no. 3, pp. 346-359, Jun. 2008.
15. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., San Diego, CA, Jun. 2005, vol. 1, pp. 886-893. (Pubitemid 43897286)
16. O. Ludwig, D. Delgado, V. Goncalves, and U. Nunes, "Trainable classifier-fusion schemes: An application to pedestrian detection," in Proc. IEEE Int. Conf. Intell. Transp. Syst., St. Louis, MO, Oct. 2009, pp. 1-6.
17. S. Maji, A. C. Berg, and J. Malik, "Classification using intersection kernel support vector machines is efficient," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Anchorage, AK, Jun. 2008, pp. 1-8.
18. L. Itti, C.Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998. (Pubitemid 128742001)
19. A. Buades, B. Coll, and J. M. Morel, "A review of image denoising algorithms, with a new one," Multisc. Model. Simul., vol. 4, no. 2, pp. 490-530, 2005. (Pubitemid 43890582)
20. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736-3745, Dec. 2006. (Pubitemid 44811686)
21. M. Aharon, M. Elad, and A. M. Bruckstein, "The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation," IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006. (Pubitemid 44637761)
22. J. Mairal, M. Elad, and G. Sapiro, "Sparse representation for color image restoration," IEEE Trans. Image Process., vol. 17, no. 1, pp. 53-69, Jan. 2008.
23. M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. New York: Springer-Verlag, 2010.
24. J. Mairal, F. Bach, and J. Ponce, "Task-driven dictionary learning," IEEE Trans. Pattern Anal. Mach. Intell., to be published, to be published.
25. Y. H. Fu, L. W. Kang, C. W. Lin, and C. T. Hsu, "Single-frame-based rain removal via image decomposition," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., Prague, Czech Republic, May 2011, pp. 1453-1456.
26. J. M. Fadili, J. L. Starck, J. Bobin, and Y. Moudden, "Image decomposition and separation using sparse representations: An overview," Proc. IEEE, vol. 98, no. 6, pp. 983-994, Jun. 2010.
27. J. M. Fadili, J. L. Starck, M. Elad, and D. L. Donoho, "MCALab: Reproducible research in signal and image decomposition and inpainting," IEEE Comput. Sci. Eng., vol. 12, no. 1, pp. 44-63, Jan./Feb. 2010.
28. J. Bobin, J. L. Starck, J. M. Fadili, Y. Moudden, and D. L. Donoho, "Morphological component analysis: An adaptive thresholding strategy," IEEE Trans. Image Process., vol. 16, no. 11, pp. 2675-2681, Nov. 2007. (Pubitemid 350147649)
29. G. Peyré, J. Fadili, and J. L. Starck, "Learning adapted dictionaries for geometry and texture separation," in Proc. SPIE, 2007, vol. 6701, p. 67 011T.
30. J. L. Starck, M. Elad, and D. L. Donoho, "Image decomposition via the combination of sparse representations and a variational approach," IEEE Trans. Image Process., vol. 14, no. 10, pp. 1570-1582, Oct. 2005. (Pubitemid 41623087)
31. B. A. Olshausen and D. J. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature, vol. 381, no. 6583, pp. 607-609, Jun. 1996. (Pubitemid 26177476)
32. S. Mallat and Z. Zhang, "Matching pursuits with time-frequency dictionaries," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
33. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online learning for matrix factorization and sparse coding," J. Mach. Learn. Res., vol. 11, pp. 19-60, Jan. 2010.
34. C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," in Proc. IEEE Int. Conf. Comput. Vis., Bombay, India, Jan. 1998, pp. 839-846.
35. M. Zhang and B. K. Gunturk, "Multiresolution bilateral filtering for image denoising," IEEE Trans. Image Process., vol. 17, no. 12, pp. 2324-2333, Dec. 2008.
36. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
37. A. M. Bruckstein, D. L. Donoho, and M. Elad, "From sparse solutions of systems of equations to sparse modeling of signals and images," SIAM Rev., vol. 51, no. 1, pp. 34-81, Feb. 2009.
38. H.-W. Chen, L.-W. Kang, and C.-S. Lu, "Dictionary learning-based distributed compressive video sensing," in Proc. Picture Coding Symp., Nagoya, Japan, Dec. 2010, pp. 210-213.
39. H. R. Sheikh and A. C. Bovik, "Image information and visual quality," IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006. (Pubitemid 43126168)
40. D. Tang, "Rain," Photoshop Tutorials [Online]. Available: http://photoshoptutorials. ws/photoshop-tutorials/photo-effects/rain.html
41. S. Patterson, Photoshop Rain Effect-Adding Rain to a Photo [Online]. Available: http://www.photoshopessentials.com/photo-effects/rain/
42. NTHU Rain Removal Project [Online]. Available: http://www.ee.nthu.edu.tw/ cwlin/Rain-Removal/Rain-Removal.htm
43. D. Lanman, Bilateral Filtering and Image Abstraction [Online]. Available: http://mesh.brown.edu/dlanman/courses.html
44. J. M. Duarte-Carvajalino and G. Sapiro, "Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization," IEEE Trans. Image Process., vol. 18, no. 7, pp. 1395-1408, Jul. 2009.
45. Y. Jia, M. Salzmann, and T. Darrell, "Factorized latent spaces with structured sparsity," in Proc. Conf. Neural Inf. Proc. Syst., Vancouver, BC, Canada, Dec. 2010, pp. 982-990.