Nonlocal mumford-shah regularizers for color image restoration

IEEE Transactions on Image Processing

Volumn 20, Issue 6, 2011, Pages 1583-1598

  Jung, Miyoun ^a   Bresson, Xavier ^c   Chan, Tony F ^d   Vese, Luminita A ^b  

^a UNIVERSITÉ PARIS DAUPHINE   (France)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^d HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

Ambrosio Tortorelli elliptic approximations; deblurring; demosaicing; denoising; impulse noise; inpainting; Mumford Shah (MS) regularizer; nonlocal operators; super resolution

Indexed keywords

DE-NOISING; DEBLURRING; DEMOSAICING; ELLIPTIC APPROXIMATION; INPAINTING; MUMFORD-SHAH; NONLOCAL OPERATOR; SUPER RESOLUTION;

COLOR IMAGE PROCESSING; IMAGE RECONSTRUCTION; IMPULSE NOISE; MATHEMATICAL OPERATORS; NOISE POLLUTION CONTROL; OPTICAL RESOLVING POWER; RESTORATION; TEXTURES;

COLOR;

ALGORITHM; ARTICLE; COLOR; COLORIMETRY; COMPUTER ASSISTED DIAGNOSIS; IMAGE ENHANCEMENT; METHODOLOGY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY;

ALGORITHMS; COLOR; COLORIMETRY; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 79957496623     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2010.2092433     Document Type: Article

References (80)

1. L. Ambrosio and V. M. Tortorelli, "Approximation of functionals depending on jumps by elliptic functionals via Γ-convergence," Commun. Pure Appl. Math., vol. 43, pp. 999-1036, 1990.
2. L. Ambrosio and V. M. Tortorelli, "On the approximation of free discontinuity problems," Bollettino dell'Unione Matematica Italiana, vol. B(7), no. 6, pp. 105-123, 1992.
3. 1 -norm of the gradient and their approximation," Interfaces and Free Boundaries, vol. 1, no. 1, pp. 17-37, 1999.
4. S. Alliney, "Digital filters as absolute norm regularizers," IEEE Trans. Signal Process., vol. 40, no. 6, pp. 1548-1562, Jun. 1992.
5. G. Aubert and P. Kornprobst, "Mathematical problems in image processing: Partial differential equations and the calculus of variations," in Applied Mathematical Sciences. New York: Springer-Verlag, 2006, vol. 147.
6. J. F. Aujol, S. Ladjal, and S. Masnou, "Exemplar-based inpainting from a variational point of view," SIAM J. Math. Anal., vol. 42, no. 3, pp. 1246-1285, 2010.
7. L. Bar, A. Brook, N. Sochen, and N. Kiryati, "Deblurring of color images corrupted by impulsive noise," IEEE Trans. Image Process., vol. 16, no. 4, pp. 1101-1111, Apr. 2007. (Pubitemid 46542396)
8. L. Bar, N. Sochen, and N. Kiryati, "Semi-blind image restoration via Mumford-Shah regularization," IEEE Trans. Image Process., vol. 15, no. 2, pp. 483-493, Feb. 2006. (Pubitemid 43126173)
9. L. Bar, N. Sochen, and N. Kiryati, "Image deblurring in the presence of impulsive noise," Int. J. Comput. Vis., vol. 70, no. 3, pp. 279-298, 2006. (Pubitemid 44127250)
10. B. E. Bayer, "Color Imaging Array," U.S. Patent 3 971 065, Jul. 20, 1976.
11. M. Bertalmío, G. Sapiro, V. Caselles, and C. Ballester, "Image inpainting," in Proc. SIGGRPAPH, 2000, pp. 417-424. (Pubitemid 32394006)
12. M. Bertalmio, L. A. Vese, G. Sapiro, and S. Osher, "Simultaneous structure and texture image inpainting," IEEE Trans. Image Process., vol. 12, no. 8, pp. 882-889, Aug. 2003.
13. M. Bertalmio, A. L. Bertozzi, and G. Sapiro, "Navier-stokes, fluid dynamics, and image and video inpainting," Proc. IEEE Comput. Vis. Pattern Recognit., vol. 1, pp. I-355, 2001.
14. P. Blomgren and T. F. Chan, "Color TV: Total variation methods for restoration of vector-valued images," IEEE Trans. Image Process., vol. 7, no. 3, pp. 304-309, Mar. 1998. (Pubitemid 128745348)
15. L. M. Bregman, "The relaxation method for finding common points of convex sets and its application to the solution of problems in convex programming," USSR Comput. Math. Math. Phys., vol. 7, no. 3, pp. 200-217, 1967.
16. X. Bresson and T. F. Chan, "Non-local unsupervised variational image segmentation models," Univ. California, Los Angeles, UCLA C.A.M. Rep. 08-67, 2008.
17. X. Bresson and T. F. Chan, "Fast minimization of the vectorial total variation norm and applications to color image processing," Inv. Problems Imag., vol. 2, no. 4, pp. 455-484, 2008.
18. A. Brook, R. Kimmel, and N. A. Sochen, "Variational restoration and edge detection for color images," JMIV, vol. 18, no. 3, pp. 247-268, 2003.
19. A. Buades, B. Coll, and J. M. Morel, "A review of image denoising algorithms, with a new one," SIAM Multiscale Model. Sim., vol. 4, no. 2, pp. 490-530, 2005. (Pubitemid 43890582)
20. A. Buades, B. Coll, and J. M. Morel, Image Enhancement by Non-Local Reverse Heat Equation, Preprint CMLA 2006-22. : , 2006.
21. A. Buades, B. Coll, J. M. Morel, and C. Sbert, "Self similarity driven demosaicking," IEEE Trans. Image Process., vol. 18, no. 6, pp. 1192-1202, Jun. 2009.
22. J. F. Cai, R. H. Chan, and M. Nikolova, "Fast two-phase image deblurring under impulse noise," JMIV, vol. 36, no. 1, pp. 46-53, 2010.
23. V. Caselles, G. Sapiro, C. Ballester, M. Bertalmio, and J. Verdera, "Filling-in by joint interpolation of vector fields and grey levels," IEEE Trans. Image Process, vol. 10, no. 8, pp. 1200-1211, Aug. 2001. (Pubitemid 32732681)
24. T. F. Chan, S. H. Kang, and J. Shen, "Euler's elastica and curvature based inpainting," SIAM J. Appl. Math., vol. 63, no. 2, pp. 564-592, 2002.
25. T. F. Chan and J. Shen, "Mathematical models for local nontexture inpaintings," SIAM J. Appl. Math., vol. 62, no. 3, pp. 1019-1043, 2002. (Pubitemid 34758536)
26. T. F. Chan and C.-K. Wong, "Total variation blind deconvolution," IEEE Trans. Image Process., vol. 7, no. 3, pp. 370-375, Mar. 1998. (Pubitemid 128745355)
27. R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F.Warner, and S. W. Zucker, "Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps," Proc. Nat. Acad. Sci., vol. 102, no. 21, pp. 7426-7431, 2005. (Pubitemid 40740988)
28. A. Criminisi, P. Pérez, and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE Trans. Image Process., vol. 13, no. 9, pp. 1200-1212, Sep. 2004.
29. D. Datsenko and M. Elad, "Example-based single image super-resolution: A global map approach with outlier rejection," J. Mult. Syst. Signal. Process., vol. 18, no. 2-3, pp. 103-121, 2007. (Pubitemid 46753782)
30. L. Demanet, B. Song, and T. Chan, "Image inpainting by correspondence maps: A deterministic approach," in Proc. 2nd IEEE Workshop Variational, Geomet. Level Set Meth. Comput. Vis., Nice, France, 2003.
31. A. A. Effros and T. K. Leung, "Texture synthesis by non-parametric sampling," in Proc. 7th IEEE Int. Conf. Comput. Vis., 1999, vol. 2, pp. 1033-1038.
32. M. Elad, J. L. Starck, D. Donoho, and P. Querre, "Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA)," Appl. Comput. Harmon. Anal., vol. 19, no. 3, pp. 340-358, 2005. (Pubitemid 41660098)
33. A. Elmoataz, O. Lezoray, and S. Bougleux, "Nonlocal discrete regularization on weighted graphs: A framework for image and manifold processing," IEEE Trans. Image Process., vol. 17, no. 7, pp. 1047-1060, Jul. 2008.
34. S. Esedoglu and J. Shen, "Digital inpainting based on the Mumford- Shah-Euler image model," Eur. J. Appl. Math., vol. 13, pp. 353-370, 2002.
35. S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in superresolution," Int. J. Imag. Syst. Tech., vol. 14, no. 2, pp. 47-57, 2004.
36. W. T. Freeman, T. R. Jones, and E. C. Pasztor, "Example-based superresolution," IEEE Comput. Graph. Appl., vol. 22, no. 2, pp. 56-65, Mar./Apr. 2002.
37. D. Geman and G. Reynolds, "Constrained restoration and the recovery of discontinuities," IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 3, pp. 367-383, Mar. 1992.
38. G. Gilboa, J. Darbon, S. Osher, and T. Chan, "Nonlocal convex functionals for image regularization," Univ. California, Los Angeles, UCLA CAM Rep. 06-57, 2006.
39. G. Gilboa and S. Osher, "Nonlocal linear image regularization and supervised segmentation," SIAM Multiscale Model. Sim., vol. 6, no. 2, pp. 595-630, 2007.
40. G. Gilboa and S. Osher, "Nonlocal operators with applications to image processing," SIAM Multiscale Model. Sim., vol. 7, no. 3, pp. 1005-1028, 2008.
41. B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, "Color plane interpolation using alternating projections," IEEE Trans. Image Process., vol. 11, no. 9, pp. 997-1013, Sep. 2002. (Pubitemid 35250384)
42. B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, "Demosaicing: Color filter array interpolation," IEEE Signal Process. Mag., vol. 22, no. 1, pp. 44-54, 2005. (Pubitemid 40319547)
43. J. F. Hamilton and J. E. Adams, "Adaptive Color Plan Interpolation in Single Sensor Color Electronic Camera," U.S. Patent 5506619, Jul. 29, 1997.
44. L. He, A. Marquina, and S. Osher, "Blind deconvolution using TV regularization and Bregman iteration," Int. J. Imag. Syst. Technol., vol. 15, no. 1, pp. 74-83, 2005. (Pubitemid 41364767)
45. K. Hirakawa and T. W. Paks, "Adaptive homogeneity-directed demosaicing algorithm," IEEE Trans. Image Process., vol. 14, no. 3, pp. 360-369, Mar. 2005. (Pubitemid 40351913)
46. K. Hirakawa and T. W. Paks, "Joint demosaicing and denoising," IEEE Trans. Image Process., vol. 15, no. 8, pp. 2146-2157, Aug. 2006. (Pubitemid 44089091)
47. M. Jung, X. Bresson, T. F. Chan, and L. A. Vese, "Color image restoration using nonlocal Mumford-Shah regularizers," LNCS, vol. 5681, pp. 373-387, 2009.
48. M. Jung, G. Chung, G. Sundaramoorthi, L. A. Vese, and A. L. Yuille, "Sobolev gradients and joint variational image segmentation, denoising and deblurring," Electron. Imag., vol. 7246, pp. 72460I-1, 2009.
49. M. Jung and L. A.Vese, "Nonlocal variational image deblurring models in the presence of Gaussian or impulse noise," LNCS, vol. 5567, pp. 402-413, 2009.
50. D. Kim, A. Tsai, M. Cetin, and A. Willsky, "A curve evolution based variational approach to simultaneous image restoration and segmentation," in Proc. IEEE Int. Conf. Image Process., 2002, vol. 1, pp. 109-112.
51. R. Kimmel, "Demosaicing: Image reconstruction from color CCDsamples," IEEE Trans. Image Process., vol. 8, no. 9, pp. 1221-1228, Sep. 1999.
52. S. Kindermann, S. Osher, and P. W. Jones, "Deblurring and denoising of images by nonlocal functionals," SIAM Multiscale Model. Sim., vol. 4, no. 4, pp. 1091-1115, 2005. (Pubitemid 44932284)
53. Y. Lou, X. Zhang, S. Osher, and A. Bertozzi, "Image recovery via nonlocal operators," J. Sci. Comput., vol. 42, no. 2, pp. 185-197, 2010.
54. W. Lu and Y.-P. Tan, "Color filter array demosaicing: New method and performance measures," IEEE Trans. Image Process., vol. 12, no. 10, pp. 1194-1210, Oct. 2003.
55. M. Mahmoudi and G. Sapiro, "Fast image and video denoising via nonlocal means of similar neighborhoods," IEEE Signal Process. Lett., vol. 12, no. 12, p. 839, Dec. 2005.
56. F. Malgouyres and F. Guichard, "Edge direction preserving image zooming: A mathematical and numerical analysis," SIAM Numer. Anal., vol. 39, no. 1, pp. 1-37, 2001.
57. S. Masnou and J.-M. Morel, "Level lines-based disocclusion," in Proc. IEEE Int. Conf. Image Process., 1998, vol. 3, pp. 259-263.
58. S. Masnou and J.-M. Morel, "Disocclusion: A variational approach using level lines," IEEE Trans. Image Process., vol. 11, no. 1, pp. 68-76, Jan. 2002. (Pubitemid 34278444)
59. Y. Meyer, "Oscillating patterns in image processing and nonlinear evolution equations," Univ. Lecture Ser. 22., Am. Math. Soc., Providence, 2002.
60. P. Milanfar [Online]. Available: http://users.soe.ucsc.edu/~milanfar/ software/
61. D. Mumford and J. Shah, "Optimal approximations by piecewise smooth functions and associated variational problems," CPAM, vol. XLII, pp. 577-685, 1989.
62. M. Nikolova, "Minimizers of cost-functions involving non-smooth data-fidelity terms. Application to the processing of outliers," SIAM Numer. Anal., vol. 40, no. 3, pp. 965-994, 2002.
63. G. Peyré, "Image processing with non-local spectral bases," SIAM Multiscale Model. Sim., vol. 7, no. 2, pp. 703-730, 2008.
64. G. Peyré, S. Bougleux, and L. Cohen, "Non-local regularization of inverse problems," LNCS, vol. 5304, pp. 57-68, 2008.
65. P. Pérez, M. Gangnet, and A. Blake, "PatchWorks: Example-based region tiling for image editing," Microsoft Corporation, Redmond, WA, Tech Rep. Microsoft Research, MSR-TR-2004-04, 2004.
66. M. Protter,M. Elad, H. Takeda, and P. Milanfar, "Generalizing the nonlocal- means to super-resolution reconstruction," IEEE Trans. Image Process., vol. 18, no. 1, pp. 36-51, Jan. 2009.
67. L. Rudin, S. Osher, and E. Fatemi, "Nonlinear total variation based noise removal algorithms," Phys. D, vol. 60, pp. 259-268, 1992.
68. L. Rudin and S. Osher, "Total variation based image restoration with free local constraints," in Proc. IEEE Int. Conf. Image Process., 1994, vol. 1, pp. 31-35.
69. J. Shah, "A common framework for curve evolution, segmentation and anisotropic diffusion," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 1996, pp. 136-142.
70. J. Shen and T. F. Chan, "Variational image inpainting," CPAM, vol. 58, no. 5, pp. 579-619, 2005.
71. S. M. Smith and J. M. Brady, "SUSAN-A new approach to low level image processing," Int. J. Comput. Vis., vol. 23, no. 1, pp. 45-78, 1997. (Pubitemid 127717027)
72. A. Szlam, M. Maggioni, and R. R. Coifman, "Regularization on graphs with function-adapted diffusion processes," J. Mach. Learn. Res., vol. 9, pp. 1711-1739, 2008.
73. E. Tadmor, S. Nezzar, and L. Vese, "Multiscale hierarchical decomposition of images with applications to deblurring, denoising and segmentation," Commun. Math. Sci., vol. 6, no. 2, pp. 281-307, 2008.
74. C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," in Proc. 6th Int. Conf. Comput. Vis., 1998, pp. 839-846.
75. D. Tschumperlé and R. Deriche, "Vector-valued image regularization with PDE's: A common framework for different applications," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 4, pp. 506-517, Apr. 2005. (Pubitemid 40556107)
76. L. Y. Wei and M. Levoy, "Fast texture synthesis using tree-structured vector quantization," Proc. SIGGRAPH 2000, pp. 479-488, 2000. (Pubitemid 32394012)
77. L. P. Yaroslavsky, Digital Image Processing: An Introduction. New York: Springer-Verlag, 1985.
78. X. Zhang, M. Burger, X. Bresson, and S. Osher, "Bregmanized nonlocal regularization for deconvolution and sparse reconstruction," SIAM J. Imag. Sci., vol. 3, no. 3, pp. 253-276, 2010.
79. D. Zhou and B. Scholkopf, "A regularization framework for learning from graph data," in Proc. Int. Conf. Mach. Learn., Banff, Canada, 2004, pp. 132-137.
80. D. Zhou and B. Scholkopf, "Regularization on discrete spaces," LNCS, vol. 3663, pp. 361-368, 2005. (Pubitemid 41520675)