Augmented Lagrangian Method, Dual Methods, and Split Bregman Iteration for ROF, Vectorial TV, and High Order Models

SIAM Journal on Imaging Sciences

Volumn 3, Issue 3, 2010, Pages 300-339

  Wu, Chunlin ^a   Tai, Xue Cheng ^a,b  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^b UNIVERSITY OF BERGEN   (Norway)

Author keywords

Augmented Lagrangian method; Dual method; ROF model; Split Bregman iteration; Total variation

Indexed keywords

AUGMENTED LAGRANGIAN METHODS; BREGMAN ITERATION; DUAL METHOD; ROF MODEL; TOTAL VARIATION;

IMAGE PROCESSING; ITERATIVE METHODS;

LAGRANGE MULTIPLIERS;

EID: 78650834156     PISSN: None     EISSN: 19364954     Source Type: Journal    
DOI: 10.1137/090767558     Document Type: Review

References (49)

1. D.P. Bertsekas, Multiplier methods: A survey, Automatica. J. IFAC, 12 (1976), pp. 133-145.
2. D.P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Athena Scientific, Belmont,MA, 1996.
3. D.P. Bertsekas and J.N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods,Athena Scientific, Belmont, MA, 1997.
4. P. Blomgren and T.F. Chan, Color TV: Total variation methods for restoration of vector-valuedimages, IEEE Trans. Image Process., 7 (1998), pp. 304-309.
5. L.M. Bregman, The relaxation method of finding the common point of convex sets and its applicationto the solution of problems in convex programming, USSR Comput. Math. Math. Phys., 7 (1967),pp. 200-217.
6. X. Bresson and T.F. Chan, Fast dual minimization of the vectorial total variation norm and applicationsto color image processing, Inverse Probl. Imaging, 2 (2008), pp. 455-484.
7. A. Buades, B. Coll, and J.M. Morel, The staircasing effect in neighborhood filters and its solution,IEEE Trans. Image Process., 15 (2006), pp. 1499-1505.
8. A. Caboussat, R. Glowinski, and V. Pons, An augmented Lagrangian approach to the numericalsolution of a non-smooth eigenvalue problem, J. Numer. Math., 17 (2009), pp. 3-26.
9. J. Cai, S. Osher, and Z. Shen, Linear Bregman iterations for compressed sensing, Math. Comp., 78(2009), pp. 1515-1536.
10. J.-F. Cai, S. Osher, and Z. Shen, Split Bregman methods and frame based image restoration, MultiscaleModel. Simul, 8 (2009), pp. 337-369.
11. J.L. Carter, Dual Methods for Total Variation-Based Image Restoration, Ph.D. thesis, Department ofMathematics, UCLA, Los Angeles, CA, 2001.
12. A. Chambolle, An algorithm for total variation minimization and applications, J. Math. Imaging Vis.,20 (2004), pp. 89-97.
13. A. Chambolle and P.L. Lions, Image recovery via total variation minimization and related problems,Numer. Math., 76 (1997), pp. 167-188.
14. T.F. Chan, S. Esedoglu, and F.E. Park, A Fourth Order Dual Method for Staircase Reduction inTexture Extraction and Image Restoration Problems, UCLA CAM Report cam05-28, Department ofMathematics, UCLA, Los Angeles, CA, 2005.
15. T. Chan, S. Esedoglu, F. Park, and A. Yip, Recent developments in total variation image restoration,in Mathematical Models of Computer Vision, N. Paragios, Y. Chen, and O. Faugeras, eds., Springer-Verlag, Berlin, 2005.
16. T.F. Chan, G.H. Golub, and P. Mulet, A nonlinear primal-dual method for total variation-basedimage restoration, SIAM J. Sci. Comput., 20 (1999), pp. 1964-1977.
17. T.F. Chan, S.H. Kang, and J.H. Shen, Total variation denoising and enhancement of color imagesbased on the CB and HSV color models, J. Visual Commun. Image Rep., 12 (2001), pp. 422-435.
18. T. Chan, A. Marquina, and P. Mulet, High-order total variation-based image restoration, SIAM J.Sci. Comput., 22 (2000), pp. 503-516.
19. D.L. Donoho, De-noising by soft-thresholding, IEEE Trans. Inform. Theory, 41 (1995), pp. 613-627.
20. J. Eckstein and D.P. Bertsekas, On the Douglas-Rachford splitting method and the proximal pointalgorithm for maximal monotone operators, Math. Programming, 55 (1992), pp. 293-318.
21. I. Ekeland and R. Témam, Convex Analysis and Variational Problems, Classics in Appl. Math. 28,SIAM, Philadelphia, 1999.
22. E. Esser, Applications of Lagrangian-Based Alternating Direction Methods and Connections to SplitBregman, UCLA CAM Report 09-31, Department of Mathematics, UCLA, Los Angeles, CA, 2009.
23. R. Glowinski and P. Le Tallec, Augmented Lagrangians and Operator-Splitting Methods in NonlinearMechanics, SIAM Stud. Appl. Math. 9, SIAM, Philadelphia, 1989.
24. T. Goldstein and S. Osher, The split Bregman method for L1-regularized problems, SIAM J. ImagingSci., 2 (2009), pp. 323-343.
25. M.R. Hestenes, Multiplier and gradient methods, J. Optim. Theory Appl., 4 (1969), pp. 303-320.
26. W. Hinterberger and O. Scherzer, Variational methods on the space of functions of bounded Hessianfor convexification and denoising, Computing, 76 (2006), pp. 109-133.
27. Y. Huang, M.K. Ng, and Y.W. Wen, A fast total variation minimization method for image restoration,Multiscale Model. Simul., 7 (2008), pp. 774-795.
28. K. Jalalzai and A. Chambolle, Enhancement of blurred and noisy images based on an original variantof the total variation, Second International Conference on Scale Space and Variational Methods inComputer Vision (SSVM 2009), Voss, Norway, 2009, Lecture Notes in Comput. Sci. 5567, Springer-Verlag, Berlin, 2009, pp. 368-376.
29. M. Lysaker, A. Lundervold, and X.-C. Tai, Noise removal using fourth-order partial differentialequations with applications to medical magnetic resonance images in space and time, IEEE Trans.Image Process., 12 (2003), pp. 1579-1590.
30. M. Lysaker and X.-C. Tai, Iterative image restoration combining total variation minimization and asecond-order functional, Int. J. Computer Vision, 66 (2006), pp. 5-18.
31. S. Osher, M. Burger, D. Goldfarb, J.J. Xu, and W.T. Yin, An iterative regularization method fortotal variation-based image restoration, Multiscale Model. Simul., 4 (2005), pp. 460-489.
32. M.J.D. Powell, A method for nonlinear constraints in minimization problems, in Optimization, R.Fletcher, ed., Academic Press, New York, 1972, pp. 283-298.
33. R.T. Rockafellar, A dual approach to solving nonlinear programming problems by unconstrained optimization,Math. Programming, 5 (1973), pp. 354-373.
34. L. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys.D, 60 (1992), pp. 259-268.
35. G. Sapiro and D.L. Ringach, Anisotropic diffusion of multivalued images with applications to colorfiltering, IEEE Trans. Image Process., 5 (1996), pp. 1582-1586.
36. O. Scherer, Denoising with higher order derivatives of bounded variation and an application to parameterestimation, Comput., 60 (1998), pp. 1-27.
37. S. Setzer, Split Bregman algorithm, Douglas-Rachford splitting and frame shrinkage, in Proceedings ofthe Second International Conference on Scale Space and Variational Methods in Computer Vision(SSVM 2009), Voss, Norway, 2009, Lecture Notes in Comput. Sci. 5567, Springer Verlag, Berlin, 2009,pp. 464-476.
38. G. Steidl, A note on the dual treatment of higher-order regularization functionals, Comput., 76 (2006),pp. 135-148.
39. X.-C. Tai and C. Wu, Augmented Lagrangian method, dual methods and split Bregman iteration forROF model, in Proceedings of the Second International Conference on Scale Space and Variational Methods in Computer Vision (SSVM 2009), Voss, Norway, 2009, Lecture Notes in Comput. Sci. 5567,Springer-Verlag, Berlin, 2009, pp. 502-513.
40. Y. Wang, J. Yang, W. Yin, and Y. Zhang, A new alternating minimization algorithm for totalvariation image reconstruction, SIAM J. Imaging Sci., 1 (2008), pp. 248-272.
41. Y.L. Wang, W.T. Yin, and Y. Zhang, A Fast Algorithm for Image Deblurring with Total VariationRegularization, UCLA CAM Report 07-22, Department of Mathematics, UCLA, Los Angeles, CA,2007.
42. J. Weickert, Anisotropic Diffusion in Image Processing, B.G. Teubner, Stuttgart, 1998.
43. J. Yang, W. Yin, Y. Zhang, and Y. Wang, A fast algorithm for edge-preserving variational multichannelimage restoration, SIAM J. Imaging Sci., 2 (2009), pp. 569-592.
44. W.T. Yin, Analysis and Generalizations of the Linearized Bregman Method, UCLA CAM Report 09-42,Department of Mathematics, UCLA, Los Angeles, CA, 2009.
45. W. Yin, S. Osher, D. Goldfarb, and J. Darbon, Bregman iterative algorithms for l1-minimizationwith applications to compressed sensing, SIAM J. Imaging Sci., 1 (2008), pp. 143-168.
46. Y.-L. You and M. Kaveh, Fourth-order partial differential equation for noise removal, IEEE Trans.Image Process., 9 (2000), pp. 1723-1730.
47. Y.-L. You, W. Xu, A. Tannenbaum, and M. Kaveh, Behavioral analysis of anisotropic diffusion inimage processing, IEEE Trans. Image Process., 5 (1996), pp. 1539-1553.
48. X. Zhang, M. Burger, X. Bresson, and S. Osher, Bregmanized nonlocal regularization for deconvolutionand sparse reconstruction, SIAM J. Imaging Sci., 3 (2010), pp. 253-276.
49. M. Zhu, S.J. Wright, and T.F. Chan, Duality-based algorithms for total-variation-regularized imagerestoration, Comput. Optim. Appl., 2008; DOI: 10.1007/s10589-008-9225-2.