A bilevel optimization approach for parameter learning in variational models

SIAM Journal on Imaging Sciences

Volumn 6, Issue 2, 2013, Pages 938-983

  Kunisch, Karl ^a   Pock, Thomas ^b  

^a UNIVERSITY OF GRAZ   (Austria)

^b GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

Author keywords

Bilevel optimization; Image denoising; Learning theory; Nondifferentiable optimization; Regularization parameter; Semismooth Newton algorithm

Indexed keywords

BI-LEVEL OPTIMIZATION; LEARNING THEORY; NON-DIFFERENTIABLE OPTIMIZATION; REGULARIZATION PARAMETERS; SEMISMOOTH NEWTON ALGORITHM;

IMAGE DENOISING; MATHEMATICAL OPERATORS; NEWTON-RAPHSON METHOD;

OPTIMIZATION;

EID: 84879933290     PISSN: None     EISSN: 19364954     Source Type: Journal    
DOI: 10.1137/120882706     Document Type: Article

References (33)

1. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, Contour detection and hierarchical image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 33 (2011), pp. 898-916.
2. J. Bard, Practical Bilevel Optimization, Kluwer Academic Publishers, Dordrecht, The Netherlands, 1998.
3. A. Chambolle and P.-L. Lions, Image recovery via total variation minimization and related problems, Numer. Math., 76 (1997), pp. 167-188.
4. A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vision, 40 (2010), pp. 120-145.
5. T. F. Chan, G. H. Golub, and P. Mulet, A nonlinear primal-dual method for total variation-based image restoration, SIAM J. Sci. Comput., 20 (1999), pp. 1964-1977.
6. R. W. Chaney, Second-order necessary conditions in constrained semismooth optimization, SIAM J. Control Optim., 25 (1987), pp. 1072-1081.
7. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image restoration by sparse 3d transform-domain collaborative filtering, in Proceedings of SPIE Electronic Imaging, 2008.
8. I. Daubechies, R. DeVore, M. Fornasier, and C. S. Güntürk, Iteratively reweighted least squares minimization for sparse recovery, Comm. Pure Appl. Math., 63 (2010), pp. 1-38.
9. J. C. De los Reyes, Optimal control of a class of variational inequalities of the second kind., SIAM J. Control Optim., 49 (2011), pp. 1629-1658.
10. A. Doicu, T. Trautmann, and T. Schreier, Numerical Regularization for Atmospheric Inverse Problems, Springer-Verlag, Heidelberg, 2010.
11. M. Elad and M. Aharon, Image denoising via sparse and redundant representations over learned dictionaries, IEEE Trans. Image Process., 15 (2006), pp. 3736-3745.
12. H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems, Math. Appl. 375, Kluwer Academic Publishers, Dordrecht, The Netherlands, 1996.
13. W. Hinterberger, M. Hintermüller, K. Kunisch, M. Von Oehsen, and O. Scherzer, Tube methods for BV regularization, J. Math. Imaging Vision, 19 (2003), pp. 219-235.
14. M. Hintermüller and G. Stadler, An infeasible primal-dual algorithm for total bounded variation-based inf-convolution-type image restoration, S IAM J. Sci. Comput., 28 (2006), pp. 1-23.
15. J. Huang and D. Mumford, Statistics of natural images and models, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR1999), Fort Collins, CO, 1999, pp. 541-547.
16. K. Ito and K. Kunisch, Newton's method for a class of weakly singular optimal control problems, SIAM J. Optim., 10 (2000), pp. 896-916.
17. K. Ito and K. Kunisch, Lagrange Multiplier Approach to Variational Problems and Applications, SIAM, Philadelphia, 2008.
18. T. Kato, Perturbation Theory for Linear Operators, 2nd ed., Springer-Verlag, Berlin, 1976.
19. C. Liu, W. T. Freeman, R. Szeliski, and S. B. Kang, Noise estimation from a single image, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR2006), 2006, pp. 901-908.
20. D. C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization, Math. Programming, 45 (1989), pp. 503-528.
21. D. Luenberger and Y. Ye, Linear and Nonline ar Programming, 3rd ed., Springer, New York, 2008.
22. J. Nocedal and S. J. Wright, Numerical Optimization, Springer, New York, 2000.
23. J. V. Outrata, A generalized mathematical program with equilibrium constraints, SIAM J. Control Optim., 38 (2000), p. 1623-1638.
24. G. Peyré and J. Fadili, Learning analysis sparsity priors, in Proceedings of the International Conference on Sampling Theory and Applications (Sampta'11), 2011.
25. T. Pock and A. Chambolle, Diagonal preconditioning for first order primal-dual algorithms, in Proceedings of the IEEE International Conference on Computer Vision (ICCV 2011), 2011.
26. S. Roth and M. J. Black, Fields of experts, Int. J. Comput. Vis., 82 (2009), pp. 205-229.
27. K. G. G. Samuel and M. Tappen, Learning optimized map estimates in continuously-valued MRF models, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR2009), 2009.
28. M. F. Tappen, Utilizing variational optimization to learn Markov random fields, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR2007), 2007.
29. M. F. Tappen, C. Liu, E. H. Adelson, and W. T. Freeman, Learning Gaussian conditional random fields for low-level vision, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR2007), 2007.
30. M. Ulbrich, Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems in Function Spaces, MOS-SIAM Ser. Optim. 11, SIAM, Philadelphia, 2011.
31. S. Vaiter, G. Peyré, C. Dossal, and J. Fadili, Robust sparse analysis regularization, IEEE Trans. Inform. Theory, 59 (2013), pp. 2001-2016.
32. C. R. Vogel and M. E. Oman, Iterative methods for total variation denoising, SIAM J. Sci. Comput., 17 (1996), pp. 227-238.
33. D. Zoran and Y. Weiss, From learning models of natural image patches to whole image restoration, in Proceedings of the IEEE International Conference on Computer Vision (ICCV 2011), 2011, pp. 479-486.