Stein Unbiased GrAdient estimator of the Risk (SUGAR) for multiple parameter selection

SIAM Journal on Imaging Sciences

Volumn 7, Issue 4, 2014, Pages 2448-2487

  Deledalle, Charles Alban ^a   Vaiter, Samuel ^b   Fadili, Jalal ^c   Peyré, Gabriel ^b  

^a UNIVERSITÉ DE BORDEAUX   (France)

^b UNIVERSITÉ PARIS DAUPHINE   (France)

^c UNIVERSITÉ DE CAEN   (France)

Author keywords

Inverse problems; Parameter selection; Proximal splitting; Risk estimation; Sparsity; SURE

Indexed keywords

ALGORITHMS; DIFFERENTIAL EQUATIONS; IMAGE RECONSTRUCTION; INVERSE PROBLEMS; ITERATIVE METHODS; MAPPING; OPTIMIZATION; RISK PERCEPTION;

PARAMETER SELECTION; PROXIMAL SPLITTING; RISK ESTIMATION; SPARSITY; SURE;

PARAMETER ESTIMATION;

EID: 84919652783     PISSN: None     EISSN: 19364954     Source Type: Journal    
DOI: 10.1137/140968045     Document Type: Article

References (73)

1. H. Attouch and B. F. Svaiter, A continuous dynamical Newton-like approach to solving monotone inclusions, SIAM J. Control Optim., 49 (2011), pp. 574-598.
2. H. Avron and S. Toledo, Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix, J. ACM, 58 (2011), 8.
3. H. Bauschke and P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, CMS Books Math./Ouvrages Math. SMC, Springer, New York, 2011.
4. A. Benazza-Benyahia and J.-C. Pesquet, Building robust wavelet estimators for multicomponent images using Stein’s principle, IEEE Trans. Image Process., 14 (2005), pp. 1814-1830.
5. J. Bennett and S. Lanning, The Netflix prize, in Proceedings of KDD Cup and Workshop 2007, ACM, New York, 2007, pp. 3-6.
6. T. Blu and F. Luisier, The SURE-LET approach to image denoising, IEEE Trans. Image Process., 16 (2007), pp. 2778-2786.
7. T. T. Cai and H. H. Zhou, A data-driven block thresholding approach to wavelet estimation, Ann. Statist., 37 (2009), pp. 569-595.
8. E. J. Candès and B. Recht, Exact matrix completion via convex optimization, Found. Comput. Math., 9 (2009), pp. 717-772.
9. E. J. Candès, C. A. Sing-Long, and J. D. Trzasko, Unbiased risk estimates for singular value thresholding and spectral estimators, IEEE Trans. Signal Process., 61 (2013), pp. 4643-4657.
10. A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vision, 40 (2011), pp. 120-145.
11. C. Chaux, L. Duval, A. Benazza-Benyahia, and J.-C. Pesquet, A nonlinear Stein-based estimator for multichannel image denoising, IEEE Trans. Signal Process., 56 (2008), pp. 3855-3870.
12. P. L. Combettes and J.-C. Pesquet, A Douglas-Rachford splitting approach to nonsmooth convex variational signal recovery, IEEE J. Sel. Topics Signal Process., 1 (2007), pp. 564-574.
13. P. L. Combettes and J.-C. Pesquet, Proximal splitting methods in signal processing, in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, Springer, New York, 2011, pp. 185-212.
14. P. L. Combettes and V. R. Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.
15. I. Daubechies, Orthonormal bases of compactly supported wavelets, Comm. Pure Appl. Math., 41 (1988), pp. 909-996.
16. C.-A. Deledalle, V. Duval, and J. Salmon, Non-local methods with shape-adaptive patches (NLM-SAP), J. Math. Imaging Vision, 43 (2012), pp. 103-120.
17. C.-A. Deledalle, G. Peyré, and J. Fadili, Stein COnsistent Risk Estimator (SCORE) for Hard Thresholding, preprint, arXiv:1301.5874v1 [math.ST], 2013.
18. C.-A. Deledalle, F. Tupin, and L. Denis, Poisson NL means: Unsupervised non local means for Poisson noise, in Proceedings of the 2010 17th IEEE International Conference on Image Processing, 2010, pp. 801-804.
19. C.-A. Deledalle, S. Vaiter, G. Peyré, J. Fadili, and C. Dossal, Proximal splitting derivatives for risk estimation, J. Phys.: Conf. Ser., 386 (2012), 012003.
20. C.-A. Deledalle, S. Vaiter, G. Peyré, J. Fadili, and C. Dossal, Risk Estimation for Matrix Recovery with Spectral Regularization, preprint, arXiv:1205.1482v3 [math.OC], 2012.
21. C.-A. Deledalle, S. Vaiter, G. Peyré, J. Fadili, and C. Dossal, Unbiased risk estimation for sparse analysis regularization, in Proceedings of the 2012 19th IEEE International Conference on Image Processing, 2012, pp. 3053-3056.
22. S. J. Dong and K. F. Liu, Stochastic estimation with Z2 noise, Phys. Lett. B, 328 (1994), pp. 130-136.
23. D. L. Donoho and I. M. Johnstone, Adapting to unknown smoothness via wavelet shrinkage, J. Amer. Statist. Assoc., 90 (1995), pp. 1200-1224.
24. C. Dossal, M. Kachour, M. J. Fadili, G. Peyré, and C. Chesneau, The degrees of freedom of the lasso for general design matrix, Statist. Sinica, 23 (2013), pp. 809-828.
25. V. Duval, J.-F. Aujol, and Y. Gousseau, A bias-variance approach for the nonlocal means, SIAM J. Imaging Sci., 4 (2011), pp. 760-788.
26. A. Edelman, Jacobians of Matrix Transforms (with Wedge Products), Handout 3, MIT, Cambridge, MA, 2005.
27. B. Efron, How biased is the apparent error rate of a prediction rule?, J. Amer. Statist. Assoc., 81 (1986), pp. 461-470.
28. Y. C. Eldar, Generalized SURE for exponential families: Applications to regularization, IEEE Trans. Signal Process., 57 (2009), pp. 471-481.
29. L. C. Evans and R. F. Gariepy, Measure Theory and Fine Properties of Functions, CRC Press, Boca Raton, FL, 1992.
30. M. Fazel, Matrix Rank Minimization with Applications, Ph.D. thesis, Department of Electrical Engineering, Stanford University, Stanford, CA, 2002.
31. D. Gilbarg and N. S. Trudinger, Elliptic Partial Differential Equations of Second Order, 2nd ed., Classics Math. 517, Springer-Verlag, Berlin, 1998.
32. D. A. Girard, A fast "Monte Carlo cross-validation" procedure for large least squares problems with noisy data, Numer. Math., 56 (1989), pp. 1-23.
33. R. Giryes, M. Elad, and Y. C. Eldar, The projected GSURE for automatic parameter tuning in iterative shrinkage methods, Appl. Comput. Harmon. Anal., 30 (2011), pp. 407-422.
34. G. H. Golub, M. Heath, and G. Wahba, Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics, 21 (1979), pp. 215-223.
35. A. Griewank and A. Walther, Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, SIAM, Philadelphia, 2008.
36. J.-B. Hiriart-Urruty and C. Lemaréchal, Convex Analysis and Minimization Algorithms, Vols. 12, Springer-Verlag, Berlin, 2001.
37. H. M. Hudson, A natural identity for exponential families with applications in multiparameter estimation, Ann. Statist., 6 (1978), pp. 473-484.
38. H. M. Hudson and T. Lee, Maximum likelihood restoration and choice of smoothing parameter in deconvolution of image data subject to Poisson noise, Comput. Statist. Data Anal., 26 (1998), pp. 393-410.
39. M. F. Hutchinson, A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines, Comm. Statist. Simulation Comput., 18 (1989), pp. 1059-1076.
40. K. Kato, On the degrees of freedom in shrinkage estimation, J. Multivariate Anal., 100 (2009), pp. 1338-1352.
41. N. Komodakis and J.-C. Pesquet, Playing with Duality: An Overview of Recent Primal-Dual Approaches for Solving Large-Scale Optimization Problems, preprint, arXiv:1406.5429v1 [cs.NA], 2014.
42. A. S. Lewis and M. L. Overton, Nonsmooth optimization via quasi-Newton methods, Math. Program., 141 (2013), pp. 135-163.
43. A. S. Lewis and H. S. Sendov, Twice differentiable spectral functions, SIAM J. Matrix Anal. Appl., 23 (2001), pp. 368-386.
44. K.-C. Li, From Stein’s unbiased risk estimates to the method of generalized cross validation, Ann. Statist., 13 (1985), pp. 1352-1377.
45. F. Luisier, T. Blu, and M. Unser, SURE-LET for orthonormal wavelet-domain video denoising, IEEE Trans. Circuits Syst. Video Technol., 20 (2010), pp. 913-919.
46. S. Mallat, A Wavelet Tour of Signal Processing. The Sparse Way, 3rd ed., Elsevier/Academic Press, Amsterdam, 2009.
47. U. Naumann, Optimal Jacobian accumulation is NP-complete, Math. Program., 112 (2008), pp. 427-441.
48. J.-C. Pesquet, A. Benazza-Benyahia, and C. Chaux, A SURE approach for digital signal/image deconvolution problems, IEEE Trans. Signal Process., 57 (2009), pp. 4616-4632.
49. J.-C. Pesquet and D. Leporini, A new wavelet estimator for image denoising, in Proceedings of the Sixth International Conference on Image Processing and Its Applications, IET, Stevenage, UK, 1997, pp. 249-253.
50. H. Raguet, J. Fadili, and G. Peyré, A generalized forward-backward splitting, SIAM J. Imaging Sci., 6 (2013), pp. 1199-1226.
51. S. Ramani, T. Blu, and M. Unser, Monte-Carlo SURE: A black-box optimization of regularization parameters for general denoising algorithms, IEEE Trans. Image Process., 17 (2008), pp. 1540-1554.
52. S. Ramani, Z. Liu, J. Rosen, J.-F. Nielsen, and J. A. Fessler, Regularization parameter selection for nonlinear iterative image restoration and MRI reconstruction using GCV and SURE-based methods, IEEE Trans. Image Process., 21 (2012), pp. 3659-3672.
53. S. Ramani, J. Rosen, Z. Liu, and J. A. Fessler, Iterative weighted risk estimation for nonlinear image restoration with analysis priors, in Proceedings of SPIE, Vol. 8296, Computational Imaging 10, SPIE, Bellingham, WA, 2012, 82960N.
54. M. Raphan and E. P. Simoncelli, Optimal denoising in redundant representations, IEEE Trans. Image Process., 17 (2008), pp. 1342-1352.
55. M. Raphan and E. P. Simoncelli, Least squares estimation without priors or supervision, Neural Comput., 23 (2011), pp. 374-420.
56. J. Rice, Choice of smoothing parameter in deconvolution problems, Contemp. Math., 59 (1986), pp. 137-151.
57. R. T. Rockafellar, Monotone operators and the proximal point algorithm, SIAM J. Control Optim., 14 (1976), pp. 877-898.
58. F. Roosta-Khorasani and U. Ascher, Improved Bounds on Sample Size for Implicit Matrix Trace Estimators, preprint, arXiv:1308.2475 [cs.NA], 2013.
59. L. I. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259-268.
60. X. Shen and J. Ye, Adaptive model selection, J. Amer. Statist. Assoc., 97 (2002), pp. 210-221.
61. V. Solo and M. Ulfarsson, Threshold selection for group sparsity, in Proceedings of the 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, 2010, pp. 3754-3757.
62. C. M. Stein, Estimation of the mean of a multivariate normal distribution, Ann. Statist., 9 (1981), pp. 1135-1151.
63. D. Sun and J. Sun, Nonsmooth Matrix Valued Functions Defined by Singular Values, Technical report, Department of Decision Sciences, National University of Singapore, Singapore, 2003.
64. R. J. Tibshirani and J. Taylor, The solution path of the generalized lasso, Ann. Statist., 39 (2011), pp. 1335-1371.
65. R. J. Tibshirani and J. Taylor, Degrees of freedom in lasso problems, Ann. Statist., 40 (2012), pp. 1198-1232.
66. S. Vaiter, C.-A. Deledalle, G. Peyré, C. Dossal, and J. Fadili, Local behavior of sparse analysis regularization: Applications to risk estimation, Appl. Comput. Harmon. Anal., 35 (2013), pp. 433-451.
67. S. Vaiter, C.-A. Deledalle, G. Peyré, J. M Fadili, and C. Dossal, The Degrees of Freedom of Partly Smooth Regularizers, preprint, arXiv:1404.5557v2 [math.ST], 2014.
68. D. Van De Ville and M. Kocher, SURE-based non-local means, IEEE Signal Process. Lett., 16 (2009), pp. 973-976.
69. D. Van De Ville and M. Kocher, Non-local means with dimensionality reduction and SURE-based parameter selection, IEEE Trans. Image Process., 9 (2011), pp. 2683-2690.
70. C. Vonesch, S. Ramani, and M. Unser, Recursive risk estimation for non-linear image deconvolution with a wavelet-domain sparsity constraint, in Proceedings of the 15th IEEE International Conference on Image Processing, 2008, pp. 665-668.
71. J. Ye, On measuring and correcting the effects of data mining and model selection, J. Amer. Statist. Assoc., 93 (1998), pp. 120-131.
72. M. Yuan and Y. Lin, Model selection and estimation in regression with grouped variables, J. R. Stat. Soc. Ser. B Stat. Methodol., 68 (2006), pp. 49-67.
73. H. Zou, T. Hastie, and R. Tibshirani, On the "degrees of freedom" of the lasso, Ann. Statist., 35 (2007), pp. 2173-2192.