Optimality conditions and a smoothing trust region newton method for nonlipschitz optimization

SIAM Journal on Optimization

Volumn 23, Issue 3, 2013, Pages 1528-1552

  Chen, Xiaojun ^a   Niu, Lingfeng ^b   Yuan, Yaxiang ^c  

^a HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

^b UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

^c INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

Author keywords

Convergence; Non Lipschitz; Nonsmooth nonconvex optimization; Penalty function; Regularized optimization; Smoothing methods; Trust region Newton method

Indexed keywords

CONVERGENCE; NON-LIPSCHITZ; NONSMOOTH NONCONVEX OPTIMIZATION; PENALTY FUNCTION; REGULARIZED OPTIMIZATIONS; SMOOTHING METHODS; TRUST REGION;

IMAGE RECONSTRUCTION; OPTIMIZATION;

NEWTON-RAPHSON METHOD;

EID: 84886296616     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/120871390     Document Type: Article

References (47)

1. T. Bannert, A trust region algorithm for nonsmooth optimization, Math. Program., 67 (1994), pp. 247-264.
2. W. Bian and X. Chen, Smoothing neural network for constrained non-Lipschitz optimization with applications, IEEE Trans. Neural Network, 23 (2012), pp. 399-411.
3. A.M. Bruckstein, D.L. Donoho, and M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Rev., 51 (2009), pp. 34-81.
4. J.V. Burke, A.S. Lewis, and M.L. Overton, A robust gradient sampling algorithm for nonsmooth, nonconvex optimization, SIAM J. Optim., 15 (2005), pp. 751-779.
5. C. Catis, N.I.M. Gould, and Ph.L. Toint, On the evaluation complexity of composite function minimization with applications to nonconvex nonlinear programming, SIAM J. Optim., 21 (2011), pp. 1721-1739.
6. R. Chartrand and V. Staneva, Restricted isometry properties and nonconvex compressive sensing, Inverse Problems, 24 (2008), pp. 1-14.
7. X. Chen, Smoothing methods for nonsmooth, nonconvex minimization, Math. Program., 134 (2012), pp. 71-99.
8. X. Chen, D. Ge, Z. Wang, and Y. Ye, Complexity of unconstrained L2-Lp minimization, Math. Program., to appear.
9. X. Chen, F. Xu, and Y. Ye, Lower bound theory of nonzero entries in solutions of l2-lp minimization, SIAM J. Sci. Comput., 32 (2010), pp. 2832-2852.
10. X. Chen and W. Zhou, Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization, SIAM J. Imag. Sci., 3 (2010), pp. 765-790.
11. F.H. Clarke, Optimization and Nonsmooth Analysis, John Wiley, New York, 1983.
12. A.R. Conn, N.I.M. Gould, and P.L. Toint, Trust-Region Methods, SIAM, Philadelphia, 2000.
13. J.E. Dennis Jr., S.-B. Li, and R.A. Tapia, A unified approach to global convergence of trust region methods for nonsmooth optimization, Math. Program., 68 (1995), pp. 319-346.
14. F. Facchinei and C. Kanzow, A nonsmooth inexact Newton method for the solution of largescale nonlinear complementarity problems, Math. Program., 76 (1997), pp. 493-512.
15. J. Fan and R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Amer. Statist. Assoc., 96 (2011), pp. 1348-1360.
16. F. Fletcher, A model algorithm for composite non-differentiable optimization problem, Math. Program. Studies, 17 (1982), pp. 67-76.
17. R. Fletcher, Practical Methods of Optimization, John Wiley, Chichester, UK, 1987.
18. S. Foucart and M.J. Lai, Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0 < q ≤ 1, Appl. Comput. Harmonic Anal., 26 (2009), pp. 395-407.
19. R. Garmanjani and L.N. Vicente, Smoothing and worst case complexity for direct-search methods in non-smooth optimization, IMA Numer. Anal., to appear.
20. S. Geman and D. McClure, Bayesian image analysis: An application to single photon emission tomography, in Proceedings of the American Statistical Association, 1985, pp. 12-18.
21. M. El Hallabi and R.A. Tapia, Global Convergence Theory for Arbitrary Norm Trust-Region Methods for Nonlinear Equations, Doctoral Thesis, Rice University, Houston, TX, 1993, pp. 93-41.
22. T. Hastie, R. Tibshirani, and J.H. Friedman, The Elements of Statistical Learning: Data mining, Inference, and Prediction, Springer-Verlag, New York, 2001.
23. J. Hiriart-Urruty, J.J. Strodiot, and V.H. Nguyen, Generalized Hessian matrix and second-order optimality conditions for problems with C1,1 data, Appl. Math. Optim., 11 (1984), pp. 43-56.
24. J. Huang, J.L. Horowitz, and S. Ma, Asymptotic properties of bridge estimators in sparse high-dimensional regression models, Ann. Stat., 36 (2008), pp. 587-613.
25. J. Huang, S. Ma, H. Xie, and C.-H. Zhang, A group bridge approach for variable selection, Biometrika, 96 (2009), pp. 339-355.
26. H. Jiang, M. Fukushima, L. Qi, and D. Sun, A trust region method for solving generalized complementarity problems, SIAM J. Optim., 8 (1998), pp. 140-157.
27. K. Knight and W.J. Fu, Asymptotics for lasso-type estimators, Ann. Statist., 28 (2000), pp. 1356-1378.
28. K.C. Kiwiel, Convergence of the gradient sampling algorithm for nonsmooth nonconvex optimization, SIAM J. Optim., 18 (2007), pp. 379-388.
29. J.M. Martínez and A.C. Moretti, A trust region method for minimization of nonsmooth functions with linear constraints, Math. Program., 76 (1997), pp. 431-449.
30. J.M. Martínez and S.A. Santos, A trust-region strategy for minimization on arbitrary domains, Math. Program., 68 (1995), pp. 267-301.
31. J.J. Moré and D.C. Sorensen, Computing a trust region step, SIAM J. Sci. Stat. Computing, 4 (1983), pp. 553-572.
32. W.K. Newey and D. McFadden, Large sample estimation and hypothesis testing, Handbook of Econometrics, Vol. IV, Handbooks in Econom. 2, North-Holland, Amsterdam, 1994, pp. 2111-2245.
33. Y. Nesterov, Smooth minimization of nonsmooth functions, Math. Program., 103 (2005), pp. 127-152.
34. Y. Nesterov, Smoothing technique and its applications in semi definite optimization, Math. Program., 110 (2007), pp. 245-259.
35. M. Nikolova, M.K. Ng, S. Zhang, and W. Ching, Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization, SIAM J. Imag. Sci., 1 (2008), pp. 2-25.
36. J. Nocedal and S.J. Wright, Numerical Optimization, 2nd ed., Springer, New York, 2006.
37. M.J.D. Powell, General algorithms for discrete nonlinear approximation calculations, Report DAMTP 1983/NA2, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, England, 1983.
38. L. Qi, Superlinearly convergent approximate Newton methods for LC1 optimization problems, Math. Program., 64 (1994), pp. 277-294.
39. L. Qi and J. Sun, A trust region algorithm for minimization of locally Lipschitzian functions, Math. Program., 66 (1994), pp. 25-43.
40. R.T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, 1970.
41. R.T. Rockafellar and R.J.-B.Wets, Variational Analysis, Springer-Verlag, New York, 1998.
42. T. Stamey, J. Kabalin, J. McNeal, I. Johnstone, F. Freiha, E. Redwine, and N. Yang, Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate. II. Radical prostatectomy treated patients, J. Urology, 141 (1989), pp. 1076-1083.
43. R. Tibshirani, Regression shrinkage and selection via the lasso, J. Roy. Statist. Soc. B, 58 (1996), pp. 267-288.
44. Y. Yuan, Conditions for convergence of a trust-region method for nonsmooth optimization, Math. Program., 31 (1985), pp. 220-228.
45. Y. Yuan, On the superliner convergence of a trust region algorithm for nonsmooth optimization, Math. Program., 31 (1985), pp. 269-285.
46. Y. Yuan, On the truncated conjugate gradient method, Math. Program., 87 (2000), pp. 561-571.
47. C.-H. Zhang, Nearly unbiased variable selection under minimax concave penalty, Ann. Statist., 38 (2010), pp. 894-942.