Worst-case complexity of smoothing quadratic regularization methods for non-lipschitzian optimization

SIAM Journal on Optimization

Volumn 23, Issue 3, 2013, Pages 1718-1741

  Bian, Wei ^a   Chen, Xiaojun ^b  

^a HARBIN INSTITUTE OF TECHNOLOGY   (China)

^b HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

Author keywords

Convergence; Nonsmooth nonconvex optimization; Quadratic regularization; Smoothing approximation; Stationary point; Worst case complexity

Indexed keywords

CONVERGENCE; NONSMOOTH NONCONVEX OPTIMIZATION; QUADRATIC REGULARIZATION; SMOOTHING APPROXIMATION; STATIONARY POINTS; WORST-CASE COMPLEXITY;

ALGORITHMS; OPTIMIZATION;

ITERATIVE METHODS;

EID: 84886302151     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/120864908     Document Type: Article

References (33)

1. A. Antoniadias and J. Fan, Regularization of wavelets approximations, J. Amer. Statist. Assoc., 96 (2001), pp. 929-967.
2. W. Bian and X. Chen, Smoothing neural network for constrained non-Lipschitz optimization with applications, IEEE Trans. Neural Netw. Learning Syst., 23 (2012), pp. 399-411.
3. J. V. Burke, A. S. Lewis, and M. L. Overton, A robust gradient sampling algorithm for nonsmooth, nonconvex optimization, SIAM J. Optim., 15 (2005), pp. 751-779.
4. E. Candes and T. Tao, Decoding by linear programming, IEEE Trans. Inform. Theory, 51 (2005), pp. 4203-4215.
5. E. Candes, M. Wakin, and S. Boyd, Enhancing sparsity by reweighted eell;1 minimization, J. Fourier Anal. Appl., 14 (2008), pp. 877-905.
6. C. Cartis, N. I. M. Gould, and Ph. L. Toint, On the evaluation complexity of composite function minimization with applications to nonconvex nonlinear programming, SIAM J. Optim., 21 (2011), pp. 1721-1739.
7. C. Cartis, N. I. M. Gould, and Ph. L. Toint, Adaptive cubic regularization methods for unconstrained optimization, Part I: Motivation, convergence and numerical results, Math. Program., 127 (2011), pp. 245-295.
8. C. Cartis, N. I. M. Gould, and Ph. L. Toint, Adaptive cubic regularization methods for unconstrained optimization, Part II: Worst-case function- and derivative-evaluation complexity, Math. Program., 130 (2011), pp. 295-319.
9. R. Chartrand and V. Staneva, Restricted isometry properties and nonconvex compressive sensing, Inverse Probl., 24 (2008), pp. 1-14.
10. X. Chen, Smoothing methods for nonsmooth, nonconvex minimization, Math. Program., 134 (2012), pp. 71-99.
11. X. Chen, D. Ge, Z. Wang, and Y. Ye, Complexity of unconstrained L2-Lp minimization, Math. Program., to appear.
12. X. Chen, M. Ng, and C. Zhang, Nonconvex lp regularization and box constrained model for image restoration, IEEE Trans. Image Process., 21 (2012), pp. 4709-4721.
13. X. Chen, F. Xu, and Y. Ye, Lower bound theory of nonzero entries in solutions of l2-lp minimization, SIAM J. Sci. Comput., 32 (2010), pp. 2832-2852.
14. X. Chen and W. Zhou, Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization, SIAM J. Imaging Sci., 3 (2010), pp. 765-790.
15. F. H. Clarke, Optimization and Nonsmooth Analysis, John Wiley, New York, 1983.
16. D. L. Donoho, Neighborly Polytopes and Sparse Solutions of Underdetermined Linear Equations, Technical report, Stanford University, Stanford, CA, 2005.
17. J. Fan, Comments on 'Wavelets in stastics: a review' by A. Antoniadis, Stat. Method. Appl., 6 (1997), pp. 131-138.
18. J. Fan and R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Amer. Statist. Assoc., 96 (2001), pp. 1348-1360.
19. R. Garmanjani and L. N. Vicente, Smoothing and worst case complexity for direct-search methods in non-smooth optimization, IMA J. Numer. Anal., to appear.
20. D. Ge, X. Jiang, and Y. Ye, A note on the complexity of Lp minimization, Math. Program., 21 (2011), pp. 1721-1739.
21. J.-L. Goffin, On the convergence rate of subgradient optimization methods, Math. Program., 13 (1977), pp. 329-347.
22. N. I. M. Gould and D. P. Robinson, A second derivative SQP method: Global convergence, SIAM J. Optim., 20 (2010), pp. 2023-2048.
23. J. Huang, J. L. Horowitz, and S. Ma, Asymptotic properties of bridge estimators in sparse high-dimensional regression models, Ann. Statist., 36 (2008), pp. 587-613.
24. J. Huang, S. Ma, H. Xue, and C. Zhang, A group bridge approach for variable selection, Biometrika, 96 (2009), pp. 339-355.
25. R. Tibshirani, Shrinkage and selection via the Lasso, J. Roy. Statist. Soc. Ser. B, 58 (1996), pp. 267-288.
26. Yu. Nesterov, Smooth minimization of non-smooth functions, Math. Program., 103 (2005),pp. 127-152.
27. Yu. Nesterov, Modified Gauss-Newton scheme with worst-case guarantees for global performance, Optim. Method Softw., 22 (2007), pp. 413-431.
28. Yu. Nesterov and B. T. Polyak, Cubic regularization of Newton's method and its global performance, Math. Program., 108 (2006), pp. 177-205.
29. M. Nikolova, M. K. Ng, S. Zhang, and W.-K. Ching, Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization, SIAM J. Imaging Sci., 1 (2008), pp. 2-25.
30. R. T. Rockafellar and R. J.-B. Wets, Variational Analysis, Springer, Berlin, 1998.
31. R. J. Tibshirani, Regression shrinkage and selection via the Lasso, J. Roy. Statist. Soc. Ser. B, 58 (1996), pp. 267-288.
32. J. V. Vazirani, Approximation Algorithms, Springer, Berlin, 2003.
33. C.-H. Zhang, Nearly unbiased variable selection under minimax concave penalty, Ann. Statist., 38 (2010), pp. 894-942.