Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization

IMA Journal of Numerical Analysis

Volumn 33, Issue 3, 2013, Pages 1008-1028

  Garmanjani, R ^a   Vicente, L N ^b  

^a UNIVERSITY OF COIMBRA   (Portugal)

^b UNIVERSITY OF COIMBRA   (Portugal)

Author keywords

derivative free optimization; direct search; nonconvex; nonsmooth; smoothing function; worst case complexity

Indexed keywords

FUNCTIONS; INVERSE PROBLEMS; STEEPEST DESCENT METHOD;

DERIVATIVE-FREE OPTIMIZATION; DIRECT SEARCH; NONCONVEX; NONSMOOTH; SMOOTHING FUNCTION; WORST-CASE COMPLEXITY;

OPTIMIZATION;

EID: 84880321255     PISSN: 02724979     EISSN: 14643642     Source Type: Journal    
DOI: 10.1093/imanum/drs027     Document Type: Review

References (36)

1. ABRAMSON, M. A. & AUDET, C. (2006) Convergence of mesh adaptive direct search to second-order stationary points. SIAM J. Optim., 17, 606-619.
2. AUDET, C. (2003) Convergence results for pattern search algorithms are tight. Optim. Eng., 5, 101-122.
3. AUDET, C. & DENNIS JR, J. E. (2006) Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim., 17, 188-217.
4. BOGANI, C., GASPARO, M. G. & PAPINI, A. (2009) Generalized set search methods for piecewise smooth problems. SIAM J. Optim., 20, 321-335.
5. BURKE, J. V., LEWIS, A. S. & OVERTON, M. L. (2005) A robust gradient sampling algorithm for nonsmooth, nonconvex optimization. SIAM J. Optim., 15, 751-779.
6. CARTIS, C., GOULD, N. I. M. & TOINT, PH. L. (2010) On the complexity of steepest descent, Newton's and regularized Newton's methods for nonconvex unconstrained optimization. SIAM J. Optim., 20, 2833-2852.
7. CARTIS, C., GOULD, N. I. M. & TOINT, PH. L. (2011) On the evaluation complexity of composite function minimization with applications to nonconvex nonlinear programming. SIAM J. Optim., 21, 1721-1739.
8. CARTIS, C., GOULD, N. I. M. & TOINT, PH. L. (2012) On the oracle complexity of first-order and derivative-free algorithms for smooth nonconvex minimization. SIAM J. Optim., 22, 66-86.
9. CARTIS, C., GOULD, N. I. M. & TOINT, PH. L. (2011) Adaptive cubic overestimation methods for unconstrained optimization. Part II: worst-case function-and derivative-evaluation complexity. Math. Program., 130, 295-319.
10. CHEN, C. & MANGASARIAN, O. L. (1996) A class of smoothing functions for nonlinear and mixed complementarity problems. Comput. Optim. Appl., 5, 97-138.
11. CHEN, X. & ZHOU, W. (2010) Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM J. Imaging Sci., 3, 765-790.
12. CLARKE, F. H. (1990) Optimization and Nonsmooth Analysis. New York: John Wiley & Sons (reprinted by SIAM, Philadelphia).
13. CONN, A. R., SCHEINBERG, K. & VICENTE, L. N. (2009) Introduction to Derivative-Free Optimization. MPS-SIAM Series on Optimization. Philadelphia: SIAM.
14. CUSTÓDIO, A. L., ROCHA, H. & VICENTE, L. N. (2010) Incorporating minimum Frobenius norm models in direct search. Comput. Optim. Appl., 46, 265-278.
15. DOLAN, E. D. & MORÉ, J. J. (2002) Benchmarking optimization software with performance profiles. Math. Program., 91, 201-213.
16. DOLAN, E. D., MORÉ, J. J. & MUNSON, T. S. (2006) Optimality measures for performance profiles. SIAM J. Optim., 16, 891-909.
17. ERMOLIEV, Y. M., NORKIN, V. I. & WETS, R. J.-B. (1995) The minimization of semicontinuous functions: mol-lifier subgradients. SIAM J. Control Optim., 32, 149-167.
18. GARMANJANI, R. (2012) Smoothing and worst case complexity for direct-search methods in non-smooth optimization. Ph.D. Thesis, Department of Mathematics, University of Coimbra, Portugal.
19. GOULD, N. I. M., ORBAN, D. & TOINT, PH. L. (2003) CUTEr (and SifDec), a constrained and unconstrained testing environment, revisited. ACM. Trans. Math. Software., 29, 373-394.
20. GRATTON, S., MOUFFE, M., TOINT, PH. L. & WEBER-MENDONCA, M. (2008a) A recursive trust-region method in infinity norm for bound-constrained nonlinear optimization. IMA J. Numer. Anal., 28, 827-861.
21. GRATTON, S., SARTENAER, A. & TOINT, PH. L. (2008b) Recursive trust-region methods for multiscale nonlinear optimization. SIAM J. Optim., 19, 414-444.
22. GUPAL, A. M. (1977) On a method for the minimization of almost-differentiable functions. Cybernet. Syst. Anal., 13, 115-117.
23. KATKOVNIK, V. Y. (1972) Method of averaging operators in iteration algorithms for stochastic optimization. Cybernet. Systems Anal., 4, 670-679.
24. KIWIEL, K. C. (2010) A nonderivative version of the gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim., 20, 1983-1994.
25. KOLDA, T. G., LEWIS, R. M. & TORCZON, V. (2003) Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev., 45, 385-482.
26. KREIMER, J. & RUBINSTEIN, R. Y. (1992) Nondifferentiable optimization via smooth approximation: general analytical approach. Ann. Oper. Res., 39, 97-119.
27. LIUZZI, G. & LUCIDI, S. (2009) A derivative-free algorithm for inequality constrained nonlinear programming via smoothing of an IQQ penalty function. SIAM J. Optim., 20, 1-29.
28. MORÉ, J. J. & WILD, S. M. (2009) Benchmarking derivative-free optimization algorithms. SIAM J. Optim., 20, 172-191.
29. NESTEROV, Y. (2004) Introductory Lectures on Convex Optimization. Dordrecht: Kluwer Academic Publishers.
30. NESTEROV, Y. (2011) Random gradient-free minimization of convex functions. Technical Report 2011/1. CORE: Center for Operations Research and Econometrics.
31. NESTEROV, Y. & POLYAK, B. T. (2006) Cubic regularization of Newton's method and its global performance. Math. Program., 108, 177-205.
32. RIOS, L. M. & SAHINIDIS, N. (2010) Derivative-free optimization: a review of algorithms and comparison of software implementations.
33. ROCKAFELLAR, R. T. & WETS, R. J.-B. (1997) Variational Analysis. Berlin: Springer, third printing in 2009.
34. VICENTE, L. N. (2010) Worst case complexity of direct search. Technical Report 10-17. Department of Mathematics, University of Coimbra, Portugal.
35. VICENTE, L. N. & CUSTÓDIO, A. L. (2012) Analysis of direct searches for discontinuous functions. Math. Program., 133, 299-325.
36. ZHANG, C. & CHEN, X. (2009) Smoothing projected gradient method and its application to stochastic linear complementarity problems. SIAM J. Optim., 20, 627-649.