A robust gradient sampling algorithm for nonsmooth, nonconvex optimization

SIAM Journal on Optimization

Volumn 15, Issue 3, 2005, Pages 751-779

  Burke, James V ^a   Lewis, Adrian S ^b   Overton, Michael L ^c  

^a UNIVERSITY OF WASHINGTON   (United States)

^b School of Operations Research and Industrial Engineering   (United States)

^c NEW YORK UNIVERSITY   (United States)

Author keywords

Generalized gradient; Gradient sampling; Nonconvex; Nonsmooth optimization; Subgradient

Indexed keywords

GENERALIZED GRADIENTS; GRADIENT SAMPLING; NONCONVEX; NONSMOOTH OPTIMIZATION; SUBGRADIENT;

CHEBYSHEV APPROXIMATION; FORTRAN (PROGRAMMING LANGUAGE); FUNCTIONS; OPTIMIZATION; PROBABILITY; PROBLEM SOLVING; SET THEORY;

ALGORITHMS;

EID: 23844446458     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/030601296     Document Type: Article

References (42)

1. [AL04] K. ANSTREICHER AND J. LEE. A masked spectral bound for maximum-entropy sampling, in MODA 7 - Advances in Model-Oriented Design and Analysis, A. di Bucchianico, H. Läuter, and H. P. Wynn, eds., Springer-Verlag, Berlin, 2004, pp. 1-10.
2. [BB90] S. BOYD AND V. BALAKRISHNAN, A regularity result for the singular values of a transfer matrix and a quadratically convergent algorithm for computing its L∞-norm, Systems Control Lett., 15 (1990), pp. 1-7.
3. [BGL95] V. BLONDEL, M. GEVERS, AND A. LINDQUIST, Survey on the state of systems and control, Eur. J. Control, 1 (1995), pp. 5-23.
4. [BLO01] J. V. BURKE, A. S. LEWIS, AND M. L. OVERTON, Optimizing matrix stability, Proc. Amer. Math. Soc., 129 (2001), pp. 1635-1642.
5. [BLO02a] J. V. BURKE, A. S. LEWIS, AND M. L. OVERTON, Approximating subdifferentials by random sampling of gradients, Math. Oper. Res., 27 (2002), pp. 567-584.
6. [BLO02b] J. V. BURKE, A. S. LEWIS, AND M. L. OVERTON, Two numerical methods for optimizing matrix stability. Linear Algebra Appl., 351-352 (2002), pp. 117-145.
7. [BLO03a] J. V. BURKE, A. S. LEWIS, AND M. L. OVERTON, Optimization and pseudospectra, with applications to robust stability, SIAM J. Matrix Anal. Appl., 25 (2003), pp. 80-104.
8. [BLO03b] J. V. BURKE, A. S. LEWIS, AND M. L. OVERTON, Robust stability and a criss-cross algorithm for pseudospectra, IMA J. Numer. Anal., 23 (2003), pp. 359-375.
9. [BO01] J. V. BURKE AND M. L. OVERTON, Variational analysis of non-Lipschitz spectral functions, Math. Program., 90 (2001), pp. 317-352.
10. [Bur85] J. V. BURKE, Descent methods for composite nondifferentiable optimization problems, Math. Program., 33 (1985), pp. 260-279.
11. [Bye88] R. BYERS, A bisection method for measuring the distance of a stable matrix to the unstable matrices, SIAM J. Sci. Stat. Comput., 9 (1988), pp. 875-881.
12. [CG78] R. CHANEY AND A. GOLDSTEIN, An extension of the method of subgradients, in Nonsmooth Optimization, C. Lemaréchal and R. Mifflin, eds., Pergamon Press, Oxford, UK, New York, 1978, pp. 51-70.
13. [Cla73] F. H. CLARKE, Necessary Conditions for Nonsmooth Problems in Optimal Control and the Calculus of Variations, Ph.D. thesis, University of Washington, Seattle, WA, 1973.
14. [Cla83] F. H. CLARKE, Optimization and Nonsmooth Analysis, Classics Appl. Math. 5, SIAM, Philadelphia, 1990.
15. [CLSW98] F. H. CLARKE, Yu. S. LEDYAEV, R. J. STERN, AND P. R WOLENSKI, Nonsmooth Analysis and Control Theory, Grad. Texts in Math. 178, Springer-Verlag, New York, 1998.
16. [Dav90] E. J. DAVISON, ED., Benchmark Problems for Control System Design: Report of the IFAC Theory Committee, Pergamon Press, Oxford, UK, 1990.
17. [DR95] V. F. DEMYANOV AND A. RUBINOV, Constructive Nonsmooth Analysis, Approx. Optim. 7, Peter Lang, Frankfurt am Main, Germany, 1995.
18. [FGG02] A. FUDULI, M. GAUDIOSO, AND G. GIALLOMBARDO, A DC piecewise affine model and a bundling technique in nonconvex nonsmooth minimization, Optim. Methods Softw., 19 (2004), pp. 89-102.
19. [Fle87] R. FLETCHER, Practical Methods of Optimization, 2nd ed., John Wiley, Chichester, UK, New York, 1987.
20. [FM68] A. V. FIACCO AND G. P. McCORMICK, Nonlinear Programming: Sequential Unconstrained Minimization Techniques, Classics Appl. Math. 4, SIAM, Philadelphia, 1990.
21. [Gol77] A. A. GOLDSTEIN, Optimization of Lipschitz continuous functions, Math. Program., 13 (1977), pp. 14-22.
22. [Gro02] A. GROTHEY, A Second Order Trust Region Bundle Method for Nonconvex Nonsmooth Optimization, Technical report MS02-001, University of Edinburgh, Edinburgh, UK, 2002.
23. [HUL93] J. B. HIRIART-URRUTY AND C. LEMARÉHAL, Convex Analysis and Minimization Algorithms II: Advanced Theory and Bundle Methods, Springer-Verlag, Berlin, 1993.
24. [Kiw85] K. C. KIWIEL, Methods of Descent for Nondifferentiable Optimization, Lecture Notes in Math. 1133, Springer-Verlag, Berlin, New York, 1985.
25. [Lem75] C. LEMARÉCHAL, An extension of Davidon methods to non-differentiable problems, in Nondifferentiable Optimization, M. L. Balinski and P. Wolfe, eds., Math. Programming Stud., 3 (1975), pp. 95-109.
26. [Lew99] A. S. LEWIS, Nonsmooth analysis of eigenvalues, Math. Program., 84 (1999), pp. 1-24.
27. [Lew02] A. S. LEWIS. Robust regularization, Math. Program., submitted; available at http://www.orie.cornell.edu/~aslewis/publications/Robust.ps.
28. [LSB91] C. LEMARÉCHAL, J.-J. STRODIOT, AND A. BIHAIN, On a bundle algorithm for nonsmooth optimization, in Nonlinear Programming 4, O. L. Mangasarian, R. R. Meyer, and S. M. Robinson, eds., Academic Press, New York, 1991, pp. 245-282.
29. [LV98] L. LUKSAN AND J. VLČEK, A bundle Newton method for nonsmooth unconstrained minimization, Math. Program., 83 (1998), pp. 373-391.
30. [Mei67] G. MEINARDUS, Approximation of Functions: Theory and Numerical Methods, Springer Tracts Nat. Philos. 13, Springer-Verlag, New York, 1967.
31. [MN92] M. M. MÄKELÄ AND P. NEITTAANMÄKI, Nonsmooth Optimization, World Scientific, River Edge, NJ, 1992.
32. [Mos03] MOSEK Optimization Software, http://www.mosek.com/, 2003.
33. [OKZ98] J. OUTRATA, M. KOCVARA, AND J. ZOWE, Nonsmooth Approach to Optimization Problems with Equilibrium Constraints, Nonconvex Optim. Appl. 28, Kluwer Academic Publishers, Dordrecht, The Netherlands, 1998.
34. [Osb85] M. R. OSBORNE, Finite Algorithms in Optimization and Data Analysis, John Wiley, Chichester, UK, New York, 1985.
35. [Roc70] R. T. ROCKAFELLAR, Convex Analysis, Princeton University Press, Princeton, NJ, 1970.
36. [RW98] R. T. ROCKAPELLAR AND R. J. B. WETS, Variational Analysis, Springer-Verlag, New York, 1998.
37. [SZ92] H. SCHRAMM AND J. ZOWE, A version of the bundle idea for minimizing a nonsmooth function: Conceptual idea, convergence analysis, numerical results, SIAM J. Optim., 2 (1992), pp. 121-152.
38. [Tre97] L. N. TREFETHEN, Pseudospectra of linear operators, SIAM Rev., 39 (1997), pp. 383-406.
39. [Wil97] J. C. WILLEMS, Generic eigenvalue assignability by real memoryless output feedback made simple, in Communications, Computation, Control and Signal Processing, A. Paulraj, V. Roychowdhury, and C. D. Schaper, eds., Kluwer Academic Publishers, Norwell, MA, 1997, pp. 343-354.
40. [Wol75] P. WOLFE, A method of conjugate subgradients for minimizing nondifferentiable functions, in Nondifferentiable Optimization, M. L. Balinski and P. Wolfe, eds., Math. Programming Stud., 3 (1975), pp. 145-173.
41. [Won85] W. M. WONHAM. Linear Multivariable Control: A Geometric Approach, 3rd ed., Appl. Math. 10, Springer-Verlag, New York, 1985.
42. [Wri02] T. G. WRIGHT, EigTool: A graphical tool for nonsymmetric eigenproblems, OxfordUniversity Computer Laboratory, Oxford, UK, http://web.comlab.ox.ac.uk/projects/pseudospectra/eigtool/ (2002).