Nonsmooth optimization via quasi-Newton methods

Mathematical Programming

Volumn 141, Issue 1-2, 2013, Pages 135-163

  Lewis, Adrian S ^a   Overton, Michael L ^b  

^a School of Chemical and Molecular Engineering   (United States)

^b NEW YORK UNIVERSITY   (United States)

Author keywords

BFGS; Clarke stationary; Line search; Nonconvex; Partly smooth; R linear convergence

Indexed keywords

BFGS; CLARKE STATIONARY; LINE SEARCHES; NONCONVEX; PARTLY SMOOTH; R-LINEAR CONVERGENCES;

ALGEBRA; LINEAR CONTROL SYSTEMS; NEWTON-RAPHSON METHOD;

OPTIMIZATION;

EID: 84884671380     PISSN: 00255610     EISSN: 14364646     Source Type: Journal    
DOI: 10.1007/s10107-012-0514-2     Document Type: Article

References (54)

1. Arzelier, D.; Gryazina, E.N.; Peaucelle, D.; Polyak, B.T.: Mixed LMI/randomized methods for static output feedback control design. Technical report 09535, LAAS-CNRS, Toulouse, Sept 2009
2. Anstreicher K.; Lee J.: A masked spectral bound for maximum-entropy sampling. In: di Bucchianico, A.; Läuter, H.; Wynn eds, H.P. (eds) MODA 7 - Advances in Model-Oriented Design and Analysis, pp. 1-10. Springer, Berlin (2004)
3. Boito P.; Dedieu J.-P.: The condition metric in the space of rectangular full rank matrices. SIAM J. Matrix Anal. Appl. 31, 2580-2602 (2010)
4. Bonnans J.; Gilbert J.; Lemaréchal C.; Sagastizábal C.: A family of variable metric proximal methods. Math. Program. 68, 15-48 (1995)
5. ∞ optimization. In: Proceedings of Fifth IFAC Symposium on Robust Control Design, Toulouse (2006)
6. Borwein J.M.; Lewis A.S.: Convex Analysis and Nonlinear Optimization: Theory and Examples. 2nd edn. Springer, New York (2005)
7. Burke J.V.; Lewis A.S.; Overton M.L.: A robust gradient sampling algorithm for nonsmooth, nonconvex optimization. SIAM J. Optim. 15, 751-779 (2005) (Pubitemid 41156914)
8. Byrd R.H.; Nocedal J.; Yuan Y.: Global convergence of a class of quasi-Newton methods on convex problems. SIAM J. Numer. Anal. 24, 1171-1190 (1987)
9. Clarke, F.H.: Optimization and Nonsmooth Analysis. John Wiley, New York, 1983. Reprinted by SIAM, Philadelphia (1990)
10. Dai Y.-H.: Convergence properties of the BFGS algorithm. SIAM J. Optim. 13, 693-701 (2002)
11. Delwiche, T.: Contribution to the design of control laws for bilateral teleoperation with a view to applications in minimally invasive surgery. Ph.D. thesis, Free University of Brussels (2009)
12. Dotta, D.; e Silva, A.S.; Decker, I.C.: Design of power systems controllers by nonsmooth, nonconvex optimization. In: IEEE Power and Energy Society General Meeting, Calgary (2009)
13. Dixon L.C.W.: Quasi-Newton techniques generate identical points. II. The proofs of four new theorems. Math. Program. 3, 345-358 (1972)
14. Gumussoy, S.; Henrion, D.; Millstone, M.; Overton, M.L.: Multiobjective robust control with HIFOO 2.0. In: Proceedings of the Sixth IFAC Symposium on Robust Control Design, Haifa (2009)
15. Gumussoy, S.; Millstone, M.; Overton, M.L.: H-infinity strong stabilization via HIFOO, a package for fixed-order controller design. In: Proceedings of the 47th IEEE Conference on Decision and Control, Cancun (2008)
16. Gumussoy, S.; Overton, M.L.: Fixed-order H-infinity controller design via HIFOO, a specialized nonsmooth optimization package. In: Proceedings of 2008 American Control Conference, Seattle (2008)
17. Gürbüzbalaban M.; Overton M.L.: On Nesterov's nonsmooth Chebyshev-Rosenbrock functions. Nonlinear Anal. Theory Methods Appl. 75, 1282-1289 (2012)
18. Haarala, M.: Large-scale nonsmooth optimization: variable metric bundle method with limited memory. Ph.D. thesis, University of Jyväskylä, Finland (2004)
19. Hiriart-Urruty J.B.; Lemaréchal C.: Convex Analysis and Minimization Algorithms, two volumes. Springer, New York (1993)
20. Kaku, A.: Implementation of high precision arithmetic in the BFGS method for nonsmooth optimization. Master's thesis, New York University, Jan 2011. http://www.cs.nyu.edu/overton/mstheses/kaku/msthesis.pdf
21. Knittel, D.; Henrion, D.; Millstone, M.; Vedrines, M.: Fixed-order and structure H-infinity control with model based feedforward for elastic web winding systems. In: Proceedings of the IFAC/ IFORS/IMACS/IFIP Symposium on Large Scale Systems, Gdansk, Poland (2007)
22. Kiwiel, K.C.: Methods of descent for nondifferentiable optimization. In: Lecture Notes in Mathematics, vol. 1133. Springer, Berlin (1985)
23. Kiwiel K.C.: Convergence of the gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim. 18, 379-388 (2007)
24. Lax P.D.: Linear Algebra. Wiley, New York (1997)
25. Lee J.: Constrained maximum-entropy sampling. Oper. Res. 46, 655-664 (1998) (Pubitemid 128655447)
26. Lemaréchal, C.: A view of line searches. In: Optimization and optimal control (Proceedings of Conference at the Mathematical Research Institute, Oberwolfach, 1980), pp. 59-78. Springer, Berlin/New York, 1981. Lecture Notes in Control and Information Sciences, vol. 30
27. Lemaréchal, C.: Numerical experiments in nonsmooth optimization. In: Nurminski, E.A. (ed.) Progress in Nondifferentiable Optimization, pp. 61-84. International Institute for Applied Systems Analysis (IIASA), Laxenburg, Austria (1982)
28. Lewis A.S.: Active sets, nonsmoothness and sensitivity. SIAM J. Optim. 13, 702-725 (2003) (Pubitemid 36977155)
29. Li D.-H.; Fukushima M.: On the global convergence of the BFGS method for nonconvex unconstrained optimization problems. SIAM J. Optim. 11, 1054-1064 (2001)
30. Lewis, A.S.; Overton, M.L.: Behavior of BFGS with an exact line search on nonsmooth examples. http://www.cs.nyu.edu/overton/papers/pdffiles/bfgs-exactLS. pdf (2008)
31. Lewis, A.S.; Overton, M.L.: Nonsmooth optimization via BFGS. http://www.cs.nyu.edu/overton/papers/pdffiles/bfgs-inexactLS.pdf (2008)
32. Lemaréchal C.; Oustry F.; Sagastizábal C.: The U-Lagrangian of a convex function. Trans. Am. Math. Soc. 352, 711-729 (2000)
33. Lemaréchal, C.; Sagastizábal, C.: An approach to variable metric bundle methods. In: IFIP Proceedings, Systems Modeling and Optimization (1994)
34. Lukšan L.; Vlček J.: Globally convergent variable metric method for convex nonsmooth unconstrained minimization. J. Optim. Theory Appl. 102, 593-613 (1999)
35. Lukšan, L.; Vlček, J.: Variable metric methods for nonsmooth optimization. Technical report 837, Academy of Sciences of the Czech Republic, May 2001
36. Mascarenhas W.F.: The BFGS method with exact line searches fails for non-convex objective functions. Math. Program. 99, 49-61 (2004) (Pubitemid 40871731)
37. Mifflin R.: An algorithm for constrained optimization with semismooth functions. Math. Oper. Res. 2, 191-207 (1977)
38. Mifflin R.; Sun D.; Qi L.: Quasi-Newton bundle-type methods for nondifferentiable convex optimization. SIAM J. Optim. 8, 583-603 (1998)
39. Nocedal J.; Wright S.J.: Nonlinear Optimization. Springer, New York (2006)
40. Osting B.: Optimization of spectral functions of Dirichlet-Laplacian eigenvalues. J. Comput. Phys. 229, 8578-8590 (2010)
41. Pouly, G.; Lauffenburger, J.-P.; Basset, M.: Reduced order H-infinity control design of a nose landing gear steering system. In: Proceedings of 12th IFAC Symposium on Control in Transportation Systems (2010)
42. Powell, M.J.D.: Some global convergence properties of a variable metric algorithm for minimization without exact line searches. In: Nonlinear Programming, pp. 53-72. American Mathematical Society, Providence. SIAM-AMS Proceedings, vol. IX (1976)
43. Rauf A.I.; Fukushima M.: Globally convergent BFGS method for nonsmooth convex optimization. J. Optim. Theory Appl. 104, 539-558 (2000)
44. Royden H.L.: Real Analysis. Macmillan, New York (1963)
45. Rockafellar R.T.; Wets R.J.B.: Variational Analysis. Springer, New York (1998)
46. Sagastizábal, C.: Composite proximal bundle method. Technical report. http://www.optimization-online.org/DB-HTML/2009/07/2356.html
47. Skajaa, A.: Limited memory BFGS for nonsmooth optimization. Master's thesis, New York University, Jan 2010. http://www.cs.nyu.edu/overton/mstheses/ skajaa/msthesis.pdf
48. Vlček J.; Lukšan L.: Globally convergent variable metric method for nonconvex nondifferentiable unconstrained minimization. J. Optim. Theory Appl. 111, 407-430 (2001) (Pubitemid 33365960)
49. Wang F.-C.; Chen H.-T.: Design and implementation of fixed-order robust controllers for a proton exchange membrane fuel cell system. Int. J. Hydrogen Energy 34, 2705-2717 (2009)
50. Wildschek, A.; Maier, R.; Hromcik, M.; Hanis, T.; Schirrer, A.; Kozek, M.; Westermayer, C.; Hemedi M.: Hybrid controller for gust load alleviation and ride comfort improvement using direct lift control flaps. In: Proceedings of Third European Conference for Aerospace Sciences (EUCASS) (2009)
51. Wolfe, P.: A method of conjugate subgradients for minimizing nondifferentiable functions. Math. Program. Stud. 3, 145-173, (1975). In: Balinski, M.L.; Wolfe, P. (eds.) Nondifferentiable Optimization
52. Yu, J.; Vishwanathan, S.V.N.; Günther, S.; Schraudolph, N.: A quasi-Newton approach to non-smooth convex optimization. In: Proceedings of the 25th International Conference on Machine Learning (2008)
53. Zhang, S.S.: Cornell University, Private Communication (2010)
54. Zhang S.; Zou X.; Ahlquist J.; Navon I.M.; Sela J.G.: Use of differentiable and nondifferentiable optimization algorithms for variational data assimilation with discontinuous cost functions. Mon. Weather Rev. 128, 4031-4044 (2000)