Smoothing methods for nonsmooth, nonconvex minimization

Mathematical Programming

Volumn 134, Issue 1, 2012, Pages 71-99

  Chen, Xiaojun ^a  

^a HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

Author keywords

Eigenvalue optimization; Nonconvex minimization; Nonsmooth; Regularized minimization problems; Smoothing methods; Stochastic variational inequality problems

Indexed keywords

EIGENVALUE OPTIMIZATION; NON-CONVEX MINIMIZATION; NON-SMOOTH; REGULARIZED MINIMIZATION PROBLEMS; SMOOTHING METHODS; STOCHASTIC VARIATIONAL INEQUALITY PROBLEMS;

ALGORITHMS; EIGENVALUES AND EIGENFUNCTIONS; FUNCTIONS; IMAGE RECONSTRUCTION; OPTIMIZATION; STOCHASTIC SYSTEMS; SUPPORT VECTOR MACHINES; VARIATIONAL TECHNIQUES;

PROBLEM SOLVING;

EID: 84865682357     PISSN: 00255610     EISSN: 14364646     Source Type: Journal    
DOI: 10.1007/s10107-012-0569-0     Document Type: Conference Paper

References (107)

1. Alefeld, G., Chen, X.: A regularized projection method for complementarity problems with non-Lipschitzian functions. Math. Comput. 77, 379-395 (2008)
2. Auslender, A.: How to deal with the unbounded in optimization: Theory and algorithm. Math. Program. 79, 3-18 (1997)
3. Ben-Tal, A., EI Ghaoui, L., Nemirovski, A.: Robust Optimization. Princeton University Press, Princeton (2009)
4. Bian,W., Chen, X.: Smoothing neural network for constrained non-Lipschitz optimizationwith applications. IEEE Trans. Neural Netw. Learn. Syst. 23, 399-411 (2012)
5. Bian,W., Chen, X.: Neural network for nonsmooth, nonconvex constrained minimization via smooth approximation. Preprint (2011)
6. Bian, W., Chen, X.: Smoothing SQP algorithm for non-Lipschitz optimization with complexity analysis. Preprint (2012)
7. Bian, W., Xue, X.P.: Subgradient-based neural network for nonsmooth nonconvex optimization problem. IEEE Trans. Neural Netw. 20, 1024-1038 (2009)
8. Bruckstein, A.M., Donoho, D.L., Elad, M.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51, 34-81 (2009)
9. Burke, J.V.: Descent methods for composite nondifferentiable optimization problems. Math. Program. 33, 260-279 (1985)
10. Burke, J.V., Lewis, A.S., Overton, M.L.: A robust gradient sampling algorithm for nonsmooth, nonconvex optimization. SIAM J. Optim. 15, 751-779 (2005)
11. Burke, J.V., Henrion, D., Lewis, A.S., Overton, M.L.: Stabilization via nonsmooth, nonconvex optimization. IEEE Trans. Automat. Control 51, 1760-1769 (2006)
12. Catis, C., Gould, N.I.M., Toint, P.: On the evaluation complexity of composite function minimization with applications to nonconvex nonlinear programming. SIAM J. Optim. 21, 1721-1739 (2011)
13. Chartrand, R.: Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Proc. Lett. 14, 707-710 (2007)
14. Chen, B., Chen, X.: A global and local superlinear continuation-smoothing method for P0 and R0 NCP or monotone NCP. SIAM J. Optim. 9, 624-645 (1999)
15. Chen, B., Harker, P.T.: A non-interior-oint continuation method for linear complementarity problems. SIAM J. Matrix Anal. Appl. 14, 1168-1190 (1993)
16. Chen, B., Xiu, N.: A global linear and local quadratic noninterior continuation method for nonlinear complementarity problems based on Chen-Mangasarian smoothing functions. SIAM J. Optim. 9, 605-623 (1999)
17. Chen, C., Mangasarian, O.L.: A class of smoothing functions for nonlinear and mixed complementarity problems. Math. Program. 71, 51-70 (1995)
18. Chen, X.: Smoothing methods for complementarity problems and their applications: A survey. J. Oper. Res. Soc. Japan 43, 32-46 (2000)
19. Chen, X.: A superlinearly and globally convergent method for reaction and diffusion problems with a non-Lipschitzian operator. Computing[Suppl] 15, 79-90 (2001)
20. Chen, X.: First order conditions for nonsmooth discretized constrained optimal control problems. SIAM J. Control Optim. 42, 2004-2015 (2004)
21. Chen, X., Fukushima, M.: Expected residual minimization method for stochastic linear complementarity problems. Math. Oper. Res. 30, 1022-1038 (2005)
22. Chen, X., Fukushima, M.: A smoothing method for a mathematical program with P-matrix linear complementarity constraints. Comput. Optim. Appl. 27, 223-246 (2004)
23. Chen, X., Ge, D., Wang, Z., Ye, Y.: Complexity of unconstrained L2-L p minimization. Preprint (2011)
24. Chen, X., Nashed, Z., Qi, L.: Smoothing methods and semismooth methods for nondifferentiable operator equations. SIAM J. Numer. Anal. 38, 1200-1216 (2000)
25. Chen, X., Niu, L., Yuan, Y.: Optimality conditions and smoothing trust region Newton method for non-Lipschitz optimization. Preprint (2012)
26. Chen, X., Qi, L.: A parameterized Newton method and a quasi-Newton method for nonsmooth equations. Comput. Optim. Appl. 3, 157-179 (1994)
27. Chen, X., Qi, L., Sun, D.: Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities.Math. Comput. 67, 519-540 (1998)
28. Chen, X.,Wets, R.J-B., Zhang, Y.: Stochastic variational inequalities: Residual minimization smoothing/sample average approximations SIAM J. Optim. (to appear)
29. Chen, X., Womersley, R., Ye, J.: Minimizing the condition number of a Gram matrix. SIAM J. Optim. 21, 127-148 (2011)
30. Chen, X., Xu, F., Ye, Y.: Lower bound theory of nonzero entries in solutions of l2 l p minimization. SIAM J. Sci. Comput. 32, 2832-2852 (2012)
31. Chen, X., Ye, Y.: On homotopy-smoothing methods for variational inequalities. SIAM J. Control Optim. 37, 587-616 (1999)
32. Chen, X., Zhang, C., Fukushima,M.: Robust solution of monotone stochastic linear complementarity problems. Math. Program. 117, 51-80 (2009)
33. Chen, X., Zhou, W.: Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 3, 765-790 (2010)
34. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)
35. Conn, A.R., Scheinberg,K.,Vicente, L.N.: Introduction toDerivative-FreeOptimization. MPS-SIAM Book Series on Optimization. SIAM, Philadelphia (2009)
36. Cottle, R.W., Pang, J.S., Stone, R.E.: The Linear Complementarity Problem. Academic Press, Boston (1992)
37. Daniilidis, A., Sagastizbal, C., Solodov, M.: Identifying structure of nonsmooth convex functions by the bundle technique. SIAM J. Optim. 20, 820-840 (2009)
38. Delage, E., Ye, Y.: Distributionally robust optimization under monment uncertainty with application to data-driven problems. Oper. Res. 58, 595-612 (2010)
39. Facchinei, F., Jiang, H., Qi, L.: A smoothing method for mathematical programs with equilibrium constraints. Math. Program. 85, 107-134 (1999)
40. Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer, New York (2003)
41. Fan, J.: Comments on "Wavelets in Statistic: A review" by A. Antoniadis. Stat. Method. Appl. 6, 131-138 (1997)
42. Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96, 1348-1360 (2001)
43. Fang, H., Chen, X., Fukushima, M.: Stochastic R0 matrix linear complementarity problems. SIAM J. Optim. 18, 482-506 (2007)
44. Ferris, M.: Extended mathematical programming: Competition and stochasticity. SIAM News 45, 1-2 (2012)
45. Fischer, A.: A special Newton-type optimization method. Optimization 24, 269-284 (1992)
46. Fukushima, M., Luo, Z.-Q., Pang, J.-S.: A globally convergent sequential quadratic programming algorithm for mathematical programs with linear complementarity constraints. Comput. Optim. Appl. 10, 5-34 (1998)
47. Fukushima, M., Luo, Z.-Q., Tseng, P.: Smoothing functions for second-order-cone complementarity problems. SIAM J. Optim. 12, 436-460 (2002)
48. Fukushima, M., Pang, J.-S.: Convergence of a smoothing continuation method for mathematical programs with complementarity constraints. In: Thera, M., Tichatschke, R. (eds.) Lecture Notes in Economics and Mathematical Systems, vol. 477, pp. 99-110. Springer, Berlin (1999)
49. Gabriel S.A., More J.J.: Smoothing of mixed complementarity problems. In: Ferris, M.C., Pang, J.S. (eds.) Complementarity and Variational Problems: State of the Art, pp. 105-116. SIAM, Philadelphia (1997)
50. Garmanjani, R., Vicente, L.N.: Smoothing and worst case complexity for direct-search methods in non-smooth optimization. Preprint (2011)
51. Ge, D., Jiang, X., Ye, Y.: A note on the complexity of L p minimization. Math. Program. 21, 1721-1739 (2011)
52. Geman, D., Reynolds, G.: Constrained restoration and the recovery of discontinuities. IEEE Trans. Pattern Anal. Mach. Intell. 14, 357-383 (1992)
53. Gürkan, G., Özge, A.Y., Robinson, S.M.: Sample-ath solution of stochastic variational inequalities. Math. Program. 84, 313-333 (1999)
54. Hamatani, K., Fukushima, M.: Pricing American options with uncertain volatility through stochastic linear complementarity models. Comput. Optim. Appl. 50, 263-286 (2011)
55. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning Data Mining, Inference, and Prediction. Springer, New York (2009)
56. Hayashi, S., Yamashita, N., Fukushima, M.: A combined smoothing and regularization method for monotone second-order cone complementarity problems. SIAM J. Optim. 15, 593-615 (2005)
57. Heinkenschloss, M., KelleyC.T. Tran, H.T.: Fast algorithms for nonsmooth compact fixed-oint problems. SIAM J. Numer. Anal. 29, 1769-1792 (1992)
58. Hintermueller, M.,Wu, T.: Nonconvex TVq -models in image restoration: Analysis and a trust-region regularization based superlinearly convergent solver. Preprint (2011)
59. Hiriart-Urruty, J.B., Lemareéchal, C.: Convex Analysis and Minimization Algorithms II: Advanced Theory and Boundle Methods. Springer, Berlin (1993)
60. Hu, M., Fukushima, M.: Smoothing approach to Nash equilibrium formulations for a class of equilibrium problems with shared complementarity constraints. Comput. Optim. Appl. (to appear)
61. Huang, J., Horowitz, J.L., Ma, S.: Asymptotic properties of bridge estimators in sparse high-dimensional regression models. Ann. Stat. 36, 587-613 (2008)
62. Huang, J., Ma, S., Xie, H., Zhang, C.-H.: A group bridge approach for variable selection. Biometrika 96, 339-355 (2009)
63. Huang, Z., Qi, L., Sun, D.: Sub-quadratic convergence of a smoothing Newton algorithm for the P0-and monotone LCP. Math. Program. 99, 423-441 (2004)
64. Jiang, H., Ralph, D.: Smooth SQP methods for mathematical programs with nonlinear complementarity constraints. Comput. Optim. Appl. 25, 123-150 (2002)
65. Jiang, H., Xu, H.: Stochastic approximation approaches to the stochastic variational inequality problem. IEEE. Trans. Autom. Control 53, 1462-1475 (2008)
66. Kanzow, C.: Some noninterior continuation methods for linear cornplementarity problems. SIAM J. Matrix Anal. Appl. 17, 851-868 (1996)
67. Knight, K., Fu, W.J.: Asymptotics for lasso-type estimators. Ann. Stat. 28, 1356-1378 (2000)
68. Kiwiel, K.C.: Methods of Descent for Nondifferentiable Optimization, Lecture Notes in Math, vol. 1133. Springer, Berlin (1985)
69. Kiwiel, K.C.: Convergence of the gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim. 18, 379-388 (2007)
70. Lewis, A.S., Pang, C.H.J.: Lipschitz behavior of the robust regularization. SIAM, J. Control Optim. 48, 3080-3104 (2010)
71. Lewis, A.S., Overton, M.L.: Eigenvalue optimization. Acta Numerica 5, 149-190 (1996)
72. Li, D., Fukushima, M.: Smoothing Newton and quasi-Newton methods for mixed complementarity problems. Comput. Optim. Appl. 17, 203-230 (2000)
73. Lin, G.H., Chen, X., Fukushima, M.: Solving stochastic mathematical programs with equilibrium constraints via approximation and smoothing implicit programming with penalization. Math. Program. 116, 343-368 (2009)
74. Lin, G.H., Fukushima, M.: Stochastic equilibrium problems and stochastic mathematical programs with equilibrium constraints: A survey. Pac. J. Optim. 6, 455-482 (2010)
75. Luo, Z-Q., Pang, J-S., Ralph, D., Wu, S.: Exact penalization and stationarity conditions of mathematical programs with equilibrium constraints. Math. Program. 75, 19-76 (1996)
76. Maréchal, P., Ye, J.J.: Optimizing condition numbers. SIAM J. Optim. 20, 935-947 (2009)
77. Martínez, J.M., Moretti, A.C.: A trust region method for minimization of nonsmooth functions with linear constraints. Math. Program. 76, 431-449 (1997)
78. Meng, K., Yang, X.: Optimality conditions via exact penalty functions. SIAM J. Optim. 20, 3205-3231 (2010)
79. Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 15, 957-972 (1977)
80. Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I and II. Springer, Berlin (2006)
81. Nesterov, Y.: Smooth minimization of nonsmooth functions. Math. Program. 103, 127-152 (2005)
82. Nesterov, Y.: Smoothing technique and its applications in semidefinite optimization. Math. Program. 110, 245-259 (2007)
83. Newey, W.K., McFadden, D.: Large sample estimation and hypothesis testing. In: Handbook of econometrics, vol. IV, pp. 2111-2245. North-Holland, Amsterdam (1994)
84. Nikolova, M.: Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. SIAM J. Multiscale Model. Simul. 4, 960-991 (2005)
85. Nikolova,M., Ng,M.K., Zhang, S., Ching,W.: Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 1, 2-25 (2008)
86. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)
87. Osborne,M.R.: Finite Algorithms in Optimizations in Optimization and Data Analysis.Wiley, Chichester (1985)
88. Polyak, R.A.: A nonlinear rescaling vs. smoothing technique in convex optimization. Math. Program. 92, 197-235 (2002)
89. Qi, H.D., Liao, L.Z.: A smoothing Newton method for extended vertical linear complementarity problems. SIAM J. Matrix Anal. Appl. 21, 45-66 (1999)
90. Qi, L., Chen, X.: A globally convergent successive approximation method for severely nonsmooth equations. SIAM J. Control Optim. 33, 402-418 (1995)
91. Qi, L., Ling, C., Tong, X., Zhou, G.: A smoothing projected Newton-type algorithm for semi-infinite programming. Comput. Optim. Appl. 42, 1-30 (2009)
92. Qi, L., Sun, D., Zhou, G.: A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities. Math. Program. 87, 1-35 (2000)
93. Rockafellar, R.T.: A property of piecewise smooth functions. Comput. Optim. Appl. 25, 247-250 (2003)
94. Rockafellar, R.T., Wets, R.J-B.: Variational Analysis. Springer, New York (1998)
95. Ruszczynski,A., Shapiro, A.: Stochastic Programming, Handbooks in Operations Research andManagement Science. Elsevier, Amsterdam (2003)
96. Smale, S.: Algorithms for solving equations. In: Proceedings of the International Congress of Mathematicians, Berkeley, CA, pp. 172-195 (1986)
97. Sun, J., Sun, D., Qi, L.: A squared smoothing Newton method for nonsmooth matrix equations and its applications in semidefinite optimization problems. SIAM J. Optim. 14, 783-806 (2004)
98. Tassa, Y., Todorov, E.: Stochastic complementarity for local control of discontinuous dynamics. In: Proceedings of Robotics: Science and Systems (RSS) (2010)
99. Wright, S.J.: Convergence of an inexact algorithm for composite nonsmooth optimization. IMA J. Numer. Anal. 9, 299-321 (1990)
100. Xu, H.: Sample average approximation methods for a class of stochastic variational inequality problems. Asia Pac. J. Oper. Res. 27, 103-119 (2010)
101. Xu, Z., Zhang, H., Wang, Y., Chang, X.: L1/2 regularizer. Sci. China Ser. F-Inf Sci. 53, 1159-1169 (2010)
102. Yuan, Y.: Conditions for convergence of a trust-region method for nonsmooth optimization. Math. Program. 31, 220-228 (1985)
103. Zang, I.: A smoothing-out technique for min-max optimization. Math. Program. 19, 61-71 (1980)
104. Zhang, C., Chen, X.: Smoothing projected gradient method and its application to stochastic linear complementarity problems. SIAM J. Optim. 20, 627-649 (2009)
105. Zhang, C., Chen, X., Sumalee, A.: Wardrop's user equilibrium assignment under stochastic environment. Transp. Res. B 45, 534-552 (2011)
106. Zhang, C.-H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38, 894-942 (2010)
107. Zhou, G.L., Caccetta, L., Teo, K.L: Asuperlinearly convergent method for a class of complementarity problems with non-Lipschitzian functions. SIAM J. Optim. 20, 1811-1827 (2010)