Neural network for nonsmooth, nonconvex constrained minimization via smooth approximation

IEEE Transactions on Neural Networks and Learning Systems

Volumn 25, Issue 3, 2014, Pages 545-556

  Bian, Wei ^a   Chen, Xiaojun ^b  

^a HARBIN INSTITUTE OF TECHNOLOGY   (China)

^b HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

Author keywords

Clarke stationary point; condition number; neural network; nonsmooth nonconvex optimization; smoothing approximation; variable selection

Indexed keywords

CONDITION NUMBERS; NONSMOOTH NONCONVEX OPTIMIZATION; SMOOTHING APPROXIMATION; STATIONARY POINTS; VARIABLE SELECTION;

CONSTRAINED OPTIMIZATION; NUMBER THEORY; ORDINARY DIFFERENTIAL EQUATIONS;

NEURAL NETWORKS;

ALGORITHM; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; HUMAN; NONLINEAR SYSTEM;

ALGORITHMS; COMPUTER SIMULATION; HUMANS; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS;

EID: 84897640798     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2278427     Document Type: Article

References (37)

1. A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing. New York, NY, USA: Wiley, 1993.
2. E. K. P. Chong, S. Hui, and S. H. Zak, "An analysis of a class of neural networks for solving linear programming problems," IEEE Trans. Autom. Control, vol. 44, no. 11, pp. 1995-2005, Nov. 1999.
3. M. Forti, P. Nistri, and M. Quincampoix, "Generalized neural network for nonsmooth nonlinear programming problems," IEEE Trans. Neural Netw., vol. 51, no. 9, pp. 1741-1754, Sep. 2004.
4. M. Forti, P. Nistri, and M. Quincampoix, "Convergence of neural networks for programming problems via a nonsmooth ?ojasiewicz inequality," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1471-1486, Nov. 2006. (Pubitemid 44824261)
5. Y. S. Xia, G. Feng, and J. Wang, "A novel recurrent neural network for solving nonlinear optimization problems with inequality constraints," IEEE Trans. Neural Netw., vol. 19, no. 8, pp. 1340-1353, Aug. 2008.
6. J. L. Liang, Z. D. Wang, and X. H. Liu, "State estimation for coupled uncertain stochastic networks with missing measurements and timevarying delays: The discrete-time case," IEEE Trans. Neural Netw., vol. 20, no. 5, pp. 781-793, May 2009.
7. W. Bian and X. P. Xue, "Subgradient-based neural network for nonsmooth nonconvex optimization problem," IEEE Trans. Neural Netw., vol. 20, no. 6, pp. 1024-1038, Jun. 2009.
8. Q. S. Liu and J. Wang, "Finite-time convergent recurrent neural network with a hard-limiting activation function for constrained optimization with piecewise-linear objective functions," IEEE Trans. Neural Netw., vol. 22, no. 4, pp. 601-613, Apr. 2011.
9. Q. S. Liu and J. Wang, "A one layer recurrent neural network for constrained nonsmooth optimization," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 41, no. 5, pp. 1323-1333, Oct. 2011.
10. L. Cheng, Z. G. Hou, Y. Z. Lin, M. Tan, W. C. Zhang, and F. X. Wu, "Recurrent neural network for non-smooth convex optimization problems with application to identification of genetic regulatory networks," IEEE Trans. Neural Netw., vol. 22, no. 5, pp. 714-726, May 2011.
11. Q. Liu, Z. Guo, and J. Wang, "A one-layer recurrent neural network for constrained pseudoconvex optimization and its application for portfolio optimization," Neural Netw., vol. 26, pp. 99-109, Feb. 2012.
12. W. Bian and X. Chen, "Smoothing neural network for constrained non-Lipschitz optimization with applications," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 3, pp. 399-411, Mar. 2012.
13. T. Yamada and T. Yabuta, "Dynamic system identification using neural networks," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 23, no. 1, pp. 204-211, Jan./Feb. 1993.
14. W. L. Lu and J. Wang, "Convergence analysis of a class of nonsmooth gradient systems," IEEE Trans. Circuits Syst. I, vol. 55, no. 11, pp. 3514-3527, Dec. 2008.
15. H. D. Qi and L. Qi, "Deriving sufficient conditions for global asymptotic stability of delayed neural networks via nonsmooth analysis," IEEE Trans. Circuits Syst. I, vol. 15, no. 1, pp. 99-109, Jan. 2004.
16. M. Forti, "M-matrices and global convergence of discontinuous neural networks," Int. J. Circuit Theory Appl., vol. 35, no. 2, pp. 105-130, 2007. (Pubitemid 46506957)
17. M. Di Marco, M. Forti, M. Grazzini, P. Nistri, and L. Pancioni, "Lyapunov method and convergence of the full-range model of CNNs," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 55, no. 11, pp. 3528-3541, Nov. 2008.
18. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. R. Stat. Soc. Ser. B, vol. 58, no. 1, pp. 267-288, 1996.
19. L. B. Montefusco, D. Lazzaro, and S. Papi, "Nonlinear filtering for sparse signal recovery from incomplete measurements," IEEE Trans. Signal Process., vol. 57, no. 7, pp. 2494-2502, Jul. 2009.
20. R. Chartrand and V. Staneva, "Restricted isometry properties and nonconvex compressive sensing," Inverse Problems, vol. 24, no. 3, pp. 1-14, 2008.
21. W. Bian and X. Chen, "Worst-case complexity of smoothing quadratic regularization methods for non-Lipschitz optimization," SIAM J. Optim., vol. 23, no. 3, pp. 1718-1741, Mar. 2013.
22. W. Bian, X. Chen, and Y. Ye, "Complexity analysis of interior point algorithms for non-Lipschitz and nonconvex minimization," 2013, preprint.
23. X. Chen, R. Womersley, and J. Ye, "Minimizing the condition number of a Gram matrix," SIAM J. Optim., vol. 21, pp. 127-148, Jan. 2011.
24. C. Zhang and X. Chen, "Smoothing projected gradient method and its application to stochastic linear complementarity problems," SIAM J. Optim., vol. 20, no. 2, pp. 627-649, 2009.
25. F. E. Curtis and M. L. Overton, "A sequential quadratic programming algorithm for nonconvex, nonsmooth constrained optimization," SIAM J. Optim., vol. 22, no. 2, pp. 474-500, 2012.
26. J. Kreimer and R. Y. Rubinstein, "Nondifferentiable optimization via smooth approximation: General analytical approach," Ann. Oper. Res., vol. 39, no. 1, pp. 97-119, 1992.
27. I. Necoara and J. A. K. Suykens, "Application of a smoothing technique to decomposition in convex optimization," IEEE Trans. Autom. Control, vol. 53, no. 11, pp. 2674-2679, Dec. 2008.
28. X. Chen, "Smoothing methods for nonsmooth, nonconvex minimization," Math. Program., vol. 134, no. 1, pp. 71-99, 2012.
29. X. Chen, "Smoothing methods for complementarity problems and their applications: A survey," J. Oper. Res. Soc. Jpn., vol. 43, no. 1, pp. 32-47, 2000.
30. R. T. Rockafellar and R. J.-B. Wets, Variational Analysis. Berlin, Germany: Springer-Verlag, 1998.
31. F. H. Clarke, Optimization and Nonsmooth Analysis. New York, NY, USA: Wiley, 1983.
32. J. Nocedal and S. J. Wright, Numerical Optimization. New York, NY, USA: Springer-Verlag, 1999.
33. L. O. Chua, C. A. Desoer, and E. S. Kuh, Linear and Nonlinear Circuits. New York, NY, USA: McGraw-Hill, 1987.
34. J. Penot and P. Quang, "Generalized convexity of functions and generalized monotonicity of set-valued maps," J. Optim. Theory Appl., vol. 92, no. 2, pp. 343-356, 1997.
35. M. Gurbuzbalaban and M. L. Overton, "On Nesterov's nonsmooth Chebyshev-Rosenbrock functions," Nonlinear Anal., Theory Methods Appl., vol. 75, no. 3, pp. 1282-1289, 2012.
36. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning Data Mining, Inference, and Prediction. New York, NY, USA: Springer-Verlag, 2009.
37. D. Betounes, Differential Equations: Theory and Applications. New York, NY, USA: Springer-Verlag, 2009.