A one-layer recurrent neural network for pseudoconvex optimization subject to linear equality constraints

IEEE Transactions on Neural Networks

Volumn 22, Issue 12 PART 1, 2011, Pages 1892-1900

  Guo, Zhishan ^a   Liu, Qingshan ^b   Wang, Jun ^c  

^a University of North Carolina   (United States)

^b SOUTHEAST UNIVERSITY   (China)

^c CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Global convergence; linear equality constraints; pseudoconvex optimization; recurrent neural networks

Indexed keywords

CHEMICAL PROCESS; EQUALITY CONSTRAINTS; FEASIBLE REGIONS; GLOBAL CONVERGENCE; GLOBAL EXPONENTIAL CONVERGENCE; ILLUSTRATIVE EXAMPLES; LINEAR EQUALITY CONSTRAINTS; OBJECTIVE FUNCTIONS; OPTIMIZATION PROBLEMS; PSEUDOCONVEX;

COMPUTER SIMULATION; OPTIMIZATION; RECURRENT NEURAL NETWORKS;

NETWORK LAYERS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; STATISTICAL MODEL;

ALGORITHMS; COMPUTER SIMULATION; LINEAR MODELS; NEURAL NETWORKS (COMPUTER);

EID: 83855161604     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2011.2169682     Document Type: Article

References (66)

1. O. L. Mangasarian, "Pseudo-convex functions," SIAM J. Control, vol. 3, no. 2, pp. 281-290, 1965.
2. J. Ponstein, "Seven kinds of convexity," SIAM Rev., vol. 9, no. 1, pp. 115-119, Jan. 1967.
3. W. Dinkelbach, "On nonlinear fractional programming," Manage. Sci., vol. 13, no. 7, pp. 492-498, Mar. 1967.
4. L. Carosi and L. Martein, "Some classes of pseudoconvex fractional functions via the Charnes-Cooper transformation," in Generalized Convexity and Related Topics. Berlin, Germany: Springer-Verlag, 2006, pp. 177-188.
5. J. R. Barber, Solid Mechanics and Its Applications, 2nd ed. 2004, ch. 26, pp. 351-357.
6. F. Forgo and I. Joó, "Fixed point and equilibrium theorems in pseudo-convex and related spaces," J. Global Optim., vol. 14, no. 1, pp. 27-54, Jan. 1999. (Pubitemid 129617285)
7. C. Olsson and F. Kahl, "Generalized convexity in multiple view geometry," J. Math Imaging Vis., vol. 38, no. 1, pp. 35-51, Sep. 2010.
8. D. W. Tank and J. J. Hopfield, "Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit," IEEE Trans. Circuits Syst., vol. 33, no. 5, pp. 533-541, May 1986.
9. S. Zhang and A. G. Constantinides, "Lagrange programming neural networks," IEEE Trans. Circuits Syst. II, vol. 39, no. 7, pp. 441-452, Jul. 1992. (Pubitemid 23579814)
10. J. Wang, "A deterministic annealing neural network for convex programming," Neural Netw., vol. 7, no. 4, pp. 629-641, 1994.
11. J. Wang, Q. Hu, and D. Jiang, "A Lagrangian network for kinematic control of redundant robot manipulators," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 1123-1132, Sep. 1999.
12. Y. Xia and J. Wang, "A recurrent neural network for solving nonlinear convex programs subject to linear constraints," IEEE Trans. Neural Netw., vol. 16, no. 2, pp. 379-386, Mar. 2005. (Pubitemid 40390821)
13. M. Forti, P. Nistri, and M. Quincampoix, "Generalized neural network for nonsmooth nonlinear programming problems," IEEE Trans. Circuits Syst. I, vol. 51, no. 9, pp. 1741-1754, Sep. 2004.
14. S. Liu and J. Wang, "A simplified dual neural network for quadratic programming with its KWTA application," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1500-1510, Nov. 2006. (Pubitemid 44824263)
15. Q. Liu and J. Wang, "A recurrent neural network for non-smooth convex programming subject to linear equality and bound constraints," in Proc. Int. Conf. Neural Inf. Process., vol. 4233. 2006, pp. 1004-1013. (Pubitemid 44699812)
16. Q. Liu and J. Wang, "A one-layer recurrent neural network with a discontinuous activation function for linear programming," Neural Comput., vol. 20, no. 5, pp. 1366-1383, May 2008.
17. Q. Liu and J. Wang, "A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming," IEEE Trans. Neural Netw., vol. 19, no. 4, pp. 558-570, Apr. 2008. (Pubitemid 351583574)
18. Q. Liu and J. Wang, "A one-layer recurrent neural network for non-smooth convex optimization subject to linear equality constraints," in Proc. Int. Conf. Neural Inf. Process., vol. 5507. 2009, pp. 1003-1010.
19. Q. Liu and J. Wang, "Finite-time convergent recurrent neural network with a hard-limiting activation function for constrained optimization with piecewise-linear objective functions," IEEE Trans. Neural Netw., vol. 22, no. 4, pp. 601-603, Apr. 2011.
20. M. P. Barbarosou and N. G. Maratos, "Non-feasible gradient projection recurrent neural network for equality constrained optimization," in Proc. IEEE Int. Joint Conf. Neural Netw., vol. 3. Jul. 2004, pp. 2251-2256. (Pubitemid 40006608)
21. Y. Xia, G. Feng, and J. Wang, "A novel recurrent neural network for solving nonlinear optimization problems with inequality constraints," IEEE Trans. Neural Netw., vol. 19, no. 8, pp. 1340-1353, Aug. 2008.
22. Y. Xia, G. Feng, and J. Wang, "A recurrent neural network with exponential convergence for solving convex quadratic program and related linear piecewise equations," Neural Netw., vol. 17, no. 7, pp. 1003-1015, Sep. 2004. (Pubitemid 39093818)
23. X. Hu, C. Sun, and B. Zhang, "Design of recurrent neural networks for solving constrained least absolute deviation problems," IEEE Trans. Neural Netw., vol. 21, no. 7, pp. 1073-1086, Jul. 2010.
24. M. P. Barbarosou and N. G. Maratos, "A nonfeasible gradient projection recurrent neural network for equality-constrained optimization problems," IEEE Trans. Neural Netw., vol. 19, no. 10, pp. 1665-1677, Oct. 2008.
25. Z. Guo and J. Wang, "A neurodynamic optimization approach to constrained sparsity maximization based on alternative objective functions," in Proc. Int. Joint Conf. Neural Netw., Barcelona, Spain, Jul. 2010, pp. 1-8.
26. X. Hu and J. Wang, "Solving pseudomonotone variational inequalities and pseudoconvex optimization problems using the projection neural network," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1487-1499, Nov. 2006. (Pubitemid 44824262)
27. Y. Xia, H. Leung, and J. Wang, "A projection neural network and its application to constrained optimization problems," IEEE Trans. Circuits Syst. I, vol. 49, no. 4, pp. 447-458, Apr. 2002. (Pubitemid 34515146)
28. X. Hu and J. Wang, "Design of general projection neural networks for solving monotone linear variational inequalities and linear and quadratic optimization problems," IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 37, no. 5, pp. 1414-1421, Oct. 2007. (Pubitemid 47534605)
29. M. S. Bazaraa, H. D. Sherali, and C. M. Shetty, Nonlinear Programming: Theory and Algorithms, 3rd ed. New York: Wiley, 2006.
30. J. Cortes, "Discontinuous dynamical systems," IEEE Control Syst. Mag., vol. 28, no. 3, pp. 36-73, Jun. 2008. (Pubitemid 351784780)
31. A. Bacciotti and F. Ceragioli, "Stability and stabilization of discontinuous systems and nonsmooth Lyapunov functions," ESAIM: Control, Optim. Calculus Variat., vol. 4, pp. 361-376, Jun. 1999. (Pubitemid 33006351)
32. J. P. Aubin and H. Frankowska, Set-Valued Analysis. Basel, Switzerland: Birkhäuser, 1990.
33. S. Karamardian and S. Schaible, "Seven kinds of monotone maps," J. Optim. Theory Appl., vol. 66, no. 1, pp. 37-46, Jul. 1990. (Pubitemid 21656369)
34. J. P. LaSalle, "Some extensions of Liapunov's second method," IRE Trans. Circuit Theory, vol. 7, no. 4, pp. 520-527, Dec. 1960.
35. H. K. Khalil, Nonlinear Systems. Englewood Cliffs, NJ: Prentice-Hall, 1996.
36. S. Zhang, X. Zhu, and L. H. Zou, "Second-order neural nets for constrained optimization," IEEE Trans. Neural Netw., vol. 3, no. 6, pp. 1021-1024, Nov. 1992. (Pubitemid 23580829)
37. J. Romagnoli and M. Sanchez, Data Processing and Reconciliation for Chemical Process Operation. New York: Academic, 1999.
38. B. Derya and W. Ralph, "Theory and practice of simultaneous data reconciliation and gross error detection for chemical processes," Comput. Chem. Eng., vol. 28, no. 3, pp. 381-402, 2004.
39. R. Serth, C. Valero, and W. Heenan, "Detection of gross errors in nonlinearly constrained data: A case study," Chem. Eng. Commun., vol. 51, nos. 1-6, pp. 89-104, 1987.
40. D. Ripps, "Adjustment of experimental data," Chem. Eng. Progr. Symp. Ser., vol. 61, no. 55, pp. 8-13, 1965.
41. D. Rollins and J. Davis, "Gross error detection when variance-covariance matrices are unknown," J. Amer. Inst. Chem. Eng., vol. 39, no. 8, pp. 1335-1341, Aug. 1993.
42. R. Serth and W. Heenan, "Gross error detection and data reconciliation in steam-metering systems," J. Amer. Inst. Chem. Eng., vol. 32, no. 5, pp. 733-742, 1986. (Pubitemid 16546834)
43. T. Edgar, D. Himmelblau, and L. Lasdon, Optimization of Chemical Processes, 2nd ed. New York: McGraw-Hill, 2002.
44. L. Cheng, Z. Hou, M. Tan, Y. Lin, W. Zhang, and F. Wang, "Recurrent neural network for non-smooth convex optimization problems with application to the identification of genetic regulatory networks," IEEE Trans. Neural Netw., vol. 22, no. 5, pp. 714-726, May 2011.
45. N. Hadjisavvas and S. Schaible, "On strong pseudomonotonicity and (semi)strict quasimonotonicity," J. Optim. Theory Appl., vol. 79, no. 1, pp. 139-155, 1993.
46. P. Huber, Robust Statistics. New York: Wiley, 1981.
47. Z. Wang, Y. Liu, M. Li, and X. Liu, "Stability analysis for stochastic Cohen-Grossberg neural networks with mixed time delays," IEEE Trans. Neural Netw., vol. 17, no. 3, pp. 814-820, May 2006.
48. S. Xu, J. Lam, D. W. C. Ho, and Y. Zou, "Novel global asymptotic stability criteria for delayed cellular neural networks," IEEE Trans. Circuits Syst. II, vol. 52, no. 6, pp. 349-353, Jun. 2005. (Pubitemid 40986454)
49. H. J. Cho and J. H. Park, "Novel delay-dependent robust stability criterion of delayed cellular neural networks," Chaos, Solitons, Fractals, vol. 32, no. 3, pp. 1194-1200, May 2007. (Pubitemid 44855571)
50. C. C. Hua, C. N. Long, and X. P. Guan, "New results on stability analysis of neural networks with time-varying delays," Phys. Lett. A, vol. 352, nos. 4-5, pp. 335-340, Apr. 2006.
51. Y. He, Q. G. Wang, and M. Wu, "LMI-based stability criteria for neural networks with multiple time-varying delays," Phys. D, vol. 212, nos. 1-2, pp. 126-136, Dec. 2005. (Pubitemid 41617928)
52. Y. He, G. P. Liu, and D. Rees, "New delay-dependent stability criteria for neural networks with time-varying delay," IEEE Trans. Neural Netw., vol. 18, no. 1, pp. 310-314, Jan. 2007. (Pubitemid 46062938)
53. W. H. Chen and W. X. Zheng, "Improved delay-dependent asymptotic stability criteria for delayed neural networks," IEEE Trans. Neural Netw., vol. 19, no. 12, pp. 2154-2161, Dec. 2008.
54. W. H. Chen, X. Lu, Z. H. Guan, and W. X. Zheng, "Delay-dependent exponential stability of neural networks with variable delay: An LMI approach," IEEE Trans. Circuits Syst. II, vol. 53, no. 9, pp. 837-842, Sep. 2006. (Pubitemid 46511462)
55. Y. R. Liu, Z. Wang, and X. H. Liu, "Design of exponential state estimators for neural networks with mixed time delays," Phys. Lett. A, vol. 364, no. 5, pp. 401-412, May 2007.
56. T. Li, L. Guo, and C. Lin, "Stability criteria with less LMI variables for neural networks with time-varying delay," IEEE Trans. Circuits Syst. II, vol. 55, no. 11, pp. 1188-1192, Nov. 2008.
57. T. Li and X. L. Ye, "Improved stability criteria of neural networks with time-varying delays: An augmented LKF approach," Neurocomputing, vol. 73, nos. 4-6, pp. 1038-1047, Jan. 2010.
58. X. L. Zhu and G. H. Yang, "New delay-dependent stability results for neural networks with time-varying delay," IEEE Trans. Neural Netw., vol. 19, no. 10, pp. 1783-1791, Oct. 2008.
59. H. Zhang, Z. W. Liu, G. B. Huang, and Z. S. Wang, "Novel weighting-delay-based stability criteria for recurrent neural networks with time-varying delay," IEEE Trans. Neural Netw., vol. 21, no. 1, pp. 91-106, Jan. 2010.
60. S. Yoshizawa, M. Morita, and S. I. Amari, "Capacity of associative memory using a nonmonotonic neural model," Neural Netw., vol. 6, no. 2, pp. 167-176, 1993. (Pubitemid 23622316)
61. M. Forti, "On global asymptotic stability of a class of nonlinear systems arising in neural network theory," J. Different. Equat., vol. 113, no. 1, pp. 246-264, 1994.
62. M. Forti and A. Tesi, "New conditions for global stability of neural networks with application to linear and quadratic programming problems," IEEE Trans. Circuits Syst. I: Fundam. Theory Appl., vol. 42, no. 7, pp. 354-366, Jul. 1995.
63. D. P. Mandic and J. A. Chambers, "Relating the slope of the activation function and the learning rate within a recurrent neural network," Neural Comput., vol. 11, no. 5, pp. 1069-1077, Jul. 1999.
64. J. J. Hopfield, "Neurons with graded response have collective computational properties like those of two state neurons," Proc. Nat. Acad. Sci. United States Amer., vol. 81, no. 10, pp. 3088-3092, May 1984. (Pubitemid 14109528)
65. J. Xu, Y. Cao, Y. Sun, and J. Tang, "Absolute exponential stability of recurrent neural networks with generalized activation function," IEEE Trans. Neural Netw., vol. 19, no. 6, pp. 1075-1089, Jun. 2008.
66. N. E. Barabanov, "Stability of systems with several monotone nonlin-earities," IEEE Trans. Autom. Control, vol. 48, no. 4, pp. 665-670, Apr. 2003.