Design of recurrent neural networks for solving constrained least absolute deviation problems

IEEE Transactions on Neural Networks

Volumn 21, Issue 7, 2010, Pages 1073-1086

  Hu, Xiaolin ^a   Sun, Changyin ^a   Zhang, Bo ^b  

^a TSINGHUA UNIVERSITY   (China)

^b SOUTHEAST UNIVERSITY   (China)

Author keywords

L1 norm optimization; least absolute deviation (LAD); minimax optimization; recurrent neural network (RNN); stability analysis

Indexed keywords

BOUND CONSTRAINTS; CONVERGENCE RATES; INEQUALITY CONSTRAINT; LEAST ABSOLUTE DEVIATIONS; LINEAR CONSTRAINTS; MINIMAX OPTIMIZATION; NUMERICAL SIMULATION; OPTIMIZATION PROBLEMS; STABILITY ANALYSIS; STRUCTURAL COMPLEXITY; THEORETICAL RESULT;

COMPUTER SIMULATION; CONSTRAINT THEORY; CONVERGENCE OF NUMERICAL METHODS; OPTIMIZATION; PROBLEM SOLVING; REINFORCEMENT LEARNING;

RECURRENT NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; HUMAN; METHODOLOGY; NONLINEAR SYSTEM;

ALGORITHMS; COMPUTER SIMULATION; HUMANS; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; PATTERN RECOGNITION, AUTOMATED;

EID: 77954563179     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2048123     Document Type: Article

References (52)

1. M. P. Barbarosou and N. G. Maratos, "A nonfeasible gradient projection recurrent neural network for equality-constrained optimization problems," IEEE Trans. Neural Netw., vol.19, no.10, pp. 1665-1677, Oct. 2008.
2. P. Bloomfield and W. L. Steiger, Least Absolute Deviations: Theory, Applications, and Algorithms. Boston, MA: Brikhäuser, 1983.
3. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge University Press, 2004.
4. L. Cheng, Z. G. Hou, and M. Tan, "A neutral-type delayed projection neural network for solving nonlinear variational inequalities," IEEE Trans. Circuits Syst. II, vol.55, no.8, pp. 806-810, Aug. 2008.
5. A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing. New York: Wiley, 1993.
6. G. Costantini, R. Perfetti, and M. Todisco, "Quasi-lagrangian neural network for convex quadratic optimization," IEEE Trans. Neural Netw., vol.19, no.10, pp. 1804-1809, Oct. 2008.
7. Y. Dodge and J. Jana, Adaptive Regression. New York: Springer, 2000.
8. M. Forti, P. Nistri, and M. Quincampoix, "Generalized neural network for nonsmooth nonlinear programming problems," IEEE Trans. Circuits Syst. I, vol.51, no.9, pp. 1741-1754, Sep. 2004.
9. A. L. Gall and V. Zissimopoulos, "Extended Hopfield models for combinatorial optimization," IEEE Trans. Neural Netw., vol.10, no.1, pp. 72-80, Jan. 1999.
10. X. Gao, "A novel neural network for nonlinear convex programming," IEEE Trans. Neural Netw., vol.15, no.3, pp. 613-621, May 2004.
11. X. Gao and L. Liao, "A novel neural network for a class of convex quadratic minimax problems," Neural Comput., vol.18, no.8, pp. 1818-1846, Aug. 2006.
12. X. Gao, L. Liao, and L. Qi, "A novel neural network for variational inequalities with linear and nonlinear constraints," IEEE Trans. Neural Netw., vol.16, no.6, pp. 1305-1317, Nov. 2005.
13. X. Gao, L. Liao, and W. Xue, "A neural network for a class of convex quadratic minimax problems with constraints," IEEE Trans. Neural Netw., vol.15, no.3, pp. 622-628, May 2004.
14. S. Haykin, Adaptive Filter Theory. Englewood Cliffs, NJ: Prentice-Hall, 1991.
15. J. J. Hopfield and D. W. Tank, "Computing with neural circuits: A model," Science, vol.233, no.4764, pp. 625-633, Aug. 1986.
16. Z. Hou, M. Gupta, P. Nikiforuk, M. Tan, and L. Cheng, "A recurrent neural network for hierarchical control of interconnected dynamic systems," IEEE Trans. Neural Netw., vol.18, no.2, pp. 466-481, Mar. 2007.
17. X. Hu and J. Wang, "Solving pseudomonotone variational inequalities and pseudoconvex optimization problems using the projection neural network," IEEE Trans. Neural Netw., vol.17, no.6, pp. 1487-1499, Nov. 2006.
18. X. Hu and J. Wang, "Solving generally constrained generalized linear variational inequalities using the general projection neural networks," IEEE Trans. Neural Netw., vol.18, no.6, pp. 1697-1708, Nov. 2007.
19. X. Hu and J. Wang, "An improved dual neural network for solving a class of quadratic programming problems and its k-winners-take-all application," IEEE Trans. Neural Netw., vol.19, no.12, pp. 2022-2031, Dec. 2008.
20. X. Hu and B. Zhang, "A new recurrent neural network for solving convex quadratic programming problems with an application to the k-winnerstake- all problem," IEEE Trans. Neural Netw., vol.20, no.4, pp. 654-664, Apr. 2009.
21. X. Hu and B. Zhang, "Another simple recurrent neural network for quadratic and linear programming," in Proc. 6th Int. Symp. Neural Netw., May 2009, pp. 116-125.
22. X. Hu and B. Zhang, "An alternative recurrent neural network for solving variational inequalities and related optimization problems," IEEE Trans. Syst., Man, Cybern. B, vol.39, no.6, pp. 1640-1645, Dec. 2009.
23. D. Kinderlehrer and G. Stampcchia, An Introduction to Variational Inequalities and Their Applications. New York: Academic, 1980.
24. W. Li and J. J. Swetits, "The linear l1 estimator and the Huber Mestimator," SIAM J. Optimization, vol.8, no.2, pp. 457-475, 1998.
25. Q. Liu and J.Wang, "A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming," IEEE Trans. Neural Netw., vol.19, no.4, pp. 558-570, Apr. 2008.
26. J. Malick, "A dual approach to semidefinite least-squares problems," SIAM J. Matrix Anal. Appl., vol.26, no.1, pp. 272-284, 2004.
27. R. K. Miller and A. N. Michel, Ordinary Differential Equations. New York: Academic, 1982.
28. M. Ohlasson, C. Pelterson, and B. Soderberg, "Neural networks for optimization problems with inequality constraints: The knapsack problem," Neural Comput., vol.5, no.2, pp. 331-339, 1993.
29. S. A. Ruzinsky and E. T. Olsen, "L1 and Lminimization via a variant of Karmarkar's algorithm," IEEE Trans. Acoust. Speech Signal Process., vol.37, no.2, pp. 245-253, Feb. 1989.
30. M. Shi and M. A. Lukas, "An L1 estimation algorithm with degeneracy and linear constraints," Comput. Statis.& Data Anal., vol.39, no.1, pp. 35-55, 2002.
31. D. W. Tank and J. J. Hopfield, "Simple neural optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit," IEEE Trans. Circuits Syst., vol.33, no.5, pp. 533-541, May 1986.
32. E. J. Teoh, K. C. Tan, H. J. Tang, C. Xiang, and C. Goh, "An asynchronous recurrent linear threshold network approach to solving the traveling salesman problem," Neurocomputing, vol.71, nos. 7-9, pp. 1359-1372, Mar. 2008.
33. Z. Wang, Z. He, and J. Chen, "Robust time delay estimation of bioelectric signals," IEEE Trans. Biomed. Eng., vol.52, no.3, pp. 454- 542, Mar. 2005.
34. Z. Wang and B. S. Peterson, "Constrained least absolute deviation neural networks," IEEE Trans. Neural Netw., vol.19, no.2, pp. 273-283, Feb. 2008.
35. Y. Xia and M. S. Kamel, "Cooperative recurrent neural networks for the constrained L1 estimator," IEEE Trans. Signal Process., vol.55, no.7, pp. 3192-3206, Jul. 2007.
36. Y. Xia, M. S. Kamel A generalized least absolute deviation method for parameter estimation of autoregressive signals
37. W. Li and J. J. Swetits, "The linear l1 estimator and the Huber Mestimator," SIAM J. Optimization, vol.8, no.2, pp. 457-475, 1998.
38. Q. Liu and J.Wang, "A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming," IEEE Trans. Neural Netw., vol.19, no.4, pp. 558-570, Apr. 2008.
39. J. Malick, "A dual approach to semidefinite least-squares problems," SIAM J. Matrix Anal. Appl., vol.26, no.1, pp. 272-284, 2004.
40. R. K. Miller and A. N. Michel, Ordinary Differential Equations. New York: Academic, 1982.
41. M. Ohlasson, C. Pelterson, and B. Soderberg, "Neural networks for optimization problems with inequality constraints: The knapsack problem," Neural Comput., vol.5, no.2, pp. 331-339, 1993.
42. S. A. Ruzinsky and E. T. Olsen, "L1 and Lminimization via a variant of Karmarkar's algorithm," IEEE Trans. Acoust. Speech Signal Process., vol.37, no.2, pp. 245-253, Feb. 1989.
43. M. Shi and M. A. Lukas, "An L1 estimation algorithm with degeneracy and linear constraints," Comput. Statis.& Data Anal., vol.39, no.1, pp. 35-55, 2002.
44. D. W. Tank and J. J. Hopfield, "Simple neural optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit," IEEE Trans. Circuits Syst., vol.33, no.5, pp. 533-541, May 1986.
45. E. J. Teoh, K. C. Tan, H. J. Tang, C. Xiang, and C. Goh, "An asynchronous recurrent linear threshold network approach to solving the traveling salesman problem," Neurocomputing, vol.71, nos. 7-9, pp. 1359-1372, Mar. 2008.
46. Z. Wang, Z. He, and J. Chen, "Robust time delay estimation of bioelectric signals," IEEE Trans. Biomed. Eng., vol.52, no.3, pp. 454- 542, Mar. 2005.
47. Z. Wang and B. S. Peterson, "Constrained least absolute deviation neural networks," IEEE Trans. Neural Netw., vol.19, no.2, pp. 273-283, Feb. 2008.
48. Y. Xia and M. S. Kamel, "Cooperative recurrent neural networks for the constrained L1 estimator," IEEE Trans. Signal Process., vol.55, no.7, pp. 3192-3206, Jul. 2007.
49. Y. Xia and M. S. Kamel, "A generalized least absolute deviation method for parameter estimation of autoregressive signals," IEEE Trans. Neural Netw., vol.19, no.1, pp. 107-118, Jan. 2008.
50. Y. Xia and J. Wang, "A recurrent neural network for nonlinear convex optimization subject to nonlinear inequality constraints," IEEE Trans. Circuits Syst. I, vol.51, no.7, pp. 1385-1394, Jul. 2004.
51. Y. Yang and J. Cao, "Solving quadratic programming problems by delayed projection neural network," IEEE Trans. Neural Netw., vol.17, no.6, pp. 1630-1634, Nov. 2006.
52. Y. Zhang and S. S. Ge, "Design and analysis of a general recurrent neural network model for time-varying matrix inversion," IEEE Trans. Neural Netw., vol.16, no.6, pp. 1477-1490, Nov. 2005.