A review of Hopfield neural networks for solving mathematical programming problems

European Journal of Operational Research

Volumn 198, Issue 3, 2009, Pages 675-687

  Wen, Ue Pyng ^a   Lan, Kuen Ming ^a   Shih, Hsu Shih ^b  

^a NATIONAL TSING HUA UNIVERSITY   (Taiwan)

^b TAMKANG UNIVERSITY   (Taiwan)

Author keywords

Energy function; Hopfield neural networks; Lagrange multiplier; Mathematical programming; penalty function; Primal and dual functions

Indexed keywords

APPROXIMATE SOLUTIONS; DISTRIBUTED PROCESS; DUAL METHODS; ELECTRONIC CIRCUITS; ENERGY FUNCTION; MATHEMATICAL PROGRAMMING PROBLEMS; NEURAL-NETWORK; NON-LINEAR; PENALTY FUNCTION; PRIMAL AND DUAL FUNCTIONS; STEADY STATE; VERY LARGE-SCALE INTEGRATIONS;

COERCIVE FORCE; ELECTRON TUBES; FREQUENCY MULTIPLYING CIRCUITS; FUNCTION EVALUATION; FUNCTIONS; HOPFIELD NEURAL NETWORKS; LAGRANGE MULTIPLIERS; MATHEMATICAL PROGRAMMING;

NEURAL NETWORKS;

EID: 64049094655     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ejor.2008.11.002     Document Type: Article

References (95)

1. Abe S. Global convergence and suppression of spurious states of the Hopfield neural networks. IEEE Transactions on Circuits Systems 40 (1993) 246-257
2. Ackley D.H., Hinton G.E., and Sejnowski T.J. A learning algorithm for Boltzmann machines. Cognitive Science 9 (1985) 147-169
3. Aiyer S.V.B., Niranjan M., and Fallside F. A theoretical investigation into the performance of the Hopfield model. IEEE Transactions on Neural Networks 1 (1990) 204-215
4. Aourid M., and Kaminska B. Minimization of the 0-1 linear programming problem under linear constraints by using neural networks: Synthesis and analysis. IEEE Transactions on Circuits and Systems - I: Fundamental Theory and Applications 43 (1996) 421-425
5. Araújo F., Rebeiro B., and Rodrigues L. A neural network for shortest path computation. IEEE Transactions on Neural Network 12 (2001) 1067-1073
6. Basheer I.A., and Hajmeer M. Artificial neural networks: Fundamentals, computing, design, and application. Journal of Microbiological Methods 43 (2000) 3-31
7. Bazaraa M.S., Jarvis J.J., and Sherali H.D. Programming and Network Flows. third ed. (2005), John-Wiley, New York
8. Bazaraa M.S., Sherali H.D., and Shetty C.M. Nonlinear Programming Theory and Algorithms. third ed. (2006), John-Wiley, New York
9. Bouzerdoum A., and Pattison T.R. Neural network for quadratic optimization with bound constraints. IEEE Transactions on Neural Networks 4 (1993) 293-304
10. Bruck J., and San J. A study on neural networks. International Journal of Intelligent Systems 3 (1988) 59-75
11. Burke L.I., and Ignizio J.P. Neural network and operations research: An overview. Computers and Operations Research 19 (1992) 179-189
12. Chen L., and Aihara K. Chaotic simulated annealing by a neural network model with transient chaos. Neural Networks 8 (1995) 915-930
13. Chong E.K.P., Hui S., and Żak S.H. An analysis of a class of neural networks for solving linear programming problems. IEEE Transactions on Automatic Control 44 (1999) 1995-2006
14. Cichocki A., and Unbehauen R. Neural Networks for Optimization and Signal Processing (1993), John Wiley, New York
15. Croes G.A. A method for solving traveling salesman problems. Operations Research 6 (1958) 791-812
16. da Silva I.N., Amaral W.C., and Arruda L.V.R. Neural approach for solving several types of optimization problems. Journal of Optimization Theory and Applications 128 (2006) 563-580
17. Dillon J.D., and O'Malley M.J. A Lagrangian augmented Hopfield network for mixed integer non-linear programming problems. Neurocomputing 42 (2002) 323-330
18. Du K.L., and Swamy M.N.S. Neural Networks in a Softcomputing Framework (2006), Springer, London, UK
19. Durbin R., and Willshaw D. An analogue approach to the traveling salesman problem using an elastic net method. Nature 326 (1987) 689-691
20. Effati S., and Baymain M. A new nonlinear neural network for solving convex nonlinear programming problems. Applied Mathematics and Computation 168 (2005) 1370-1379
21. El-Bouri A., Balakrishnan S., and Popplewell N. A neural network to enhance local search in the permutation flowshop. Computers and Industrial Engineering 49 (2005) 182-196
22. Gee A.H., and Prager R.W. Limitations of neural networks for solving travelings salesman problems. IEEE Transactions on Neural Networks 6 (1995) 280-282
23. Gill P.E., Murray W., and Wright M.H. Practical Optimization (1981), Academic, London
24. Gong D., Gen M., Yamazaki G., and Xu W. Lagrangian ANN for convex programming with linear constraints. Computers and Industrial Engineering 32 (1997) 429-443
25. Hagan M.T., Demuth H.B., and Beale M. Neural Network Design (1996), PWS Publishing Co, Massachusetts
26. Ham F.M., and Kostanic I. Principles of Neurocomputing for Science and Engineering (2001), McGraw-Hill, New York
27. Hasam G.-O., and Nezam M.-A. An efficient simplified neural network for solving linear and quadratic programming problems. Applied Mathematics and Computation 175 (2006) 452-464
28. Haykin S. Neural Networks: A Comprehensive Foundation. second ed. (1999), Prentice-Hall, New Jersey
29. Hopfield J.J. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences 79 (1982) 2554-2558
30. Hopfield J.J. Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Sciences 81 (1984) 3088-3092
31. Hopfield J.J. Artificial neural networks. IEEE Circuits and Devices Magazine September (1988) 3-10
32. Hopfield J.J., and Tank D.W. Neural computation of decisions in optimization problems. Biological Cybernetics 52 (1985) 141-152
33. Kamgar-Parsi B., and Kamgar-Parsi B. On problem solving with Hopfield neural networks. Biological Cybernet 62 (1990) 415-423
34. Kennedy M.P., and Chua L.O. Neural networks for nonlinear programming. IEEE Transportation Circuits System 35 (1988) 554-562
35. Klimasauskas C.C. Neural networks: A new technology for information processing. Data Base 20 (1989) 21-23
36. Kohonen T. Self-organized formation of topologically correct feature maps. Biological Cybernetics 43 (1982) 59-69
37. Kumar, S., 2005. Neural Networks: A Classroom Approach. International Edition, McGraw-Hill, Singapore.
38. Kurita N., and Funahashi K. On the Hopfield neural networks and mean field theory. Neural Networks 9 (1996) 1531-1540
39. Lan K.M., Wen U.P., Shih H.S., and Lee E.S. A hybrid neural network approach to bilevel programming. Applied Mathematics Letters 20 8 (2007) 880-884
40. Li S.Z. Improving convergence and solution quality of Hopfield-type neural networks with augmented Lagrange multipliers. IEEE Transactions on Neural Networks 7 (1996) 1507-1516
41. Lin C.L., Lai C.C., and Huang T.H. A neural network for linear matrix inequality problems. IEEE Transactions on Neural Networks 11 (2000) 1078-1092
42. Looi C.K. Neural network methods in combinatorial optimization. Computers and Operations Research 19 3/4 (1992) 191-208
43. Maa C.Y., and Shanblatt M.A. Linear and quadratic programming neural network analysis. IEEE Transaction on Neural Networks 3 (1992) 380-394
44. Malek A., and Tari A. Primal-dual solution for the linear programming problems using neural networks. Applied Mathematics and Computation 167 (2005) 198-211
45. Marti{dotless}́n-Valdivia M., Ruiz-Sepúlveda A., and Triguero-Ruiz F. Improving local minima of Hopfield networks with augmented Lagrange multipliers for large scale TSPs. Neural Networks 13 (2000) 283-285
46. Méida-Casermeiro E., Galán-Mari{dotless}́n G., and Muñoz-Peréz J. An efficient multivalued Hopfield network for the traveling salesman problem. Neural Processing Letters 14 (2001) 203-216
47. Munehisa T., Kobayashi M., and Yamazaki H. Cooperative updating in the Hopfield model. IEEE Transactions on Neural Networks 12 (2001) 1243-1251
48. Nygard K.E., Jueli P., and Kadaba N. Neural networks for selecting vehicle routing heuristic. ORSA Journal of Computing 2 (1990) 353-364
49. Papageorgiou G., Likas A., and Stafylopatis A. Improved exploration in Hopfield network state-space through parameter perturbation driven by simulated annealing. European Journal of Operations Research 108 (1998) 283-292
50. Pavlik P. Parallel Hopfield machine. Neurocomputing 4 (1992) 89-91
51. Peng M.K., Gupta N.K., and Armitage A.F. An investigation into the improvement of local minima of the Hopfield network. Neural Networks 9 (1996) 1241-1253
52. Qu H., Yi Z., and Tang H. Improving local minima of columnar competitive model for TSPs. IEEE Transactions on Circuit and Systems I 53 (2006) 1353-1362
53. Reinelt G. TSPLIB - A traveling salesman problem library. ORSA Journal on Computing 3 (1991) 376-384
54. Ricanek, K., Lebby, G.L., Haywood, K., 1999. Hopfield like networks for pattern recognition with application to face recognition. In: International Joint Conference on Neural Network, vol. 5, pp. 3265-3269.
55. Rodri{dotless}́guez-Vázquez A., Domi{dotless}́nguez-Castro R., Rueda A., Huertas J.L., and Sánchez-Sinencio E. Switched-capacitor neural networks for linear programming. Electronics Letters 24 (1988) 496-498
56. Rodri{dotless}́guez-Vázquez A., Domi{dotless}́nguez-Castro R., Rueda A., Huertas J.L., and Sánchez-Sinencio E. Nonlinear switched-capacitor 'neural networks' for optimization problems. IEEE Transactions on Circuits Systems 37 (1990) 384-397
57. Sexton R.S., Aldaee B., Dorsey R.E., and Johnson J.D. Global optimization for artificial neural networks: A tabu search application. European Journal of Operational Research 106 (1998) 570-584
58. Sexton R.S., Dorsey R.E., and Johnson J.D. Optimization of neural networks: A comparative analysis of the genetic algorithm and simulated annealing. European Journal of Operational Research 114 (1999) 589-601
59. Sharda R. Neural networks for the MS/OR analyst: An application bibliography. Interfaces 24 (1994) 116-130
60. Shih H.S., Wen U.P., Lee E.S., Lan K.M., and Hsiao H.C. A neural network approach to multiobjective and multilevel programming problems. Computers and Mathematics with Applications 48 (2004) 95-108
61. Shirazi, B., Yih, S., 1989. Critical analysis of applying Hopfield neural net model to optimization problems. In: IEEE International Conference on Systems, Man and Cybernetics, vol. 1, 14-17 November, pp. 210-215.
62. Silva I.N., Amaral W.C., and Arruda L.V. Design and analysis of an efficient neural network model for solving nonlinear optimization problems. International Journal of Systems Science 36 (2005) 833-843
63. Sima, J., Orponen, P., 2001. A Computational Taxonomy and Survey of Neural Network Models. Available from: .
64. Smith K.A. Neural networks for combinatorial optimization: A review on more than a decade of research. INFORMS Journal on Computing 11 (1999) 15-34
65. Smith K.A., and Gupta J.N.D. Neural networks in business: Techniques and applications for the operations research. Computers and Operations Research 27 (2000) 1023-1044
66. Smith K., Palaniswami M., and Krishnamoorthy M. Traditional heuristic versus Hopfield neural network approaches to a car sequencing problem. European Journal of Operational Research 93 (1996) 300-316
67. Smith K., Palaniswami M., and Krishnamoorthy M. Neural techniques for combinatorial optimization with applications. IEEE Transactions on Neural Networks 9 (1998) 1301-1318
68. Takahashi Y. Mathematical improvement of the Hopfield model for TSP feasible solutions by synapse dynamic systems. IEEE Transactions on Systems, Man, and Cybernetics, Part B 28 (1998) 906-919
69. Takahashi Y. A neural network theory for constrained optimization. Neurocomputing 24 (1999) 117-161
70. Talaván P.M., and Yáñez J. Parameter setting of the Hopfield network applied to TSP. Neural Networks 15 (2002) 363-373
71. Tan K.C., Tang H., and Ge S.S. On parameter settings of Hopfield networks applied to traveling salesman problems. IEEE Transactions on Circuits and Systems - I 52 (2005) 994-1002
72. Tank D.W., and Hopfield J.J. Simple 'neural' optimization networks: An A/D converter, signal decision network and a linear programming circuit. IEEE Transactions on Circuits Systems 33 (1986) 533-541
73. Walsh M.P., Flynn M.E., and O'Malley M.J. Augmented Hopfield network for mixed integer programming. IEEE Transactions on Neural Networks 10 (1998) 456-458
74. Wang J. Analogue neural network for solving the assignment problem. Electronics Letters 28 (1992) 1047-1050
75. Wang J. Analysis and design of a recurrent neural network for linear programming. IEEE Transactions on Circuits and Systems - I: Fundamental Theory and Applications 40 (1993) 613-618
76. Wang J. A recurrent neural network for solving the shortest path problem. IEEE Transactions on Circuits and Systems - I: Fundamental Theory and Applications 43 (1996) 482-486
77. Wang J. Primal and dual assignment networks. IEEE Transactions on Neural Networks 8 (1997) 784-790
78. Wang J. Primal and dual neural networks for shortest-path routing. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 28 (1998) 864-869
79. Watta P.B., and Hassoun M.H. A coupled gradient network approach for static and temporal mixed-integer optimization. IEEE Transactions on Neural Networks 7 (1996) 578-593
80. Widrow B., Rumelhart D.E., and Lehr M.A. Neural networks: Applications in industry, business and science. Communication of the ACM 37 (1994) 93-105
81. Wilson G.V., and Pawley G.S. On the stability of the traveling salesman problem algorithm of Hopfield and Tank. Biological Cybernetics 58 (1988) 63-70
82. Wilson R.L., and Sharda R. Neural networks. OR/MS Today 19 (1992) 36-42
83. Wong B.K., Lai V.S., and Lam J. A bibliography of neural networks in business: Techniques and applications research: 1994-1998. Computers and Operations Research 27 (2000) 1045-1076
84. Wu A.I., and Tam P.K.S. A neural network methodology and strategy of quadratic optimization. Neural Computing and Applications 8 (1999) 283-289
85. Wu X.Y., Xia Y.S., Li J., and Chen W.K. A high-performance neural network for solving linear and quadratic programming problems. IEEE Transactions on Neural Network 7 (1996) 643-651
86. Xia Y. A new neural network for solving linear programming and its application. IEEE Transactions on Neural Networks 7 (1996) 525-529
87. Xia Y., and Wang J. Neural network for solving linear programming problem with bound variables. IEEE Transactions on Neural Networks 6 (1995) 515-519
88. Xia Y., and Wang J. A general methodology for designing globally convergent optimization neural networks. IEEE Transactions on Neural Networks 9 (1998) 1331-1334
89. Xia Y., Feng G., and Wang J. A primal-dual neural network for online resolving constrained kinematic redundancy in robot motion control. IEEE Transactions on Systems Man and Cybernetics Part B - Cybernetics 35 (2005) 54-64
90. Yalcinoz T., and Short M.J. Large-scale economic dispatch using an improved Hopfield neural network. IEE Proceedings Generation, Transmission and Distribution 144 (1997) 181-185
91. Zak S.H., Upatising V., and Hui S. Solving linear programming problems with neural networks: A comparative study. IEEE Transactions on Neural Networks 6 (1995) 94-104. http://www.mathworks.com/ The MatheWorks:
92. Zhang X.S. Neural Networks in Optimization (2000), Kluwer Academic, London
93. Zhang S., and Constantinides A.G. Lagrange programming neural networks. IEEE Transactions on Circuits Systems 39 (1992) 441-452
94. Zhang H.C., and Huang S.H. Applications of neural networks in manufacturing: A state-of-the-art survey. International Journal of Production Research 33 (1995) 705-728
95. Zurada, J.M., 1992. Introduction to Artificial Neural Systems. West, New York.