Neural network models for combinatorial optimization: A survey of deterministic, stochastic and chaotic approaches

Control and Cybernetics

Volumn 31, Issue 2, 2002, Pages 183-216

  Smith, K A ^a   Potvin, J Y ^b   Kwok, T ^a  

^a MONASH UNIVERSITY   (Australia)

^b UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

Chaos; Combinatorial optimization; Deformable templates; Hopfield networks; Neural networks; Self organizing maps

Indexed keywords

GRADIENT METHODS; HEURISTIC METHODS; PROBLEM SOLVING; STOCHASTIC CONTROL SYSTEMS;

COMBINATORIAL OPTIMIZATION PROBLEMS (COP);

NEURAL NETWORKS;

EID: 0036909592     PISSN: 03248569     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (87)

1. Aarts, E.H.L. and Korst, J. (1989) Simulated Annealing and Boltzmann Machines. John Wiley & Sons, Essex.
2. Aihara, K., Takabe, T. and Toyoda, M. (1990) Chaotic neural networks. Physics Letters A, 144, 333-340.
3. Aihara, K. (1994) Chaos in neural response and dynamical neural network models: towards a new generation of analog computing. In: M. Yamaguti, Towards the Harnessing of Chaos. Elsevier Science Publishers B.V., Amsterdam, 83-98.
4. Aiyer, S.V.B., Niranjan, M. and Fallside, F. (1990) A theoretical investigation into the performance of the Hopfield model. IEEE Transactions on Neural Networks, 1, 204-215.
5. Akiyama, Y., Yamashita, A., Kajiura, M. and Aiso, H. (1989) Combinatorial Optimization with Gaussian Machines. Proceedings IEEE International Joint Conference on Neural Networks, 1, 533-540.
6. Amartur, S.C., Piraino, D. and Takefuji, Y. (1992) Optimization Neural Networks for the Segmentation of Magnetic Resonance Images. IEEE Transactions on Medical Imaging, 11, 215-220.
7. Angeniol, B., Vaubois, G. and Le Texier, J.Y. (1988) Self-Organizing Feature Maps and the Travelling Salesman Problem. Neural Networks, 1, 289-293.
8. Asai, H., Onodera, K., Kamio, T. and Ninomiya, H. (1995) A study of Hopfield neural networks with external noises. Proceedings of the IEEE International Conference on Neural Networks, 4, 1584-1589.
9. Barr, R.S., Golden, B.L., Kelly, J.P., Resende, M.G.C. and Stewart, W.R. (1995) Designing and Reporting on Computational Experiments with Heuristic Methods. Journal of Heuristics, 1, 9-32.
10. Boeres, M.S.C., De Carvalho, L.A.V. and Barbosa, V.C. (1992) A Faster Elastic Net Algorithm for the Traveling Salesman Problem. Proceedings of the Int. Joint Conf. on Neural Networks, 2, 215-220.
11. Brandt, R.D., Wang, Y., Laub, A.J. and Mitra, S.K. (1988) Alternative networks for Solving the Travelling Salesman Problem and the List-Matching Problem. Proceedings International Conference on Neural Networks, 2, 333-340.
12. Burke, L.I. (1994) Adaptive Neural Networks for the Traveling Salesman Problem: Insights from Operations Research. Neural Networks, 7, 681-690.
13. Burke, L.I. and Damany, P. (1992) The Guilty Net for the Traveling Salesman Problem. Computers & Operations Research, 19, 255-265.
14. Burke, L.I. and Ignizio, J.P. (1992) Neural Networks and Operations Research: An Overview. Computers & Operations Research, 19, 179-189.
15. Chen, L. and Aihara, K. (1995) Chaotic Simulated Annealing by a Neural Network Model with Transient Chaos. Neural Networks, 8, 915-930.
16. Desieno, D. (1988) Adding a Conscience Mechanism to Competitive Learning. Proceedings of the IEEE Int. Conf. on Neural Networks, 1, 117-124.
17. Durbin, R. and Willshaw, D.J. (1987) An Analogue Approach to the Traveling Salesman Problem using an Elastic Net Method. Nature, 326, 689-691.
18. Favata, F. and Walker, R. (1991) A Study of the Application of Kohonen-type Neural Networks to the Traveling Salesman Problem. Biological Cybernetics, 64, 463-468.
19. Foo, Y.P.S. and Szu, H. (1989) Solving large-scale optimization problems by divide-and-conquer neural networks. Proceedings IEEE International Joint Conference on Neural Networks, 1, 507-511.
20. Garey, M.R. and Johnson, D.S. (1979) Computers and Intractability. W.H. Freeman, New York.
21. Gee, A.H. (1993) Problem Solving with Optimization Networks. Ph.D. dissertation, Queen's College, Cambridge University, U.K.
22. Ghaziri, H. (1991) Solving Routing Problems by a Self-Organizing Map. In: T. Kohonen, K. Makisara, O. Simula, J. Kangas, eds., Artificial Neural Networks. North-Holland, Amsterdam, 829-834.
23. Ghaziri, H. (1996) Supervision in the Self-Organizing Feature Map: Application to the Vehicle Routing Problem. In: I.H. Osman and J.P. Kelly, eds., Meta-Heuristics: Theory & Applications. Kluwer, Boston, 651-660.
24. Goldstein, M. (1990) Self-Organizing Feature Maps for the Multiple Traveling Salesmen Problem. Proceedings of the International Neural Network Conference, 1, 258-261.
25. Guerrero, F., Smith, K.A. and Lozano, S. (1999) Self-organizing neural approach for solving the quadratic assignment problem. Proc. Int. Workshop on Soft Computing in Industry, 13-18.
26. Guerrero, F., Lozano, S., Smith, K.A. and Eguia, I. (2000) Facility Location using Neural Networks. In: Suzuki, Y. Ovaska, T. Furuhashi, R. Roy, and Y. Dote, eds., Soft Computing in Industrial Applications. Springer-Verlag, London, 171-179.
27. Guerrero, F., Lozano, S., Canca, D. amd Smith, K.A. (1998) Machine Grouping in Cellular Manufacturing: A Self-Organizing Neural Network. Proceedings 4th Int. Conf. on Engineering Applications of Neural Networks. Systems Engineering Association, Turku, Finland, 374-377.
28. Guerrero, F., Lozano, S., Smith, K.A. and Kwok, T. (1999) Manufacturing cell formation using a new self-organizing neural network. Proc. Int. Conf. on Computers and Industrial Engineering, 1, 668-672.
29. Hasegawa, M., Ikeguchi, T. and Aihara, K. (1997a) Combination of chaotic neurodynamics with the 2-opt algorithm to solve traveling salesman problems. Physical Review Letters, 79, 2344-2347.
30. Hasegawa, M., Ikeguchi, T., Matozaki, T. and Aihara, K. (1997b) An analysis on additive effects of nonlinear dynamics for combinatorial optimization. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, E80-A, 205-213.
31. Hasegawa, M., Ikeguchi, T. and Aihara, K. (1998) Solving quadratic assignment problems by chaotic neurodynamics. Proceedings of the 1997 International Conference on Neural Information Processing and Intelligent Information Systems, 1, 182-185.
32. Hayakawa, Y., Marumoto, A. and Sawada, Y. (1995) Effects of the chaotic noise on the performance of neural network model for optimization problems. Physical Review E, 51, R2693-R2696.
33. Hegde, S., Sweet, J. and Levy, W. (1988) Determination of Parameters in a Hopfield/Tank Computational Network. Proceedings IEEE International Conference on Neural Networks, 2, 291-298.
34. Hinton, G.E., Sejnowshi, T.J. and Ackley, D.H. (1984) Boltzmann machines: constraint satisfaction networks that learn. Carnegie Mellon University Technical Report CMU-CS-84-119.
35. Hopfield, J.J. (1982) Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proceedings of the National Academy of Sciences, 79, 2554-2558.
36. Hopfield, J.J. (1984) Neurons with Graded Response have Collective Computational Properties like those of Two-State Neurons. Proceedings of the National Academy of Sciences, 81, 3088-3092.
37. Hopfield, J.J. amd Tank, D.W. (1985) Neural Computation of Decisions in Optimization Problems. Biological Cybernetics, 52, 141-152.
38. Hooker, J.N. (1995) Testing Heuristics: We Have it All Wrong. Journal of Heuristics, 1, 33-42.
39. Johnson, D.S. (1990) Local Optimization and the Traveling Salesman Problem. Volume 443 of Lecture Notes in Computer Science. "Automata, Languages and Programming". Springer-Verlag, Berlin, 446-461.
40. Kamgar-Parsi, B. and Kamgar-Parsi, B. (1992) Dynamical Stability and Parameter Selection in Neural Optimization. Proceedings IEEE International Conference on Neural Networks, 4, 566-571.
41. Kirkpatrick S., Gelatt, C.D. and Vecchi, M.P. (1983) Optimization by Simulated Annealing. Science, 220, 671-680.
42. Kohonen, T. (1982) Self-Organized Formation of Topologically Correct Feature Maps. Biological Cybernetics, 43, 59-69.
43. Kohonen, T. (1988) Self-Organization and Associative Memory. Springer, Berlin.
44. Kwok, T., Smith, K. and Wang, L. (1998a) Solving combinatorial optimization problems by chaotic neural networks. In: C. Dagli et al., eds., Intelligent Engineering Systems Through Artificial Neural Networks: Neural Networks, Fuzzy Logic, Evolutionary Programming, Data Mining, and Rough Sets, 8, 317-322, New York: ASME Press.
45. Kwok, T., Smith, K. and Wang, L. (1998b) Incorporating chaos into the Hopfield neural network for combinatorial optimization. World Multiconference on Systemics, Cybernetics and Informatics, 1, 659-665.
46. Kwok, T. and Smith, K. (1999a) A unified framework for chaotic neural network approaches to combinatorial optimization. IEEE Transactions on Neural Networks, 10, 978-981.
47. Kwok, T. and Smith, K.A. (2001) A noisy self-organizing neural network with bifurcation dynamics for combinatorial optimization. IEEE Transactions on Neural Networks (submitted).
48. Lai, W.K. and Coghill, G.G. (1992) Genetic Breeding of Control Parameters for the Hopfield/Tank Neural Net. Proceedings International Joint Conference on Neural Networks, 4, 618-623.
49. Lin, S. and Kernighan, B.W. (1973) An effective heuristic algorithm for the travelling salesman problem. Operations Research, 21, 498-516.
50. Lo, J.T.-H. (1992) A new approach to global optimization and its applications to neural networks. Proceedings IEEE International Joint Conference on Neural Networks, 4, 600-605.
51. Looi, C.K. (1992) Neural Network Methods in Combinatorial Optimization. Computers & Operations Research, 19, 191-208.
52. Lozano, S., Guerrero, F., Eguia, I. and Smith, K.A. (1998) Cell Formation using Two Neural Networks in Series. In: C. Dagli et al., eds., Intelligent Engineering Systems Through Artificial Neural Networks, 8, ASME Press, 341-346.
53. Matsuyama, Y. (1991) Self-Organization via Competition, Cooperation and Categorization Applied to Extended Vehicle Routing Problems. Proceedings of the Int. Joint Conf. on Neural Networks, 1, 385-390.
54. McCulloch, W.S. and Pitts, W. (1943) A Logical Calculus of Ideas Immanent in Nervous Activity. Bulletin of Mathematical Biophysics, 5, 115-133.
55. Minsky, M. and Papert, S. (1969) Perceptrons, MIT Press, Cambridge, MA.
56. Nemhauser, G.L. and Wolsey, L.A. (1988) Integer and Combinatorial Optimization, John Wiley & Sons, Canada.
57. Nozawa, H. (1992) A neural network model as a globally coupled map and applications based on chaos. Chaos, 2, 377-386.
58. Peterson, C. and Soderberg, B. (1989) A New Method for Mapping Optimization Problems onto Neural Networks. International Journal of Neural Systems, 1, 3-22.
59. Peterson, C. and Soderberg, B. (1993) Artificial Neural Networks. In: C.R. Reeves, ed., Modern Heuristic Techniques for Combinatorial Optimization, Chapter 5, Blackwell Scientific Publishers, Oxford, UK.
60. Potvin, J.Y. (1993) The Traveling Salesman Problem: A Neural Network Perspective. ORSA Journal on Computing, 5, 328-348.
61. Potvin, J.Y. and Robillard C. (1995) Clustering for Vehicle Routing with a Competitive Neural Network Model. Neurocomputing, 8, 125-139.
62. Rosenblatt, F. (1958) The perceptron: A Probabilistic Model for Information Storage and Organization in the Brain. Psychological Review, 65, 386-408.
63. Rumelhart, D.E. and McClelland, J.L. (1986) Parallel Distributed Processing: Explorations in the Microstructure of Cognition. MIT Press, Cambridge, MA.
64. Simmen, M.W. (1991) Parameter Sensitivity of the Elastic Net Approach to the Traveling Salesman Problem. Neural Computation, 3, 363-374.
65. Smith, K.A. (1995) Solving the Generalized Quadratic Assignment Problem using a Self-Organizing Process. Proceedings of the IEEE Int. Conf. on Neural Networks, 4, 1876-1879.
66. Smith, K.A. (1996) An argument for abandoning the Travelling Salesman Problem as a neural network benchmark. IEEE Trans. on Neural Networks, 7, 1542-1544.
67. Smith, K.A. (1999) Neural Networks for Combinatorial Optimization: A Review of More than a Decade of Research. INFORMS Journal on Computing, 11, 15-34.
68. Smith, K.A., Abramson, D. and Duke, D. (1999) Efficient Timetabling Formulations for Hopfield Neural Networks. In: C. Dagli et al., eds., Smart Engineering System Design: Neural Networks, Fuzzy Logic, Evolutionary Programming, Data Mining, and Complex Systems, 9, ASME Press, 1027-1032.
69. Smith, K.A. and Gupta, J.N.D. (2001) Continuous function optimisation via gradient descent on a neural network approximation function. Volume 2084 of Lecture Notes in Computer Science. "Connectionist Models of Neurons, Learning Processes, and Artificial Intelligence". Springer-Verlag, Berlin, 741-748.
70. Smith, K.A., Palaniswami, M. and Krishnamoorthy, M. (1996) A Hybrid Neural Approach to Combinatorial Optimization. Computers & Operations Research, 23, 597-610.
71. Smith, K., Palaniswami, M. and Krishnamoorthy, M. (1998) Neural Techniques for Combinatorial Optimization with Applications. IEEE Transactions on Neural Networks, 9, 1301-1318.
72. Szu, H. and Hartley, R. (1987) Fast Simulated Annealing. Physics Letters A, 122, 157-162.
73. Szu, H. (1988) Fast TSP algorithm based on binary neuron output and analog input using zero-diagonal interconnect matrix and necessary and sufficient conditions of the permutation matrix. Proceedings IEEE International Conference on Neural Networks, 2, 259-266.
74. Takefuji, Y. and Szu, H. (1989) Design of parallel distributed Cauchy machines. Proceedings IEEE International Joint Conference on Neural Networks, 1, 529-532.
75. Tank, D.W. and Hopfield, J.J. (1986) Simple neural optimization networks: An A/D converter, signal decision circuit and a linear programming circuit. IEEE Transactions on Circuit Systems, 33, 533-541.
76. Tokuda, I., Nagashima, T. and Aihara, K. (1997) Global bifurcation structure of chaotic neural networks and its application to Travelling Salesman Problems. Neural Networks, 10, 1673-1690.
77. Vakhutinsky, A.I. and Golden, B.L. (1994) Solving Vehicle Routing Problems using Elastic Nets. Proceedings of the IEEE Int. Conf. On Neural Networks, 7, 4535-4540.
78. Vakhutinsky, A.I. and Golden, B.L. (1995) A Hierarchical Strategy for Solving Travelling Salesman Problems using Elastic Nets. Journal of Heuristics, 1, 67-76.
79. Van Der Bout, D.E. and Miller, T.K. (1988) A Travelling Salesman Objective Function That Works. Proceedings IEEE International Conference on Neural Networks, 2, 299-303.
80. Van Der Bout, D.E. and Miller, T.K. (1989) Improving the Performance of the Hopfield-Tank Neural Network through Normalization and Annealing. Biological Cybernetics, 62, 129-139.
81. Van Der Bout, D.E. and Miller, T.K. (1990) Graph partitioning using annealed neural networks. IEEE Transactions on Neural Networks, 1, 192-203.
82. Wang, L. and Smith, K.A. (1998) On chaotic simulated annealing. IEEE Transactions on Neural Networks, 9, 716-718.
83. Willshaw, D.J. and Von Der Malsburg, C. (1979) A Marker Induction Mechanism for the Establishment of Ordered Neural Mappings: Its Application to the Retinotectal Problem. Philosophical Transactions of the Royal Society, Series B 287, 203-243.
84. Wilson, G.V. and Pawley, G.S. (1988) On the stability of the TSP algorithm of Hopfield and Tank. Biological Cybernetics, 58, 63-70.
85. Wolfe, W.J. (1999) A fuzzy Hopfield-Tank traveling salesman problem model. INFORMS Journal on Computing, 11, 329-44.
86. Yamada, T., Aihara, K. and Kotani, M. (1993) Chaotic neural networks and the Travelling Salesman Problem. Proceedings of the International Joint Conference on Neural Networks, 2, 1549-1552.
87. Zhou; D. Yasuda, K. and Yokoyama, R. (1997) A method to combine chaos and neural network based on the fixed point theory. Transactions of the Institute of Electrical Engineers of Japan, 117-C, 599-608.