A stochastic connectionist approach for global optimization with application to pattern clustering

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 30, Issue 1, 2000, Pages 10-24

  Babu, G Phanendra ^a,b   Murty, M Narasimha ^c   Keerthi, S Sathiya ^d  

^a IEEE   (United States)

^b Technology Deployment International Inc   (United States)

^c INDIAN INSTITUTE OF SCIENCE   (India)

^d NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Clustering; Connectionist approaches; Function optimization; Global optimization

Indexed keywords

ALGORITHMS; ERROR ANALYSIS; MODAL ANALYSIS; PARALLEL PROCESSING SYSTEMS; PATTERN RECOGNITION; RANDOM PROCESSES; SIMULATED ANNEALING;

GLOBAL OPTIMIZATION; STOCHASTIC CONNECTIONIST APPROACH;

NEURAL NETWORKS;

EID: 0033884869     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/3477.826943     Document Type: Article

References (70)

1. E. Aarts and J. Korst, Simulated Annealing and Boltzmann Machine: A Stochastic Approach to Combinatorial Optimization and Neural Computing. New York: Wiley, 1989.
2. D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, "A learning algorithm for Boltzmann machines," Cogn. Sci., vol. 9, pp. 147-169, 1985.
3. M. R. Anderberg, Cluster Analysis for Applications. New York: Academic, 1973.
4. D. F. Andrews, "Plots of high dimensional data," Biometrics, vol. 28, pp. 125-136, 1972.
5. L. Aristidis and S. L. Andreas, "Group updates and multi scaling: An efficient neural network approach to combinatorial optimization," IEEE Trans. Syst., Man, Cybern. B., vol. 26, pp. 222-232, Apr. 1996.
6. G. P. Babu, "Pattern Clustering with Connectionist and Evolutionary Approaches," Ph.D. dissertation, Comput. Sci. Automat., Ind. Inst. Sci., Bangalore, 1994.
7. G. P. Babu and M. N. Murty, "A near-optimal initial seed value selection in k-means algorithm using a genetic algorithm," Pattern Recognit. Lett., vol. 14, pp. 763-769, 1993.
8. G. P. Babu and M. N. Murty, "Clustering with evolution strategies," Pattern Recognit., vol. 27, no. 2, pp. 321-329, 1994.
9. _, "Simulated annealing for selecting initial seeds in k-means algorithm," Ind. J. Pure Appl. Math., vol. 24, no. 1 and 2, pp. 85-94, 1994.
10. R. E. Bellman and S. E. Dreyfus, Applied Dynamic Programming. Princeton, NJ: Princeton Univ. Press, 1962.
11. J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms. New York: Plenum, 1981.
12. J. C. Bezdek and S. K. Pal, Eds., Fuzzy Models for Pattern Recognition. New York: IEEE Press, 1992.
13. J. N. Bhuyan, V. V. Raghavan, and K. E. Venkatesh, "Genetic algorithm for clustering with an ordered representation," in Proc . 4th Int. Conf. Genetic Algorithms, 1991, pp. 408-415.
14. G. L. Bilbro and W. E. Synder, "Optimization of functions with many minima," IEEE Trans. Syst., Man, Cybern., vol. 21, no. 4, pp. 840-849, 1991.
15. I. O. Bohachevsky, M. E. Johnson, and M. L. Stein, "Generalized simulated annealing for function optimization," Technometrics, vol. 28, no. 3, pp. 209-217, 1986.
16. A. Bouzerdoum and T. R Pattison, "Neural network for quadratic optimization with bound constraints," IEEE Trans. Neural Networks, vol. 4, no. 2, pp. 293-304, 1993.
17. L. Breiman and A. Cutler, "Deterministic algorithm for global optimization," J. Math. Programm., vol. 58, pp. 179-199, 1993.
18. D. E. Brown and C. L. Huntley, "A practical application of simulated annealing to clustering," Pattern Recognit., vol. 25, no. 4, pp. 401-412, 1992.
19. D. F. Chernoff, "The use of faces to represent points in k-dimensional space graphically," J. Amer. Stat. Assoc., vol. 58, no. 342, pp. 361-368, 1973.
20. Y. T. Chien, Interactive Pattern Recognition. New York: Marcel Dekker, 1978.
21. L. O. Chua and G. Lin, "Nonlinear programming without computation," IEEE Trans. Circuits Syst., vol CAS-31, no. 2, pp. 182-188, 1984.
22. G. B. Coleman and H. C. Andrews, "Image segmentation by clustering," in Proc. IEEE, 1967, pp. 773-785.
23. A. Corana, M. Marchesi, C. Martini, and S. Ridella, "Minimizing multimodal functions of continuous variables with the simulated annealing algorithm," ACM Trans. Math. Softw., vol. 13, no. 3, pp. 262-280, 1987.
24. L. C. W. Dixon and G. P. Szego, Eds., Toward Global Optimization. Amsterdam, The Netherlands: North-Holland, 1975.
25. _, Toward Global Optimization 2. Amsterdam, The Netherlands: North-Holland, 1978.
26. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
27. A. Rodriguez-Vazquez et al., "Nonlinear switched-capacitor neural networks for optimization problems," IEEE Trans. Circuits Syst., vol. 37, pp. 384-398, Apr. 1990.
28. W. Feller, An Introduction to Probability Theory and Its Applications 1. New York: Wiley, 1950.
29. D. B. Fogel and P. K. Simpson, "Experiments with evolving fuzzy clusters," in Proc. 2nd Annu. Conf. Evol. Prog., CA, 1993, pp. 90-97.
30. E. W. Forgy, "Cluster analysis of multivariate data: Efficiency versus interpretability of classification," Biometrics, vol. 21, no. 768, 1965.
31. Y. Fukada, "Spatial clustering procedures for region analysis," Pattern Recognit., vol. 12, pp. 395-403, 1980.
32. R. Ge and Y. Qin, "The globally convexized filled functions for global optimization," Appl. Math. Comput., vol. 35, pp. 131-158, 1990.
33. S. Geman and D. Geman, "Stochastic relaxation, Gibbs distribution and Bayesian restoration of images," IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-6, pp. 721-741, 1984.
34. S. Geman and C. Hwang, "Diffusions for global optimization," SIAM J. Contr. Optim., vol. 24, no. 5, pp. 1031-1043, 1986.
35. A. D. Gordon and J. T. Henderson, "Algorithm for Euclidean sum of squares classification," Biometrics, vol. 33, pp. 355-362, 1977.
36. K. C. Gowda and E. Diday, "Symbolic clustering using a new dissimilarity measure," Pattern Recognit., vol. 24, no. 6, pp. 567-578, 1991.
37. R. M. Gray, "Vector quantization," IEEE ASSP Mag., pp. 4-29, 1984.
38. L. O. Hall, I. B. Ozyurt, and J. C. Bezdek, "Clustering with a genetically optimized approach," IEEE Trans. Evol. Comput., vol. 3, no. 2, pp. 103-112, 1999.
39. R. M. Haralick and G. L. Kelly, "Pattern recognition with measurement space and spatial clustering," IEEE Trans. Circuits Syst., vol. CAS-22, pp. 440-450, 1975.
40. J. H. Holland, Adaptation in Natural and Artificial Systems. Ann Arbor, MI: Univ. Michigan Press, 1975.
41. K. Hwang and D. Kim, "Parallel pattern clustering on a multiprocessor with orthogonally shared memory," presented at the Int. Conf. Parallel Processing, 1987.
42. M. A. Ismail and M. S. Kamel, "Multi dimensional data clustering utilizing hybrid search strategies," Pattern Recognit., vol. 22, pp. 75-89, 1989.
43. A. K. Jain and R. C. Dubes, Algorithms for Clustering Data. Englewood Cliffs, NJ: Prentice-Hall, 1988.
44. D. Jones and M. A. Beltramo, "Clustering with Genetic Algorithm," General Motors Research Labs., Warren, MI, GMR-7156, 1990. Research Publication.
45. M. P. Kennedy and L. O. Chua, "Neural networks for nonlinear programming," IEEE Trans. Circuits Syst., vol. 35, pp. 554-562, May 1988.
46. C. D. Kirkpatrick, Gelatt Jr., and M. P. Vecchi, "Optimization by simulated annealing," Science, vol. 220, pp. 671-680, 1983.
47. R. W. Klein and R. C. Dubes, "Experiments in projection and clustering by simulated annealing," Pattern Recognit., vol. 22, no. 2, pp. 213-220, 1989.
48. T. Kohonen, Self-Organization and Associative Memory. New York: Spinger-Verlag, 1984.
49. W. L. G. Koontz, P. M. Narendra, and K. Fukunaga, "A branch and bound clustering algorithm," IEEE Trans. Comput., vol. C-23, pp. 908-914, 1975.
50. X. Li and Z. Fang, "Parallel clustering algorithms," Parallel Comput., vol. 11, pp. 275-290, 1989.
51. L. M. Ni and A. K. Jain, "A VLSI systolic architecture for pattern clustering," IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-7, Jan. 1985.
52. B. K. Parsi, J. A. Gualtieri, J. A. Devaney, and K. K. Parsi, "Clustering with neural networks," Biol. Cybern., vol. 63, pp. 201-208, 1990.
53. V. V. Raghavan and B. Agarwal, "Optimal determination of user-oriented clusters: An application for the reproductive plan," in Genetic Algorithms and Their Applications: Proc. 2nd Int. Conf. Genetic Algorithms, 1987, pp. 241-246.
54. V. V. Raghavan and K. Birchand, "A clustering strategy based on a formalism of the reproductive process in natural system," in Proc. 2nd Int. Conf. Information Storage and Retrieval, 1979, pp. 10-22.
55. S. Ranka and S. Sahni, "Clustering on hypercubes," IEEE Trans. Parallel Distrib. Processing, pp. 128-138, 1991.
56. M. R. Rao, "Cluster analysis and mathematical programming," J. Amer. Stat. Assoc., vol. 66, pp. 622-626, 1971.
57. K. Rose, E. Gurewitz, and G. Fox, "A deterministic annealing approach to clustering," Pattern Recognit. Lett., vol. 11, pp. 589-594, 1990.
58. S. Z. Selim and K. Alsulton, "A simulated annealing algorithm for the clustering problem," Pattern Recognit., vol. 10, no. 24, pp. 1003-1008, 1991.
59. S. Z. Selim and M. A. Ismah, "Soft clustering of multi-dimensional data: A semifuzzy approach," Pattern Recognit., vol. 17, no. 5, pp. 559-568, 1984.
60. H. Spath, Cluster Analysis Algorithms for Data Reduction and Classification. West Sussex, U.K.: Ellis Horwood, 1980.
61. V. Sridhar and M. N. Murty, "Clustering algorithms for library comparison," Pattern Recognit., vol. 24, no. 9, pp. 815-823, 1991.
62. D. W. Tank and J. J. Hopfield, "Simple neural optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit," IEEE Trans. Circuits Syst., vol. CAS-33, pp. 533-541, 1986.
63. Center for development of advanced computing, in C-DAC, 1993. PARAS Tutorial.
64. R. T. Rockafeller, "Lagrange multipliers and optimality," SIAM Rev., pp. 183-238, 1993.
65. D. E. Van den bout and T. K. Miller, "Graph partitioning using annealed neural networks," IEEE Trans. Neural Networks, vol. 1, no. 2, pp. 192-203, 1990.
66. D. Vanderbilt and S. G. Louie, "A Monte-Carlo simulated annealing approach to optimize over continuous variables," J. Comput. Phys., vol. 56, pp. 259-271.
67. H. D. Vinod, "Integer programming and theory of grouping," J. Amer. Stat. Assoc., pp. 506-519, 1969.
68. P. Withold, "Fuzzy clustering with partial supervision," IEEE Trans. Syst., Man, Cybern. B, vol. 27, pp. 787-795, Oct. 1997.
69. E. L. Zapata, F. F. Rivera, O. G. Plata, and M. A. Ismail, "Parallel fuzzy clustering on fixed size hypercube simd computers," Parallel Comput., vol. 11, pp. 291-303, 1989.
70. S. Zhang, X. Zhu, and L. Zou, "Second-order neural nets for constrained optimization," IEEE Trans. Neural Networks, vol. 3, no. 6, pp. 1021-1024, 1992.