An adaptive multiobjective approach to evolving ART architectures

IEEE Transactions on Neural Networks

Volumn 21, Issue 4, 2010, Pages 529-550

  Kaylani, Assem ^a   Georgiopoulos, Michael ^a   Mollaghasemi, Mansooreh ^b   Anagnostopoulos, Georgios C ^c   Sentelle, Christopher ^a   Zhong, Mingyu ^a  

^a University of Central Florida   (United States)

^b UNIVERSITY OF CENTRAL FLORIDA   (United States)

^c Florida Institute of Technology   (United States)

Author keywords

ARTMAP; Category proliferation; Classification; Genetic algorithms (GAs); Genetic operators; Machine learning

Indexed keywords

ADAPTIVE RESONANCE THEORY; ART NEURAL NETWORKS; ARTMAP; CATEGORY PROLIFERATION; CLASSIFICATION AND REGRESSION TREE; EVOLUTIONARY APPROACH; FUZZY ARTMAP; GAUSSIANS; GENETIC OPERATORS; MACHINE-LEARNING; MULTI OBJECTIVE; MULTIOBJECTIVE APPROACH; NETWORK PARAMETERS; USER INTERVENTION;

ARTS COMPUTING; CLASSIFIERS; LEARNING ALGORITHMS; MATHEMATICAL OPERATORS; MULTIOBJECTIVE OPTIMIZATION; NETWORK ARCHITECTURE; SUPPORT VECTOR MACHINES;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; FUZZY LOGIC; HUMAN; NORMAL DISTRIBUTION;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; FUZZY LOGIC; HUMANS; NEURAL NETWORKS (COMPUTER); NORMAL DISTRIBUTION; PATTERN RECOGNITION, AUTOMATED;

EID: 77950859398     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2009.2037813     Document Type: Article

References (61)

1. S. Grossberg, "Adaptive pattern classification and universal recoding, II: Feedback, expectation, olfaction, and illusions," Biol. Cybern., vol.23, no.4, pp. 187-202, Dec. 1976.
2. G. A. Carpenter, S. Grossberg, N. Markuzon, J. H. Reynolds, and D. B. Rosen, "Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps," IEEE Trans. Neural Netw., vol.3, no.5, pp. 698-713, Sep. 1992.
3. G. Anagnostopoulos, "Novel approaches in adaptive resonance theory for machine learning," Ph.D. dissertation, Dept. Electr. Comput. Eng., Univ. Central Florida, Orlando, FL, 2001.
4. J. R. Williamson, "Gaussian ARTMAP: A neural network for fast incremental learning of noisy multidimensional maps," Neural Netw., vol.9, no.5, pp. 881-897, 1996.
5. A. Al-Daraiseh, A. Kaylani, M. Georgiopoulos, A. S. Wu, M. Mollaghasemi, and G. C. Anagnostopoulos, "GFAM: Evolving fuzzy ARTMAP neural networks," Neural Netw., vol.20, no.8, pp. 873-891, Oct. 2007.
6. A. Kaylani, A. Al-Daraiseh, M. Georgiopoulos, M. Mollaghasemi, G. C. Anagnostopoulos, and A. S. Wu, "Genetic optimization of ART neural network architectures," in Proc. Int. Joint Conf. Neural Netw., Aug. 2007, pp. 1114-1119.
7. D. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, 1989.
8. Y. Jin, T. Okabe, and B. Sendhoff, "Dynamic weighted aggregation for evolutionary multi-objective optimization: Why does it work and how?," in Proc. Genetic Evol. Comput. Conf., L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke, Eds., San Francisco, CA, 2001, pp. 1042-1049.
9. Y. Jin, T. Okabe, and B. Sendhoff, "Neural network regularization and ensembling using multi-objective evolutionary algorithms," in Proc. Congr. Evol. Comput., 2004, DOI: 10.1109/CEC.2004.1330830.
10. H. Ishibuchi, K. Nozaki, N. Yamamoto, and H. Tanaka, "Construction of fuzzy classification systems with rectangular fuzzy rules using genetic algorithms," Fuzzy Sets Syst., vol. 65, no. 2/3, pp. 237-253, 1994.
11. H. Ishibuchi, K. Nozaki, N. Yamamoto, and H. Tanaka, "Selecting fuzzy-if-then rules for classification problems using genetic algorithms," IEEE Trans. Fuzzy Syst., vol.3, no.3, pp. 260-270, Aug. 1995.
12. H. Ishibuchi, T. Murata, and I. B. Turksen, "Single-objective and twoobjective genetic algorithms for selecting linguistic rules for pattern classification problems," Fuzzy Sets Syst., vol.89, no.2, pp. 135-150, 1997.
13. H. Ishibuchi, T. Nakashima, and T. Murata, "Three objective genetics based machine learning for linguistic rule extraction," in Proc. Evol. Multi-Objective Optim., 2001, pp. 588-602.
14. H. Ishibuchi, T. Nakashima, and T. Murata, "Three objective genetics based machine learning for linguistic rule extraction," Inf. Sci., vol.136, no.1-4, pp. 109-133, 2001.
15. H. Ishibuchi and Y. Nojima, "Multi-objective optimization in linguistic rule extraction from numerical data," Int. J. Approx. Reason., vol.44, pp. 4-31, 2007.
16. R. M. Everson and J. E. Fieldsend, "Multi-class ROC analysis from a multi-objective optimization perspective," Pattern Recognit. Lett., vol.27, no.8, pp. 918-927, 2006.
17. Y. Jin, R. Wen, and B. Sendhoff, "Evolutionary multi-objective optimization of spiking neural networks," in Proceedings of the International Conference of Artificial Neural Networks (ICANN), ser. Lecture Notes in Computer Science. Berlin, Germany: Springer-Verlag, 2007, vol.4668, pp. 370-379.
18. L. Graning, Y. Jin, and B. Sendhoff, "Generalization improvement in multi-objective learning," in Proc. Int. Joint Conf. Neural Netw., 2006, pp. 9983-19900
19. Y. Jin and B. Sendhoff, "Pareto-based multiobjective machine learning: An overviewand case studies," IEEE Trans. Syst. Man Cybern. C, Appl. Rev., vol.38, no.3, pp. 397-415, May 2008.
20. G. C. Anagnostopoulos, M. Bharadwaj, M. Georgiopoulos, S. J. Verzi, and G. L. Heileman, "Exemplar-based pattern recognition via semisupervised learning," in Proc. Int. Joint Conf. Neural Netw., Portland, OR, 2003, vol.4, pp. 2782-2787.
21. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees . New York: Chapman & Hall, 1984.
22. G. A. Carpenter and S. Grossberg, "A massively parallel architecture for a self-organizing neural pattern recognition machine," Comput. Vis. Graph. Image Process., vol.37, pp. 54-115, 1987.
23. G. A. Carpenter and S. Grossberg, "ART 2: Self-organization of stable category recognition codes for analog input patterns," Appl. Opt., vol.26, no.23, 1987.
24. G. A. Carpenter, S. Grossberg, and J. H. Reynorlds, "ARTMAP: Supervised real-time learning and classification of non-stationary data by a self-organizing neural network," Neural Netw., vol.6, pp. 565-588, 1991.
25. G. A. Carpenter, S. Grossberg, and D. B. Rosen, "Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system," Neural Netw., vol.4, no.6, pp. 759-771, 1991.
26. S. Marriott and R. F. Harrison, "A modified fuzzy ARTMAP architecture for the approximation of noisy mappings," Neural Netw., vol.8, pp. 619-641, 1995.
27. E. Gomez-Sanchez, Y. Dimitriadis, J. Cano-Izquierdo, and J. Lopez- Coronado, "MicroARTMAP: Use of mutual information for category reduction in Fuzzy ARTMAP," IEEE Trans. Neural Netw., vol.13, no.1, pp. 1045-9227, Jan. 2002.
28. E. Gomez-Sanchez, Y. Dimitriadis, J. Cano-Izquierdo, and J. Lopez-Coronado, "Safe-μ ARTMAP: A new solution for reducing category proliferation in Fuzzy ARTMAP," in Proc. Int. Joint Conf. Neural Netw., Washington, DC, 2001, vol.2, pp. 1197-1202.
29. A. Koufakou, M. Georgiopoulos, G. Anagnostopoulos, and T. Kasparis, "Cross-validation in Fuzzy ARTMAP for large databases," Neural Netw., vol.14, pp. 1279-1291, 2001.
30. G. A. Carpenter, B. L. Milenova, and B. W. Noeske, "Distributed ARTMAP: A neural network for fast distributed supervised learning," Neural Netw., vol.11, no.5, pp. 793-813, 1998.
31. J. R. Williamson, "A constructive, incremental-learning network for mixture modeling and classification," Neural Comput., vol.9, no.7, pp. 1517-1543, 1997.
32. E. Parrado-Hernadez, E. Gomez-Sanchez, and Y. Dimitriadis, "Study of distributed learning as a solution to category proliferation in Fuzzy ARTMAP based neural systems," Neural Netw., vol.16, pp. 1039-1057.
33. K. A. D. Jong, "An analysis of the behavior of a class of genetic adaptive systems," Ph.D. dissertation, Comput. Commun. Sci., Univ. Michigan, Ann Arbor, MI, 1975.
34. H. Muhlenbein and D. Schlierkamp-Voosen, "Predictive models for the breeder genetic algorithm I: Continuous parameter optimization," Evol. Comput., vol.1, no.1, pp. 25-49, 1993.
35. T. Back, "Self-adaptation in genetic algorithms," in Proc. 1st Eur. Conf. Artif. Life , Washington, DC, 1992, pp. 263-271.
36. R. Hinterding, Z. Michalewicz, and A. E. Eiben, "Adaptation in evolutionary computation: A survey," in Proc. IEEE Int. Conf. Evol. Comput./IEEE World Congr. Comput. Intell., I. Alekser and E. J. Taylor, Eds., Amsterdam, The Netherlands, Apr. 1997, pp. 65-69.
37. T. C. Fogarty, "Varying the probability of mutation in the genetic algorithm," in Proc. 3rd Int. Conf. Genetic Algorithms , San Francisco, CA, 1989, pp. 104-109.
38. M. Laumanns, L. Thiele, K. Deb, and E. Zitzler, "Combining convergence and diversity in evolutionary multiobjective optimization," Evol. Comput., vol.10, no.3, pp. 263-282, 2002.
39. P. J. Angeline, G. M. Saunders, and J. P. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Trans. Neural Netw., vol.5, no.1, pp. 54-65, Jan. 1994.
40. N. Srinivas and L. M. Patnaik, "Adaptive probabilities of crossover and mutation in genetic algorithms," IEEE Trans. Syst. Man Cybern., vol.24, no.2, pp. 656-667, Apr. 1994.
41. C. C. Coello, "Evolutionary multi-objective optimization: A historical view of the field," IEEE Comput. Intell. Mag., vol.1, no.1, pp. 28-36, Feb. 2006.
42. A. Kaylani, M. Georgiopoulos, M. Mollaghasemi, and G. Anagnostopoulos, "Genetic optimization of ART neural network architectures," in Proc. Artif. Intell. Soft Comput. Conf., 2007, pp. 225-230.
43. R. Santos, J. Nievola, and A. Freitas, "Extracting comprehensible rules from neural networks via genetic algorithms," in Proc. IEEE Symp. Combinat. Evol. Comput. Neural Netw., 2000, pp. 130-139.
44. I. Das and J. Dennis, "A closer look at drawbacks of minimizing weighted sums of objectives for Pareto set generation in multicriteria optimization problems," Structural Optim., vol.14, pp. 63-69, 1997.
45. J. D. Schaffer, "Multiple objective optimization with vector evaluated genetic algorithms," in Proc. 1st Int. Conf. Genetic Algorithms , Mahwah, NJ, 1985, pp. 93-100.
46. N. Srinivas and K. Deb, "Multiobjective optimization using nondominated sorting in genetic algorithms," Evol. Comput., vol.2, no.3, pp. 221-248, 1994.
47. C. M. Fonseca and P. J. Fleming, "Genetic algorithms for multiobjective optimization: Formulation, discussion and generalization," in Proc. 5th Int. Conf. Genetic Algorithms , San Mateo, CA, 1993, pp. 416-423.
48. J. Horn, N. Nafpliotis, and D. E. Goldberg, "A niched Pareto genetic algorithm for multiobjective optimization," in Proc. 1st IEEE Conf. Evol. Comput./IEEE World Congr. Comput. Intell., Piscataway, NJ, 1994, vol.1, pp. 82-87.
49. M. Erickson, A. Mayer, and J. Horn, "The niched Pareto genetic algorithm 2 applied to the design of groundwater remediation systems," in Proc. 1st Int. Conf. Evol. Multi-Criterion Optim. , London, U.K., 2001, pp. 681-695.
50. H. Ishibuchi and T. Murata, "Multi-objective genetic local search algorithm," in Proc. IEEE Int. Conf. Evol. Comput., May 1996, pp. 119-124.
51. K. Deb, S. Agrawal, A. Pratab, and T. Meyarivan, "A fast elitist nondominated sorting genetic algorithm for multi-objective optimization: NSGA-II," in Proc. Parallel Problem Solving Nature VI Conf., Paris, France, 2000, pp. 849-858.
52. K. Deb, A. Pratab, S. Agrawal, and T. Meyarivan, "A fast and elitist multiobjective genetic algorithm: NSGA-II," IEEE Trans. Evol. Comput., vol.6, no.2, pp. 182-197, Apr. 2002.
53. E. Zitzler and L. Thiele, "Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach," IEEE Trans. Evol. Comput., vol.3, no.4, pp. 257-271, Nov. 1999.
54. E. Zitzler, M. Laumanns, and L. Thiele, "SPEA2: Improving the strength Pareto evolutionary algorithm," Comput. Eng. Netw. Lab. (TIK), Swiss Federal Inst. Technol. (ETH), Zurich, Switzerland, Tech. Rep. 103, 2001.
55. J. E. Fieldsend, R. M. Everson, and S. Singh, "Using unconstrained elite archives for multiobjective optimization," IEEE Trans. Evol. Comput., vol.7, no.3, pp. 305-323, Jun. 2003.
56. G. A. Carpenter and H. A. Tan, "Rule extraction: From neural architecture to symbolic representation," Connection Sci., vol.7, pp. 3-27, 1995.
57. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowl. Disc., vol.2, no.2, pp. 121-167, Jun. 1998.
58. D. J. Newman, S. Hettich, C. L. Blake, and C. J. Merz, UCI Repository of Machine Learning Databases, Univ. California at Irvine, Irvine, CA, 1998 [Online]. Available: http://www.ics.uci.edu/∼mlearn/MLRepository. html
59. C.-C. Chang and C.-J. Lin, LIBSVM: A Library for Support Vector Machines, Nat. Taiwan Univ., Taiwan, 2001 [Online]. Available: http:// www.csie.ntu.edu.tw/cjlin/libsvm
60. J. C. Platt, "Fast training of support vector machines using sequential minimal optimization," in Advances in Kernel Methods: Support Vector Learning, B. Schölkopf, C. J. Burges, and A. J. Smola, Eds. Cambridge, MA: MIT Press, 1999, pp. 185-208.
61. R. Fan, P. Chen, and C. J. Lin, "Working set selection using second order information for training SVM," J. Mach. Learn. Res., vol.6, pp. 1889-1918, 2005