Sensitivity versus accuracy in multiclass problems using memetic pareto evolutionary neural networks

IEEE Transactions on Neural Networks

Volumn 21, Issue 5, 2010, Pages 750-770

  Fernández Caballero, Juan Carlos ^a   Martinez, Francisco José ^b   Hervas, César ^a   Gutierrez, Pedro Antonio ^a  

^a UNIVERSITY OF CÓRDOBA   (Spain)

^b UNIVERSIDAD LOYOLA ANDALUCÍA   (Spain)

Author keywords

Accuracy; Local search; Multiclassification; Multiobjective evolutionary algorithms; Neural networks; Sensitivity

Indexed keywords

BENCH-MARK PROBLEMS; BEST MODEL; CLASSIFICATION ,; CLASSIFICATION RATES; EVOLUTIONARY APPROACH; EVOLUTIONARY NEURAL NETWORK; HIGH PRECISION; LOCAL SEARCH; MACHINE LEARNING PROBLEM; MEMETIC; MEMETIC EVOLUTIONARY ALGORITHMS; MULTI OBJECTIVE EVOLUTIONARY ALGORITHMS; MULTI-CLASS PROBLEMS; MULTI-CLASSIFICATION; MULTI-LAYER PERCEPTRON NEURAL NETWORKS; MULTICLASSIFIERS; PARETO FRONT; REAL PROBLEMS; UNIVERSITY OF CALIFORNIA;

LEARNING ALGORITHMS; MULTIOBJECTIVE OPTIMIZATION; NEURAL NETWORKS; PARETO PRINCIPLE;

EVOLUTIONARY ALGORITHMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; COMPUTER NETWORK; COMPUTER SIMULATION; EVOLUTION; GENETICS; HUMAN; MUTATION; RECEIVER OPERATING CHARACTERISTIC; REPRODUCIBILITY; STATISTICS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER COMMUNICATION NETWORKS; COMPUTER SIMULATION; EVOLUTION; HUMANS; MODELS, GENETIC; MUTATION; NEURAL NETWORKS (COMPUTER); REPRODUCIBILITY OF RESULTS; ROC CURVE;

EID: 77951937354     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2041468     Document Type: Article

References (75)

1. H. A. Abbass, "A memetic pareto evolutionary approach to artificial neural networks," in Proc. 14th Australian Joint Conf. Artif. Intell.: Adv. Artif. Intell., London, U.K., Dec. 2001, pp. 1-12.
2. H. A. Abbass, "An evolutionary artificial neural networks approach for breast cancer diagnosis," Artif. Intell. Med., vol.25, pp. 265-281, 2002.
3. H. A. Abbass, "Pareto neuro-evolution: Constructing ensemble of neural networks using multiobjective optimization," in Proc. IEEE Congr. Evol. Comput., Canberra, Australia, Jun. 2003, pp. 2074-2080.
4. H. A. Abbass, "Speeding up backpropagation using multiobjective evolutionary algorithms," Neural Comput., vol.15, pp. 2705-2726, 2003.
5. H. A. Abbass, R. Sarker, and C. Newton, "PDE: A Pareto-frontier differential evolution approach for multi-objective optimization problems," in Proc. Congr. Evol. Comput., Seoul, South Korea, May 2001, pp. 971-978.
6. R. Albuquerque, A. Pádua, R. H. C. Takahashi, and R. R. Saldanha, "Improving generalization of MLPs with multi-objective optimization," Neurocomputing, vol.35, pp. 189-194, 2000.
7. P. J. Angeline, G. M. Saunders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Trans. Neural Netw., vol.5, no.1, pp. 54-65, Jan. 1994.
8. A. Asuncion and D. J. Newman, 2007, UCI Machine Learning Repository, [Online]. Available: http://www.ics.uci.edu/~mlearn/MLRepository. html
9. C. M. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Oxford Univ. Press, 1995.
10. C. M. Bishop, Pattern Recognition and Machine Learning. New York: Springer-Verlag, 2006.
11. A. P. Braga, R. H. C. Takahashi, M. A. Costa, and R. A. Teixeira, "Multi-objective algorithms for neural networks learning," Studies Comput. Intell., vol.16, pp. 151-171, 2006.
12. A. Chandra and X. Yao, "DIVACE: Diverse and accurate ensemble learning algorithm," in Proc. 5th Int. Conf. Intell. Data Eng. Autom. Learn., Exeter, U.K., Aug. 2005, pp. 619-625.
13. N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, "SMOTE: Synthetic minority over-sampling technique," J. Artif. Intell. Res., vol.16, pp. 321-357, 2002.
14. C. A. Coello, G. B. Lamont, and D. A. Van Veldhuizen, Evolutionary Algorithms for Solving Multi-Objective Problems. New York: Springer-Verlag, 2007.
15. C. Cotta and P. Moscato, "A gentle introduction to memetic algorithms," in Handbook on Metaheuristics, F. Glover and G. A. Kochenberger, Eds. Norwell, MA: Kluwer, 2001.
16. K. Deb, Multi-Objective Optimization Using Evolutionary Algorithms. New York: Wiley, 2004.
17. K. Deb, A. Pratab, S. Agarwal, and T. Meyarivan, "A fast and elitist multiobjective genetic algorithm: NSGA2," IEEE Trans. Evol. Comput., vol.6, no.2, pp. 182-197, Apr. 2002.
18. O. J. Dunn, "Multiple comparisons among means," J. Amer. Stat. Assoc., vol.56, pp. 52-56, 1961.
19. R. M. Everson and J. E. Fieldsend, "Multi-class ROC analysis from a multi-objetive optimisation perspective," Pattern Recognit. Lett., vol.27, pp. 918-927, 2006.
20. T. Fawcett, "An introduction to ROC analysis," Pattern Recognit. Lett., vol.27, pp. 861-874, 2006.
21. J. E. Fieldsend and S. Singh, "Pareto evolutionary neural networks," IEEE Trans. Neural Netw., vol.16, no.2, pp. 338-354, Mar. 2005.
22. J. E. Fieldsend and S. Singh, "Pareto multi-objective non-linear regression modelling to aid CAPM analogous forecasting," in Proc. Int. Joint Conf. Neural Netw., Honolulu, HI, May 2002, pp. 388-393.
23. M. Friedman, "A comparison of alternative tests of significance for the problem of m rankings," Ann. Math. Stat., vol.11, no.1, pp. 86-92, 1940.
24. C. Fu, C. Chien-Hsing, L. Chin-Chin, and C. Chun-Jen, "A prototype classification method and its application to handwritten character recognition," in Proc. IEEE Int. Conf. Syst. Man Cybern., Mar. 2004, vol.5, pp. 4738-4743.
25. E. Gelenbe and K. F. Hussain, "Learning in the multiple class random neural network," IEEE Trans. Neural Netw., vol.13, no.6, pp. 1257-1267, Nov. 2002.
26. A. Gepperth and S. Roth, "Applications of multi-objective structure optimization," Neurocomputing, vol.69, pp. 701-703, 2006.
27. T. Van Gestel, B. Baesens, J. Suykens, M. Espinoza, D. E. Baestaens, J. Vanthienen, and B. De Moor, "Bankruptcy prediction with least squares support vector machine classifiers," in Proc. IEEE Int. Conf. Comput. Intell. Financial Eng., Hong Kong, Mar. 2003, pp. 1-8.
28. J. González, I. Rojas, J. Ortega, H. Pomares, F. J. Fernández, and A. F. Días, "Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation," IEEE Trans. Neural Netw., vol.14, no.6, pp. 1478-1495, Nov. 2003.
29. C. Hervás, M. Silva, P. A. Gutiérrez, and A. Serrano, "Multilogistic regression by evolutionary neural network as a classification tool to discriminate highly overlapping signals: Qualitative investigation of volatile organic compounds in polluted waters by using headspacemass spectrometric analysis," Chemometrics Intell. Lab. Syst., vol.92, pp. 179-185, 2008.
30. T. K. Ho and M. Basu, "Complexity measures of supervised classification problems," IEEE Trans. Pattern Anal. Mach. Intell., vol.24, no.3, pp. 289-300, Mar. 2002.
31. D. C. Hoaglin, F. Mosteller, and J. W. Tukey, Understanding Robust and Exploratory Data Analysis. New York: Wiley, 1983.
32. Y. Hochberg and A. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987.
33. C. R. Houck, J. A. Joines, M. G. Kay, and J. R. Wilson, "Empirical investigation of the benefits of partial lamarckianism," Evol. Comput., vol.5, pp. 31-60, 1997.
34. C. Igel and M. Hu&Die;sken, "Empirical evaluation of the improved Rprop learning algorithms," Neurocomputing, vol.50, no.6, pp. 105-123, 2003.
35. C. Igel and M. Hu&Die;sken, "Improving the Rprop learning algorithm," in Proc. 2nd Int. ICSC Symp. Neural Comput., Berlin, Germany, May 2000, pp. 115-121.
36. J. Ilonen, J. K. Kamarainen, and J. Lampinen, "Differential evolution training algorithm for feed-forward neural networks," Neural Process. Lett., vol.17, pp. 93-105, 2003.
37. A. K. Jain, R. P. W. Duin, and M. Jianchang, "Statistical pattern recognition: A review," IEEE Trans. Pattern Anal. Mach. Intell., vol.22, no.1, pp. 4-37, Jan. 2000.
38. Y. Jin, T. Okabe, and B. Sendhoff, "Neural network regularization and ensembling using multi-objective evolutionary algorithms," in Proc. Congr. Evol. Ensembling Using Multi-Objective Evol. Algorithms, Portland, OR, Jan. 2004, pp. 1-8.
39. Y. Jin, T. Okabe, and B. Sendhoff, "Evolutionary multi-objective approach to constructing neural network ensembles for regression," in Applications of Evolutionary Multi-Objective Optimization. Singapore: World Scientific, 2004, pp. 653-672.
40. Y. Jin and B. Sendhoff, "Pareto-based multiobjective machine learning: An overviewand case studies," IEEE Trans. Syst. Man Cybern. C, Appl. Rev., vol.38, no.3, pp. 397-415, Jun. 2008.
41. Y. Jin, B. Sendhoff, and E. Ko&Die;rner, "Evolutionary multi-objective optimization for simultaneous generation of signal-type and simbol-type representations," in Proc. 3rd Int. Conf. Evol. Multi-Criterion Optim., Guanajuato, Mexico, Mar. 2005, pp. 752-766.
42. H. Kaizhu, Y. Haiqin, K. Irwin, and M. R. Lyu, "Maximizing sensitivity in medical diagnosis using biased minimax probability Machine," IEEE Trans. Biomed. Eng., vol.53, no.5, pp. 821-831, May 2006.
43. T. C. W. Langrebe and R. P. W. Duin, "Efficient multiclass ROC approximation by decomposition via confusion matrix perturbation analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol.30, no.5, pp. 810-822, May. 2008.
44. Y. Liu and X. Yao, "Ensemble learning via negative correlation," Neural Netw., vol.12, no.10, pp. 1399-1404, 1999.
45. T. B. Ludermir, A. Yamazaki, and C. Zanchettin, "An optimization methodology for neural network weights and architectures," IEEE Trans. Neural Netw., vol.17, no.6, pp. 1452-1459, Nov. 2006. (Pubitemid 44824259)
46. E. Lughofer, J. E. Smith, M. A. Tahir, P. Caleb-Solly, C. Eitzinger, D. Sannen, and M. Nuttin, "Human-machine interaction issues in quality control based on online image classification," IEEE Trans. Syst. Man Cybern. A, Syst. Humans, vol.39, no.5, pp. 960-971, Sep. 2009.
47. A. C. Martínez-Estudillo, C. Hervás-Martínez, F. J. Martínez-Estudillo, and N. García, "Hybridation of evolutionary algorithms and local search by means of a clustering method," IEEE Trans. Syst. Man Cybern. B, Cybern., vol.36, no.3, pp. 534-546, Jun. 2006.
48. F. J. Martínez-Estudillo, P. A. Gutiérrez, C. Hervás, and J. C. Fernández, "Evolutionary learning by a sensitivity-accuracy approach for multi-class problems," in Proc. IEEE World Congr. Comput. Intell., Hong-Kong, Jun. 2008, pp. 1581-1588.
49. D. Mossman, "Three-way ROCs," Med. Decision Making, vol.19, no.1, pp. 78-89, 1999.
50. G. Ou and Y. L. Murphey, "Multi-class pattern classification using neural networks," Pattern Recognit., vol.40, pp. 4-18, 2007.
51. P. Palmes, T. Hayasaka, and S. Usui, "Mutation-based genetic neural network," IEEE Trans. Neural Netw., vol.16, no.3, pp. 587-600, May 2005.
52. R. Parekh, J. Yang, and V. Honavar, "Constructive neural-network learning algorithms for pattern classification," IEEE Trans. Neural Netw., vol.11, no.2, pp. 436-450, Mar. 2000.
53. L. Prechelt, "A set of neural network benchmark problems and benchmarking rules," Fakulta&Die;t fu&Die;r Informatik, Universita&Die;t Karlsruhe, Karlsruhe, Germany, Tech. Rep. 21/94, 1994.
54. K. V. Price, R. M. Storn, and J. A. Lampinen, Differential Evolution. A Practical Approach to Global Optimization. New York: Springer- Verlag, 2005.
55. F. Provost and T. Fawcett, "Robust classification system for imprecise environments," in Proc. 15th Nat. Conf. Artif. Intell.,Madison, WI, Jul. 1998, pp. 706-713.
56. F. Provost and T. Fawcett, "Analysis and visualization of the classifier performance: Comparison under imprecise class and cost distribution," in Proc. 3rd Int. Conf. Knowl. Disc. Data Mining, Aug. 1997, pp. 43-48.
57. R. Reed, "Pruning algorithms. A survey," IEEE Trans. Neural Netw., vol.4, no.5, pp. 740-747, Sep. 1993.
58. A. J. F. van Rooij, L. C. Jain, and R. P. Johnson, Neural Networks Training Using Genetic Algorithms. Singapore: World Scientific, 1996.
59. S. Roth, A. Gepperth, and C. Igel, "Multi-objective neural network optimization for visual object detection," Multi-Objective Mach. Learn., Studies Comput. Intell., vol.16, pp. 629-655, 2006.
60. E. J. Scheme, B. Hudgins, and P. A. Parker, "Myoelectric signal classification for phoneme-based speech recognition," IEEE Trans. Biomed. Eng., vol.54, no.4, pp. 694-699, Apr. 2007. (Pubitemid 46469941)
61. R. Setiono, W. K. Leow, and J. M. Zurada, "Extraction of rules from artificial neural networks for nonlinear regression," IEEE Trans. Neural Netw., vol.13, no.3, pp. 564-577, May 2002.
62. J. E. Smith, "Coevolving memetic algorithms: A review and progress report," IEEE Trans. Syst. Man Cybern. B, Cybern., vol.37, no.1, pp. 6-17, Feb. 2007.
63. R. Storn and K. Price, "Differential evolution. A fast and efficient heuristic for global optimization over continuous spaces," J. Global Optim., vol.11, pp. 341-359, 1997.
64. Y. Sun, M. S. Kamel, A. K. C. Wong, and Y. Wang, "Cost-sensitive boosting for classification of imbalanced data," Pattern Recognit., vol.40, pp. 3358-3378, 2007.
65. X. Tian and F. Deng, "An improved multi-class SVM algorithm and its application to the credit scoring model," in Proc. 5th World Congr. Intell. Control Autom., Hangzhou, China, Jun. 2004, pp. 1940-1944.
66. K. Tumer and J. Ghosh, "Analysis of decision boundaries in linearly combined neural classifiers," Pattern Recognit., vol.29, no.2, pp. 341-348, 1996.
67. S. Ventura, C. Romero, A. Zafra, J. A. Delgado, and C. Hervás, "JCLEC: A Java framework for evolutionary computation," Soft Comput.-A Fusion Found. Methodol. Appl., vol.12, pp. 381-392, 2008.
68. D. L. Whitley, V. S. Gordon, and K. E. Mathias, "Lamarckian evolution, the baldwing effect and function optimization," in Proc. Int. Conf. Evol. Comput./3rd Conf. Parallel Probl. Solving From Nature, Jerusalem, Israel, Oct. 1994, pp. 6-15.
69. W. Yan, Z. Zhu, and R. Hu, "Hybrid genetic/BP algorithm and its application for radar target classification," in Proc. IEEE Nat. Aerosp. Electron. Conf., Dayton, OH, Jul. 1997, pp. 981-984.
70. X. Yao, "Evolving artificial neural networks," Proc. IEEE, vol.87, no.9, pp. 1423-1447, Sep. 1999.
71. X. Yao, "Evolutionary artificial neural networks," Int. J. Intell. Syst., vol.4, no.3, pp. 203-222, 1993.
72. X. Yao and Y. Liu, "A new evolutionary system for evolving artificial neural networks," IEEE Trans. Neural Netw., vol.8, no.3, pp. 694-713, May 1997.
73. X. Yao and Y. Liu, "Making use of population information in evolutionary artificial neural networks," IEEE Trans. Syst. Man Cybern. B, Cybern., vol.28, no.3, pp. 417-425, Jun. 1998.
74. Y. J. Yau, J. Teo, and P. Anthony, "Pareto evolution and co-evolution in cognitive game AI synthesis," in Proc. 4th Int. Conf. Evol. Multi- Criterion Optim., Matsushima, Japan, Mar. 2007, pp. 227-241.
75. G. P. Zhang, "Neural networks for classification: A survey," IEEE Trans. Syst. Man Cybern. C, Appl. Rev., vol.30, no.4, pp. 451-462, Nov. 2000.