A study of statistical techniques and performance measures for genetics-based machine learning: Accuracy and interpretability

Soft Computing

Volumn 13, Issue 10, 2009, Pages 959-977

  Garcia S ^a   Fernández, Alberto ^b   Luengo, Julian ^b   Herrera, F ^b  

^a UNIVERSITY OF JAÉN   (Spain)

^b UNIVERSITY OF GRANADA   (Spain)

Author keywords

Classification; Cohen's kappa; Genetic algorithms; Genetics based machine learning; Interpretability; Non parametric tests; Statistical tests

Indexed keywords

CLASSIFICATION; COHEN'S KAPPA; GENETICS-BASED MACHINE LEARNING; INTERPRETABILITY; NON-PARAMETRIC TESTS;

BIOLOGY; CHROMOSOMES; CLASSIFIERS; DECODING; EDUCATION; IMAGE RETRIEVAL; LEARNING ALGORITHMS; ROBOT LEARNING; STATISTICS;

STATISTICAL TESTS;

EID: 64549120231     PISSN: 14327643     EISSN: 14337479     Source Type: Journal    
DOI: 10.1007/s00500-008-0392-y     Document Type: Article

References (46)

1. Aguilar-Ruiz JS, Giráldez R, Riquelme JC (2000) Natural encoding for evolutionary supervised learning. IEEE Trans Evol Comput 11(4):466-479
2. Alcalá-Fdez J, Sánchez L, García S, del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM, Fernández JC, Herrera F (2009) KEEL: A software tool to assess evolutionary algorithms to data mining problems. Soft Comput 13(3):307-318
3. Alpaydin E (2004) Introduction to machine learning, vol 452. MIT Press, Cambridge
4. Anglano C, Botta M (2002) NOW G-Net: Learning classification programs on networks of workstations. IEEE Trans Evol Comput 6(13):463-480
5. Asuncion A, Newman DJ (2007) UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA. http://www.ics.uci.edu/~mlearn/MLRepository.htm
6. Bacardit J (2004) Pittsburgh genetic-based machine learning in the data mining era: Representations, generalization and run-time, Dept. Comput. Sci., University Ramon Llull, Barcelona, Spain
7. Bacardit J, Garrell JM (2003) Evolving multiple discretizations with adaptive intervals for a pittsburgh rule-based learning classifier system. In: Proceedings of the genetic and evolutionary computation conference (GECCO'03), vol 2724. LNCS, Germany, pp 1818-1831
8. Bacardit J, Garrell JM (2004) Analysis and improvements of the adaptive discretization intervals knowledge representation. In: Proceedings of the genetic and evolutionary computation conference (GECCO'04), vol 3103. LNCS, Germany, pp 726-738
9. Bacardit J, Garrell JM (2007) Bloat control and generalization pressure using the minimum description length principle for Pittsburgh approach learning classifier system. In: Kovacs T, Llorá X, Takadama K (eds) Advances at the frontier of learning classifier systems, vol 4399. LNCS, USA, pp 61-80
10. Barandela R, Sánchez JS, García V, Rangel E (2003) Strategies for learning in class imbalance problems. Pattern Recognit 36(3):849-851
11. Ben-David A (2007) A lot of randomness is hiding in accuracy. Eng Appl Artif Intell 20:875-885
12. Bernadó-Mansilla E, Garrell JM (2003) Accuracy-based learning classifier systems: Models, analysis and applications to classification tasks. Evol Comput 11(3):209-238
13. Bernadó-Mansilla E, Ho TK (2005) Domain of competence of XCS classifier system in complexity measurement space. IEEE Trans Evol Comput 9(1):82-104
14. Clark P, Niblett T (1989) The CN2 induction algorithm. Machine Learn 3(4):261-283
15. Cohen JA (1960) Coefficient of agreement for nominal scales. Educ Psychol Meas 37-46
16. Corcoran AL, Sen S (1994) Using real-valued genetic algorithms to evolve rule sets for classification. In: Proceedings of the IEEE conference on evolutionary computation, pp 120-124
17. De Jong KA, Spears WM, Gordon DF (1993) Using genetic algorithms for concept learning. Machine Learn 13:161-188
18. Demŝar J (2006) Statistical comparisons of classifiers over multiple data sets. J Machine Learn Res 7:1-30
19. Drummond C, Holte RC (2006) Cost curves: An improved method for visualizing classifier performance. Machine Learn 65(1):95-130
20. Freitas AA (2002) Data mining and knowledge discovery with evolutionary algorithms, vol 264. Springer, Berlin
21. Grefenstette JJ (1993) Genetic algorithms for machine learning, vol 176. Kluwer, Norwell
22. Guan SU, Zhu F (2005) An incremental approach to genetic-algorithms-based classification. IEEE Trans Syst Man Cybern B 35(2):227-239
23. Hekanaho J (1998) An evolutionary approach to concept learning. Dissertation, Department of Computer Science, Abo akademi University, Abo, Finland
24. Hochberg Y (2000) A sharper bonferroni procedure for multiple tests of significance. Biometrika 75:800-803
25. Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65-70
26. Huang J, Ling CX (2005) Using AUC and accuracy in evaluating learning algorithms. IEEE Trans Knowl Data Eng 17(3):299-310
27. Iman RL, Davenport JM (1980) Approximations of the critical region of the Friedman statistic. Commun Stat 18:571-595
28. Jiao L, Liu J, Zhong W (2006) An organizational coevolutionary algorithm for classification. IEEE Trans Evol Comput 10(1):67-80
29. Koch GG (1970) The use of non-parametric methods in the statistical analysis of a complex split plot experiment. Biometrics 26(1):105-128
30. Landgrebe TCW, Duin RPW (2008) Efficient multiclass ROC approximation by decomposition via confusion matrix perturbation analysis. IEEE Trans Pattern Anal Mach Intell 30(5):810-822
31. Lim T-S, Loh W-Y, Shih Y-S (2000) A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms. Machine Learn 40(3):203-228
32. Markatou M, Tian H, Biswas S, Hripcsak G (2005) Analysis of variance of cross-validation estimators of the generalization error. J Machine Learn Res 6:1127-1168
33. Rivest RL (1987) Learning decision lists. Machine Learn 2:229-246
34. Sheskin DJ (2006) Handbook of parametric and nonparametric statistical procedures, vol 1736. Chapman & Hall/CRC, London/West Palm Beach
35. Shaffer JP (1995) Multiple hypothesis testing. Ann Rev Psychol 46:561-584
36. Sigaud O, Wilson SW (2007) Learning classifier systems: A survey. Soft Comput 11:1065-1078
37. Sokolova M, Japkowicz N, Szpakowicz S (2006) Beyond accuracy, F-score and ROC: A family of discriminant measures for performance evaluation. In: Australian conference on artificial intelligence, vol 4304. LNCS, Germany, pp 1015-1021
38. Tan KC, Yu Q, Ang JH (2006) A coevolutionary algorithm for rules discovery in data mining. Int J Syst Sci 37(12):835-864
39. Tulai AF, Oppacher F (2004) Multiple species weighted voting - a genetics-based machine learning system. In: Proceedings of the genetic and evolutionary computation conference (GECCO'03), vol 3103. LNCS, Germany, pp 1263-1274
40. Venturini G (1993) SIA: A supervised inductive algorithm with genetic search for learning attributes based concepts. In: Proceedings of the machine learning ECML'93, vol 667. LNAI, Germany, pp 280-296
41. Wilson SW (1994) ZCS: A zeroth order classifier system. Evol Comput 2:1-18
42. Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149-175
43. Witten IH, Frank E (2005) Data mining: Practical machine learning tools and techniques, 2nd edn, vol 525. Morgan Kaufmann, San Francisco
44. Wright SP (1992) Adjusted p-values for simultaneous inference. Biometrics 48:1005-1013
45. Youden W (1950) Index for rating diagnostic tests. Cancer 3:32-35
46. Zar JH (1999) Biostatistical analysis, vol 929. Prentice Hall, Englewood Cliffs