A meta-heuristic approach for improving the accuracy in some classification algorithms

Computers and Operations Research

Volumn 38, Issue 1, 2011, Pages 174-189

  Pham, Huy Nguyen Anh ^a   Triantaphyllou, Evangelos ^a  

^a Louisiana State University   (United States)

Author keywords

Classification; Convex region; False negative; False positive; Fitting; Generalization; Genetic algorithms; Optimization; Unclassifiable

Indexed keywords

CLASSIFICATION (OF INFORMATION); DATA HANDLING; GENETIC ALGORITHMS; HEURISTIC METHODS; OPTIMIZATION;

CONVEX REGION; FALSE NEGATIVES; FALSE POSITIVE; FITTING; GENERALIZATION; UNCLASSIFIABLE;

HEURISTIC ALGORITHMS;

EID: 77956059614     PISSN: 03050548     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cor.2010.04.011     Document Type: Conference Paper

References (100)

1. O. Pujol, P. Radeva, and J. Vitri Discriminant ECOC: a heuristic method for application dependent design of error correcting output codes IEEE Transactions on Pattern Analysis and Machine Intelligence 28 6 2006 1001 1007
2. T.G. Dietterich, and G. Bakiri Solving multiclass learning problems via error-correcting output codes Journal of Artificial Intelligence Research 2 1995 263 286
3. K. Crammer, and Y. Singer On the learnability and design of output codes for multiclass problems Machine Learning 47 23 2002 201 233
4. V. Vapnik Statistical learning theory 1998 Wiley-Interscience [p. 375567]
5. H. Abdi A Neural Network primer Journal of Biological Systems 2 2003 247 281
6. Hecht-Nielsen R. Theory of the backpropagation neural network. In: Proceedings of the international joint conference on neural networks, Washington, DC, USA, 1989. p. 593605.
7. J.R. Quinlan C4.5: programs for machine learning 1993 Morgan Kaufmann Publisher San Mateo, CA, USA [p. 3542]
8. J.R. Quinlan Simplifying decision trees International Journal of ManMachine Studies 27 1987 221 234
9. J.R. Quinlan Improved use of continuous attributes in C4.5 Artificial Intelligence Research 4 1996 77 90
10. L. Breiman Bagging predictors Journal of Machine Learning 24 1996 123 140
11. H.N.A. Pham, and E. Triantaphyllou Prediction of diabetes by employing a new data mining approach which balances fitting and generalization R. Yin Lee, Studies in computation intelligence vol. 131 2008 Springer Berlin, Germany [p. 1126, chapter 2]
12. H.N.A. Pham, and E. Triantaphyllou An application of a new meta-heuristic for optimizing the classification accuracy when analyzing some medical datasets Expert Systems with Applications 36 5 2009 9240 9249
13. H.N.A. Pham, and E. Triantaphyllou The impact of overfitting and overgeneralization on the classification accuracy in data mining O. Maimon, L. Rokach, Soft computing for knowledge discovery and data mining 2007 Springer New York, NY, USA [p. 391431, part 4, chapter 5]
14. P. Greig-Smith The use of random and contiguous quadrats in the study of the structure of plant communities Annals of Botany 16 1952 293 316
15. S. Cohen, L. Rokach, and O. Maimon Decision-tree instance-space decomposition with grouped gain-ratio Journal of Information Science 177 17 2007 3592 3612
16. Kohavi R. Scaling up the accuracy of naive-Bayes classifiers: a decision-tree hybrid. In: Proceedings of the second international conference on knowledge discovery and data mining, Portland, OR, USA, 1996. p. 2027.
17. K.A. Polat, S. Sahan, H. Kodaz, and S. Gunes Breast cancer and liver disorders classification using artificial immune recognition system (airs) with performance evaluation by fuzzy resource allocation mechanism Expert Systems with Applications 32 2007 172 183
18. L. Rokach, O. Maimon, and O. Arad Improving supervised learning by sample decomposition Journal of Computational Intelligence and Applications 5 1 2005 37 54
19. Z. Zhou, and C. Chen Hybrid decision tree Journal of Knowledge-Based Systems 15 2002 515 528
20. L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone Classification and regression trees 1984 Chapman & Hall/CRC Publisher [p. 27993]
21. L. Breiman Random forests Journal of Machine Learning 45 1 2001 5 32
22. Mansour Y, McAllester D. Generalization bounds for decision trees. In: Proceedings of the 13th annual conference on computer learning theory, San Francisco, CA, USA, 2000. p. 6980.
23. G.I. Webb Further experimental evidence against the utility of Occam's razor Journal of Artificial Intelligence Research 4 1996 397 417
24. P. Clark, and R. Boswell Rule induction with CN2: some recent improvements Y. Kodratoff, Machine learningEWSL-91 1991 Springer Berlin, Germany [p. 15163]
25. P. Clark, and T. Niblett The CN2 algorithm Journal of Machine Learning 3 1989 261 283
26. Cohen WW. Fast effective rule induction. In: Proceedings of the 12th international conference on machine learning, Tahoe City, CA, USA, 1995. p. 11523.
27. N. Mastrogiannis, B. Boutsinas, and I. Giannikos A method for improving the accuracy of data mining classification algorithms Computers and Operations Research 36 10 2009 2829 2839
28. T.M. Cover, and P.E. Hart Nearest neighbor pattern classification Institute of Electrical and Electronics Engineers Transactions on Information Theory 13 1 1967 21 27
29. B.V. Dasarathy, and B.V. Sheela A composite classifier system design: concepts and methodology Proceedings of the IEEE 67 5 1979 708 713
30. S. Dudani The distance-weighted k-nearest-neighbor rule IEEE Transactions on Systems, Man, and Cybernetics 6 4 1976 325 327
31. J.M. Keller, M.R. Gray, and J.A. Givens Jr. A fuzzy K-nearest neighbor algorithm Journal of IEEE Transactions on Systems, Man, and Cybernetics 15 4 1985 580 585
32. P.N. Tan, S. Michael, and K. Vipin Introduction to data mining 2005 Addison-Wesley Publishers [p. 145315, chapters 4 and 5]
33. R.O. Duda, and P.E. Hart Pattern classification and scene analysis 1973 Wiley-Interscience [p. 5664]
34. N. Friedman, D. Geiger, and M. Goldszmidt Bayesian network classifiers Journal of Machine Learning 29 1997 131 161
35. R. Kohavi, and J.H. George Wrappers for feature subset selection Journal of Artificial Intelligence 97 12 1997 273 324 [special issue on relevance]
36. I. Kononenko Semi-nave Bayesian classifier Y. Kodratoff, Proceedings of the sixth European working session on learning 1991 Springer Berlin, Germany 206 219
37. Langley P, Sage S. Induction of selective Bayesian classifiers. In: Proceedings of UAI-94, Seattle, WA, USA, 1994. p. 399406.
38. Pazzani MJ. Searching for dependencies in Bayesian classifiers. In: Proceedings of AI&STAT'95, 1995. p. 23948.
39. S. Geman, E. Bienenstock, and R. Doursat Neural networks and the bias/variance dilemma Journal of Neural Computation 4 1992 1 58
40. J.E. Moody The effective number of parameters: an analysis of generalization and regularization in non-linear learning systems Journal of Advances in Neural Information Processing Systems 4 1992 847 854
41. Weigend A. On overfitting and the effective number of hidden units. In: Proceedings of the 1993 connectionist models summer school, 1993. p. 33542.
42. M. Smith Neural networks for statistical modeling 1996 ITP New Media Publisher 1-850-32842-0 [p. 11729]
43. C. Cortes, and V. Vapnik Support-vector networks Journal of Machine Learning 20 3 1995 273 297
44. N. Cristianini, and S.T. John An introduction to support vector machines and other kernel-based learning methods 2000 Cambridge University Press
45. D. De Vaus Analyzing social science data: 50 key problems in data analysis 1st ed 2002 Sage Publications Ltd.
46. O. Melnik Decision region connectivity analysis: a method for analyzing high-dimensional classifiers Machine Learning 48 2002 321 351
47. N. Tichy An analysis of clique formation and structure in organizations Administrative Science Quarterly 18 2 1973 194 208
48. Karp RM. Reducibility among combinatorial problems. In: Proceedings of the symposium, IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA, 1972. p. 85103.
49. J. Seo, and B. Shneiderman Interactively exploring hierarchical clustering results Computer 35 7 2002 80 86 July
50. G. Karypis, E.H. Han, and V. Kumar CHAMELEON: a hierarchical clustering algorithm using dynamic modeling IEEE Computer 32 8 1999 68 75 August
51. Moore AW. K-means and hierarchical clustering. Online tutorial at the following URL: 〈 http://www.autonlab.org/tutorials/kmeans.html 〉, Carnegie Mellon University, USA, 2010.
52. Ritter J. An efficient bounding sphere. In: Graphics Gems, 1990. p. 3013.
53. D.E. Goldberg Genetic algorithms in search, optimization, and machine learning 1989 Addison-Wesley Publishers
54. K. De Jong Genetic algorithms: a 30 year perspective L. Booker, S. Forrest, M. Mitchell, R. Riolo, Perspectives on adaptation in natural and artificial systems 2005 Oxford University Press
55. Asuncion A, Newman DJ. UCI-machine learning repository. Website 〈 archive.ics.uci.edu/ml/ 〉, University of California, Irvine, School of Information and Computer Sciences, CA, USA, 2010.
56. Tin KH, Eugene MK. Building projectable classifiers of arbitrary complexity. In: Proceedings of the 13th international conference on pattern recognition, Vienna, Austria, August, 1996. p. 8805.
57. Weiss SM, Kapouleas I. An empirical comparison of pattern recognition, neural nets, and machine learning classification methods. In: Proceedings of the 11th international joint conference on artificial intelligence, Detroit, MI, USA, 1989. p. 7817.
58. Artificial Neural Network Toolbox 6.0 and Statistics Toolbox 6.0. Matlab Version 7.0. Website: 〈 www.mathworks.com/products/ 〉.
59. Smith JW, Everhart JE, Dickson WC, Knowler WC, Johannes RS. Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In: Proceedings of the 12th symposium on computer applications and medical care, Los Angeles, CA, USA, 1988. p. 2615.
60. Jankowski N, Kadirkamanathan V. Statistical control of RBF-like networks for classification. In: Proceedings of the seventh international conference on artificial neural networks, ICANN, Lausanne, Switzerland, 1997. p. 38590.
61. Au WH, Chan KCC. Classification with degree of membership: a fuzzy approach. In: Proceedings of the first IEEE international conference on data mining, San Jose, CA, USA, 2001. p. 3542.
62. L. Rutkowski, and K. Cpalka Flexible neuro-fuzzy systems IEEE Transactions on Neural Networks 14 2003 554 574
63. Leon IV WD. Enhancing pattern classification with relational fuzzy neural networks and square BK-products. PhD dissertation, Computer Science, Florida State University, FL, USA, 2006. p. 714.
64. D. Michie, D.J. Spiegelhalter, and C.C. Taylor Machine learning, neural and statistical classification, series artificial intelligence 1994 Prentice Hall Englewood Cliffs, NJ, USA [p. 15760, chapter 9]
65. V. Kecman, and T. Arthanari Comparisons of QP and LP based learning from empirical data L. Monostori, J. V′ancza, M. Ali, LNCS and LNAI 2001 Springer New York, USA 326 332
66. G. Fung, and O.L. Mangasarian Proximal support vector machine classifiers Proceedings of the seventh ACM SIGKDD international conference on knowledge discovery and data mining 2001 ACM Press San Francisco, CA, USA 77 86
67. Domm M, Engel A, Louis PP, Goldberg J. An integer support vector machine. In: Proceedings of the sixth international conference on software engineering, artificial intelligence, networking and parallel/distributed computing, 2005, Towson, MD, USA. p. 1449.
68. Shevked Z, Dakovski L. Learning and classification with prime implicants applied to medical data diagnosis. In: Proceedings of the 2007 international conference on computer systems and technologies, Rousse, Bulgaria, June 2007.
69. Hamilton HJ, Shan N, Cercone N. RIAC: a rule induction algorithm based on approximate classification. Technical report no. CS 96-06, University of Regina, Regina, Canada, 1996.
70. Ster B, Dobnikar A. Neural Networks in medical diagnosis comparison with other methods. In: Proceedings of the international conference on engineering applications of neural networks, EANN'96, London, UK, 1996. p. 42730.
71. Bennet KP, Blue JA. A support vector machine approach to decision trees. Math report, no. 97-100, Rensselaer Polytechnic Institute, Troy, NY, USA, 1997.
72. D. Nauck, and R. Kruse Obtaining interpretable fuzzy classification rules from medical data Artificial Intelligence in Medicine 16 1999 149 169
73. C.A. Pena-Reyes, and M. Sipper A fuzzy-genetic approach to breast cancer diagnosis Artificial Intelligence in Medicine 17 1999 131 155
74. R. Setiono Generating concise and accurate classification rules for breast cancer diagnosis Artificial Intelligence in Medicine 18 2000 205 219
75. J. Abonyi, and H. Szeifert Supervised fuzzy clustering for the identification of fuzzy classifiers Pattern Recognition Letters 24 2003 2195 2207
76. Pham DT, Dimov SS, Salem Z. Technique for selecting examples in inductive learning. In: Proceedings of the European symposium on intelligent techniques, ESIT 2000, Aachen, Germany, 2000. p. 11927.
77. Cheung N. Machine learning techniques for medical analysis. BSc thesis, School of Information Technology and Electrical Engineering, University of Queensland, Australia, 2001.
78. Lee YJ, Mangasarian OL. RSVM: reduced support vector machines. In: Proceedings of the first SIAM international conference on data mining, Chicago, IL, USA, 2001.
79. Y.J. Lee, and O.L. Mangasarian SSVM: a smooth support vector machine for classification Computational Optimization and Applications 20 1 2001 5 22
80. G.T. Van, J.A.K. Suykens, G. Lanckriet, A. Lambrechts, B. De Moor, and J. Vandewalle Bayesian framework for least squares support vector machine classifiers, Gaussian processes and kernel Fisher discriminant analysis Neural Computation 14 2002 1115 1147
81. E. Comaka, K. Polatb, S. Gunesb, and A. Arslana A new medical decision making system: least square support vector machine (LSSVM) with fuzzy weighting pre-processing Expert Systems with Applications 32 2 2007 409 414
82. Heart S. Website: 〈 www.is.umk.pl/projects/datasets-stat.html#Heart 〉, August 2008.
83. S. Ozen, and S. Gunes Attribute weighting via genetic algorithms for attribute weighted artificial immune system (AWAIS) and its application to heart disease and liver disorders problems Expert Systems with Applications 36 1 2009 386 392
84. Sakprasat S, Sinclair MC. Classification rule mining for automatic credit approval using genetic programming. In: Proceedings of the IEEE congress on evolutionary computation, Singapore, 2007. p. 54855.
85. Statlog Australia Credit Approval. Website: 〈 www.is.umk.pl/ projects/datasets-stat.html#Australian 〉, 8/2009.
86. C.C. Hsu, Y.P. Huang, and K.W. Chang Extended naive Bayes classifier for mixed data Expert Systems with Applications 35 2008 1080 1083
87. C.L. Huang, M.C. Chen, and C.J. Wang Credit scoring with a data mining approach based on support vector machines Expert Systems with Applications 33 4 2007 847 856
88. Nakashima T, Nakai G, Ishibuchi H. Constructing fuzzy ensembles for pattern classification problems. In: Proceedings of the international conference on systems, man and cybernetics, Washington, DC, USA, vol. 4, October 2003. p. 32005.
89. M. Blachnik, and W. Duch Prototype-based threshold rules. In: Neural Information Processing, LNCS vol. 4234 2006 Springer Berlin, German [p. 102837]
90. Lei G, Hui-Zhong W, Liang X. A novel classification algorithm based on fuzzy kernel multiple hyperspheres. In: Proceedings of the fourth international conference on fuzzy systems and knowledge discovery, FSKD 2007, Haikou, Hainan, China, vol. 2, 2007. p. 1148.
91. Segata N, Blanzieri E. Empirical assessment of classification accuracy of local SVM. Technical report # DISI-08-014, University of Trento, Italy, March 2008.
92. Gonzalez JA, Holder LB, Cook DJ. Graph-based concept learning. In: Proceedings of the 14th international FAIRS conference, FL, USA, 2001. p. 37781.
93. Eggermont J, Kok JN, Kosters WA. Genetic programming for data classification: partitioning the search space. In: Proceedings of the 2004 symposium on applied computing, 2004. p. 10015.
94. D. Gavrilis, L.G. Tsoulos, and E. Dermatas Selecting and constructing features using grammatical evolution Pattern Recognition Letters 29 2008 1358 1365
95. K. Kianmehr, M. Alshalalfa, and R. Alhajj Effectiveness of fuzzy discretization for class association rule-based classification. In: LNCS vol. 4994 2008 Springer Berlin, Germany [p. 298308]
96. M. Ene Neural network-based approach to discriminate healthy people from those with Parkinson's disease Annals of the University of Craiova, Mathematics and Computer Science Series 35 2008 112 116
97. M.A. Little, P.E. McSharry, E.J. Hunter, J. Spielman, and L.O. Ramig Suitability of dysphonia measurements for telemonitoring of Parkinson's disease IEEE Transactions on Biomedical Engineering 2009
98. L.A. Kurgan, K.J. Cios, R. Tadeusiewicz, M. Ogiela, and L.S. Goodenday Knowledge discovery approach to automated cardiac SPECT diagnosis Artificial Intelligence in Medicine 23 2 2001 149 169
99. Z. Zhanga, Y. Shib, and G. Gao A rough set-based multiple criteria linear programming approach for the medical diagnosis and prognosis Expert Systems with Applications 36 5 2009 8932 8937
100. J.W. Thomas, and J.P. Hofer Accuracy of risk-adjusted mortality rate as a measure of hospital quality of care Medical Care 37 1 1999 83 92