Probabilistic combination of classification rules and its application to medical diagnosis

Machine Learning

Volumn 101, Issue 1-3, 2015, Pages 105-135

  Tomczak, Jakub M ^a   Zięba, Maciej ^a  

^a WROCLAW UNIVERSITY OF TECHNOLOGY   (Poland)

Author keywords

Classification rules; Imbalanced data; Medical diagnosis; Probabilistic combination

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTER AIDED DIAGNOSIS; DECISION TREES; GRAPHIC METHODS; LEARNING ALGORITHMS; LEARNING SYSTEMS; RANDOM VARIABLES;

BENCHMARK DATASETS; CLASS IMBALANCE PROBLEMS; CLASSIFICATION RULES; IMBALANCED DATA; INDEPENDENT RANDOM VARIABLES; MEDICAL DATA SETS; PROBABILISTIC COMBINATION; SUFFICIENT STATISTICS;

DIAGNOSIS;

EID: 84942373924     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-015-5508-x     Document Type: Article

References (75)

1. Alcalá, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., et al. (2010). KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. Journal of Multiple-Valued Logic and Soft Computing, 17, 255–287.
2. Berzal, F., Cubero, J., Sánchez, D., & Serrano, J. (2004). ART: A hybrid classification model. Machine Learning, 54, 67–92.
3. Błaszczyński, J., Słowiński, R., & Szeląg, M. (2011). Sequential covering rule induction algorithm for variable consistency rough set approaches. Information Sciences, 181(5), 987–1002.
4. Blinova, V., Dobrynin, D., Finn, V., Kuznetsov, S. O., & Pankratova, E. (2003). Toxicology analysis by means of the JSM-method. Bioinformatics, 19(10), 1201–1207.
5. Brazdil, P. B., Soares, C., & Da Costa, J. P. (2003). Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results. Machine Learning, 50(3), 251–277.
6. Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123–140.
7. Brown, I., & Mues, C. (2012). An experimental comparison of classification algorithms for imbalanced credit scoring data sets. Expert Systems with Applications, 39(3), 3446–3453.
8. Brzezinska, I., Greco, S., & Slowinski, R. (2007). Mining Pareto-optimal rules with respect to support and confirmation or support and anti-support. Engineering Applications of Artificial Intelligence, 20(5), 587–600.
9. Buntine, W. L. (1992). A theory of learning classification rules. PhD thesis, University of Technology, Sydney.
10. Cawley, G. C. (2006). Leave-one-out cross-validation based model selection criteria for weighted LS-SVMs. In International Joint Conference on Neural Networks (IJCNN’06) (pp. 1661–1668). IEEE
11. Ceglar, A., & Roddick, J. F. (2006). Association mining. ACM Computing Surveys (CSUR), 38(2), 5.
12. Cerf, L., Gay, D., Selmaoui-Folcher, N., Crémilleux, B., & Boulicaut, J. F. (2013). Parameter-free classification in multi-class imbalanced data sets. Data & Knowledge Engineering, 87, 109–129.
13. Cestnik, B. (1990). Estimating probabilities: A crucial task in machine learning. In ECAI (pp. 147–149).
14. Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357.
15. Chen, Z., & Chen, G. (2008). Building an associative classifier based on fuzzy association rules. International Journal of Computational Intelligence Systems, 1(3), 262–273.
16. Cohen, W. (1995). Fast effective rule induction. In Machine learning: Proceedings of the Twelfth International Conference (pp. 1–10).
17. Cooper, G. F., & Herskovits, E. (1992). A Bayesian method for the induction of probabilistic networks from data. Machine learning, 9(4), 309–347.
18. Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273–297.
19. Daemen, A., & De Moor, B. (2009). Development of a kernel function for clinical data. In Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2009) (pp. 5913–5917). IEEE
20. Damianou, A., Ek, C., Titsias, M., & Lawrence, N. (2012). Manifold relevance determination. In ICML (pp. 145–152).
21. Dembczyński, K., Kotłowski, W., & Słowiński, R. (2008). Maximum likelihood rule ensembles. In ICML (pp. 224–231). ACM
22. Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 7, 1–30.
23. Domingos, P. (1997). Bayesian model averaging in rule induction. In Preliminary papers of the sixth international workshop on artificial intelligence and statistics (pp. 157–164).
24. Domingos, P. (2000). Bayesian averaging of classifiers and the overfitting problem. In ICML (pp. 223–230).
25. Domingos, P., & Pazzani, M. (1997). On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning, 29(2), 103–130.
26. Džeroski, S., Cestnik, B., & Petrovski, I. (1993). Using the m-estimate in rule induction. Journal of Computing and Information Technology, 1(1), 37–46.
27. Ek, C. H., Rihan, J., Torr, P. H., Rogez, G., & Lawrence, N. D. (2008). Ambiguity modeling in latent spaces. In Machine learning for multimodal interaction (pp. 62–73). Springer
28. Elkan, C. (2001). The foundations of cost-sensitive learning. In IJCAI (Vol. 17, pp. 973–978).
29. Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874.
30. Fawcett, T., & Provost, F. (1997). Adaptive fraud detection. Data Mining and Knowledge Discovery, 1(3), 291–316.
31. Fayyad, U., & Irani, K. (1993). Multi-interval discretization of continuous-valued attributes for classification learning. In IJCAI (pp. 1022–1029).
32. Frank, A., & Asuncion, A. (2010). UCI machine learning repository. http://archive.ics.uci.edu/ml
33. Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119–139.
34. Fürnkranz, J. (1999). Separate-and-conquer rule learning. Artificial Intelligence Review, 13(1), 3–54.
35. Fürnkranz, J., & Flach, P. A. (2003). An analysis of rule evaluation metrics. In Proceedings of the 20th international conference on machine learning (ICML-03) (pp. 202–209).
36. Galar, M., Fernández, A., Barrenechea, E., Bustince, H., & Herrera, F. (2012). A review on ensembles for the class imbalance problem: Bagging-, boosting-, and hybrid-based approaches. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 42(4), 463–484.
37. Gama, J. (2000). A cost-sensitive iterative Bayes. In ICML.
38. Hall, M. (1999). Correlation-based feature selection for machine learning. PhD thesis, The University of Waikato.
39. He, H., & Garcia, E. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284.
40. Holte, R. (1993). Very simple classification rules perform well on most commonly used datasets. Machine Learning, 11, 63–91.
41. Japkowicz, N. (2001). Concept-learning in the presence of between-class and within-class imbalances. In Advances in artificial intelligence (pp. 67–77). Springer
42. Japkowicz, N., & Stephen, S. (2002). The class imbalance problem: A systematic study. Intelligent Data Analysis, 6(5), 429–449.
43. Jaynes, E. T. (1968). Prior probabilities. IEEE Transactions on Systems Science and Cybernetics, 4(3), 227–241.
44. Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London Series A Mathematical and Physical Sciences, 186(1007), 453–461.
45. Kononenko, I. (1992). Combining decisions of multiple rules. In AIMSA (pp. 87–96).
46. Kononenko, I. (2001). Machine learning for medical diagnosis: History, state of the art and perspective. Artificial Intelligence in Medicine, 23(1), 89–109.
47. Kotsiantis, S. (2007). Supervised machine learning: A review of classification techniques. Informatica, 31, 249–268.
48. Kubat, M., & Matwin, S. (1997). Addressing the curse of imbalanced training sets: One-sided selection. In ICML (pp. 179–186).
49. Kubat, M., Holte, R., & Matwin, S. (1997). Learning when negative examples abound. In ECML (pp. 146–153).
50. Lavrač, N. (1999). Selected techniques for data mining in medicine. Artificial Intelligence in Medicine, 16(1), 3–23.
51. Liu, B., Hsu, W., & Ma, Y. (1998). Integrating classification and association rule mining. In 4th International conference on knowledge discovery and data mining (KDD98) (pp. 80–86).
52. Mac Namee, B., Cunningham, P., Byrne, S., & Corrigan, O. (2002). The problem of bias in training data in regression problems in medical decision support. Artificial Intelligence in Medicine, 24(1), 51–70.
53. Mansoori, E., Zolghadri, M., & Katebi, S. (2008). SGERD: A steady-state genetic algorithm for extracting fuzzy classification rules from data. IEEE Transactions on Fuzzy Systems, 16(4), 1061–1071.
54. Masnadi-Shirazi, H., & Vasconcelos, N. (2010). Risk minimization, probability elicitation, and cost-sensitive SVMs. In ICML (pp. 204–213).
55. Michalski, R. S., Mozetic, I., Hong, J., & Lavrac, N. (1986). The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. In Proc AAAI (pp. 1–41).
56. Minka, T. P. (2000). Bayesian model averaging is not model combination (pp. 1–2). http://research.microsoft.com/en-us/um/people/minka/papers/minka-bma-isnt-mc
57. Mitchell, T. (1997). Machine learning. Boston: McGraw Hill.
58. Pawlak, Z., Grzymala-Busse, J., Slowinski, R., & Ziarko, W. (1995). Rough sets. Communications of the ACM, 38(11), 88–95.
59. Pearson, R., Goney, G., & Shwaber, J. (2003). Imbalanced clustering for microarray time-series. In ICML.
60. Quinlan, J. R. (1993). C4.5: Programs for machine learning. Burlington: Morgan Kaufmann.
61. Rückert, U., & Kramer, S. (2008). Margin-based first-order rule learning. Machine Learning, 70(2–3), 189–206.
62. Shon, A., Grochow, K., Hertzmann, A., & Rao, R. P. (2005). Learning shared latent structure for image synthesis and robotic imitation. In Advances in neural information processing systems (pp. 1233–1240).
63. Stefanowski, J. (1998). On rough set based approaches to induction of decision rules. Rough Sets in Knowledge Discovery, 1(1), 500–529.
64. Stefanowski, J., & Wilk, S. (2006). Rough sets for handling imbalanced data: Combining filtering and rule-based classifiers. Fundamenta Informaticae, 72(1), 379–391.
65. Štrumbelj, E., Bosnić, Z., Kononenko, I., Zakotnik, B., & Grašič Kuhar, C. (2010). Explanation and reliability of prediction models: The case of breast cancer recurrence. Knowledge and Information Systems, 24(2), 305–324.
66. Tenenbaum, J., & Griffiths, T. (2001). Generalization, similarity, and Bayesian inference. Behavioral and Brain Sciences, 24(4), 629–640.
67. Tomczak, J., & Gonczarek, A. (2013). Decision rules extraction from data stream in the presence of changing context for diabetes treatment. Knowledge and Information Systems, 34, 521–546.
68. Vamplew, P., Dazeley, R., Berry, A., Issabekov, R., & Dekker, E. (2011). Empirical evaluation methods for multiobjective reinforcement learning algorithms. Machine Learning, 84(1–2), 51–80.
69. Viswanathan, M., & Webb, G. I. (1998). Classification learning using all rules. In Machine learning: ECML-98 (pp. 149–159).
70. Vorontsov, K., & Ivahnenko, A. (2011). Tight combinatorial generalization bounds for threshold conjunction rules. In Pattern recognition and machine intelligence (pp. 66–73). Springer
71. Vreeken, J., Van Leeuwen, M., & Siebes, A. (2011). Krimp: Mining itemsets that compress. Data Mining and Knowledge Discovery, 23(1), 169–214.
72. Wang, B., & Japkowicz, N. (2010). Boosting support vector machines for imbalanced data sets. Knowledge and Information Systems, 25(1), 1–20.
73. Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5(2), 241–259.
74. Zadrozny, B., & Elkan, C. (2001). Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers. In ICML (pp. 609–616).
75. Zięba, M., Tomczak, J. M., Lubicz, M., & Świątek, J. (2014). Boosted SVM for extracting rules from imbalance data in application to prediction of the post-operative life expectancy in the lung cancer patients. Applied Soft Computing, 14, 99–108.