Ensemble of software defect predictors: An AHP-based evaluation method

International Journal of Information Technology and Decision Making

Volumn 10, Issue 1, 2011, Pages 187-206

  Peng, Yi ^a   Kou, Gang ^a   Wang, Guoxun ^a   Wu, Wenshuai ^a   Shi, Yong ^b,c  

^a UNIVERSITY OF ELECTRONIC SCIENCE AND TECHNOLOGY OF CHINA   (China)

^b UNIVERSITY OF NEBRASKA AT OMAHA   (United States)

^c INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

Author keywords

classification; Ensemble; software defect prediction; the analytic hierarchy process (AHP)

Indexed keywords

EID: 78751522102     PISSN: 02196220     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0219622011004282     Document Type: Article

References (66)

1. NIST Planning Report 02-3, The Economic Impacts of Inadequate Infrastructure for Software Testing (U.S. Department of Commerce's National Institute of Standards & Technology, 2002), .
2. S. Lessmann, B. Baesens, C. Mues and S. Pietsch, Benchmarking classification models for software defect prediction: A proposed framework and novel findings, IEEE Transactions on Software Engineering 34(4) 2008, 485-496.
3. J. C. Munson and T. M. Khoshgoftaar, The detection of fault-prone programs, IEEE Transactions on Software Engineering 18(5) 1992 ,423-433.
4. T. M. Khoshgoftaar, A. S. Pandya and D. L. Lanning, Application of neural networks for predicting faults, Annals of Software Engineering 1(1) (1995) 141-154.
5. T. M. Khoshgoftaar, E. B. Allen, J. P. Hudepohl and S. J. Aud, Application of neural networks to software quality modeling of a very large telecommunications system, IEEE Transactions on Neural Networks 8(4) 1997, 902-909.
6. T. M. Khoshgoftaar, E. B. Allen, W. D. Jones and J. P. Hudepohl, Classification-tree models of software-quality over multiple releases, IEEE Transactions on Reliability 49(1) 2000, 4-11.
7. T. M. Khoshgoftaar, E. B. Allen and J. Deng, Using regression trees to classify faultprone software modules, IEEE Transactions on Reliability 51(4) 2002, 455-462.
8. T. M. Khoshgoftaar and N. Seliya, Analogy-based practical classification rules for software quality estimation, Empirical Software Engineering 8(4) (2003) 325-350.
9. T. Menzies, J. DiStefano, A. Orrego and R. Chapman, Assessing predictors of software defects, in Proceedings of Workshop Predictive Software Models (2004).
10. A. A. Porter and R. W. Selby, Evaluating techniques for generating metric-based classification trees, Journal of Systems and Software 12(3) (1990) 209-218.
11. K. El-Emam, S. Benlarbi, N. Goel and S. N. Rai, Comparing case-based reasoning classifiers for predicting high risk software components, Journal of Systems and Software 55(3) (2001) 301-310.
12. K. Ganesan, T. M. Khoshgoftaar and E. B. Allen, Case-based software quality prediction, International Journal of Software Engineering and Knowledge Engineering 10(2) (2000) 139-152.
13. K. O. Elish and M. O. Elish, Predicting defect-prone software modules using support vector machines, Journal of Systems and Software 81(5) (2008) 649-660.
14. Y. Peng, G. Kou, G. Wang, H. Wang and F. Ko, Empirical evaluation of classifiers for software risk management, International Journal of Information Technology and Decision Making 8(4) (2009) 749-768.
15. Y. Peng, G. Wang and H. Wang, User p based software defect detection algorithms selection using MCDM, Information Sciences (2010), doi: 10.1016/j.ins.2010.04.019.
16. L. Guo, Y. Ma, B. Cukic and H. Singh, Robust prediction of fault-proneness by random forests, in Proceedings of 15th International Symposium on Software Reliability Engineering (2004).
17. N. E. Fenton and M. Neil, A critique of software defect prediction models, IEEE Transactions on Software Engineering 25(5) 1999, 675-689.
18. I. Myrtveit and E. Stensrud, A controlled experiment to assess the benefits of estimating with analogy and regression models, IEEE Transactions on Software Engineering 25(4) 1999,510-525.
19. I. Myrtveit, E. Stensrud and M. Shepperd, Reliability and validity in comparative studies of software prediction models, IEEE Transactions on Software Engineering 31(5) 2005,380-391.
20. M. Shepperd and G. Kadoda, Comparing software prediction techniques using simulation, IEEE Transactions on Software Engineering 27(11) 2001, 1014-1022. (Pubitemid 33106029)
21. T. L. Saaty, The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation (McGraw-Hill, Columbus, OH, 1980).
22. M. Chapman, P. Callis and W. Jackson, Metrics Data Program, NASA IV and V Facility, http://mdp.ivv.nasa.gov/, 2004.
23. T. G. Dietterich, An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization, Machine Learning 40(2) (2000) 139-157.
24. Y. Freund and R. E. Schapire, Experiments with a new boosting algorithm, in Proc. 13th International Conference on Machine Learning (Morgan Kaufmann, San Francisco, 1996), pp. 148-156.
25. T. G. Dietterich, Machine learning research: Four current directions, AI Magazine 18 (1997) 97-136.
26. K. M. Ting and Z. Zheng, A study of AdaBoost with Naïive Bayesian classifiers: Weakness and improvement, Computational Intelligence 19(2) (2003) 186-200.
27. T. Wilson, J. Wiebe and R. Hwa, Recognizing strong and weak opinion clauses, Computational Intelligence 22(2) (2006) 73-99.
28. J. Han and M. Kamber, Data Mining: Concepts and Techniques, 2nd edn. (Morgan Kaufmann, 2006).
29. D. Opitz and R. Maclin, Popular ensemble methods: An empirical study, Journal of Artificial Intelligence Research 11 (1999) 169-198.
30. T. G. Dietterich, Ensemble methods in machine learning, in J. Kittler and F. Roli (eds.) First International Workshop on Multiple Classifier Systems, Lecture Notes in Computer Science, Vol. 1857 (New York, Springer-Verlag, 2000b), pp. 1-15.
31. E. Bauer and R. Kohavi, An empirical comparison of voting classification algorithms: Bagging, boosting, and variants, Machine Learning 36(1/2) (1999) 105-139.
32. L. Breiman, Bagging predictors, Machine Learning 24(2) (1996) 123-140.
33. L. Breiman, Heuristics of instability in model selection, The Annals of Statistics 24(6) (1994) 2350-2383.
34. I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. (Morgan Kaufmann, San Francisco, 2005).
35. R. Schapire, The strength of weak learnability, Machine Learning 5(2) (1990) 197-227.
36. Y. Freund and R. E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences 55 (1997) 119-139.
37. D. H. Wolpert, Stacked generalization, Neural Networks 5 (1992) 241-259.
38. Y. Peng, G. Kou, Y. Shi and Z. Chen, A descriptive framework for the field of data mining and knowledge discovery, International Journal of Information Technology and Decision Making 7(4) (2008) 639-682.
39. L. Hansen and P. Salamon, Neural network ensembles, IEEE Transactions on Pattern Analysis and Machine Intelligence 12 (1990) 993-1001.
40. A. Krogh and J. Vedelsby, Neural network ensembles, cross validation, and active learning, in Tesauro, G., Touretzky, D. and Leen, T. (eds.), Advances in Neural Information Processing Systems, Vol. 7 (Cambridge, MA, MIT Press, 1995), pp. 231-238.
41. L. Breiman, J. H. Friedman, R. A. Olshen and C. J. Stone, Classification and Regression Trees (Wadsworth International Group, Belmont, California, 1984).
42. R. Kohavi, Scaling up the accuracy of Naïive Bayes classifiers: A decision tree hybrid, in E. Simoudis, J. W. Han and U. Fayyad (eds.), Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (Portland, OR. Menlo Park, CA, AAAI Press, 1996), pp. 202-207.
43. J. R. Quinlan, C4.5: Programs for Machine Learning (Morgan Kaufmann, 1993).
44. S. Le Cessie and J. C. Houwelingen, Ridge estimators in logistic regression, Applied Statistics 41(1) (1992) 191-201.
45. C. M. Bishop, Neural Networks for Pattern Recognition (Oxford University Press, 1995).
46. J. C. Platt, Fast training of support vector machines using sequential minimal optimization, in B. Schotolkopf, C. J. C. Burges and A. Smola (eds.), Advances in Kernel Methods-Support Vector Learning (MIT Press, 1998), pp. 185-208.
47. V. N. Vapnik, The Nature of Statistical Learning Theory (Springer, New York, USA, 1995).
48. P. Domingos and M. Pazzani, On the optimality of the simple Bayesian classifier under zero-one loss, Machine Learning 29(203) (1997) 103-130.
49. S. M. Weiss and C. A. Kulikowski, Computer Systems that Learn: Classification and Predication Methods from Statistics, Neural Nets, Machine Learning and Expert Systems (Morgan Kaufmann, 1991).
50. B. V. Dasarathy, Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques (IEEE Computer Society Press, 1991).
51. R. Kohavi, The power of decision tables, in N. Lavrac and S. Wrobel (eds.), Proceedings of the Eighth European Conference on Machine Learning (Springer-Verlag, Iraklion, Crete, Greece, 1995), pp. 174-189.
52. W. W. Cohen, Fast effective rule induction, in Proceedings of the Twelfth International Conference on Machine Learning (Morgan Kaufmann, 1995), pp. 115-123.
53. T. L. Saaty, How to make a decision: The analytic hierarchy process, European Journal of Operational Research 48 (1990) 9-26.
54. T. L. Saaty and M. Sagir, Extending the measurement of tangibles to intangibles, International Journal of Information Technology & Decision Making 8(1) (2009) 7-27.
55. T. L. Saaty, Decision making with the analytic hierarchy process, International Journal of Services Sciences 1(1) (2008) 83-98.
56. T. L. Saaty, A scaling method for priorities in hierarchical structures, Journal of Mathematical Psychology 15(3) (1977) 234-281.
57. F. Zahedi, The analytic hierarchy process-a survey of the method and its applications, Interfaces 16(4) (1986) 96-108.
58. W. Ho, Integrated analytic hierarchy process and its applications - A literature review, European Journal of Operational Research 186(1) 211-228.
59. K. Sugihara and H. Tanaka, Interval evaluations in the analytic hierarchy process by possibility analysis, Computational Intelligence 17(3) (2001) 567-579.
60. H. Li and L. Ma, Ranking decision alternatives by integrated DEA, AHP and gower plot techniques, International Journal of Information Technology & Decision Making 7(2) (2008) 241-258.
61. D. K. Despotis and D. Derpanis, A min-max goal programming approach to priority derivation in AHP with interval judgments, International Journal of Information Technology & Decision Making 7(1) (2008) 175-182.
62. C. Ferri, J. Hernandezorallo and R. Modroiu, An experimental comparison of performance measures for classification, Pattern Recognition Letters (2009) 27-38.
63. C. Mair, G. Kadoda, M. Leflel, L. Phapl, K. Schofield, M. Shepperd and S. Webster, An investigation of machine learning based prediction systems, Journal of Systems Software 53(1) (2000) 23-29.
64. V. U. B. Challagulla, F. B. Bastani, I. Y. Raymond and A. Paul, Empirical assessment of machine learning based software defect prediction techniques, International Journal on Artificial Intelligence Tools 17(2) (2008) 389-400.
65. R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval (Addison Wesley, 1999).
66. L. I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms (Wiley, 2004).