Techniques for evaluating fault prediction models

Empirical Software Engineering

Volumn 13, Issue 5, 2008, Pages 561-595

  Jiang, Yue ^a   Cukic, Bojan ^a   Ma, Yan ^b  

^a West Virginia University   (United States)

^b WEST VIRGINIA UNIVERSITY   (United States)

Author keywords

Empirical studies; Fault prediction models; Model evaluation; Predictive models in software engineering

Indexed keywords

ENGINEERING RESEARCH; FORECASTING; MATHEMATICAL MODELS; MODELS; NONLINEAR CONTROL SYSTEMS; RISK ASSESSMENT; SOFTWARE ENGINEERING; STATISTICAL METHODS; SYSTEMS ENGINEERING; TECHNOLOGY; VERIFICATION;

COST CURVES; DEVELOPMENT COSTS; DEVELOPMENT ENVIRONMENTS; EMPIRICAL SOFTWARE ENGINEERING; EMPIRICAL STUDIES; FAULT PREDICTION; FAULT-PREDICTION MODELS; FAULT-PRONE; MODEL EVALUATION; PERFORMANCE ASSESSMENT; PERFORMANCE EVALUATION; PERFORMANCE MEASURES; PREDICTIVE MODELS IN SOFTWARE ENGINEERING; PROGRAM MODULES; PROJECT COSTS; SOFTWARE MODULES; STATISTICAL TECHNIQUES; SYSTEM FAILURES; TO MANY; VERIFICATION ACTIVITIES;

COMPUTER SOFTWARE SELECTION AND EVALUATION;

EID: 52549091028     PISSN: 13823256     EISSN: 15737616     Source Type: Journal    
DOI: 10.1007/s10664-008-9079-3     Document Type: Article

References (39)

1. NM Adams DJ Hand 1999 Comparing classifiers when the misallocation costs are uncertain Pattern Recognit 32 1139 1147
2. Arisholm E, Briand LC (2006) Predicting fault-prone components in a java legacy system. Proceedings of the 2006 ACM/IEEE International Symposium on Empirical Software Engineering (ISESE'06)
3. Azar D, Precup D, Bouktif S, Kegl B, Sahraoui H (2002) Combining and adapting software quality predictive models by genetic algorithms. 17th IEEE International Conference on Automated Software Engineering. IEEE Computer Society
4. VR Basili LC Briand WL Melo 1996 A validation of object-oriented design metrics as quality indicators IEEE Trans Softw Eng 22 10 751 761
5. Boetticher GD (2005) Nearest neighbor sampling for better defect prediction. ACM SIGSOFT Software Engineering Notes, 30(4). ACM, New York, NY, pp 1-6
6. Braga AC, Costa L, Oliveira P (2006) A nonparametric method for the comparison of areas under two ROC curves. International Conference on Robust Statistics (ICORS06). Technical University of Lisbon, 16-21 July 2006, Lisbon, Portugal
7. L Breiman 2001 Random forests Mach Learn 45 5 32
8. Challagulla VUB, Bastani FB, Yen I-L, Paul RA (2005) Empirical assessment of machine learning based software defect prediction techniques. Proceedings of the 10th IEEE International Workshop on Object-Oriented Real-Time Dependable Systems (WORDS'05), pp 263-270
9. Conover WJ (1999) Practical nonparametric statistics. Wiley, New York
10. Davis J, Goadrich M (2006) The relationship between precision-recall and ROC curves. Proceedings of the 23rd International Conference on Machine Learning. Pittsburgh, PA, pp 233-240
11. J Demsar 2006 Statistical comparisons of classifiers over multiple data sets J Mach Learn Res 7 1 30
12. C Drummond RC Holte 2006 Cost curves: an improved method for visualizing classifier performance Mach Learn 65 1 95 130
13. K El-Emam S Benlarbi N Goel SN Rai 2001 Comparing case-based reasoning classifiers for predicting high-risk software components J Syst Softw 55 3 301 320
14. Fenton N, Neil M (1999) Software metrics and risk. The 2nd European Software Measurement Conference (FESMA 99), TI-KVIV, Amsterdam, pp 39-55
15. Gokhale SS, Lyu MR (1997) Regression tree modeling for the prediction of software quality. In: Pham H (ed) The third ISSAT International Conference on Reliability and Quality in Design. Anaheim, CA, pp 31-36
16. Guo L, Ma Y, Cukic B, Singh H (2004) Robust prediction of fault-proneness by random forests. Proceedings of the 15th IEEE International Symposium on Software Reliability Engineering (ISSRE 2004), IEEE Press
17. TM Khoshgoftaar DL Lanning 1995 A neural network approach for early detection of program modules having high risk in the maintenance phase J Syst Softw 29 1 85 91
18. Khoshgoftaar TM, Allen EB, Ross FD, Munikoti R, Goel N, Nandi A (1997) Predicting fault-prone modules with case-based reasoning. The Eighth International Symposium on Software Engineering (ISSRE '07). IEEE Computer Society, pp 27-35
19. Khoshgoftaar TM, Seliya N (2002) Tree-based software quality estimation models for fault prediction. The 8th IEEE Symposium on Software Metrics (METRICS'02), IEEE Computer Society, pp 203-214
20. TM Khoshgoftaar B Cukic N Seliya 2007 An empirical assessment on program module-order models Qual Technol Quant Manag 4 2 171 190
21. AG Koru H Liu 2005 Building effective defect-prediction models in practice IEEE Softw 22 6 23 29
22. M Kubat RC Holte S Matwin 1998 Machine learning for the detection of oil spills in satellite radar images Mach Learn 30 2-3 195 215
23. Lewis D, Gale W (1994) A sequential algorithm for training text classifiers. Annual ACM Conference on Research and Development in Information Retrieval, the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Springer-Verlag, New York, NY, pp 3-12
24. Ling CX, Li C (1998) Data mining for direct marketing: problems and solutions. Proc. of the 4th Intern. Conf. on Knowledge Discovery and Data Mining, New York, pp 73-79
25. Ma Y (2007) An empirical investigation of tree ensembles in biometrics and bioinformatics. West Virginia University, PhD thesis, January 2007
26. Macskassy S, Provost F, Rosset S (2005a) Pointwise ROC confidence bounds: an empirical evaluation. Proceedings of the Workshop on ROC Analysis in Machine Learning (ROCML-2005)
27. Macskassy S, Provost F, Rosset S (2005b) ROC confidence bands: an empirical evaluation. Proceedings of the 22nd International Conference on Machine Learning (ICML). Bonn, Germany
28. Menzies T, Stefano JD, Ammar K, Chapman RM, McGill K, Callis P et al (2003) When can we test less? Proceedings of the Ninth International Software Metrics Symposium (METRICS'03), IEEE Computer Society
29. T Menzies J Greenwald A Frank 2007 Data mining static code attributes to learn defect predictors IEEE Trans Softw Eng 33 1 2 13
30. TJ Ostrand EJ Weyuker RM Bell 2005 Predicting the location and number of faults in large software systems IEEE Trans Softw Eng 31 4 340 355
31. N Ohlsson H Alberg 1996 Predicting fault-prone software modules in telephone switches IEEE Trans Softw Eng 22 12 886 894
32. N Ohlsson AC Eriksson ME Helander 1997 Early risk-management by identification of fault-prone modules Empir Softw Eng 2 2 166 173
33. RW Selby AA Porter 1988 Learning from examples: generation and evaluation of decision trees for software resource analysis IEEE Trans Softw Eng 14 12 1743 1757
34. Siegel S (1956) Nonparametric statistics. McGraw-Hill, New York
35. M Vuk T Curk 2006 ROC curve, lift chart and calibration plot Metodoloski zvezki 3 89 108
36. Witten IH, Frank E (2005) Data mining: practical machine learning tools and techniques. Morgan Kaufmann
37. W Youden 1950 Index for rating diagnostic tests Cancer 3 32 35
38. Yousef WA, Wagner RF, Loew MH (2004) Comparison of non-parametric methods for assessing classifier performance in terms of ROC parameters. In Proceedings of Applied Imagery Pattern Recognition Workshop, vol. 33, issue 13-15, pp 190-195
39. H Zhang X Zhang 2007 Comments on 'data mining static code attributes to learn defect predictors' IEEE Trans Softw Eng 33 9 635 637