Replication of defect prediction studies: Problems, pitfalls and recommendations

ACM International Conference Proceeding Series

Volumn , Issue , 2010, Pages

  Mende, Thilo ^a  

^a UNIVERSITY OF BREMEN   (Germany)

Author keywords

Defect prediction model; Replication

Indexed keywords

DATA SETS; DEFECT PREDICTION; DEFECT PREDICTION MODELS; EVALUATION MEASURES; MISSING INFORMATION; PERFORMANCE MEASURE; RANDOM MODEL; REPLICATION; REPLICATION STUDY; REPRODUCIBILITIES;

FORECASTING; LEARNING ALGORITHMS; MATHEMATICAL MODELS; MODELS; PREDICTIVE CONTROL SYSTEMS; RESEARCH; SOFTWARE ENGINEERING;

DEFECTS;

EID: 78649770544     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1868328.1868336     Document Type: Conference Paper

References (26)

1. E. Arisholm, L. C. Briand, and M. Fuglerud. Data mining techniques for building fault-proneness models in telecom java software. In ISSRE '07: Proceedings of the 18th IEEE International Symposium on Software Reliability Engineering, pages 215-224. IEEE Press, 2007.
2. E. Arisholm, L. C. Briand, and E. B. Johannessen. A systematic and comprehensive investigation of methods to build and evaluate fault prediction models. Technical Report TR 2008-06, Simula Research Laboratory, 2008.
3. E. Dimitriadou, K. Hornik, F. Leisch, D. Meyer, and A. Weingessel. e1071: Misc functions of the department of statistics (e1071), TU Wien, 2009. R package version 1.5-19.
4. C. Drummond. Replicability is not reproducibility: Nor is it good science. In Proceedings of the Twenty-Sixth International Conference on Machine Learning: Workshop on Evaluation Methods for Machine Learning IV, 2009.
5. M. D'Ambros, M. Lanza, and R. Robbes. An extensive comparison of bug prediction approaches. In 7th IEEE Working Conference on Mining Software Repositories, 2010.
6. K. El Emam, S. Benlarbi, N. Goel, and S. N. Rai. The confounding effect of class size on the validity of object-oriented metrics. IEEE Transactions on Software Engineering, 27(7):630-650, 2001.
7. G. Gay, T. Menzies, B. Cukic, and B. Turhan. How to build repeatable experiments. In PROMISE '09: Proceedings of the 5th International Conference on Predictor Models in Software Engineering, pages 1-9, New York, NY, USA, 2009. ACM.
8. A. E. Hassan. Predicting faults using the complexity of code changes. In International Conference on Software Engineering, pages 78-88, Washington, DC, USA, 2009. IEEE Computer Society.
9. Y. Jiang, B. Cukic, and Y. Ma. Techniques for evaluating fault prediction models. Empirical Software Engineering, 13(5):561-595, 2008.
10. B. Kitchenham and E. Mendes. Why comparative effort prediction studies may be invalid. In PROMISE '09: Proceedings of the 5th International Conference on Predictor Models in Software Engineering, pages 1-5, New York, NY, USA, 2009. ACM.
11. A. G. Koru, K. E. Emam, D. Zhang, H. Liu, and D. Mathew. Theory of relative defect proneness. Empirical Software Engineering, 13(5):473-498, 2008.
12. S. Lessmann, B. Baesens, C. Mues, and S. Pietsch. Benchmarking classiffcation models for software defect prediction: A proposed framework and novel findings. IEEE Transactions on Software Engineering, 34(4):485-496, 2008.
13. T. Mende and R. Koschke. Revisiting the evaluation of defect prediction models. In PROMISE '09: Proceedings of the 5th International Conference on Predictor Models in Software Engineering, pages 1-10, New York, NY, USA, 2009.
14. T. Mende and R. Koschke. Effort-aware defect prediction models. In European Conference on Software Maintenance and Reengineering, pages 109-118, 2010.
15. T. Menzies, A. Dekhtyar, J. Distefano, and J. Greenwald. Problems with precision: A response to "comments on 'data mining static code attributes to learn defect predictors'". IEEE Transactions on Software Engineering, 33(9):637-640, 2007.
16. T. Menzies, J. Greenwald, and A. Frank. Data mining static code attributes to learn defect predictors. IEEE Transactions on Software Engineering, 33(1):2-13, 2007.
17. T. Menzies, Z. Milton, B. Turhan, B. Cukic, Y. Jiang, and A. Bener. Defect prediction from static code features: current results, limitations, new approaches. Automated Software Engineering, 2010.
18. N. Nagappan, T. Ball, and A. Zeller. Mining metrics to predict component failures. In International Conference on Software Engineering, November 2006.
19. N. Ohlsson and H. Alberg. Predicting fault-prone software modules in telephone switches. IEEE Transactions on Software Engineering, 22(12):886-894, 1996.
20. T. Ostrand, E. Weyuker, and R. Bell. Predicting the location and number of faults in large software systems. IEEE Transactions on Software Engineering, 31(4):340-355, 2005.
21. R Development Core Team. R: A Language and Environment for Statistical Computing.
22. T. Sing, O. Sander, N. Beerenwinkel, and T. Lengauer. ROCR: visualizing classiffer performance in R. Bioinformatics, 21(20):3940-3941, 2005.
23. A. Tosun, B. Turhan, and A. Bener. Validation of network measures as indicators of defective modules in software systems. In PROMISE '09: Proceedings of the 5th International Conference on Predictor Models in Software Engineering, pages 1-9, New York, NY, USA, 2009. ACM.
24. H. Zhang and X. Zhang. Comments on "data mining static code attributes to learn defect predictors". IEEE Transactions on Software Engineering, 33(9):635-637, 2007.
25. T. Zimmermann and N. Nagappan. Predicting defects using network analysis on dependency graphs. In International Conference on Software Engineering, pages 531-540, New York, NY, USA, 2008. ACM.
26. T. Zimmermann, R. Premraj, and A. Zeller. Predicting defects for eclipse. In Predictor Models in Software Engineering, May 2007.