Ensemble missing data techniques for software effort prediction

Intelligent Data Analysis

Volumn 14, Issue 3, 2010, Pages 299-331

  Twala, Bhekisipho ^a   Cartwright, Michelle ^b  

^a UNIVERSITY OF JOHANNESBURG   (South Africa)

^b BRUNEL UNIVERSITY   (United Kingdom)

Author keywords

Decision tree; Ensemble; Imputation; Incomplete data; Machine learning; Missing data techniques; Software prediction; Supervised learning

Indexed keywords

EID: 77953511815     PISSN: 1088467X     EISSN: 15714128     Source Type: Journal    
DOI: 10.3233/IDA-2010-0423     Document Type: Article

References (74)

1. E. Bauer and R. Kohavo, An empirical comparison of voting classification algorithms: Bagging, boosting and variants, Machine Learning, 36 1/2 (1989), 105-139.
2. L. Breiman, Bagging predictors, Machine Learning 26 (2) (1996), 123-140.
3. L. Breiman, Bias, variance, and arcing classifiers, Technical Report 460, Statistics Department, University of California at Berkeley, 1996b.
4. L. Breiman, J. Friedman, R. Olshen and C. Stone, Classification and Regression Trees, Wadsworth, 1984.
5. L. Briand, V. Basili and W. Thomas, A pattern recognition approach to software engineering data analysis, IEEE Transactions on Software Engineering 18(11) (1992), 931-942.
6. M. Cartwright and M.J. Shepperd, Building predictive models from object-oriented metrics. 8th European Software Control and Metrics Conference, Berlin, 1997.
7. M. Cartwright, M.J. Shepperd and Q. Song, Dealing with Missing Software Project Data, In Proceedings of the 9th International Symposium on Software Metrics, 2003, 154-165.
8. B. Cestnik, I. Kononenko and I. Bratko, Assistant 86: a knowledge-elicitation tool for sophisticated users. In I. Bratko and N. Lavrac, editors, European Working Session on Learning -EWSL87, Sigma Press, Wilmslow, England, 1987.
9. P. Cheeseman, J. Kelly,M. Self, J. Stutz,W. Taylor and D. Freeman, Bayesian Classification. In Proceedings of American Association of Artificial Intelligence (AAAI), Morgan Kaufmann Publishers: San Meteo, CA, 1988, 607-611.
10. J. Chen and J. Shao. Nearest Neighbour Imputation for Survey Data. Journal of Official Statistics 16(2) (2000), 113-131.
11. P. Ciaccia, M. Patella and P. Zezula. M-tree: An Efficient Access Method for Similarity Search in Metric Spaces, In VLDB '97, 1997, 426-435.
12. A.P. Dempster, N.M. Laird and D.B. Rubin, Maximum likelihood estimation from incomplete data via the EM algorithm, Journal of the Royal Statistical Society, Series B 39 (1977), 1-38.
13. T.G. Dietterich, An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization, Machine Learning 40(2) (2000), 139-158.
14. T.G. Dietterich, Ensemble Methods in Machine Learning, in: First InternationalWorkshop on Multiple Classifier Systems, Lecture Notes in Computer Science, J. Kittler and F. Roli, eds, 2000, 1-15.
15. J. Dougherty, R. Kohavi and M. Sahami, Supervised and Unsupervised Discretization of Continuous Features, In Proceedings of the 12th International Conference on Machine Learning, Morgan Kauffmann, Los Altos, CA, 1995.
16. K. El-Emam and A. Birk, Validating the ISO/IEC 15504 Measures of Software Development Process Capability, Journal of Systems and Software 51(2) (2000), 119-149.
17. M. Evett, T.M. Khoshgoftaar, P. Cheien and E. Allen, GP-based software quality prediction, In Proceedings of the 3rd Annual Genetic Programming Conference, 1998, 60-65.
18. N. Fenton and M. Neil, A critique of software defect prediction models, IEEE Transactions on Software Engineering 25(5) (1999), 675-689.
19. Y. Freund and R. Schapire, Experiments with a new boosting algorithm, In Machine Learning: Proceedings of the 13th International Conference, 1996, 148-156.
20. J.W. Graham and S.M. Hofer, EMCOV.EXE user's Guide (Unpublished Manuscript), University Park, PA: Pennsylvania State University Department of Bahavioral Health, 1993.
21. T.K. HO, Random decision forests. In Proceedings of the 3rd International Conference on Document Analysis and Recognition, 1995, 278-282.
22. N. Hollander, W. Sauerbrei and M. Schumacher, Confidence intervals for the effect of a prognostic factor after selection of an optimal cutpoint, Statistics in Medicine 23 (2004), 1701-1713.
23. ISBSG Data Disk, Release 7, June 2001.
24. I. Jolliffe, Principal Component Analysis, Springer Verlag, 1986.
25. P. Jönsson and C. Wohlin, An Evaluation of k-Nearest Neighbour Imputation Using Likert Data, In Proceedings of the 10th International Symposium on Software Metrics, 108-118, September 11-17, 2004.
26. G.F. Kadoda, M. Cartwright and M. Shepperd, Issues on the Effective Use of CBR Technology for Software Project Prediction, ICCBR (2001), 276-290.
27. G. Kalton, Compensating for Missing Survey Data, Michigan, 1983.\
28. T.M. Khoshgoftaar and N. Seliya, Comparative assessment of software quality classification techniques: an empirical case study, Empirical Software Engineering Journal 9(3) (2004), 229-257.
29. T.M. Khoshgoftaar and N. SELIYA, Tree-based Software Quality Estimation Models for Fault Prediction, In the Proceedings of the 8th IEEE International Symposium on Software Metrics, 2002.
30. T.M. Khoshgoftaar and E.B. Allen, Modelling Software Quality with Classification Trees, In Recent Advances in Reliability and Quality Engineering, Hoang Pham Editor. World Scientific, Singapore, 1999.
31. T.M. Khoshgoftaar, N. Seliya and A. Herzberg, Resource-oriented software quality classification models, Journal of Systems and Software 76 (2004), 111-126.
32. R.E. Kirk, Experimental Design (2nd Ed.), Monterey, CA: Brooks, Cole Publishing Company, 1982.
33. J.O. Kim and J. Curry, The treatment of missing data in multivariate analysis, Sociological Methods and Research 6 (1977), 215-240.
34. K.K. Lai, L. Yu, S.Y.Wang and L.G. Zhou, Credit risk analysis using a reliability-based neural network ensemble model, Lecture Notes in Computer Science 4132 (2006), 682-690.
35. K. Lakshminarayan, S.A. HARP and T. Samad, Imputation of Missing Data in Industrial Databases, Applied Intelligence 11 (1999), 259-275.
36. R.J.A. Little and D.B. Rubin, Statistical Analysis with Missing Data, New York: Wiley, 1987.
37. M. Mendonca amd N.L. Sunderhaft, Mining Software Engineering Data: A Survey. DACS-SOAR-99-103 A DACS State-of-theArt Report. DoD Data and Analysis Center for Software, 1999.
38. T. Menzies, Practical Machine Learning for Software Engineering and Knowledge Engineering. In Handbook of Software Engineering and Knowledge Engineering, 2001. [Available from http://tim.menzies.com/pdf/00ml.pdf, January 2009].
39. MINITAB. MINITAB Statistical Software for Windows 9.0, MINITAB, Inc., PA, USA, 2002.
40. MULTIPLE IMPUTATION SOFTWARE. [Available from http://www.stat.psu.edu/ jls/misoftwa.html or http:// methcenter.psu.edu/EMCOV.html; January 2009].
41. O.J. Murphy and R.L. McCraw, Designing storage efficient decision trees, IEEE Transactions on Computing 40(3) (1991), 315-319.
42. S.K. Murthy and S. Salzberg, Lookahead and pathology in decision tree induction, Proceedings of the 14th International Joint Conference on Artificial Intelligence, (1992), 309-347, Montreal, Canada: Morgan Kauffman.
43. S. Murthy, S. Kasif and R. Biegel, OC1: Randomised induction of oblique decisions trees, In Proceedings of the 11th National Conference on Artificial Intelligence, (1993), 322-327.
44. I. Myrtveit, E. Stensrud and U. Olsson, Analyzing Data Sets with Missing Data: An Empirical Evaluation of Imputation Methods and Likelihood-Based Methods, IEEE Transactions on Software Engineering 27(11) (2001), 1999-11013
45. D.E. Neumann, An Enhanced Neural Network Technique for Software Risk Analysis, IEEE Transactions on Software Engineering, (2002), 904-912.
46. E. Papatheocharous and A.S. Andreou, Classification and Prediction of Software Cost through Fuzzy Decision Trees, Lecture Notes in Business Information Processing 24 (2009), 234-247.
47. A.A. Porter and R.W. Selby, Empirically Guided Software Development Using Metric-Based Classification Trees, IEEE Software 7(2) (1990), 46-54.
48. A.A. Porter and R.W. Selby, Evaluating Techniques for Generating Metric-based Classification Trees, Journal of Systems Software (1990), 209-218.
49. J.R. Quinlan, C.4.5: Programs for Machine Learning. Los Altos, California: Morgan Kauffman Publishers, INC., 1993.
50. J.R. Quinlan, Induction to Decision Trees, Machine Learning 1 (1986), 81-106.
51. J.R. Quinlan, Induction over Large Databases: Technical Report HPP-79-114, Stanford University.
52. D.B. Rubin, Multiple Imputation After 18+ Years, Journal of the American Statistical Association 91 (1996), 473-489.
53. S.R. Safavian and D. Landgrebe,Asurvey of decision tree classifiers, IEEE Transactions on Systems, Man and Cybernetics 21 (1991), 660-674.
54. S.J. Sayyad and T.J. Menzies, The PROMISE Repository of Software Engineering Databases. School of Information Technology and Engineering, University of Ottawa, Canada, 2005 [Available from http://promise.site.uottawa. ca/SERepository; January 2009].
55. J.L. Schafer, Analysis of Incomplete Multivariate Data, Chapman and Hall, London, 1997.
56. J.L. Schafer and M.K. Olsen, Multiple Imputation for multivariate missing data problems: a data analyst's perspective, Multivariate Behavioral Research 33(4) (1998), 545-571.
57. J.L. Schafer and J.W. Graham, Missing data: Our view of the state of the art, Psychological Methods 7(2) (2002), 147-177.
58. R.W. Selby and A.A. Porter, Learning from Examples: Generation and Evaluation of Decision Trees for Software Resource Analysis, IEEE Trans on Soft Eng 14(12) (1988), 1743-1757.
59. P. Sentas, A. Lefteris and I. Stamelos, Multiple Logistic Regression as Imputation method Applied on Software Effort prediction, In Proceedings of the 10th International Symposium on Software Metrics, Chicago, 14-16 September 2004.
60. A, Shapiro, Structured Induction in Expert Systems, Addison Wesley, London, 1987.
61. M.J. Sheppered and C. Schofiled, Estimating software project using analogies, IEEE Transaction on Software Engineering 23(12) (1997), 736-743.
62. K. Strike, K.E. El-Emam and K.E., Madhavjim, Software Cost Estimation with Incomplete Data, IEEE Transaction on Software Engineering 27(10) (2001), 890-908.
63. Q. Song and M. Sheppered, A Short Note on Safest Default Missingness Mechanism Assumptions, Empirical Software Engineering 10(2) (2005), 235-243.
64. K. Srinivasan and D. Fisher, Machine Learning Approaches to Estimating Software Development Effort, IEEE Transaction on Software Engineering 21(2) (1995), 126-137.
65. J. Tian, Integrating Time Domain and Input Domain Analyses of Software Reliability Using Tree-Based Models, IEEE Transactions on Software Engineering 21(12) (1995), 945-958.
66. J. Tian and J. Palma, Analyzing and Improving Reliability: A Tree-based Approach, IEEE Software 15(2) (1998), 97-104.
67. B. Twala, An Empirical Comparison of Techniques Handling Incomplete Data Using Decision Trees, Applied Artificial Intelligence 23(5) (2009), 373-405.
68. B. Twala, Effective Techniques for Handling Incomplete Data Using Decision Trees, Unpublished P.hD. Dissetation, Open University, Milton Keynes, UK, 2005.
69. B. Twala, M.C. Jones and D.J. Hand, Good methods for coping with missing data in decision trees, Pattern Recognition Letters 29 (2008), 950-956.
70. B. Twala, M. Cartwright and G. Liebchen, Classifying Incomplete Software Engineering Data Using Decision Trees: An Improved Probabilistic Approach, In Proceedings of Software Engineering Applications, November 13-15, 2006, Dallas, TX, USA.
71. B. Twala, M.H. Cartwright and M. Shepperd, Comparison of Various Methods for Handling Incomplete Data in Software Engineering Databases, In Proceedings of the 4th International Symposium on Empirical Software Engineering, Noosa Heads, Australia, November 2005.
72. B. Twala and M.H. Cartwright, Ensemble imputation methods for missing software engineering data, In: Proceedings of 11th IEEE International Software Metrics Symposium, 2005, 30-40.
73. W.N. Venables and B.D. Ripley, Modern Applied Statistics with S-PLUS, New York: Springer, 1994.
74. A.P. White and W.Z. Liu, Bias in information-based measures in decision tree induction, Machine Leaning 15 (1994), 321-329.