An empirical analysis of data preprocessing for machine learning-based software cost estimation

Information and Software Technology

Volumn 67, Issue , 2015, Pages 108-127

  Huang, Jianglin ^a   Li, Yan Fu ^b   Xie, Min ^a  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^b CENTRALESUPÉLEC   (France)

Author keywords

Case selection; Data preprocessing; Feature selection; Missing data treatments; Scaling; Software cost estimation

Indexed keywords

APPLICATION PROGRAMS; COST BENEFIT ANALYSIS; FEATURE EXTRACTION; FORECASTING; MACHINE LEARNING; SOFTWARE DESIGN;

CASE SELECTIONS; DATA PREPROCESSING; MISSING DATA; SCALING; SOFTWARE COST ESTIMATIONS;

COST ESTIMATING;

EID: 84942029324     PISSN: 09505849     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.infsof.2015.07.004     Document Type: Conference Paper

References (95)

1. J. Wen, S. Li, Z. Lin, Y. Hu, and C. Huang Systematic literature review of machine learning based software development effort estimation models Inform. Softw. Technol. 54 2012 41 59
2. B.W. Boehm Software engineering economics IEEE Trans. Softw. Eng. SE-10 1984 4 21
3. L.H. Putnam, and W. Myers Measures for Excellence: Reliable Software on Time, within Budget 1991 Prentice Hall Professional Technical Reference
4. A. Bou, D. Ho, and L. Fernando Towards an early software estimation using log-linear regression and a multilayer perceptron model J. Syst. Softw. 86 2013 144 160
5. K. Srinivasan, and D. Fisher Machine learning approaches to estimating software development effort IEEE Trans. Softw. Eng. 21 1995 126 137
6. A. Heiat Comparison of artificial neural network and regression models for estimating software development effort Inform. Softw. Technol. 44 2002 911 922
7. C. Mair, G. Kadoda, M. Lefley, K. Phalp, C. Schofield, M. Shepperd, and S. Webster Investigation of machine learning based prediction systems J. Syst. Softw. 53 2000 23 29
8. I.F. de Barcelos Tronto, J.D.S. da Silva, and N. Sant'Anna An investigation of artificial neural networks based prediction systems in software project management J. Syst. Softw. 81 2008 356 367
9. G.R. Finnie, G.E. Wittig, and J.M. Desharnais A comparison of software effort estimation techniques: using function points with neural networks, case-based reasoning and regression models J. Syst. Softw. 39 1997 281 289
10. Y.F. Li, M. Xie, and T.N. Goh A study of mutual information based feature selection for case based reasoning in software cost estimation Expert Syst. Appl. 36 2009 5921 5931
11. E. Kocaguneli, S. Member, and T. Menzies Exploiting the essential assumptions of analogy-based effort estimation IEEE Trans. Softw. Eng. 38 2012 425 439
12. M. Azzeh, D. Neagu, and P.I. Cowling Analogy-based software effort estimation using fuzzy numbers J. Syst. Softw. 84 2011 270 284
13. E. Kocaguneli, T. Menzies, and J.W. Keung Kernel methods for software effort estimation: effects of different kernel functions and bandwidths on estimation accuracy Empirical Softw. Eng. 18 2013 1 24
14. N.H. Chiu, and S.J. Huang The adjusted analogy-based software effort estimation based on similarity distances J. Syst. Softw. 80 2007 628 640
15. S.J. Huang, and N.H. Chiu Optimization of analogy weights by genetic algorithm for software effort estimation Inform. Softw. Technol. 48 2006 1034 1045
16. J. Li, and G. Ruhe Analysis of attribute weighting heuristics for analogy-based software effort estimation method AQUA+ Empirical Softw. Eng. 13 2008 63 96
17. A.L.I. Oliveira, P.L. Braga, R.M.F. Lima, and M.L. Cornélio GA-based method for feature selection and parameters optimization for machine learning regression applied to software effort estimation Inform. Softw. Technol. 52 2010 1155 1166
18. S. Zhang, C. Zhang, and Q. Yang Data preparation for data mining Appl. Artif. Intell. 17 2003 375 381
19. B. Twala, and M. Cartwright Ensemble missing data techniques for software effort prediction Intell. Data Anal. 14 2010 299 331
20. L.L. Minku, X. Yao, A principled evaluation of ensembles of learning machines for software effort estimation, in: the 7th International Conference on Predictive Models in Software Engineering (PROMISE'11), Banff, Canada, 2011, pp. 1-10.
21. M. Azzeh, Software effort estimation based on optimized model tree, in: the 7th International Conference on Predictive Models in Software Engineering (PROMISE'11), Banff, Canada, 2011.
22. M. Shepperd, and G. Kadoda Comparing software prediction techniques using simulation IEEE Trans. Softw. Eng. 27 2001 1014 1022
23. J.W. Keung, B.A. Kitchenham, and D.R. Jeffery Analogy-X: providing statistical inference to analogy-based software cost estimation IEEE Trans. Softw. Eng. 34 2008 471 484
24. K. Vinay Kumar, V. Ravi, M. Carr, and N. Raj Kiran Software development cost estimation using wavelet neural networks J. Syst. Softw. 81 2008 1853 1867
25. Y.F. Li, M. Xie, and T.N. Goh A study of project selection and feature weighting for analogy based software cost estimation J. Syst. Softw. 82 2009 241 252
26. K. Strike, K.E. Emam, and N. Madhavji Software cost estimation with incomplete data IEEE Trans. Softw. Eng. 27 2001 890 908
27. J. Van Hulse, and T.M. Khoshgoftaar A comprehensive empirical evaluation of missing value imputation in noisy software measurement data J. Syst. Softw. 81 2008 691 708
28. I. Myrtveit, E. Stensrud, and U.H. Olsson Analyzing data sets with missing data: an empirical evaluation of imputation methods and likelihood-based methods IEEE Trans. Softw. Eng. 27 2001 999 1013
29. P. Sentas, and L. Angelis Categorical missing data imputation for software cost estimation by multinomial logistic regression J. Syst. Softw. 79 2006 404 414
30. Y.S. Seo, and D.H. Bae On the value of outlier elimination on software effort estimation research Empirical Softw. Eng. 18 2013 659 698
31. M. Tsunoda, T. Kakimoto, A. Monden, K.I. Matsumoto, An empirical evaluation of outlier deletion methods for analogy-based cost estimation, in: The 7th International Conference on Predictive Models in Software Engineering (PROMISE '11), Banff, Canada, 2011, pp. 1-10.
32. J. Keung, E. Kocaguneli, and T. Menzies Finding conclusion stability for selecting the best effort predictor in software effort estimation Autom. Softw. Eng. 20 2013 543 567
33. M. Azzeh, D. Neagu, and P.I. Cowling Fuzzy grey relational analysis for software effort estimation Empirical Softw. Eng. 15 2009 60 90
34. N. Mittas, and L. Angelis LSEbA: least squares regression and estimation by analogy in a semi-parametric model for software cost estimation Empirical Softw. Eng. 15 2010 523 555
35. J. Li, G. Ruhe, A. Al-Emran, and M.M. Richter A flexible method for software effort estimation by analogy Empirical Softw. Eng. 12 2007 65 106
36. L. Angelis, and I. Stamelos A simulation tool for efficient analogy based cost estimation Empirical Softw. Eng. 5 2000 35 68
37. A. Corazza, S. Di Martino, F. Ferrucci, C. Gravino, and E. Mendes Investigating the use of support vector regression for web effort estimation Empirical Softw. Eng. 16 2011 211 243
38. B.A. Kitchenham, E. Mendes, A comparison of cross-company and within-company effort estimation models for web applications, in: The 8th International Conference on Empirical Assessment in Software Engineering (EASE'04), Edinburgh, Scotland, UK, 2004, pp. 47-55.
39. S. Di Martino, F. Ferrucci, C. Gravino, E. Mendes, Comparing size measures for predicting web application development effort: a case study, in: The 1st International Symposium on Empirical Software Engineering and Measurement (ESEM'07), IEEE, Madrid, Spain, 2007, pp. 324-333.
40. G. Costagliola, S. Di Martino, F. Ferrucci, C. Gravino, G. Tortora, and G. Vitiello Effort estimation modeling techniques: a case study for web applications The 6th International Conference on Web Engineering (ICWE'06) 2006 ACM Palo Alto, CA, USA 9 16
41. L.C. Briand, T. Langley, I. Wieczorek, A replicated assessment and comparison of common software cost modeling techniques, in: The 22nd International Conference on Software Engineering (ICSE'00), Limerick, Ireland, 2000, pp. 377-386.
42. E. Kocaguneli, S. Member, and T. Menzies On the value of ensemble effort estimation IEEE Trans. Softw. Eng. 38 2012 1403 1416
43. A. Corazza, S. Di Martino, F. Ferrucci, C. Gravino, F. Sarro, E. Mendes, How effective is Tabu search to configure support vector regression for effort estimation? in: The 6th International Conference on Predictive Models in Software Engineering (PROMISE'10), Timisoara, Romania, 2010, pp. 1-10.
44. A. Corazza, S. Di Martino, F. Ferrucci, C. Gravino, E. Mendes, Applying support vector regression for Web effort estimation using a cross-company dataset, in: The 3rd International Symposium on Empirical Software Engineering and Measurement (ESEM'09), Lake Buena Vista, Florida, USA, 2009, pp. 191-202.
45. N. Mittas, M. Athanasiades, and L. Angelis Improving analogy-based software cost estimation by a resampling method Inform. Softw. Technol. 50 2008 221 230
46. J. Li, A. Al-Emran, G. Ruhe, Impact analysis of missing values on the prediction accuracy of analogy-based software effort estimation method AQUA, in: The 1st International Symposium on Empirical Software Engineering and Measurement (ESEM'07), Madrid, Spain, 2007, pp. 126-135.
47. E. Mendes, A comparison of techniques for web effort estimation, in: The 1st International Symposium on Empirical Software Engineering and Measurement (ESEM'07), Madrid, Spain, 2007, pp. 334-343.
48. J. Keung, Empirical evaluation of analogy-X for software cost estimation, in: The 2nd ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM'08), Kaiserslautern, Germany, 2008, pp. 294-296.
49. F. Ferrucci, E. Mendes, F. Sarro, Web effort estimation: the value of cross-company data set compared to single-company data set, in: The 8th International Conference on Predictive Models in Software Engineering (PROMISE'12), Lund, Sweden, 2012, pp. 29-38.
50. P.C. Pendharkar, G.H. Subramanian, and J.A. Rodger A probabilistic model for predicting software development effort IEEE Trans. Softw. Eng. 31 2005 615 624
51. M. Auer, A. Trendowicz, B. Graser, E. Haunschmid, and S. Biffl Optimal project feature weights in analogy-based cost estimation: improvement and limitations IEEE Trans. Softw. Eng. 32 2006 83 92
52. T. Menzies, A. Butcher, D. Cok, A. Marcus, L. Layman, F. Shull, B. Turhan, and T. Zimmermann Local versus global lessons for defect prediction and effort estimation IEEE Trans. Softw. Eng. 39 2013 822 834
53. A. Brady, T. Menzies, Case-based reasoning vs parametric models for software quality optimization, in: The 6th International Conference on Predictive Models in Software Engineering (PROMISE'10), Timisoara, Romania, 2010, pp. 1-10.
54. E. Kocaguneli, T. Menzies, How to find relevant data for effort estimation? in: The 5th International Symposium on Empirical Software Engineering and Measurement (ESEM'11), Banff, Canada, 2011, pp. 255-264.
55. Y.F. Li, M. Xie, and T.N. Goh A study of the non-linear adjustment for analogy based software cost estimation Empirical Softw. Eng. 14 2009 603 643
56. R. Borges, T. Menzies, Learning to change projects, in: The 8th International Conference on Predictive Models in Software Engineering (PROMISE'12), Lund, Sweden, 2012, pp. 11-18.
57. E. Kocaguneli, T. Menzies, J. Hihn, B.H. Kang, Size doesn't matter? On the value of software size features for effort estimation, in: The 8th International Conference on Predictive Models in Software Engineering (PROMISE'12), Lund, Sweden, 2012, pp. 89-98.
58. Q. Liu, W.Z. Qin, R. Mintram, and M. Ross Evaluation of preliminary data analysis framework in software cost estimation based on ISBSG R9 data Softw. Qual. J. 16 2008 411 458
59. V. Khatibi Bardsiri, D.N.A. Jawawi, S.Z.M. Hashim, and E. Khatibi A PSO-based model to increase the accuracy of software development effort estimation Softw. Qual. J. 21 2013 501 526
60. A. Corazza, S. Di Martino, F. Ferrucci, C. Gravino, F. Sarro, and E. Mendes Using tabu search to configure support vector regression for effort estimation Empirical Softw. Eng. 18 2013 506 546
61. J. Li, G. Ruhe, Decision support analysis for software effort estimation by analogy, in: The 3rd International Conference on Predictor Models in Software Engineering (PROMISE'07), Minneapolis, MN, USA, 2007, pp. 6-16.
62. L.L. Minku, and X. Yao Ensembles and locality: insight on improving software effort estimation Inform. Softw. Technol. 55 2013 1512 1528
63. M. Azzeh A replicated assessment and comparison of adaptation techniques for analogy-based effort estimation Empirical Softw. Eng. 17 2012 90 127
64. C.J. Hsu, and C.Y. Huang Comparison of weighted grey relational analysis for software effort estimation Softw. Qual. J. 19 2010 165 200
65. E. Kocaguneli, T. Menzies, J. Keung, D. Cok, and R. Madachy Active learning and effort estimation: finding the essential content of software effort estimation data IEEE Trans. Softw. Eng. 39 2013 1040 1053
66. N. Mittas, and L. Angelis Ranking and clustering software cost estimation models through a multiple comparisons algorithm IEEE Trans. Softw. Eng. 39 2013 537 551
67. C. Lopez-Martin, C. Isaza, and A. Chavoya Software development effort prediction of industrial projects applying a general regression neural network Empirical Softw. Eng. 17 2011 738 756
68. A. Bakir, B. Turhan, and A.B. Bener A new perspective on data homogeneity in software cost estimation: a study in the embedded systems domain Softw. Qual. J. 18 2009 57 80
69. A. Bakir, B. Turhan, and A. Bener A comparative study for estimating software development effort intervals Softw. Qual. J. 19 2011 537 552
70. N. Ramasubbu, R.K. Balan, Overcoming the challenges in cost estimation for distributed software projects, in: The 34th International Conference on Software Engineering (ICSE'12), Zurich, Switzerland, 2012, pp. 91-101.
71. M.V. Kosti, N. Mittas, L. Angelis, Alternative methods using similarities in software effort estimation, in: The 8th International Conference on Predictive Models in Software Engineering (PROMISE'12), Lund, Sweden, 2012, pp. 59-68.
72. D. Rodríguez, M.A. Sicilia, E. García, and R. Harrison Empirical findings on team size and productivity in software development J. Syst. Softw. 85 2012 562 570
73. J. Van Hulse, and T.M. Khoshgoftaar Incomplete-case nearest neighbor imputation in software measurement data Inform. Sci. 259 2014 596 610
74. Z. Chen, T. Menzies, D. Port, B. Boehm, Feature subset selection can improve software cost estimation accuracy, in: The 2005 International Conference on Predictor Models in Software Engineering (PROMISE'05), St. Louis, MO, USA, 2005, pp. 1-6.
75. S. Das, Filters, wrappers and a boosting-based hybrid for feature selection, in: The 18th International Conference on Machine Learning (ICML'01), Williamstown, MA, USA, 2001, pp. 74-81.
76. E. Mendes, I. Watson, C. Triggs, N. Mosley, and S. Counsell A comparative study of cost estimation models for web hypermedia applications Empirical Softw. Eng. 8 2003 163 196
77. ISBSG, The ISBSG Development & Enhancement Project Data in: ISBSG (Ed.), 2013. < http://www.isbsg.org >.
78. I. Guyon, and A. Elisseeff An introduction to variable and feature selection J. Mach. Learn. Res. 3 2003 1157 1182
79. H. Peng, F. Long, and C. Ding Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy IEEE Trans. Pattern Anal. Mach. Intell. 27 2005 1226 1238
80. P.L. Braga, A.L.I. Oliveira, S.R.L. Meira, A GA-based feature selection and parameters optimization for support vector regression applied to software effort estimation, in: The 23rd Annual ACM Symposium on Applied Computing (SAC'08), Fortaleza, Ceara, Brazil, 2008, pp. 1788-1792.
81. F. Ferrucci, M. Harman, and F. Sarro Search-based software project management G. Ruhe, C. Wohlin, Software Project Management in a Changing World 2014 Springer 373 399
82. C. Kirsopp, and M. Shepperd Case and feature subset selection in case-based software project effort prediction M. Bramer, A. Preece, F. Coenen, Research and Development in Intelligent Systems XIX 2003 Springer London 61 74
83. M. Fernández-Diego, and F. González-Ladrón-de-Guevara Potential and limitations of the ISBSG dataset in enhancing software engineering research: a mapping review Inform. Softw. Technol. 56 2014 527 544
84. N. Mittas, and L. Angelis A permutation test based on regression error characteristic curves for software cost estimation models Empirical Softw. Eng. 17 2011 34 61
85. J.M. Desharnais Analyse statistique de la productivitie des projets informatique a partie de la technique des point des fonction 1989 University of Montreal
86. M. Jørgensen, and M. Shepperd A systematic review of software development cost estimation studies IEEE Trans. Softw. Eng. 33 2007 33 53
87. F. Walkerden, and R. Jeffery Empirical study of analogy-based software effort estimation Empirical Softw. Eng. 4 1999 135 158
88. A.R. Gray, and S.G. MacDonell A comparison of techniques for developing predictive models of software metrics Inform. Softw. Technol. 39 1997 425 437
89. Y.F. Li, M. Xie, and T.N. Goh Adaptive ridge regression system for software cost estimating on multi-collinear datasets J. Syst. Softw. 83 2010 2332 2343
90. T. Foss, E. Stensrud, B. Kitchenham, and I. Myrtveit A simulation study of the model evaluation criterion MMRE IEEE Trans. Softw. Eng. 29 2003 985 995
91. I. Myrtveit, E. Stensrud, and M. Shepperd Reliability and validity in comparative studies software prediction models IEEE Trans. Softw. Eng. 31 2005 380 391
92. M.A. Ahmed, I. Ahmad, and J.S. AlGhamdi Probabilistic size proxy for software effort prediction: a framework Inform. Softw. Technol. 55 2013 241 251
93. B. Kitchenham, and E. Mendes Why comparative effort prediction studies may be invalid The 5th International Conference on Predictor Models in Software Engineering (PROMISE'09) 2009 ACM Vancouver, Canada 4
94. E. Mendes, S. Di Martino, F. Ferrucci, and C. Gravino Cross-company vs. single-company web effort models using the Tukutuku database: an extended study J. Syst. Softw. 81 2008 673 690
95. M. Shepperd, and S. MacDonell Evaluating prediction systems in software project estimation Inform. Softw. Technol. 54 2012 820 827