A framework for comparing multiple cost estimation methods using an automated visualization toolkit

Information and Software Technology

Volumn 57, Issue 1, 2015, Pages 310-328

  Mittas, Nikolaos ^a   Mamalikidis, Ioannis ^b   Angelis, Lefteris ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

^b TECHNOLOGICAL EDUCATIONAL INSTITUTE OF KAVALA   (Greece)

Author keywords

Automated tools; Graphical comparison; Prediction models; REC curves; Software Cost Estimation

Indexed keywords

AUTOMATION; PREDICTIVE ANALYTICS;

AUTOMATED TOOLS; GRAPHICAL COMPARISON; PREDICTION MODEL; REC CURVES; SOFTWARE COST ESTIMATIONS;

COST ESTIMATING;

EID: 84922596154     PISSN: 09505849     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.infsof.2014.05.010     Document Type: Article

References (51)

1. B. Boehm, Software Engineering Economics, Prentice-Hall, Englewood Cliffs, NJ, 1981.
2. B. Boehm, C. Abts, A.W. Brown, S. Chulani, B.K. Clark, E. Horowitz, R. Madachy, D.J. Reifer, B. Steece, Software Cost Estimation with COCOMO II (with CDROM), Prentice-Hall, Englewood Cliffs, NJ, 2000.
3. B. Boehm, C. Abts, S. Chulani, Software development cost estimation approaches-A survey, J. Ann. Softw. Eng. 10 (1-4) (2000) 177-205.
4. J. Bi, K.P. Bennet, Regression error characteristics curves, in: Proceedings of the AIII 20th International Conference on Machine Learning (ICML'03), 2003, pp. 43-50.
5. L. Briand, I. Wieczorek, Resource modeling in software engineering, in: J. Marciniak (Ed.), Second Edition of the Encyclopedia of Software Engineering, Wiley, 2002.
6. L. Buglione, C. Ebert, Estimation tools and techniques, IEEE Trans. Softw. Eng. 28 (3) (2011) 91-94.
7. S. Conte, H. Dunsmore, V.Y. Shen, Software Engineering Metrics and Models, Benjamin Cummings, Menlo Park, CA, 1986.
8. B. Efron, R.J. Tibshirani, An Introduction to the Bootstrap, Chapman & Hall, 1993.
9. J. Egan, Signal Detection Theory and ROC Analysis, Series in Cognition and Perception, Academic Press, 1975.
10. P. Ellis, The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and The Interpretation of Research Results, Cambridge University Press, 2010.
11. T. Foss, E. Stensrud, B. Kitchenham, I. Myrtveit, A simulation study of the model evaluation criterion MMRE, IEEE Trans. Softw. Eng. 29 (11) (2003) 985-995.
12. Galorath, Inc. http://www.galorath.com.
13. I. Good, Permutation Tests: A Practical Guide to Resampling Methods For Testing Hypotheses. Springer Series in Statistics, second ed., 2000.
14. F. Heemstra, Software cost estimation, Inf. Softw. Technol. 34 (10) (1992) 627-639.
15. International Software Benchmarking Standards Group (ISBSG), 2012. https://www.isbsg.org.
16. R. Jensen, An improved macrolevel software development resource estimation model, in: Proceedings of 5th ISPA Conference, 1983, pp. 88-92.
17. C. Jones, Applied Software Measurement: Assuring Productivity and Quality, McGraw Hill, 1996.
18. M. Jorgensen, M. Shepperd, A systematic review of software development cost estimation studies, IEEE Trans. Softw. Eng. 33 (1) (2007) 33-53.
19. V. Kampenes, T. Dyba, J. Hannay, D. Sjoberg, A systematic review of effect size in software engineering experiments, Inf. Softw. Technnol. 49 (11-12) (2007) 1073-1086.
20. B. Kitchenham, L. Pickard, S. MacDonell, M. Shepperd, What accuracy statistics really measure, IEE Proc. Softw. 148 (3) (2001) 81-85.
21. B. Kitchenham, E. Mendes, Why comparative effort prediction studies may be invalid, in: Proceedings of the ACM 5th International Conference on Predictor Models in, Software Engineering, (PROMISE'09), 2009, pp. 1-5.
22. E. Kocaguneli, T. Menzies, J.W. Keung, On the value of ensemble effort estimation, IEEE Trans. Softw. Eng. 38 (6) (2012) 1403-1416.
23. E. Mendes, B. Kitchenham, Further comparison of cross-company and withincompany effort estimation models for web applications, in: Proceedings of the 10th IEEE International Symposium on Software Metrics, IEEE Computer Society, 2004, pp. 348-357.
24. T. Menzies, M. Shepperd, Special issue on repeatable results in software engineering prediction, Emp. Softw. Eng. 17 (2012) 1-17.
25. R. Miller, Simultaneous statistical inference, Springer Series in Statistics, second ed., New York, McGraw-Hill, 1981.
26. N. Mittas, L. Angelis, Comparing cost prediction models by resampling techniques, J. Syst. Softw. 81 (5) (2008) 616-632.
27. N. Mittas, L. Angelis, Bootstrap confidence intervals for regression error characteristic curves evaluating the prediction error of software cost estimation models, in: Proceedings of the 2nd Artificial Intelligence Techniques in Software Engineering Workshop (AISEW2009), at the 5th IFIP Conference on Artificial Intelligence Applications and Innovations, 2009, pp. 221-230.
28. N. Mittas, L. Angelis, Visual comparison of software cost estimation models by regression error characteristic analysis, J. Syst. Softw. 83 (2010) 621-637.
29. N. Mittas, L. Angelis, A permutation test based on regression error characteristic curves for software cost estimation models, Emp. Softw. Eng. 17 (2012) 34-61.
30. N. Mittas, L. Angelis, Ranking and clustering Software cost estimation models through a multiple comparisons algorithm, IEEE Trans. Softw. Eng. 39 (4) (2013) 537-551.
31. N. Mittas, I. Mamalikidis, L. Angelis, StatREC: A graphical user interface tool for visual hypothesis testing of cost prediction models, in: Proceedings of the ACM 8th International Conference on Predictor Models in, Software Engineering, (PROMISE'12), 2012, pp. 39-48.
32. Y. Miyazaki, M. Terakado, K. Ozaki, H. Nozaki, Robust regression for developing software estimation models, J. Syst. Softw. 27 (1994) 3-16.
33. D.S. Moore, G.P. McCabe, W.D. Duckworth, S.L. Sclove, The Practice of Business Statistics, Freeman, New York, 2003.
34. I. Myrtveit, E. Stensrud, M. Shepperd, Reliability and validity in comparative studies of software prediction models, IEEE Trans. Softw. Eng. 31 (5) (2005) 380-391.
35. NASA93 Dataset, 2007. http://promisedata.org/repository/#nasa93.
36. PRICE Systems L.L.C. http://www.pricesystems.com.
37. L. Putnam, W. Myers, Measures for Excellence, Yourdon Press Computing Series, 1992.
38. Quantitative Software Management (QSM), Inc. http://www.qsm.com.
39. R Development Core Team, 2011. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org.
40. R.NET, 2012. http://rdotnet.codeplex.com.
41. C.R. Rao, H. Toutenburg, A. Fieger, C. Heumann, M. Schomaker, Linear Models: Least Squares and Alternatives, Springer Series in Statistics, 2007.
42. A. Scott, M. Knott, A cluster analysis method for grouping means in the analysis of variance, Biometrics 30 (3) (1974) 507-512.
43. Y. Shan, R. McKay, C. Lokan, D. Essam, Software project effort estimation using genetic programming, in: Proceedings of the ICCCAS' 02 International Conference on Communications, Circuits and Systems, 2002, pp. 1108-1112.
44. M. Shepperd, G. Kadoda, Comparing software prediction techniques using simulation, IEEE Trans. Softw. Eng. 27 (11) (2001) 1014-1022.
45. M. Shepperd, S. MacDonell, Evaluating prediction systems in software project estimation, Inf. Softw. Technnol. 54 (8) (2012) 820-827.
46. M. Shepperd, C. Schofield, Estimating software project effort using analogies, IEEE Trans. Softw. Eng. 23 (11) (1997) 736-743.
47. D. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures, third ed., Chapman & Hall/CRC, 2004.
48. Statistics and Information Systems Group, STAINS, http://stains.csd.auth.gr.
49. I. Stamelos, L. Angelis, E. Sakellaris, BRACE: BootstRap based analogy cost estimation, in: Proceedings of the 12th European Software Control Metrics, 2001, pp. 17-23.
50. L. Torgo, Regression error characteristic surfaces, in: Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2005), 2005.
51. Visual Basic.NET, Microsoft Visual Studio. http://www.microsoft.com/visualstudio/en-us.