Inconsistency of expert judgment-based estimates of software development effort

Journal of Systems and Software

Volumn 80, Issue 11, 2007, Pages 1770-1777

  Grimstad, Stein ^a,b   Jørgensen, Magne ^a  

^a SIMULA RESEARCH LABORATORY   (Norway)

^b UNIVERSITY OF OSLO   (Norway)

Author keywords

Effort estimation; Expert judgment; Inconsistency; Software development

Indexed keywords

COMPUTER SOFTWARE SELECTION AND EVALUATION; CORRELATION METHODS; EXPERT SYSTEMS; INFORMATION ANALYSIS; PROFESSIONAL ASPECTS;

EFFORT ESTIMATION; EXPERT JUDGMENT; INCONSISTENCY;

SOFTWARE ENGINEERING;

EID: 34548646952     PISSN: 01641212     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jss.2007.03.001     Document Type: Article

References (31)

1. Aranda J., and Easterbrook S. Anchoring and adjustment in software estimation. European Software Engineering Conference (2005), ACM Press, Lisbon, Portugal
2. Briand L.C., and Wieczorek I. Resource estimation in software engineering. In: Marcinak J.J. (Ed). Encyclopedia of Software Engineering (2002), John Wiley & Sons, New York 1160-1196
3. Chulani S., Boehm B., et al. Bayesian analysis of empirical software engineering cost models. IEEE Transactions on Software Engineering 25 4 (1999) 573-583
4. Conte S.D., Dunsmore H.E., et al. Software Engineering Metrics and Models (1986), Menlo Park, California, Benjamin Cummings
5. Coombs P. IT Project Estimation - A Practical Guide to the Costing of Software (2003), Cambridge University Press, Cambridge
6. Grimstad S., and Jørgensen M. A Framework for Analysis of Software Cost Estimation Error. ISESE (2006), ACM Press, Rio de Janeiro, Brazil
7. Grimstad, S., Jørgensen, M., (forthcoming). The Impact of Irrelevant Information on Software Effort Estimates. In: 18th Australian Conference on Software Engineering, Melbourne, Australia.
8. Gruschke, T.M., Jørgensen, M., (in preparation). How much does feedback improve software cost estimation? An Empirical Study, in preparation.
9. Harvey N. Why are judgments less consistent in less predictable task situations?. Organizational Behaviour and Human Decision Processes 63 (1995) 247-263
10. Heemstra, F.J., Kusters, R.J., (1989). Controlling software development costs: a field study. In: International Conference on Organisation and Information Systems, Bled, Yugoslavia.
11. Höst M., and Wohlin C. An experimental study of individual subjective effort estimations and combinations of the estimates. International Conference on Software Engineering, Kyoto, Japan (1998), IEEE Comput. Soc., Los Alamitos, CA, USA
12. Jenkins A.M., Naumann J.D., et al. Empirical investigation of systems development practices and results. Information and Management 7 2 (1984) 73-82
13. Jørgensen M. A review of studies on expert estimation of software development effort. Journal of Systems and Software 70 1-2 (2004) 37-60
14. Jørgensen, M., (forthcoming). Estimation of Software Development Work Effort:Evidence on Expert Judgment and Formal Models. In: International Journal of Forecasting.
15. Jørgensen M., and Sjøberg D.I.K. Impact of effort estimates on software project work. Information and Software Technology 43 15 (2001) 939-948
16. Jørgensen, M., Moløkken, K., (2002). Combination of software development effort prediction intervals: Why, when and how? Conference on Software Engineering and Knowledge Engineering, Italy.
17. Jørgensen M., and Sjøberg D.I.K. The impact of customer expectation on software development effort estimates. International Journal of Project Management 22 (2004) 317-325
18. Jørgensen, M., Faugli, B., et al. (in press). Characteristics of software engineers with optimistic predictions. Journal of Systems and Software, doi:10.1016/j.jss.2006.09.047.
19. Kusters R.J., Genuchten M.J.I.M., et al. Are software cost-estimation models accurate?. Information and Software Technology 32 3 (1990) 187-190
20. Levi K. Expert systems should be more accurate than human experts: Evaluation procedures from human judgment and decision making. IEEE Transactions on Systems, Man, and Cybernetics 19 3 (1989) 647-657
21. Lusk C.M., and Hammond K.R. Judgment in a dynamic task: microburst forecasting. Journal of Behavioral Decision Making 4 (1991) 55-73
22. Meehl P.E. When shall we use our heads instead of the formula?. Journal of Counseling Psychology 4 4 (1957) 268-273
23. Moløkken, K., Jørgensen, M. (2003). A review of software surveys on software effort estimation. In: International Symposium on Empirical Software Engineering, Rome, Italy, Simula Res. Lab. Lysaker Norway.
24. Moløkken-Østvold, K., Jørgensen, M., et al. (2004). A Survey on Software Estimation in the Norwegian Industry. Metrics '04, Chicago, Illinois.
25. Morera O.F., and Budescu D.V. Random error reduction in analytic hierarchies: a comparison of holistic and decompositional decision strategies. Journal of Behavioral Decision Making 14 (2001) 223-242
26. Rothstein H.G. The effects of time pressure on judgment in multiple cue probability learning. Organizational Behaviour and Human Decision Processes 37 (1986) 83-92
27. Shanteau J., Weiss D.J., et al. Performance-based assessment of expertise: how can you tell if someone is an expert?. European Journal of Operations Research 136 (2002) 253-263
28. Simonsohn, U., (in press). Clouds make nerds look good: field evidence of the impact of incidental factors on decision making. Journal of Behavioral Decision Making, doi:10.1002/bdm.545.
29. Sommerville. Software Engineering (2004), Addison-Wesley
30. Stewart T.R. Improving reliability of judgmental forecasts. In: Armstrong J.S. (Ed). Principles of Forecasting (2001), Kluwer Academic Publishers, Boston 81-106
31. Taff L.M., Borchering J.W., et al. Estimeetings: development estimates and a front-end process for a large project. IEEE Transactions on Software Engineering 17 8 (1991) 839-849