The role of outcome feedback in improving the uncertainty assessment of software development effort estimates

ACM Transactions on Software Engineering and Methodology

Volumn 17, Issue 4, 2008, Pages

  Gruschke, Tanja M ^a,b   Jørgensen, Magne ^a,b  

^a SIMULA RESEARCH LABORATORY   (Norway)

^b UNIVERSITY OF OSLO   (Norway)

Author keywords

Cost estimation; Effort prediction intervals; Judgment based uncertainty assessment; Overconfidence; Software cost estimation; Software development management

Indexed keywords

COST ESTIMATION; EFFORT PREDICTION INTERVALS; JUDGMENT-BASED UNCERTAINTY ASSESSMENT; OVERCONFIDENCE; SOFTWARE COST ESTIMATION; SOFTWARE DEVELOPMENT MANAGEMENT;

UNCERTAINTY ANALYSIS;

EID: 50649116352     PISSN: 1049331X     EISSN: 15577392     Source Type: Journal    
DOI: 10.1145/13487689.13487693     Document Type: Article

References (56)

1. BENGTSSON, C., PERSSON, M., AND WILLENHAG, P. 2005. Gender and overconfidence. Economics Lett. 86, 2 (Feb.), 199-203.
2. BOLGER, F. AND WRIGHT, G. 1994. Assessing the quality of expert judgment: Issues and analysis. Decision Support Syst. 11, 1, 1-24.
3. BRYAN, G. E. 1994. Not All Programmers Are Created Equal. In Proceedings of the IEEE Aerospace Applications Conference, 55-62.
4. BUEHLER, R. AND GRIFFIN, D. 2003. Planning, personality, and prediction: The role of future focus in optimistic time predictions. Organiza. Behav. Hum. Decision Proc. 92, 80-90.
5. BUEHLER, R., GRIFFIN, D., AND ROSS, M. 1994. Exploring the "Planning fallacy": Why people underestimate their task completion times. J. Personality Soc. Psych. 67, 3, 366-381.
6. BUEHLER, R., GRIFFIN, D., AND MACDONALD, H. 1997. The role of motivated reasoning in optimistic time predictions. Personality Soc. Psych. Bull. 23, 3, 238-247.
7. CONNOLLY, T. AND DEAN, D. 1997. Decomposed versus holistic estimates of effort required for software writing tasks. Manag. Sci. 43, 7, 1029-1045.
8. EINHORN, H. J. AND HOGARTH, R. M. 1978. Confidence in judgment: Persistence of the illusion of validity. Psych. Rev. 85, 5, 395-416.
9. ERICSON, K. A. AND KRAMPE, R. T. 1993. The role of deliberate practice in the acquisition of expert performance. Psych. Rev. 100, 3, 363-406.
10. GAGNE, R. M. AND MEDSKER, K. L. 1996. The Conditions of Learning: Training Applications. Harcourt Brace.
11. GIGERENZER, G. AND HOFFRAGE, U. 1995. How to improve Bayesian reasoning without instruction: Frequency formats. Psych. Rev. 102, 4, 684-704.
12. GRAY, A., MACDONELL, S. G., AND SHEPPERD, M. J. 1999. Factors systematically associated with errors in subjective estimates of software development effort: the stability of expert judgment. In Proceedings of the IEEE 6th International Metrics Symposium, IEEE Computer Society, 216-227.
13. GREEN, K. W. J., MEDLIN, B., AND WHITTEN, D. 2004. Developing optimism to improve performance: an approach for the manufacturing sector. Industrial Manag. Data Sys. 104, 2, 106-114.
14. HAMMOND, K. R. 1996. Human Judgement and Social Policy: Irreducible Uncertainty, Inevitable Error; Unavoidable Injustice. Oxford University Press, Oxford, UK.
15. HARRISON, W. 1997. Editorial. Empirical Softw. Engin. 2, 1, 7-10.
16. HEINE, S. J. AND LEHMAN, D. R. 1995. Cultural variation in unrealistic optimism: Does the West feel more invulnerable than the East? J. Personality Soc. Psych. 68, 4, 595-607.
17. HOGARTH, R. M. 2001. Educating Intuition. University of Chicago Press, Chicago.
18. HUGHES, R. T. 1996. Expert judgement as an estimating method. Inform. Softw. Tech. 38, 2, 67-75.
19. HUMPHREY, W. S. 1996. The PSP and personal project estimating. Amer. Program,. 9, 6, 2-15.
20. JOHNSON, D. D. P. 2004. Overconfidence and War. Harvard University Press, Cambridge, MA.
21. JØRGENSEN, M. 2004a. Top-down and bottom-up expert estimation of software development effort. Inform. Softw. Tech. 46, 1, 3-16.
22. JØRGENSEN, M. 2004b. A review of studies on expert estimation of software development effort. J. Syst. Softw. 70, 1-2, 37-60.
23. JØRGENSEN, M. 2004c. Realism in assessment of effort estimation uncertainty: It matters how you ask. IEEE Trans. Softw. Engin. 30, 4, 209-217.
24. JØRGENSEN, M. 2005. Evidence-Based Guidelines for assessment of software development cost uncertainty. IEEE Trans. Softw. Engin. 31, 11, 924-954.
25. JØRGENSEN, M. AND MOLØKKEN- ØSTVOLD, K. 2004a. Eliminating over-confidence in software development effort estimates. In Conference on Product Focused Software Process Improvement (PROFES) Springer-Verlag 174-184.
26. JØRGENSEN, M. AND MOLØKKEN- ØSTVOLD, K. 2004b. Reasons for software effort estimation error: impact of respondent role, information collection approach, and data analysis method. IEEE Trans. Softw. Engin. 30, 12, 993-1007.
27. JØRGENSEN, M. AND SJØBERG, D. I. K. 2002. Impact of experience on maintenance skills. J. Softw. Main. Evol. Res. Practice 14, 2, 123-146.
28. JØRGENSEN, M. AND SJØBERG, D. I. K. 2003a. An effort prediction interval approach based on the empirical distribution of previous estimation accuracy. Inform. Softw. Tech. 45, 3, 123-136.
29. JØRGENSEN, M. AND SJØBERG. D. 2003b. Generalization and theory-building in software engineering research. In Proceedings of Empirical Assessment in Software Engineering (EASE'04) 29-36.
30. JØRGENSEN, M., FAUGLI, B., AND GRUSCHKE, T. M. Characteristics of Software Engineers with Optimistic Predictions. J. Syst. Softw. To appear.
31. JØRGENSEN, M., INDAHL, U., AND SJØBERG, D. 2003. Software effort estimation by analogy and regression toward the mean. J. Syst. Softw. 68, 3, 253-262.
32. JØRGENSEN, M., TEIGEN, K. H., AND MOLØKKEN, K. 2004. Better sure than safe? Over-confidence in judgement based software development effort prediction intervals. J. Syst. Softw. 70, 1-2, 79-93.
33. KITCHENHAM, B., PFLEEGER, S. L., MCC OLL, B., AND EAGAN, S. 2002. An empirical study of maintenance and development estimation accuracy. J. Syst. Softw. 64, 1, 57-77.
34. KLAYMAN, J., SOLL, J. B., GONZÁLEZ- VALLEJO, C., AND BARLAS, S. 1999. Overconfidence: It depends on how, what and whom you ask. Organizat. Behau. Hum. Decision Proc. 79, 3, 216-247.
35. KRUGLANSKI, A. W. 1996. Motivated social cognition: Principles of the interface. In Social Psychology: Handbook of Basic Principles, E.T. Higgins and A. W. Kurglanski, Eds. Guilford Press, New York, 493-520.
36. LI, H., SHEN, Q. P., AND LOVE, P. E. D. 2005. Cost modelling of office buildings in Hong Kong: An exploratory study. Facilities 23, 9/10, 438-452.
37. LI, S. AND FANG, Y. 2004. Respondents in Asian cultures (e.g., Chinese) are more risk-seeking and more overconfident than respondents in other cultures (e.g., in United States) but the reciprocal predictions are in total opposition: how and why? J Cognition Culture 4, 2, 263-292.
38. LICHTENSTEIN, S. AND FISCHHOFF, B. 1977. Do those who know more also know more about how much they know? Organizat. Behav. Hum. Decision Proc. 20, 2, 159-183.
39. MANOVE, M. 1995. Entrepeneurs, optimism and the competitive edge. UFAE and IAE Working Papers 296.95,
40. MOLØKKEN, K. AND JØRGENSEN, M. 2003. A review of software surveys on software effort estimation. In Proceedings of the International Symposium, on Empirical Software Engineering. IEEE Computer Society, 223-230.
41. MOLØKKEN, K. AND JØRGENSEN, M. 2005. Expert estimation of Web-development projects: Are software professionals in technical roles more optimistic than those in non-technical roles? Empirical Softw. Engin. 10, 1, 7-30.
42. MOLØKKEN-ØSTVOLD, K. AND JØRGENSEN, M. 2004. Group processes in software effort estimation. Empirical Softw. Engin. 9, 4, 315-334.
43. OSKAMP, S. 1965. Overconfidence in case-study judgments. J. Consult. Psych. 29, 3, 261-265.
44. PIETRASANTA, A. M. 1968. Current methodological research. In Proceedings of the 23rd ACM National Conference, 341-346.
45. PRECHELT, L. AND UNGER, B. 2001. An experiment measuring the effects of personal software process (PSP) training. IEEE Trans. Softw. Engin. 27, 5, 465-472.
46. PUNCOCHAR, J. M. AND FOX, P. W. 2004. Confidence in individual and group decision making: When "two heads" are worse than one. J. Educat. Psych. 96, 3, 582-591.
47. ROZENBLIT, L. AND KEIL, F.C. 2002. The misunderstood limits of folk science: an illusion of explanatory depth. Cognitive Sci. 26, 5, 521-562.
48. SHADISH, W. R., COOK, T. D., AND CAMPBELL, D.T. 2002. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin.
49. SHENHAR, A. J. 2001. One size does not fit all projects: Exploring classical contingency domains. Manag. Sci. 47, 3, 394-414.
50. SHEPPERD, J. A., CARROLL, P., GRACE, J., AND TERRY, M. 2002. Exploring the causes of comparative optimism. Psychologica Belgica 42, 1/2, 65-98.
51. SHEPPERD, J. A., FERNANDEZ, J. K. AND QUELLETTE, J. A. 1996. Abandoning unrealistic optimism: Performance estimates and the temporal proximity of self-relevant feedback. J. Personality Social Psych. 70, 4, 844-855.
52. STEEN, v. D. E. 2004. Rational overoptimism (and other biases). Amer. Econ. Rev. 94, 4, 1141-1151.
53. WEBSTER JR, M. 1994. Experimental methods. In Group Processes, M. Foschi and E. J. Lawler, Eds., Nelson-Hall Publishers, 43-69.
54. WHITECOTTON, S. M., SANDERS, D. E., AND NORRIS, K. B. 1998. Improving predictive accuracy with a combination of human intuition and mechanical decision aids. Organizat. Behav. Hum. Decision Proces. 76, 3, 325-348.
55. WILSON, L. F. AND BREKKE, N. 1994. Mental contamination and mental correction: unwanted influence on judgments and evaluation. Psych. Bull. 116, 1, 117-142.
56. ZENDLER, A. E., HORN, E., SCHWARTZEL, H., AND PLODERERDER, E. 2001. Demonstrating the usage of single-case designs in experimental software engineering. Inform. Softw. Tech. 43, 12, 681-691.