A review of studies on expert estimation of software development effort

Journal of Systems and Software

Volumn 70, Issue 1-2, 2004, Pages 37-60

  Jorgensen M ^a  

^a SIMULA RESEARCH LABORATORY   (Norway)

Author keywords

Effort estimation; Expert judgment; Project planning; Software development

Indexed keywords

COMPUTER SOFTWARE; DATA REDUCTION; ERROR ANALYSIS; PERSONNEL; PLANNING; PROGRAM DOCUMENTATION; PROJECT MANAGEMENT; SOCIETIES AND INSTITUTIONS;

ESTIMATION TRAINING; EXPERT ESTIMATION; PROJECT PLANNING;

SOFTWARE ENGINEERING;

EID: 0141838535     PISSN: 01641212     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0164-1212(02)00156-5     Document Type: Article

References (191)

1. Abdel-Hamid T.K., Madnik S.E. The dynamics of software project scheduling. Communications of the ACM. 26(5):1983;340-346.
2. Abdel-Hamid T.K., Sengupta K., Ronan D. Software project control: an experimental investigation of judgment with fallible information. IEEE Transactions on Software Engineering. 19(6):1993;603-612.
3. Abdel-Hamid T.K., Sengupta K., Swett C. The impact of goals on software project management: an experimental investigation. MIS Quarterly. 23(4):1999;531-555.
4. Alpert M., Raiffa H. A progress report on the training of probability assessors. Tversky A. Judgment under Uncertainty: Heuristics and Biases. 1982;294-305 Cambridge University Press, Cambridge.
5. Arkes H.R. Overconfidence in judgmental forecasting. Armstrong J.S. Principles of Forecasting: A Handbook for Researchers and Practitioners. 2001;495-515 Kluwer Academic Publishers, Boston.
6. Armstrong J.S. Combining forecasts. Armstrong J.S. Principles of Forecasting: A Handbook for Researchers and Practitioners. 2001;417-440 Kluwer Academic Publishers, Boston.
7. Armstrong J.S. The forecasting dictionary. Armstrong J.S. Principles of Forecasting: A Handbook for Researchers and Practitioners. 2001;761-824 Kluwer Academic Publishers, Boston.
8. Armstrong J.S. Selecting forecasting methods. Armstrong J.S. Principles of Forecasting: A handbook for Researchers and Practitioners. 2001;365-386 Kluwer Academic Publishers, Boston.
9. Armstrong J.S. Standards and practices for forecasting. Armstrong J.S. Principles of Forecasting: A Handbook for Researchers and Practitioners. 2001;679-732 Kluwer Academic Publishers, Boston.
10. Armstrong J.S., Denniston W.B. Jr., Gordon M.M. The use of the decomposition principle in making judgments. Organizational Behavior and Human Decision Processes. 14(2):1975;257-263.
11. Ashton R.H. Combining the judgments of experts: How many and which ones? Organizational Behaviour and Human Decision Processes. 38(3):1986;405-414.
12. Atkinson, K., Shepperd, M., 1994. Using function points to find cost analogies. European Software Cost Modelling Meeting, Ivrea, Italy.
13. Ayton A. How bad is human judgment? Wright G., Goodwin P. Forecasting with Judgment. 1998;237-268 Johh Wiley & Sons, New York.
14. Balzer W.K., Doherty M.E., O'Connor R.J. Effects of cognitive feedback on performance. Psychological Bulletin. 106(3):1989;410-433.
15. Basili V., Caldierea H., Rombach D. The Experience Factory. Marciniak J.J. Encyclopedia of Software Engineering. 1994;469-476 Wiley, New York.
16. Benson P.G. The effects of feedback and training on the performance of probability forecasters. International Journal of Forecasting. 8(4):1992;559-573.
17. Betteridge R. Successful experience of using function points to estimate project costs early in the life-cycle. Information and Software Technology. 34(10):1992;655-658.
18. Birk A., Dingsøyr T., Stalhane T. Postmortem: Never leave a project without it. IEEE Software. 19(3):2002;43-45.
19. Blattberg R.C., Hoch S.J. Database models and managerial intuition: 50%. model+50% manager Management Science. 36:1990;887-899.
20. Blocher E., Bouwman M.J., Daves C.E. Learning from experience in performing analytical procedures. Training Research Journal. 3:1997;59-79.
21. Boehm B.W. Software Engineering Economics. 1981;Prentice-Hall, New Jersey.
22. Boehm B.W. Software engineering economics. IEEE Transactions on Software Engineering. 10(1):1984;4-21.
23. Boehm B., Sullivan K. Software economics: status and prospects. Information and Software Technology. 41:1999;937-946.
24. Boehm B., Abts C., Chulani S. Software development cost estimation approaches - a survey. Annals of Software Engineering. 10:2000;177-205.
25. Bolger F., Wright G. Assessing the quality of expert judgment: Issues and analysis. Decision Support Systems. 11(1):1994;1-24.
26. Bongaarts J., Bulatao R.A. Beyond Six Billion: Forecasting the World's Population. 2000;National Academy Press, Washington, DC.
27. Bowden P., Hargreaves M., Langensiepen C.S. Estimation support by lexical analysis of requirements documents. Journal of Systems and Software. 51(2):2000;87-98.
28. Braun P.A., Yaniv I. A case study of expert judgment: economists' probabilities versus base-rate model forecasts. Journal of Behavioral Decision Making. 5(3):1992;217-231.
29. Brehmer B. In one word: not from experience. Acta Psychologica. 45:1980;223-241.
30. Brenner L.A., Koehler D.J., Tversky A. On the evaluation of one-sided evidence. Journal of Behavioral Decision Making. 9(1):1996;59-70.
31. Briand L.C., El Emam K., Surmann D., Wieczorek I., Maxwell K.D. An assessment and comparison of common software cost estimation modeling techniques. International Conference on Software Engineering, Los Angeles, USA. 1999;313-323 ACM, New York.
32. Briand L.C., Langley T., Wieczorek I. A replicated assessment and comparison of common software cost modeling techniques. International Conference on Software Engineering, Limerick, Ireland. 2000;377-386 ACM, New York.
33. Briand L.C., Wieczorek I. Resource estimation in software engineering. Marcinak J.J. Encyclopedia of Software Engineering. 2002;John Wiley & Sons, New York.
34. Buehler R., Griffin D., MacDonald H. The role of motivated reasoning in optimistic time predictions. Personality and Social Psychology Bulletin. 23(3):1997;238-247.
35. Buehler R., Griffin D., Ross M. Exploring the "Planning fallacy": Why people underestimate their task completion times. Journal of Personality and Social Psychology. 67(3):1994;366-381.
36. Camerer C.F., Johnson E.J. The process-performance paradox in expert judgment: How can experts know so much and predict so badly? Ericsson K.A., Smith J. Towards a General Theory of Expertise. 1991;195-217 Cambridge University Press.
37. Chatterjee S., Chatterjee S. On combining expert opinion. American Journal of Mathematical and Management Sciences. 7(3 & 4):1987;271-295.
38. Christensen R. Analysis of variance, design and regression. Applied statistical methods. 1998;Chapman & Hall/CRC.
39. Chulani S., Boehm B., Steece B. Bayesian analysis of empirical software engineering cost models. IEEE Transactions on Software Engineering. 25(4):1999;573-583.
40. Clemen R.T. Combining forecasts: a review and annotated bibliography. International Journal of Forecasting. 5(4):1989;559-583.
41. Connolly T., Dean D. Decomposed versus holistic estimates of effort required for software writing tasks. Management Science. 43(7):1997;1029-1045.
42. Cosier R.A., Rose G.L. Cognitive conflict and goal conflict effects on task performance. Organizational Behaviour and Human Performance. 19(2):1977;378-391.
43. Cuelenaere A.M.E., Genuchten M.J.I.M., Heemstra F.J. Calibrating a software cost estimation model: why and how. Information and Software Technology. 29(10):1987;558-567.
44. Dawes R.M. Proper and improper linear models. International Journal of Forecasting. 2:1986;5-14.
45. Dolado J.J. On the problem of the software cost function. Information and Software Technology. 43(1):2001;61-72.
46. Edwards J.S., Moores T.T. A conflict between the use of estimating and planning tools in the management of information systems. European Journal of Information Systems. 3(2):1994;139-147.
47. Einhorn H.J., Hogarth R.M. Confidence in judgment: persistence of the illusion of validity. Psychological Review. 85(5):1978;395-416.
48. Engelkamp S., Hartkopf S., Brössler P. Project experience database: a report based on first practical experience. PROFES, Oulu, Finland. 2000;204-215 Springer-Verlag, Berlin.
49. Ericsson K.A., Lehmann A.C. Expert and exceptional performance: Evidence of maximal adaptation to task constraints. Annual Review of Psychology. 47:1996;273-305.
50. Ettenson R., Shanteau J., Krogstad J. Expert judgment: Is more information better. Psychological Reports. 60(1):1987;227-238.
51. Fairley, R.E., 1992. Recent advantages in software estimation techniques. In: International Conference on Software Engineering, Melbourne, Australia, pp. 382-391.
52. Fischer I., Harvey N. Combining forecasts: what information do judges need to outperform the simple average. International Journal of Forecasting. 15(3):1999;227-246.
53. Fischhof B. Hindsight. <> foresight: the effect of outcome knowledge on judgement under uncertainty Journal of Experimental Psychology: Human Perception and Performance. 1:1975;288-299.
54. Fischhoff B., Slovic P., Lichtenstein S. Fault trees: sensitivity of estimated failure probabilities to problem representation. Journal of Experimental Psychology: Human Perception and Performance. 4(2):1978;330-334.
55. Fisher G.W. When oracles fail - a comparison of four procedures for aggregating subjective probability forecasts. Organizational Behaviour and Human Performance. 28(1):1981;96-110.
56. Getty D.J., Pickett R.M., D'Orsi S.J., Swets J.A. Enhanced interpretation of diagnostic images. Investigative Radiology. 23:1988;244-252.
57. Gigerenzer G., Todd P.M. Simple Heuristics that Make us Smart. 1999;Oxford University Press, New York.
58. Goodman P.A. Application of cost-estimation techniques: industrial perspective. Information and Software Technology. 34(6):1992;379-382.
59. Goodwin P. Enhancing judgmental sales forecasting: the role of laboratory research. Wright G., Goodwin P. Forecasting with Judgment. 1998;91-112 John Wiley & Sons, New York.
60. Goodwin P., Wright G. Improving judgmental time series forecasting: a review of the guidance provided by research. International Journal of Forecasting. 9:1990;147-161.
61. Griffin D., Buehler R. Frequency, probability, and prediction: easy solutions to cognitive illusions? Cognitive Psychology. 38(1):1999;48-78.
62. Hagafors R., Brehmer B. Does having to justify one's judgments change nature of the judgment process? Organizational Behaviour and Human Decision Processes. 31(2):1983;223-232.
63. Hammond K.R. Human judgement and social policy: irreducible uncertainty, inevitable error, unavoidable injustice. 1996;Oxford University Press, New York.
64. Hammond K.R., Hamm R.M., Grassia J., Pearson T. Direct comparison of the efficacy of intuitive and analytical cognition in expert judgment. IEEE Transactions on Systems, Man, and Cybernetics. 17(5):1987;753-770.
65. Harvey N. Improving judgment in forecasting. Armstrong J.S. Principles of Forecasting: A Handbook for Researchers and Practitioners. 2001;59-80 Kluwer Academic Publishers, Boston.
66. Heemstra F.J. Software cost estimation. Information and Software Technology. 34(10):1992;627-639.
67. Heemstra F.J., Kusters R.J. Function point analysis: evaluation of a software cost estimation model. European Journal of Information Systems. 1(4):1991;223-237.
68. Henry R.A. Improving group judgment accuracy: information sharing and determining the best member. Organizational Behaviour and Human Decision Processes. 62:1995;190-197.
69. Hihn J., Habib-Agahi H. Cost estimation of software intensive projects: a survey of current practices. International Conference on Software Engineering. 1991;276-287 IEEE Computer Society Press, Los Alamitos, CA, USA.
70. Hihn, J., Habib-Agahi, H., 1991b. Cost estimation of software intensive projects: a survey of current practices. In: International Conference on Software Engineering, pp. 276-287.
71. Hill J., Thomas L.C., Allen D.E. Experts' estimates of task durations in software development projects. International Journal of Project Management. 18(1):2000;13-21.
72. Hoch S.J., Schkade D.A. A psychological approach to decision support systems. Management Science. 42(1):1996;51-64.
73. Hogarth R.M. A note on aggregating opinions. Organizational Behaviour and Human Performance. 21(1):1978;40-46.
74. Houdek, F.S.K., W.E, 1998. Establishing experience factories at Daimler-Benz. An experience report. In: International Conference on Software Engineering, Kyoto, Japan, pp. 443-447.
75. Hughes R.T. Expert judgement as an estimating method. Information and Software Technology. 38(2):1996;67-75.
76. Höst M., Wohlin C. A subjective effort estimation experiment. Information and Software Technology. 39(11):1997;755-762.
77. Höst M., Wohlin C. An experimental study of individual subjective effort estimations and combinations of the estimates. International Conference on Software Engineering, Kyoto, Japan. 1998;332-339 IEEE Computer Society, Los Alamitos, CA, USA.
78. Jeffery D.R., Low G. Calibrating estimation tools for software development. Software Engineering Journal. 5(4):1990;215-221.
79. Jeffery D.R., Ruhe M., Wieczorek I. A comparative study of two software development cost modeling techniques using multi-organizational and company-specific data. Information and Software Technology. 42(14):2000;1009-1016.
80. Johnson E.J. Expertise and decision under uncertainty: performance and process. Chi M.T.H., Glaser R., Farr M.J. The Nature of Expertise. 1998;209-228 Lawrence Erlbaum, Hillsdale, NJ.
81. Josephs R., Hahn E.D. Bias and accuracy in estimates of task duration. Organizational Behaviour and Human Decision Processes. 61(2):1995;202-213.
82. Jørgensen M. The quality of questionnaire based software maintenance studies. ACM SIGSOFT - Software Engineering Notes. 20(1):1995;71-73.
83. Jørgensen, M., 1997. An empirical evaluation of the MkII FPA estimation model. In: Norwegian Informatics Conference, Voss, Norway, Tapir, Oslo, pp. 7-18.
84. Jørgensen M. Software Effort Estimation by Analogy and Regression Toward the Mean. Journal of Systems and Software. 68(3):2003;253-262.
85. Jørgensen, M., Moløkken, K., 2002. Combination of software development effort prediction intervals: Why, when and how? In: Fourteenth IEEE Conference on Software Engineering and Knowledge Engineering (SEKE'02), Ischia, Italy.
86. Jørgensen, M., Sjøberg, D., 2000. The importance of not learning from experience. In: European Software Process Improvement 2000 (EuroSPI'2000), Copenhagen, pp. 2.2-2.8.
87. Jørgensen M., Sjøberg D.I.K. Impact of effort estimates on software project work. Information and Software Technology. 43(15):2001;939-948.
88. Jørgensen M., Sjøberg D.I.K. Software process improvement and human judgement heuristics. Scandinavian Journal of Information Systems. 13:2001;99-121.
89. Jørgensen, M., Sjøberg, D.I.K., 2002a. The impact of customer expectation on software development effort estimates. International Journal of Project Management (submitted).
90. Jørgensen M., Sjøberg D.I.K. Impact of experience on maintenance skills. Journal of Software Maintenance and Evolution: Research and practice. 14(2):2002;123-146.
91. Jørgensen, M., Teigen, K.H., 2002. Uncertainty intervals versus interval uncertainty: an alternative method for eliciting effort prediction intervals in software development projects. In: Proceedings of the International conference on Project Management (ProMAC), Singapore, pp. 343-352.
92. Jørgensen, M., Sjøberg, D.I.K., Conradi, R., 1998. Reuse of software development experience at Telenor Telecom Software. In: European Software Process Improvement Conference (EuroSPI'98), Gothenburg, Sweden, pp. 10.19-10.31.
93. Jørgensen, M., Moen, L., Løvstad, N., 2002. Combining quantitative software development cost estimation precision data with qualitative data from Project Experience Reports at Ericsson Design Center in Norway. In: Proceedings of the Conference on Empirical Assessment in Software Engineering, Keele, England, Keele University.
94. Jørgensen, M., Teigen, K.H., Moløkken, K., 2002. Better sure than safe? Overconfidence in judgment based software development effort prediction intervals. Journal of Systems and Software (submitted).
95. Kahneman D., Lovallo D. Timid choices and bold forecasts: a cognitive perspective on risk taking. Management Science. 39(1):1993;17-31.
96. Kahneman D., Slovic P., Tversky A. Judgment under Uncertainty: Heuristics and biases. 1982;Cambridge University Press, Cambridge, UK.
97. Kahneman D., Tversky A. On the psychology of prediction. Psychological Review. 80(4):1973;237-251.
98. Kahneman D., Tversky A. Intuitive predictions: biases and corrective procedures. TIMS Studies in Management Science. 12:1979;313-327.
99. Keen P.G.W. Information systems and organizational change. Social Impacts of Computing. 24(1):1981;24-33.
100. Keren G., Teigen K.H. Why is. p=0.90 better than p=0.70 ? preference for definitive predictions by lay consumers of probability judgments Psychonomic Bulletin and Reviews. 8(2):2001;191-202.
101. Kernaghan J.A., Cooke R.A. The contribution of the group process to successful project planning in R & D settings. IEEE Transactions on Engineering Management. 33(3):1986;134-140.
102. Kernaghan J.A., Cooke R.A. Teamwork in planning innovative projects: improving group performance by rational and interpersonal interventions in group process. IEEE Transactions on Engineering Management. 37(2):1990;109-116.
103. Kitchenham B., Pfleeger S.L., McColl B., Eagan S. A case study of maintenance estimation accuracy. Journal of Systems and Software. 64(1):2002;57-77.
104. Klayman J., Soll J.B., Gonzalez V.C., Barlas S. Overconfidence: it depends on how, what and whom you ask. Organizational Behaviour and Human Decision Processes. 79(3):1999;216-247.
105. Klein W.M., Kunda Z. Exaggerated self-assessments and the preference for controllable risks. Organizational Behavior and Human Decision Processes. 59(3):1994;410-427.
106. Koehler D.J. Explanation, imagination, and confidence in judgment. Psychological Bulletin. 110(3):1991;499-519.
107. Koehler D.J., Harvey N. Confidence judgments by actors and observers. Journal of Behavioral Decision Making. 10(3):1997;221-242.
108. Koriat A., Lichtenstein S., Fischhoff B. Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory. 6(2):1980;107-118.
109. Kruger J., Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology. 77(6):1999;1121-1134.
110. Kusters R.J. Are software cost-estimation models accurate? Information and software technology. 32:1990;187-190.
111. Kusters R.J., Genuchten M.J.I.M., Heemstra F.J. Are software cost-estimation models accurate? Information and Software Technology. 32(3):1990;187-190.
112. Lawrence M., O'Connor M. Judgement or models: the importance of task differences. Omega, International Journal of Management Science. 24(3):1996;245-254.
113. Lederer A.L., Mirani R., Neo B.S., Pollard C., Prasad J., Ramamurthy K. Information system cost estimating: a management perspective. MIS Quarterly. 14(2):1990;159-176.
114. Lederer A.L., Prasad J. Nine management guidelines for better cost estimating. Communications of the ACM. 35(2):1992;51-59.
115. Lederer A.L., Prasad J. Information systems software cost estimating: a current assessment. Journal of Information Technology. 8(1):1993;22-33.
116. Lederer A.L., Prasad J. A causal model for software cost estimating error. IEEE Transactions on Software Engineering. 24(2):1998;137-148.
117. Lederer A.L., Prasad J. Software management and cost estimating error. Journal of Systems and Software. 50(1):2000;33-42.
118. Libby R., Blashfield R.K. Performance of a composite as a function of the number of judges. Organizational Behaviour and Human Performance. 21(2):1978;121-129.
119. Lichtenstein S., Fischhoff B. Do those who know more also know more about how much they know. Organizational Behaviour and Human Decision Processes. 20(2):1977;159-183.
120. Lichtenstein S., Fischhoff B., Phillips L.D. Calibration of probabilities: the state of the art to 1980. Tversky A. Judgment under Uncertainty: Heuristics and Biases. 1982;Cambridge University Press, Cambridge.
121. Lim J.S., O'Connor M. Judgmental forecasting with time series and causal information. International Journal of Forecasting. 12(1):1996;139-153.
122. Londeix B. Deploying realistic estimation (field situation analysis). Information and Software Technology. 37(12):1995;655-670.
123. MacGregor D.G. Decomposition for judgmental forecasting and estimation. Armstrong J.S. Principles of Forecasting: A Handbook for Researchers and Practitioners. 2001;107-123 Kluwer Academic Publishers, Boston.
124. MacGregor D.G., Lichtenstein S. Problem structuring aids for quantitative estimation. Journal of Behavioral Decision Making. 4(2):1991;101-116.
125. Maines L.A. An experimental examination of subjective forecast combination. International Journal of Forecasting. 12(2):1996;223-233.
126. Makridakis S., Wheelwright S.C., Hyndman R.J. Forecasting: Methods and Applications. 1998;John Wiley & Sons, New York.
127. Marouane R., Mili A. Economics of software project management in Tunisia: basic TUCOMO. Information and Software Technology. 31(5):1989;251-257.
128. Marwane R., Mili A. Building tailor-made software cost model: Intermediate TUCOMO. Information and Software Technology. 33(3):1991;232-238.
129. McClelland A.G.R., Bolger F. The calibration of subjective probabilities: theories and models 1980-94. Ayton P. Subjective Probability. 1994;John Wiley, Chichester.
130. McConnel S. Software Project Survival Guide. 1998;Microsoft Press.
131. Meehl P.E. When shall we use our heads instead of the formula? Journal of Counseling Psychology. 4(4):1957;268-273.
132. Mendes E., Counsell S., Mosley N. Measurement and effort prediction for Web applications. 2001;Springer-Verlag, Berlin, Germany.
133. Meyer M.A., Booker J.M. Eliciting and Analyzing Expert Judgment: A Practical Guide. 1991;SIAM, Philadelphia, Pennsylvania.
134. Mizuno O., Kikuno T., Inagaki K., Takagi Y., Sakamoto K. Statistical analysis of deviation of actual cost from estimated cost using actual project data. Information and Software Technology. 42:2000;465-473.
135. Mohanty S.N. Software cost estimation: Present and future. Software - Practice and Experience. 11(2):1981;103-121.
136. Moløkken, K., 2002. Expert estimation of Web-development effort: individual biases and group processes (Master Thesis). Department of Informatics, University of Oslo.
137. Mukhopadhyay T., Vicinanza S.S., Prietula M.J. Examining the feasibility of a case-based reasoning model for software effort estimation. MIS Quarterly. 16(2):1992;155-171.
138. Murali C.S., Sankar C.S. Issues in estimating real-time data communications software projects. Information and Software Technology. 39(6):1997;399-402.
139. Myrtveit I., Stensrud E. A controlled experiment to assess the benefits of estimating with analogy and regression models. IEEE Transactions on Software Engineering. 25:1999;510-525.
140. NASA, 1990. Manager's handbook for software development. Goddard Space Flight Center, Greenbelt, MD, NASA Software Engineering Laboratory.
141. Newby-Clark I.R., Ross M., Buehler R., Koehler D.J., Griffin D. People focus on optimistic scenarios and disregard pessimistic scenarios when predicting task completion times. Journal of Experimental Psychology: Applied. 6(3):2000;171-182.
142. Niessink F., van Vliet H. Predicting maintenance effort with function points. International Conference on Software Maintenance, Bari, Italy. 1997;32-39 IEEE Computer Society, Los Alamitos, CA, USA.
143. Nisbett R.E., Ross L. Human inference: Strategies and shortcomings of social judgment. 1980;Prentice-Hall, Englewood Cliffs, NJ.
144. O'Connor M., Remus W., Griggs K. Judgmental forecasting in times of change. International Journal of Forecasting. 9(2):1993;163-172.
145. Ohlsson N., Wohlin C., Regnell B. A project effort estimation study. Information and Software Technology. 40(14):1998;831-839.
146. Ordonez L., Benson L. III Decisions under time pressure: how time constraint affects risky decision making. Organizational Behaviour and Human Decision Processes. 71(2):1997;121-140.
147. Park R.E. A manager's checklist for validating software cost and schedule estimates. American Programmer. 9(6):1996;30-35.
148. Paynter J. Project estimation using screenflow engineering. International Conference on Software Engineering: Education and Practice, Dunedin, New Zealand. 1996;150-159 IEEE Computer Society Press, Los Alamitos, CA, USA.
149. Pelham B.W., Neter E. The effect of motivation of judgment depends on the difficulty of the judgment. Journal of Personality and Social Psychology. 68(4):1995;581-594.
150. Pengelly A. Performance of effort estimating techniques in current development environments. Software Engineering Journal. 10(5):1995;162-170.
151. Reagan-Cirincione P. Improving the accuracy of group judgment: a process intervention combining group facilitation, social judgment analysis, and information technology. Organizational Behaviour and Human Decision Processes. 58(2):1994;246-270.
152. Remus W., O'Connor M., Griggs K. Does reliable information improve the accuracy of judgmental forecasts? International Journal of Forecasting. 11(2):1995;285-293.
153. Ringuest J.L., Tang K. Simple rules for combining forecasts: Some empirical results. Socio-Economical Planning Science. 21(4):1987;239-243.
154. Roth P.L. Research trends in judgment and their implications for the Schmidt-Hunter global estimation procedure. Organizational Behaviour and Human Decision Processes. 54(2):1993;299-319.
155. Rowe G., Wright G. Expert opinions in forecasting: The role of the Delphi process. Armstrong J.S. Principles of Forecasting: A Handbook for Researchers and Practitioners. 2001;125-144 Kluwer Academic Publishers, Boston.
156. Sanbonmatsu D.M., Sharon A.A., Biggs E. Overestimating causality: Attributional effects of confirmatory processing. Journal of Personality and Social Psychology. 65(5):1993;892-903.
157. Sanders D.E., Ritzman L.P. On knowing when to switch from quantitative to judgemental forecasts. International Journal of Forecasting. 11(6):1991;27-37.
158. Sanders G.S. Self-presentation and drive in social facilitation. Journal of Experimental Social Psychology. 20(4):1984;312-322.
159. Sanders N.R., Ritzman L.P. Judgmental adjustment of statistical forecasts. Armstrong J.S. Principles of Forecasting: A Handbook for Researchers and Practitioners. 2001;405-416 Kluwer Academic Publishers, Boston.
160. Schmitt N., Coyle B.W., King L. Feedback and task predictability as determinants of performance in multiple cue probability learning tasks. Organizational Behaviour and Human decision processes. 16(2):1976;388-402.
161. Seaver D.A., Winterfeldt D., Edwards W. Eliciting subjective probability distributions on continuous variables. Organizational Behaviour and Human Decision Processes. 21(3):1978;379-391.
162. Shanteau J. Competence in experts: the role of task characteristics. Organizational Behaviour and Human Decision Processes. 53(2):1992;252-266.
163. Shepperd J.A., Fernandez J.K., Quellette J.A. Abandoning unrealistic optimism: performance estimates and the temporal proximity of self-relevant feedback. Journal of Personality and Social Psychology. 70(4):1996;844-855.
164. Sieber J.E. Effects of decision importance on ability to generate warranted subjective uncertainty. Journal of Personality and Social Psychology. 30(5):1974;688-694.
165. Simon H.A. Making management decisions: the role of intuition and emotion. Academy of Management Executive. 1:1987;57-63.
166. Skitmore R.M., Stradling S.G., Tuohy A.P. Human effects in early stage construction contract price forecasting. IEEE Transactions on Engineering Management. 41(1):1994;29-40.
167. Soll J.B. Determinants of overconfidence and miscalibration: the roles of random error and ecological structure. Organizational Behaviour and Human Decision Processes. 65(2):1996;117-137.
168. Stahlberg D., Eller F., Maass A., Frey D. We knew it all along: hindsight bias in groups. Organizational Behaviour and Human Decision Processes. 63(1):1995;46-58.
169. Stone E.R., Opel R.B. Training to improve calibration and discrimination: the effects of performance and environmental feedback. Organizational Behaviour and Human Decision Processes. 83(2):2000;282-309.
170. Subbotin V. Outcome feedback effects on under- and overconfident judgments (general knowledge tasks). Organizational Behaviour and Human Decision Processes. 66(3):1996;268-276.
171. Taff L.M., Borchering J.W., Hudgins J.W.R. Estimeetings: development estimates and a front-end process for a large project. IEEE Transactions on Software Engineering. 17(8):1991;839-849.
172. Tan H.-T., Lipe M.G. Outcome effects: the impact of decision process and outcome controllability. Journal of Behavioral Decision Making. 10(4):1997;315-325.
173. Tausworthe R.C. The work breakdown structure in software project management. Journal of Systems and Software. 1(3):1980;181-186.
174. Thomsett R. Double Dummy Spit and other estimating games. American Programmer. 9(6):1996;16-22.
175. Todd P., Benbasat I. Inducing compensatory information processing through decision aids that facilitate effort reduction: an experimental assessment. Journal of Behavioral Decision Making. 13(1):2000;91-106.
176. Tversky A., Kahneman D. Judgment under uncertainty: Heuristics and biases. Science. 185:1974;1124-1131.
177. van Genuchten M., Koolen H. On the use of software cost models. Information and Management. 21:1991;37-44.
178. Verner J.M., Overmyer S.P., McCain K.W. In the 25 years since The Mythical Man-Month what have we learned about project management. Information and Software Technology. 41:1999;1021-1026.
179. Vicinanza S.S., Mukhopadhyay T., Prietula M.J. Software effort estimation: an exploratory study of expert performance. Information Systems Research. 2(4):1991;243-262.
180. Walkerden F., Jeffery D.R. Software cost estimation: a review of models, process, and practice. Advances in Computers. 44:1997;59-125.
181. Walkerden F., Jeffery R. An empirical study of analogy-based software effort estimation. Journal of Empirical Software Engineering. 4(2):1999;135-158.
182. Webby R.G., O'Connor M.J. Judgemental and statistical time series forecasting: a review of the literature. International Journal of Forecasting. 12(1):1996;91-118.
183. Weinberg G.M., Schulman E.L. Goals and performance in computer programming. Human Factors. 16(1):1974;70-77.
184. Whitecotton S.M., Sanders D.E., Norris K.B. Improving predictive accuracy with a combination of human intuition and mechanical decision aids. Organizational Behaviour and Human Decision Processes. 76(3):1998;325-348.
185. Winklhofer H., Diamantopoulos A., Witt S.F. Forecasting practice: a review of the empirical literature and an agenda for future research. International Journal of Forecasting. 12(2):1996;193-221.
186. Wolverton R.W. The cost of developing large-scale software. IEEE Transactions on Software Engineering. C-23(6):1974;615-636.
187. Wright G., Ayton P. Subjective probability. 1994;John Wiley, West Sussex, England.
188. Yaniv I., Foster D.P. Precision and accuracy of judgmental estimation. Journal of Behavioral Decision Making. 10:1997;21-32.
189. Yau C., Gan L.-Y. Comparing the top-down and bottom-up approaches of function point analysis: a case study. Software Quality Journal. 4(3):1995;175-187.
190. Zajonc R.B. Social facilitation. Science. 149:1965;269-274.
191. Zajonc R.B., Heingarner A., Herman E.M. Social enhancement and impairment of performance in the cockroach. Journal of Personality and Social Psychology. 13(2):1969;83-92.