Quantifying IT forecast quality

Science of Computer Programming

Volumn 74, Issue 11-12, 2009, Pages 934-988

  Eveleens, J L ^a   Verhoef, C ^a  

^a VU UNIVERSITY AMSTERDAM   (Netherlands)

Author keywords

Cone of uncertainty; Estimating Quality Factor; Estimation; Forecast; Forecast to actual ratio; IT portfolio management; Reference cone

Indexed keywords

ESTIMATING QUALITY FACTOR; FORECAST; FORECAST-TO-ACTUAL RATIO; IT PORTFOLIO MANAGEMENT; REFERENCE CONE;

DECISION MAKING; ESTIMATION; FINANCIAL DATA PROCESSING; INFORMATION TECHNOLOGY; INVESTMENTS; Q FACTOR MEASUREMENT; UNCERTAINTY ANALYSIS;

FORECASTING;

EID: 70449095884     PISSN: 01676423     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.scico.2009.09.005     Document Type: Article

References (77)

1. Arkes H.R. Overconfidence in judgmental forecasting. In: Armstrong J.S. (Ed). Principles of Forecasting: A Handbook for Researchers and Practitioners (2001), Springer 495-515
2. Armour P.G. The inaccurate conception. Communications of the ACM 51 3 (2008) 13-16
3. Armstrong J.S. The forecasting dictionary. In: Armstrong J.S. (Ed). Principles of Forecasting: A Handbook for Researchers and Practitioners (2001), Springer 761-819
4. Augustine N.R. Augustine's laws and major system development programs. Defense Systems Management Review 2 (1979) 50-76
5. Bergeron F., and St-Arnaud J.-Y. Estimation of information systems development efforts. Information & Management 22 (1992) 239-254
6. Boehm B. Software Engineering Economics (1981), Prentice Hall PTR
7. Boehm B. Making a difference in the software century. Computer 41 3 (2008) 32-38
8. Boehm B., Clark B., Horowitz E., and Westland C. Cost models for future software life cycle processes: COCOMO 2.0. Annals of Software Engineering 1 (1995)
9. Lionel Briand, Khaled El Emam, Dagmar Surmann, Isabella Wieczorek, Katrina D. Maxwell, An assessment and comparison of common software cost estimation modeling techniques, in: Proceedings of the 21st International Conference on Software Engineering, 1999, pp. 313-322
10. Murray Cantor, Estimation variance and governance, 2006. www-128.ibm.com/developerworks/rational/library/mar06/cantor/
11. Chatfield C. Prediction intervals for time-series forecasting. In: Armstrong J.S. (Ed). Principles of Forecasting: A Handbook for Researchers and Practitioners (2001), Springer 475-494
12. Cohn M. Agile Estimating and Planning (2005), Prentice Hall
13. Conte S.D., Dunsmore H.E., and Shen V.Y. Software Engineering Metrics and Models (1986), The Benjamin/Cummings Publishing Company
14. DeMarco T. Controlling Software Projects (1982), Prentice Hall PTR
15. DeMarco T., and Lister T. Waltzing with Bears (2003), Dorset House Publishing
16. Dreger J.B. Function Point Analysis (1989), Prentice Hall
17. J.L. Eveleens, P. Kampstra, C. Verhoef, Quantifying changes in IT metrics after outsourcing transitions. Under review. Available via www.cs.vu.nl/~x/sps/sps.pdf
18. J.L. Eveleens, C. Verhoef, The rise and fall of the Chaos report figures, IEEE Software, in press (doi:10.1109/MS.2009.154). Available via www.cs.vu.nl/~x/chaos/chaos.pdf
19. Fairley D. Making accurate estimates. IEEE Software 19 6 (2002) 61-63
20. Feller W. On the Kolmogorov-Smirnov limit theorems for empirical distributions. The Annals of Mathematical Statistics 19 2 (1948) 177-189
21. Fisz M. Probability Theory and Mathematical Statistics (1963), John Wiley & Sons
22. Flyvbjerg B., Bruzelius N., and Rothengatter W. Megaprojects and Risk: An Anatomy of Ambition (2003), Cambridge University Press
23. Garmus D., and Herron D. Function Point Analysis-Measurement Practices for Successful Software Projects (2001), Addison-Wesley
24. Glass R. IT failure rates-70% or 10-15%. IEEE Software May (2005) 110-112
25. Glass R. The Standish report: Does it really describe a software crisis. Communications of the ACM 49 8 (2006)
26. Gryphon S., Kruchten P., McConnell S., and Little T. The Cone of Uncertainty. IEEE Software 23 5 (2006) 8-10
27. Deborah Hartmann, Interview: Jim Johnson of the Standish Group, 2006. http://www.infoq.com/articles/Interview-Johnson-Standish-CHAOS
28. Harvey N. Improving judgment in forecasting. In: Armstrong J.S. (Ed). Principles of Forecasting: A Handbook for Researchers and Practitioners (2001), Springer 59-80
29. The Standish Group International Inc. Chaos, Technical Report, The Standish Group International Inc., 1994
30. The Standish Group International Inc. Chaos: A recipe for success, Technical Report, The Standish Group International Inc., 1999
31. The Standish Group International Inc. Extreme chaos, Technical Report, The Standish Group International Inc., 2001
32. Jenkins M., Naumann J., and Wetherbe J. Empirical investigation of systems development practices and results. Information & Management 7 (1984) 73-82
33. Jim Johnson, Standish: Why were project failures up and cost overruns down in 1998, 2006. http://www.infoq.com/articles/chaos-1998-failure-stats
34. Jørgensen M. Experience with the accuracy of software maintenance task effort prediction models. IEEE Transactions on Software Engineering 21 (1995) 674-681
35. Jørgensen M. A review of studies on expert estimation of software development effort. Journal of Systems and Software 70 (2004) 37-60
36. Jørgensen M. Practical guidelines for expert-judgment-based software effort estimation. IEEE Software 22 3 (2005) 57-63
37. Jørgensen M., and Moløkken K. How large are software cost overruns? A Review of the 1994 Chaos report. Information and Software Technology 48 (2006) 297-301
38. K. Kavoussanakis, Terry Sloan, UKHEC Report on software estimation, Technical Report, University of Edinburgh, 2001
39. Kemerer C. An empirical validation of software cost estimation models. Communications of the ACM 30 5 (1987) 416-429
40. Kitchenham B., MacDonell S., Pickard L., and Shepperd M. What accuracy statistics really measure. IEE Proceedings - Software 148 (2001) 81-85
41. Kulk G.P., and Verhoef C. Quantifying requirements volatility effects. Science of Computer Programming 72 (2008) 136-175
42. G.P. Kulk, C. Verhoef, Quantifying IT estimation risks (in press) Available via http://www.cs.vu.nl/~x/qier/qier.pdf, 2009
43. Laird L.M. The limitations of estimation. IT Professional 8 6 (2006) 40-45
44. Laird L.M., and Carol Brennan M. Software Measurement and Estimation: A Practical Approach (2006), John Wiley & Sons
45. Laranjeira L.A. Software size estimation of object-oriented systems. IEEE Transactions on Software Engineering 16 5 (1990) 510-522
46. Lederer A.L., Mirani R., Siong Neo B., Pollard C., Prasad J., and Ramamurthy K. Information system cost estimating: A management perspective. MIS Quarterly 14 2 (1990) 159-176
47. Tim Lister, Becoming a better estimator - An Introduction to using the EQF Metric, 2002. Available via http://www.stickyminds.com
48. Little T. Context-adaptive agility: Managing complexity and uncertainty. IEEE Software 22 3 (2005) 28-35
49. Little T. Schedule estimation and uncertainty surrounding the cone of uncertainty. IEEE Software 23 3 (2006) 48-54
50. Karen Lum, Michael Bramble, Jairus Hihn, John Hackney, Mori Khorrami, Erik Monson, Handbook for software cost estimation, Technical Report, Jet Propulsion Laboratory, 2003
51. Maplesoft. http://www.maplesoft.com/Products/Maple/
52. McConnell S. Rapid Development: Taming wild Software Schedules (1996), Microsoft Press
53. McConnell S. Software Estimation: Demystifying the black art (2006), Microsoft Press
54. M. Shahid Mujtaba, Robert Ritter, Enterprise modeling system: Inventory exposure and delivery performance, Technical Report, Hewlett-Packard company, 1994
55. Myrtveit I., and Stensrud E. A controlled experiment to assess the benefits of estimating with analogy and regression models. IEEE Transactions on Software Engineering 25 (1999) 510-525
56. Carlo Pescio, Realistic and useful: Toward better estimates, 2007. Available via http://www.eptacom.net/betterestimate/RealisticAndUseful.pdf
57. Phan D., Vogel D., and Nunamaker J. The search for perfect project management. Computerworld 26 (1988) 95-100
58. Putnam L.H., and Myers W. Measures for Excellence-Reliable Software on Time, Within Budget (1992), Yourdon Press Computing Series
59. Putnam L.H., and Fitzsimmons A. Estimating software costs. Datamation September-October-November (1979)
60. Putnam L.H. A macro-estimation methodology for software development. Proceedings IEEE COMPCON 76 Fall (1976), IEEE Computer Society Press 138-143
61. Putnam L.H. A general empirical solution to the macro software sizing and estimating problem. IEEE Transactions of Software Engineering 4 4 (1978) 345-381
62. R-project. http://www.r-project.org/
63. Sauer C., Gemino A., and Horner Reich B. The impact of size and volatility on IT project performance. Communications of the ACM 50 11 (2007) 79-84
64. Shelley C.C. Practical experience of implementing software measurement programmes in industry. Transactions on Information and Communications Technologies 4 (1993)
65. Shepperd M., and Schofield C. Estimating software project effort using analogies. IEEE Transactions of Software Engineering 23 11 (1997) 736-743
66. Michael Smithson, Confidence intervals, Sage University Papers Series on Quantitative Applications in Social Sciences, vol. 07-140, Thousand Oaks, 2003
67. Sommerville I. Software Engineering. 7th edition (2004), Pearson Educcation Limited
68. Stensrud E., Foss T., Kitchenham B., and Myrtveit I. A further empirical investigation of the relationship between MRE and project size. Empirical Software Engineering 8 (2003) 139-161
69. Tockey S. Return on Software (2005), Addison-Wesley
70. Oxford University Press. Concise oxford english dictionary Oxford University Press, 1999
71. van Vliet H. Software Engineering: Principles and Practice. 3rd edition (2008), John Wiley & Sons
72. Verhoef C. Quantifying the effects of IT-governance rules. Science of Computer Programming 67 2-3 (2007) 247-277. http://www.cs.vu.nl/~x/gov/gov.pdf Available via http://www.cs.vu.nl/~x/gov/gov.pdf
73. Frank Vogelezang, Scope management-how uncertainty is your certainty, in: International Workshop on Software Measurement, 2007
74. Weinberg G.M., and Schulman E.L. Goals and performance in computer programming. Human Factors 16 1 (1974) 70-77
75. Yang D., Boehm B., Yang Y., Wang Q., and Li M. Coping with the Cone of Uncertainty: An Empirical Study of the SAIV Process Model. Lecture Notes in Computer Science vol. 4470/2007 (2007), Springer, Berlin, Heidelberg
76. Yourdon. The ten most important ideas in software engineering, 2006. http://www.yourdonreport.com/index.php/2006/10/17/the-ten-most-important -ideas-in-software-engineering/
77. Zvegintzov N. Frequently begged questions and how to answer them. IEEE Software 20 2 (1998) 93-96