A systematic literature review of software quality cost research

Journal of Systems and Software

Volumn 84, Issue 3, 2011, Pages 415-427

  Karg, Lars M ^a   Grottke, Michael ^a   Beckhaus, Arne ^b  

^a UNIVERSITY OF ERLANGEN NUREMBERG   (Germany)

^b UNIVERSITY OF FREIBURG   (Germany)

Author keywords

Prevention appraisal failure cost scheme; Quality costs; Software development; Systematic literature review

Indexed keywords

COMPUTING JOURNALS; COST IMPACT; ECONOMIC ASPECTS; FAILURE COST; FUTURE RESEARCH DIRECTIONS; PREVENTION COSTS; QUALITY COSTS; RESEARCH APPROACH; RESEARCH COMMUNITIES; RESEARCH TOPICS; SOFTWARE DEVELOPMENT; SOFTWARE QUALITY; SYSTEMATIC LITERATURE REVIEW;

COSTS; RESEARCH; SOFTWARE DESIGN;

COMPUTER SOFTWARE SELECTION AND EVALUATION;

EID: 78951481193     PISSN: 01641212     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jss.2010.11.904     Document Type: Article

References (117)

1. T.K. Abdel-Hamid The economics of software quality assurance: a simulation-based case study MIS Q. 12 3 1988 395 411
2. J. Ahire, R. Landeros, and D. Golhar Total quality management: a literature review and an agenda for future research Prod. Operations Manage. 4 3 1995 277 306
3. J. Antony, and C. Fergusson Six sigma in the software industry: results from a pilot study Manage. Audit. J. 19 2004 1025 1032
4. A. Arora, J.P. Caulkins, and R. Telang Research note: Sell first, fix later: impact of patching on software quality Manage. Sci. 52 3 2006 465 471
5. L.J. Arthur Quantum improvements in software system quality Commun. ACM 40 6 1997 46 52
6. R.D. Banker, S.M. Datar, C.F. Kemerer, and D. Zweig Software complexity and maintenance costs Commun. ACM 36 11 1993 81 94
7. R.D. Banker, and S.A. Slaughter A field study of scale economies in software maintenance Manage. Sci. 43 12 1997 1709 1725
8. S. Beecham, N. Baddoo, T. Hall, H. Robinson, and H. Sharp Motivation in software engineering: a systematic literature review Inf. Softw. Technol. 50 9-10 2008 860 878
9. S. Biffl, A. Aurum, B. Boehm, H. Erdogmus, P. Grünbacher, Value-Based Software Engineering 2006 Springer Berlin
10. S. Biffl, and M. Halling Investigating the defect detection effectiveness and cost benefit of nominal inspection teams IEEE Trans. Softw. Eng. 29 5 2003 385 397
11. D. Binkley Semantics guided regression test cost reduction IEEE Trans. Softw. Eng. 23 8 1997 498 516
12. B.W. Boehm Software Engineering Economics 1981 Prentice-Hall Englewood Cliffs
13. B.W. Boehm, and P.N. Papaccio Understanding and controlling software costs IEEE Trans. Softw. Eng. 14 10 1988 1462 1477
14. K.V. Bourgeois Process insights from a large-scale software inspection data analysis CrossTalk, J. Defense Softw. Eng. 9 10 1996 17 23
15. J. Brodman, and D. Johnson Return on investment from software process improvement as measured by U.S. industry CrossTalk, J. Defense Softw. Eng. 9 4 1996 23 28
16. D.B. Brown, S. Maghsoodloo, and W.H. Deason A cost model for determining the optimal number of software test cases IEEE Trans. Softw. Eng. 15 2 1989 218 221
17. F. Calzolari, P. Tonella, and G. Antoniol Maintenance and testing effort modeled by linear and nonlinear dynamic systems Inf. Softw. Technol. 43 8 2001 477 486
18. J.W. Cangussu, R.A. DeCarlo, and A.P. Mathur A formal model of the software test process IEEE Trans. Softw. Eng. 28 8 2002 782 796
19. T. Chavez A decision-analytic stopping rule for validation of commercial software systems IEEE Trans. Softw. Eng. 26 9 2000 907 918
20. J.S. Collofello, and S.N. Woodfield Evaluating the effectiveness of reliability-assurance techniques J. Syst. Softw. 9 3 1989 191 195
21. B.G. Dale Managing Quality 4th edition 2003 Wiley-Blackwell Oxford, UK
22. T. Daughtrey The search for software quality Quality Progress 21 11 1988 29 31
23. F. Deissenboeck, and M. Pizka Probabilistic analysis of process economics Softw. Process: Improve. Pract. 13 1 2008 5 17
24. M. Diaz, and J. Sligo How software process improvement helped Motorola IEEE Softw. 14 5 1997 75 81
25. R. Dion Process improvement and the corporate balance sheet IEEE Softw. 10 4 1993 28 35
26. H. Do, G. Rothermel, and A. Kinneer Prioritizing JUnit test cases: an empirical assessment and cost-benefits analysis Empirical Softw. Eng. 11 1 2006 33 70
27. C. Ebert, and R. Dumke Software Measurement: Establish-Extract-Evaluate- Execute 2010 Springer Berlin
28. W. Ehrlich, B. Prasanna, J. Stampfel, and J. Wu Determining the cost of a stop-testing decision IEEE Softw. 10 2 1993 33 42
29. S. Elbaum, G. Rothermel, S. Kanduri, and A.G. Malishevsky Selecting a cost-effective test case prioritization technique Softw. Quality J. 12 3 2004 185 210
30. M. Ellims, J. Bridges, and D.C. Ince The economics of unit testing Empirical Softw. Eng. 11 1 2006 5 31
31. A. Engel, and M. Last Modeling software testing costs and risks using fuzzy logic paradigm J. Syst. Softw. 80 6 2007 817 835
32. L.A. Franz, and J.C. Shih Estimating the value of inspections and early testing for software projects Hewlett-Packard J. 45 6 1994 60 67
33. B. Freimut, L.C. Briand, and F. Vollei Determining inspection cost-effectiveness by combining project data and expert opinion IEEE Trans. Softw. Eng. 31 12 2005 1074 1092
34. D. Galin Software Quality Assurance 2003 Pearson Harlow
35. D. Galin Toward an inclusive model for the costs of software quality Softw. Quality Professional 6 4 2004 25 31
36. R. Geist, M. Chetuparambil, S. Hedetniemi, and A.J. Turner Computing research programs in the U.S. Commun. ACM 39 12 1996 96 99
37. R.L. Glass, V. Ramesh, and I. Vessey An analysis of research in computing disciplines Commun. ACM 47 6 2004 89 94
38. R.B. Grady, and T. von Slack Key lessons in achieving widespread inspection use IEEE Softw. 11 4 1994 46 57
39. J.C. Granja-Alvarez, and M.J. Barranco-Garcia A method for estimating maintenance cost in a software project: A case study J. Softw. Maint.: Res. Pract. 9 3 1997 161 175
40. M. Grottke, and C. Graf Modeling and predicting software failure costs Proc. 33rd Annual IEEE International Computer Software and Applications Conference 2009 IEEE Computer Society Los Alamitos 180 189
41. W.J. Gutjahr Optimal test distributions for software failure cost estimation IEEE Trans. Softw. Eng. 21 3 1995 219 228
42. B. Hansen, J. Rose, and G. Tjornehoj Prescription, description, reflection: the shape of the software process improvement field Int. J. Inf. Manage. 24 6 2004 457 472
43. R. Hewett, and P. Kijsanayothin On modeling software defect repair time Empirical Softw. Eng. 14 2 2009 165 186
44. I.P. Hollingsworth, W. Keogh, and M.H. Atkins Applying quality costs in a software development environment Aust. J. Inf. Syst. 6 2 1999 64 75
45. C.T. Horngren, G. Foster, S.M. Datar, M. Rajan, and C. Ittner Cost Accounting - A Managerial Emphasis 13th edition 2008 Prentice-Hall Englewood Cliffs
46. R.-H. Hou, S.-Y. Kuo, and Y.-P. Chang Needed resources for software module test using the hyper-geometric software reliability growth model IEEE Trans. Rel. 45 4 1996 451 459
47. D. Houston, and J.B. Keats Cost of software quality: a means of promoting software process improvement Quality Eng. 10 3 1998 563 573
48. P. Hsia, C.-T. Hsu, D.C. Kung, and E.J. Byrne Incremental delivery reduces maintenance cost: a COCOMO-based study J. Softw. Maint.: Res. Pract. 10 4 1998 225 247
49. C.-Y. Huang Cost-reliability-optimal release policy for software reliability models incorporating improvements in testing efficiency J. Syst. Softw. 77 2 2006 139 155
50. C.-Y. Huang, and M.R. Lyu Optimal release time for software systems considering cost, testing-effort, and test efficiency IEEE Trans. Rel. 54 4 2005 583 591
51. C.P. Hullocker Finding the cost of software quality IEEE Trans. Eng. Manage. 33 4 1986 223 228
52. A.A. Issa, F.A. Abu Rub, and F.F. Thabata Using test case patterns to estimate software development and quality management cost Softw. Quality J. 17 3 2009 263 281
53. C.D. Ittner Activity-based costing concepts for quality improvement Eur. Manage. J. 17 5 1999 492 500
54. A. Jones, and M. Tabberer Process quality costing adapted to software development Empirical Softw. Eng. 2 3 1993 199 208
55. C. Jones The economics of software process improvement IEEE Comp. 29 1 1996 95 97
56. M. Jorgensen, and M. Shepperd A systematic review of software development cost estimation studies IEEE Trans. Softw. Eng. 33 1 2007 33 53
57. L.M. Karg, and A. Beckhaus Modelling software quality costs by adapting established methodologies of mature industries Proc. 2007 IEEE International Conference on Industrial Engineering and Engineering Management 2007 IEEE Piscataway 267 271
58. T.M. Khoshgoftaar, E.B. Allen, W.D. Jones, and J.P. Hudepohl Cost-benefit analysis of software quality models Empirical Softw. Eng. 9 1 2001 9 30
59. M. Kimura, T. Toyota, and S. Yamada Economic analysis of software release problems with warranty cost and reliability requirement Reliability Eng. Syst. Saf. 66 1 1999 49 55
60. S. King, J. Hammond, R. Chapman, and A. Pryor Is proof more cost-effective than testing? IEEE Trans. Softw. Eng. 26 8 2000 675 686
61. B. Kitchenham, O.P. Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman Systematic literature reviews in software engineering - a systematic literature review Inf. Softw. Technol. 51 1 2009 7 15
62. Kitchenham, B., Charters, S., 2007. Guidelines for performing systematic literature reviews in software engineering. Tech. Rep. EBSE 2007-001, Keele University and Durham University Joint Report.
63. S.T. Knox Modeling the cost of software quality Digital Tech. J. 5 4 1993 9 17
64. H. Krasner, and D. Houston Using the cost of quality approach for software CrossTalk, J. Defense Softw. Eng. 11 11 1998 6 11
65. S. Kusumoto, K. Matsumoto, T. Kikuno, and K. Torii A new metric for cost effectiveness of software reviews IEICE Trans. Inf. Syst. E75-D 5 1992 674 680
66. O. Laitenberger Cost-effective detection of software defects through perspective-based inspections Empirical Softw. Eng. 6 1 2001 81 84
67. R.J. Leach Methods of measuring software reuse for the prediction of maintenance effort J. Softw. Maint.: Res. Pract. 8 5 1996 309 320
68. Y.-W. Leung Optimum software release time with a given cost budget J. Syst. Softw. 17 3 1992 233 242
69. W.C. Lim Effects of reuse on quality, productivity, and economics IEEE Softw. 11 5 1994 23 30
70. C.-T. Lin, and C.-Y. Huang Enhancing and measuring the predictive capabilities of testing-effort dependent software reliability models J. Syst. Softw. 81 6 2008 1025 1038
71. W. Mandeville Software cost of quality IEEE J. Selected Areas Commun. 8 2 1990 315 318
72. R.T. McCann How much code inspection is enough? CrossTalk, J. Defense Softw. Eng. 14 7 2001 9 12
73. P. Middleton, and J. Sutton Lean Software Strategies 2005 Productivity Press
74. P. Mohagheghi, and R. Conradi Quality, productivity and economic benefits of software reuse: a review of industrial studies Empirical Softw. Eng. 12 5 2007 472 516
75. N.A. Mylonopoulos, and V. Theoharakis On site: global perceptions of IS journals Commun. ACM 44 9 2001 29 33
76. K. Okumoto, and A. Goel Optimal release time for software systems based on reliability and cost criteria J. Syst. Softw. 1 4 1980 315 318
77. D. O'Neill Determining return on investment using software inspections CrossTalk, J. Defense Softw. Eng. 16 3 2003 16 21
78. C.L. Pettijohn Achieving quality in the development process AT&T Tech. J. 65 2 1986 85 93
79. H. Pham A software cost model with imperfect debugging, random life cycle and penalty cost Int J. Syst. Sci. 27 5 1996 455 463
80. H. Pham System Software Reliability 2006 Springer Heidelberg
81. H. Pham, and X. Zhang A software cost model with warranty and risk costs IEEE Trans. Comput. 48 1 1999 71 75
82. A.A. Porter, H.P. Siy, C.A. Toman, and L.G. Votta An experiment to assess the cost-benefits of code inspections in large scale software development IEEE Trans. Softw. Eng. 23 6 1997 329 346
83. C.K. Prahalad, and M.S. Krishnan The new meaning of quality in the information age Harvard Bus. Rev. 77 5 1999 109 118
84. R.S. Pressman Software Engineering: A Practitioner's Approach 7th edition 2010 McGraw-Hill New York
85. A. Rai, H. Song, and M. Troutt Software quality assurance: an analytical survey and research prioritization J. Syst. Softw. 40 1 1998 67 83
86. E. Rivard, and K. Kaiser The benefits of quality IS Datamation 35 1 1989 53 58
87. G. Rothermel, S. Elbaum, A.G. Malishevsky, P. Kallakuri, and X. Qiu On test suite composition and cost-effective regression testing ACM Trans. Softw. Eng. Methodol. 13 3 2004 277 331
88. RTI, 2002. The economic impacts of inadequte infrastructure for software testing. Planning report 02-3, National Institute of Standards and Technology, Gaithersburg, MD.
89. G. Sabaliauskaite, S. Kusumoto, and K. Inoue Extended metrics to evaluate cost effectiveness of software inspections IEICE Trans. Inf. Syst. E87-D 2 2004 475 480
90. S.R. Schach The economic impact of software reuse on maintenance J. Softw. Maint.: Res. Pract. 6 4 1994 185 196
91. A. Schiffauerova, and V. Thomson A review of research on cost of quality models and best practices Int. J. Quality Reliability Manage. 23 6 2006 647 669
92. S.A. Sherer A cost-effective approach to testing IEEE Softw. 8 2 1991 34 40
93. P. Simmons Quality outcomes: determining business value IEEE Softw. 13 1 1996 25 32
94. N.D. Singpurwalla Determining an optimal time interval for testing and debugging software IEEE Trans. Softw. Eng. 17 4 1991 313 319
95. S.A. Slaughter, D.E. Harter, and M.S. Krishnan Evaluating the cost of software quality Commun. ACM 41 8 1998 67 73
96. H.M. Sneed Economics of software re-engineering J. Softw. Maint.: Res. Pract. 3 3 1991 163 182
97. Q. Song, M. Shepperd, M. Cartwright, and C. Mair Software defect association mining and defect correction effort prediction IEEE Trans. Softw. Eng. 32 2 2006 69 82
98. R. Spiewak, and K. McRitchie Using software quality methods to reduce cost and prevent defects CrossTalk, J. Defense Softw. Eng. 21 12 2008 23 27
99. N. Stewart Software error costs Quality Progress 21 11 1988 48 49
100. X. Teng, and H. Pham A software cost model for quantifying the gain with considerations of random field environments IEEE Trans. Comput. 53 3 2004 380 384
101. T.H. Tse, T.Y. Chen, and R.L. Glass An assessment of systems and software engineering scholars and institutions (2000-2004) J. Syst. Softw. 79 6 2006 816 819
102. R. van Megen, and D.B. Meyerhoff Costs and benefits of early defect detection: experiences from developing client server and host applications Softw. Quality J. 4 4 1995 247 256
103. R. van Solingen Measuring the ROI of software process improvement IEEE Softw. 21 3 2004 32 38
104. W.T. Ward Calculating the real cost of software defects Hewlett-Packard J. 42 5 1991 55 58
105. G. Webb, and N. Patton Quality and cost - it's not either/or: making the case with cost of quality CrossTalk, J. Defense Softw. Eng. 21 11 2008 23 28
106. J.C. Westland The cost of errors in software development: evidence from industry J. Syst. Softw. 62 1 2002 1 9
107. E.J. Weyuker The cost of data flow testing: an empirical study IEEE Trans. Softw. Eng. 16 2 1990 121 128
108. E.J. Weyuker Using failure cost information for testing and reliability assessment ACM Trans. Softw. Eng. Methodol. 5 2 1996 87 98
109. E.J. Weyuker Evaluation techniques for improving the quality of very large software systems in a cost-effective way J. Syst. Softw. 47 2-3 1999 97 103
110. J.A. Whittaker, and J.M. Voas 50 years of software: key principles for quality IT Pro Nov/Dec 2002 28 35
111. A. Williams, A. Wiele, and B. Dale Quality costing: a management review Int. J. Manage. Rev. 1 4 1999 441 460
112. C. Wohlin, and U. Koerner Software faults: spreading, detection and costs Softw. Eng. J. 5 1 1990 33 42
113. S. Yamada, H. Ohtera, and H. Narihisa Software reliability growth models with testing effort IEEE Trans. Rel. 35 1 1986 19 23
114. S. Yamada, and S. Osaki Cost-reliability optimal release policies for software systems IEEE Trans. Rel. 34 4 1985 422 424
115. S. Yamada, and S. Osaki Optimal software release policies with simultaneous cost and reliability requirements Eur. J. OR 31 1 1987 46 51
116. J. Yong, and A. Wilkinson The long and winding road: the evolution of quality management Total Quality Manage. 13 1 2002 101 121
117. X. Zhang, and H. Pham A software cost model with error removal times and risk costs Int J. Syst. Sci. 29 4 1998 435 442