Comparing the defect reduction benefits of code inspection and test-driven development

IEEE Transactions on Software Engineering

Volumn 38, Issue 3, 2012, Pages 547-560

  Wilkerson, Jerod W ^a   Nunamaker Jr , Jay F ^b   Mercer, Rick ^c  

^a Penn State Erie   (United States)

^b UNIVERSITY OF ARIZONA   (United States)

^c University of Arizona   (United States)

Author keywords

Agile programming; code inspections and walk throughs; empirical study; reliability; test driven development; testing strategies

Indexed keywords

AGILE PROGRAMMING; CODE INSPECTIONS; EMPIRICAL STUDIES; TEST DRIVEN DEVELOPMENT; TESTING STRATEGIES;

COMPUTER SCIENCE; DEFECTS; INSPECTION; RELIABILITY; TESTING;

SOFTWARE TESTING;

EID: 84861863036     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2011.46     Document Type: Article

References (47)

1. G. Tassey, "The Economic Impact of Inadequate Infrastructure for Software Testing," technical report, Nat'l Inst. of Standards and Technology, 2002.
2. B. George and L. Williams, "A Structured Experiment of Test-Driven Development," Information and Software Technology, vol. 46, no. 5, pp. 337-342, 2004.
3. E.M. Maximilien and L. Williams, "Assessing Test-Driven Development at IBM," Proc. 25th Int'l Conf. Software Eng., pp. 564-9, 2003.
4. D.L. Parnas and M. Lawford, "The Role of Inspections in Software Quality Assurance," IEEE Trans. Software Eng., vol. 29, no. 8, pp. 674-676, Aug. 2003.
5. F. Shull, V.R. Basili, B.W. Boehm, A.W. Brown, P. Costa, M. Lindvall, D. Port, I. Rus, R. Tesoriero, and M. Zelkowitz, "What We Have Learned about Fighting Defects," Proc. Eighth IEEE Symp. Software Metrics, pp. 249-58, 2002.
6. M.E. Fagan, "Design and Code Inspections to Reduce Errors in Program Development," IBM Systems J., vol. 15, no. 3, pp. 182-211, 1976.
7. N. Nagappan, M.E. Maximilien, T. Bhat, and L. Williams, "Realizing Quality Improvement through Test Driven Development: Results and Experiences of Four Industrial Teams," Empirical Software Eng., vol. 13, no. 3, pp. 289-302, 2008.
8. P. Runeson, C. Andersson, T. Thelin, A. Andrews, and T. Berling, "What Do We Know about Defect Detection Methods?" IEEE Software, vol. 23, no. 3, pp. 82-90, May/June 2006.
9. M.E. Fagan, "Advances in Software Inspections," IEEE Trans. Software Eng., vol. 12, no. 7, pp. 744-51, July 1986.
10. T. Gilb and D. Graham, Software Inspection. Addison-Wesley, 1993.
11. O. Laitenberger and J.-M. DeBaud, "An Encompassing Life Cycle Centric Survey of Software Inspection," J. Systems and Software, vol. 50, no. 1, pp. 5-31, 2000. (Pubitemid 32212306)
12. A. Aurum, H. Petersson, and C. Wohlin, "State-of-the-Art: Software Inspections after 25 Years," Software Testing, Verification and Reliability, vol. 12, no. 3, pp. 133-54, 2002.
13. A.F. Ackerman, L.S. Buchwald, and F.H. Lewski, "Software Inspections: An Effective Verification Process," IEEE Software, vol. 6, no. 3, pp. 31-36, May 1989.
14. W.S. Humphrey, A Discipline for Software Eng., ser. the SEI Series in Software Engineering. Addison-Wesley Publishing Company, 1995.
15. R.C. Linger, "Cleanroom Software Engineering for Zero-Defect Software," Proc. 15th Int'l Conf. Software Eng., pp. 2-13, 1993.
16. T. Thelin, P. Runeson, and B. Regnell, "Usage-Based Reading-An Experiment to Guide Reviewers with Use Cases," Information and Software Technology, vol. 43, no. 15, pp. 925-38, 2001. (Pubitemid 33143374)
17. T. Thelin, P. Runeson, and C. Wohlin, "An Experimental Comparison of Usage-Based and Checklist-Based Reading," IEEE Trans. Software Eng., vol. 29, no. 8, pp. 687-704, Aug. 2003.
18. T. Thelin, P. Runeson, C. Wohlin, T. Olsson, and C. Andersson, "Evaluation of Usage-Based Reading-Conclusions after Three Experiments," Empirical Software Eng., vol. 9, nos. 1/2, pp. 77-110, 2004.
19. V.R. Basili, S. Green, O. Laitenberger, F. Lanubile, F. Shull, S. Sørumgard, and M.V. Zelkowitz, "The Empirical Investigation of Perspective-Based Reading," Empirical Software Eng., vol. 1, no. 2, pp. 133-64, 1996. (Pubitemid 126708443)
20. C. Denger, M. Ciolkowski, and F. Lanubile, "Investigating the Active Guidance Factor in Reading Techniques for Defect Detection," Proc. Third Int'l Symp. Empirical Software Eng., 2004.
21. O. Laitenberger and J.-M. DeBaud, "Perspective-Based Reading of Code Documents at Robert Bosch GmbH," Information and Software Technology, vol. 39, no. 11, pp. 781-791, 1997. (Pubitemid 127371616)
22. J. Miller, M. Wood, and M. Roper, "Further Experiences with Scenarios and Checklists," Empirical Software Eng., vol. 3, no. 1, pp. 37-64, 1998. (Pubitemid 128512973)
23. V.R. Basili, G. Caldiera, F. Lanubile, and F. Shull, "Studies on Reading Techniques," Proc. 21st Ann. Software Eng. Workshop, pp. 59-65, 1996.
24. V.R. Basili and R.W. Selby, "Comparing the Effectiveness of Software Testing Strategies," IEEE Trans. Software Eng., vol. 13, no. 12, pp. 1278-1296, Dec. 1987.
25. A.A. Porter and L.G. Votta, "An Experiment to Assess Different Defect Detection Methods for Software Requirements Inspections," Proc. 16th Int'l Conf. Software Eng., pp. 103-12, 1994.
26. A.A. Porter, L.G. Votta Jr., and V.R. Basili, "Comparing Detection Methods for Software Requirements Inspections: A Replicated Experiment," IEEE Trans. Software Eng., vol. 21, no. 6, pp. 563-575, June 1995.
27. F. Shull, F. Lanubile, and V.R. Basili, "Investigating Reading Techniques for Object-Oriented Framework Learning," IEEE Trans. Software Eng., vol. 26, no. 11, pp. 1101-1118, Nov. 2000. (Pubitemid 32076247)
28. J.F. Nunamaker Jr., R.O. Briggs, D.D. Mittleman, D.R. Vogel, and P.A. Balthazard, "Lessons from a Dozen Years of Group Support Systems Research: A Discussion of Lab and Field Findings," J. Management Information Systems, vol. 13, no. 3, pp. 163-207, 1997.
29. J.F. Nunamaker Jr., A.R. Dennis, J.S. Valacich, D.R. Vogel, and J.F. George, "Electronic Meeting Systems to Support Group Work," Comm. ACM, vol. 34, no. 7, pp. 40-61, 1991.
30. P.M. Johnson, "An Instrumented Approach to Improving Software Quality through Formal Technical Review," Proc. 16th Int'l Conf. Software Eng., pp. 113-22, 1994.
31. M. van Genuchten, C. van Dijk, H. Scholten, and D. Vogel, "Using Group Support Systems for Software Inspections," IEEE Software, vol. 18, no. 3, pp. 60-65, May/June 2001. (Pubitemid 32469737)
32. S. Biffli, P. Grünbacher, and M. Halling, "A Family of Experiments to Investigate the Effects of Groupware for Software Inspection," Automated Software Eng., vol. 13, no. 3, pp. 373-394, 2006.
33. F. Lanubile, T. Mallardo, and F. Calefato, "Tool Support for Geographically Dispersed Inspection Teams," Software Process: Improvement and Practice, vol. 8, no. 4, pp. 217-231, 2003. (Pubitemid 40290382)
34. C.K. Tyran and J.F. George, "Improving Software Inspections with Group Process Support," Comm. ACM, vol. 45, no. 9, pp. 87-92, 2002. (Pubitemid 135699754)
35. M. van Genuchten, W. Cornelissen, and C. van Dijk, "Supporting Inspections with an Electronic Meeting System," J. Management Information Systems, vol. 14, no. 3, pp. 165-78, 1997.
36. P. Vitharana and K. Ramamurthy, "Computer-Mediated Group Support, Anonymity, and the Software Inspection Process: An Empirical Investigation," IEEE Trans. Software Eng., vol. 29, no. 2, pp. 167-80, Feb. 2003.
37. B.W. Boehm, Software Eng. Economics, Prentice Hall, 1981.
38. K. Beck, Test Driven Development: By Example. Addison-Wesley Professional, 2002.
39. M.M. Müller and O. Hagner, "Experiment about Test-First Programming," IEE Proc. Software, vol. 149, no. 5, pp. 131-136, Oct. 2002. (Pubitemid 35313947)
40. H. Erdogmus, M. Morisio, and M. Torchiano, "On the Effectiveness of the Test-First Approach to Programming," IEEE Trans. Software Eng., vol. 31, no. 3, pp. 226-37, Mar. 2005. (Pubitemid 40890004)
41. W.S. Humphrey, Managing the Software Process, ser. The SEI Series in Software Engineering. Addison-Wesley Publishing Company, 1989.
42. T.L. Rodgers, D.L. Dean, and J.F. Nunamaker Jr., "Increasing Inspection Efficiency through Group Support Systems," Proc. 37th Ann. Hawaii Int'l Conf. System Sciences, 2004.
43. C. Fox, "Java Inspection Checklist," 1999.
44. R.G. Ebenau and S.H. Strauss, Software Inspection Process, ser. Systems Design and Implementation. McGraw Hill, 1994.
45. L.S. Meyers, G. Gamst, and A.J. Guarino, Applied Multivariate Research: Design and Interpretation. Sage Publications, Inc., 2006.
46. J.F. Hair, B. Black, B. Babin, R.E. Anderson, and R.L. Tatham, Multivariate Data Analysis, sixth ed. Prentice Hall, 2005.
47. S.R. Searle, F.M. Speed, and G.A. Milliken, "Population Marginal Means in the Linear Model: An Alternative to Least Squares Means," The Am. Statistician, vol. 34, no. 4, pp. 216-221, 1980.