A multiple comparative study of test-with development product changes and their effects on team speed and product quality

Empirical Software Engineering

Volumn 16, Issue 2, 2011, Pages 177-210

  Bannerman, Steve ^a   Martin, Andrew ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Multiple comparative study; Product quality; Team speed; Test with development

Indexed keywords

AUTOMATED TEST; COMPARATIVE STUDIES; OPEN SOURCE PROJECTS; PRODUCT CHANGES; PRODUCT QUALITY; PRODUCT'S COMPLEXITY; SHORT DURATIONS; TEAM SPEED; TEST-FIRST DEVELOPMENT; TEST-WITH DEVELOPMENT;

QUALITY CONTROL;

TESTING;

EID: 79953171863     PISSN: 13823256     EISSN: 15737616     Source Type: Journal    
DOI: 10.1007/s10664-010-9137-5     Document Type: Article

References (47)

1. Abrahamsson P, Hanhineva A, Jaalinoja J (2005) Improving business agility through technical solutions: a case study on test-driven development in mobile software development. In: IFIP 2005: business agility and information technology diffusion, pp 1-17
2. Auer K, Miller R (2002) Extreme programming applied. Addison-Wesley, Boston
3. N Ayewah D Hovemeyer JD Morgenthaler J Penix W Pugh 2008 Using static analysis to find bugs IEEE Softw 25 5 22 29 10.1109/MS.2008.130 10.1109/MS.2008.130
4. Bannerman S, Martin A (2010) A multiple comparative study of test-with development product changes and their effects on team speed and product quality. Tech. Rep. RR-10-03, The University of Oxford. http://www.comlab.ox.ac.uk/ techreports/cs/2010.html
5. VR Basili RW Selby DH Hutchens 1986 Experimentation in software engineering IEEE Trans Softw Eng 12 7 733 743
6. VR Basili F Shull F Lanubile 1999 Building knowledge through families of experiments IEEE Trans Softw Eng 25 4 456 473 10.1109/32.799939 10.1109/32.799939 (Pubitemid 129304112)
7. K Beck 1999 Embracing change with extreme programming IEEE Comput 32 10 70 77
8. Beck K (1999b) Extreme programming explained: embrace change. Addison-Wesley
9. Beck K (2003) Test-driven development. Addison-Wesley, Boston
10. I Benbasat DK Goldstein M Mead 1987 The case research strategy in studies of information systems JSTOR MIS Q 11 3 369 386 10.2307/248684 10.2307/248684
11. Bhat T, Nagappan N (2006) Evaluating the efficacy of test-driven development: industrial case studies. In: ISESE '06: proceedings of the 2006 ACM/IEEE international symposium on empirical software engineering. ACM, New York, pp 356-363. doi: 10.1145/1159733.1159787 (Pubitemid 46634470)
12. Boehm BW, Abts C, Brown AW, Chulani S, Clark BK, Horowitz E, Madachy R, Reifer DJ, Steece B (2000) Software cost estimation with Cocomo II. Prentice Hall, Englewood Cliffs
13. Canfora G, Cimitile A, Garcia F, Piattini M, Visaggio CA (2006) Evaluating advantages of test driven development: a controlled experiment with professionals. In: ISESE '06: proceedings of the 2006 ACM/IEEE international symposium on empirical software engineering. ACM, New York, pp 364-371. doi: 10.1145/1159733.1159788 (Pubitemid 46634471)
14. DN Card FE McGarry GT Page 1987 Evaluating software engineering technologies IEEE Trans Softw Eng 13 7 845 851 10.1109/TSE.1987.233495 10.1109/TSE.1987.233495
15. JE Cook LG Votta AL Wolf 1998 Cost-effective analysis of in-place software processes IEEE Trans Softw Eng 24 8 650 663 10.1109/32.707700 10.1109/32.707700 (Pubitemid 128739724)
16. LO Damm L Lundberg 2006 Results from introducing component-level test automation and test-driven development J Syst Softw 79 7 1001 1014 10.1016/j.jss.2005.10.015 10.1016/j.jss.2005.10.015 (Pubitemid 43765969)
17. S Diehl AE Hassan RC Holt 2005 Report on msr 2005: international workshop on mining software repositories SIGSOFT Softw Eng Notes 30 5 1 3 10.1145/1095430.1095433 10.1145/1095430.1095433
18. Diehl S, Gall H, Pinzger M, Hassan AE (2006) Msr 2006: the 3rd international workshop on mining software repositories. In: ICSE '06: proceedings of the 28th international conference on software engineering. ACM, New York, pp 1021-1021. doi: 10.1145/1134285.1134483
19. H Erdogmus M Morisio M Torchiano 2005 On the effectiveness of the test-first approach to programming IEEE Trans Softw Eng 31 3 226 237 10.1109/TSE.2005.37 10.1109/TSE.2005.37 (Pubitemid 40890004)
20. Fenton N, Pfleeger S (1997) Software metrics: a rigorous and practical approach. PWS Publishing Company, Boston
21. Gall H, Lanza M, Zimmermann T (2007) 4th international workshop on mining software repositories (msr 2007). In: ICSE COMPANION '07: companion to the proceedings of the 29th international conference on software engineering. IEEE Computer Society, Washington, DC, pp 107-108. doi: 10.1109/ICSECOMPANION.2007.8 (Pubitemid 47430763)
22. George B, Williams L (2003) An initial investigation of test driven development in industry. In: SAC '03: proceedings of the 2003 ACM symposium on applied computing. ACM, New York, pp 1135-1139. doi: 10.1145/952532.952753
23. Geras A, Smith M, Miller J (2004) A prototype empirical evaluation of test driven development. In: METRICS '04: proceedings of the software metrics, 10th international symposium. IEEE Computer Society, Washington, DC, pp 405-416. doi: 10.1109/METRICS.2004.2 (Pubitemid 40338179)
24. Gupta A, Jalote P (2007) An experimental evaluation of the effectiveness and efficiency of the test driven development. In: ESEM '07: proceedings of the first international symposium on empirical software engineering and measurement. IEEE Computer Society, Washington, DC, pp 285-294. doi: 10.1109/ESEM.2007.20
25. JE Hannay DIK Sjoberg T Dyba 2007 A systematic review of theory use in software engineering experiments IEEE Trans Softw Eng 33 2 87 107 10.1109/TSE.2007.12 10.1109/TSE.2007.12 (Pubitemid 46223696)
26. AE Hassan RC Holt A Mockus 2005 Report on msr 2004: international workshop on mining software repositories SIGSOFT Softw Eng Notes 30 1 4 10.1145/1039174.1039188 10.1145/1039174.1039188
27. Janzen DS, Saiedian H (2006) On the influence of test-driven development on software design. In: CSEET '06: proceedings of the 19th conference on software engineering education & training. IEEE Computer Society, Washington, DC, pp 141-148. doi: 10.1109/CSEET.2006.25 (Pubitemid 44592196)
28. Jeffries R, Anderson A, Hendrickson C (2001) Extreme programming installed. Addison-Wesley, Boston
29. B Kitchenham L Pickard SL Pfleeger 1995 Case studies for method and tool evaluation IEEE Softw 12 4 52 62 10.1109/52.391832 10.1109/52.391832
30. B Kitchenham S Pfleeger L Pickard P Jones D Hoaglin KE Emam J Rosenberg 2002 Preliminary guidelines for empirical research in software engineering IEEE Trans Softw Eng 28 8 721 734 10.1109/TSE.2002.1027796
31. Lanza M, Godfrey MW, Kim S (2008) Msr 2008-5th working conference on mining software repositories. In: ICSE companion '08: companion of the 30th international conference on software engineering. ACM, New York, pp 1037-1038. doi: 10.1145/1370175.1370235
32. TC Lethbridge SE Sim J Singer 2005 Studying software engineers: data collection techniques for software field studies Empir Software Eng 10 3 311 341 10.1007/s10664-005-1290-x 10.1007/s10664-005-1290-x (Pubitemid 40940550)
33. Madeyski L (2005) Preliminary analysis of the effects of pair programming and test-driven development on the external code quality. In: Proceeding of the 2005 conference on software engineering: evolution and emerging technologies. IOS Press, Amsterdam, pp 113-123
34. Maximilien EM, Williams L (2003) Assessing test-driven development at IBM. In: ICSE '03: proceedings of the 25th international conference on software engineering. IEEE Computer Society, Washington, DC, pp 564-569
35. TJ McCabe 1976 A complexity measure IEEE Trans Softw Eng 2 4 308 320 445904 10.1109/TSE.1976.233837
36. M Muller O Hagner 2002 Experiment about test-first programming IEE Proc, Softw 149 5 131 136 10.1049/ip-sen:20020540 (Pubitemid 35313947)
37. N Nagappan EM Maximilien T Bhat L Williams 2008 Realizing quality improvement through test driven development: results and experiences of four industrial teams Empir Software Eng 13 3 289 302 10.1007/s10664-008-9062-z 10.1007/s10664-008-9062-z
38. Pancur M, Ciglaric M, Trampus M, Vidmar T (2003) Towards empirical evaluation of. test-driven development in a university environment. In: EUROCON 2003: IEEE region 8 proceedings, vol 2. IEEE Press, pp 83-86
39. A Pinsonneault KL Kraemer 1993 Survey research methodology in management information systems: an assessment J Manage Inf Syst 10 2 75 105
40. Raymond ES (1999) The magic cauldron. In: The cathedral & the bazaar. O'Reilly & Associates, pp 137-194
41. Robson C (2002) Real world research. Blackwell Publishers, Cambridge
42. P Runeson M Host 2009 Guidelines for conducting and reporting case study research in software engineering Empir Software Eng 14 2 131 164 10.1007/s10664-008-9102-8
43. Sanchez JC, Williams L, Maximilien EM (2007) On the sustained use of a test-driven development practice at IBM. In: AGILE '07: proceedings of the AGILE 2007. IEEE Computer Society, Washington, DC, pp 5-14. doi: 10.1109/AGILE.2007. 43
44. Siniaalto M, Abrahamsson P (2007) A comparative case study on the impact of test-driven development on program design and test coverage. In: ESEM '07: proceedings of the first international symposium on empirical software engineering and measurement. IEEE Computer Society, Washington, DC, pp 275-284. doi: 10.1109/ESEM.2007.2
45. DIK Sjoberg JE Hannay O Hansen VB Kampenes A Karahasanovic N Liborg AC Rekdal 2005 A survey of controlled experiments in software engineering IEEE Trans Softw Eng 31 9 733 753 10.1109/TSE.2005.97 10.1109/TSE.2005.97 (Pubitemid 41555415)
46. Williams L, Maximilien EM, Vouk M (2003) Test-driven development as a defect-reduction practice. In: ISSRE '03: proceedings of the 14th international symposium on software reliability engineering. IEEE Computer Society, Washington, DC, pp 34-45
47. Zaidman A, Van Rompaey B, Demeyer S, van Deursen A (2008) Mining software repositories to study co-evolution of production & test code. In: ICST '08: proceedings of the 2008 international conference on software testing, verification, and validation. IEEE Computer Society, Washington, DC, pp 220-229. doi: 10.1109/ICST.2008.47