Empirical investigation towards the effectiveness of Test First programming

Information and Software Technology

Volumn 51, Issue 1, 2009, Pages 182-194

  Huang, Liang ^a   Holcombe, Mike ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

Agile methods; Empirical software engineering; Programming paradigms; Software engineering process; Software testing; Testing strategies

Indexed keywords

COMPUTER SOFTWARE SELECTION AND EVALUATION; ELECTRIC LINES; PROGRAMMING THEORY; SOFTWARE ENGINEERING; TESTING; THERMOLUMINESCENCE;

AGILE METHODS; EMPIRICAL SOFTWARE ENGINEERING; PROGRAMMING PARADIGMS; SOFTWARE ENGINEERING PROCESS; TESTING STRATEGIES;

SOFTWARE TESTING;

EID: 56649121999     PISSN: 09505849     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.infsof.2008.03.007     Document Type: Article

References (55)

1. Ambler S.W. Introduction to Test Driven Development. Agile Database Techniques: Effective Strategies for the Agile Software Developer (2003), John Wiley & Sons
2. Arisholm E., and Sjøberg D.I.K. Evaluating the effect of a delegated versus centralized control style on the maintainability of object-oriented software. IEEE Transactions on Software Engineering 30 8 (2004) 521-534
3. Basili V.R., Selby R.W., and Hutchens D.H. Experimentation in software engineering. IEEE Transactions on Software Engineering SE-12 7 (1986) 733-743
4. Basili V.R., and Rombach H.D. The TAME project: towards improvement-oriented software environments. IEEE Transactions on Software Engineering 14 6 (1988) 758-773
5. Beck K. Extreme Programming Explained: Embrace Change (2000), Addison Wesley, Longman, Reading, Mass, USA
6. Beck K. Test Driven Development: By Example (2003), Addison Wesley, Reading, MA
7. Boehm B. Software Engineering Economics (1981), Prenctice Hall, Englewood Cliffs, NJ
8. Cederqvist P. Version Management with CVS (1993), Network Theory Ltd., Bristol, UK
9. D. Chaplin, Test first Programming, TechZone, 2001.
10. Cohen J., and Cohen P. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences (1975), Erlbaum, Hillsdale, NJ
11. Cohen J. Statistical Power Analysis for the Behavioral Sciences. second ed. (1988), Lawrence Earlbaum and Associates, Hillsdale, NJ
12. Cook T.D., and Campbell D.T. Quasi-Experimentation: Design and Analysis Issues for Field Settings (1979), Houghton Mifflin Co., Boston
13. Crispin L. Driving software quality: how test-driven development impacts software quality. IEEE Software 23 6 (2006) 70-71
14. Deming W.E. Quality, Productivity, and Competitive Position (1982), Center for Advanced Engineering Study, MIT
15. Delgado R.F. Planning for quality software. SAM Advanced Management Journal 57 2 (1992) 24-28
16. Dustin E., Rashka J., and Paul J. Automated software testing: Introduction, management, and performance (1999), Addison Wesley
17. "Eclipse Platform Technical Overview", Object Technology Int'l, 2003.
18. S.H. Edwards, Using test-driven development in the classroom: providing students with automatic, concrete feedback on performance, in: Proceedings of the International Conference on Education and Information Systems: Technologies and Applications (EISTA'03), August 2003.
19. Erdogmus H., Morisio M., and Torchiano M. On the effectiveness of the Test-First approach to programming. IEEE Transactions on Software Engineering 31 3 (2005)
20. Fenton N., and Pfleeger S. Software Metrics: A Rigorous and Practical Approach. second ed. (1997), International Thomson Computer Press, London
21. D. Gelperin, W. Hetzel, Software quality engineering, in: Proceedings of 4th International Conference on Software Testing, Washington DC, June 1987.
22. George B., and Williams L. A structured experiment of test-driven development. Information and Software Technology 46 (2004) 337-342
23. Gossett W.S. The probable error of a mean. Biometrika 6 1 (1908) 1-25
24. Holcombe M., and Parker H. Keeping our clients happy: myths and management issues in 'client-led' student software projects. Computer Science Education 9 3 (1999) 230-241
25. M. Holcombe, M. Gheorghe, F. Macias, Teaching XP for real: Some initial observations and plans, in: Proceedings of 2nd International Conference on Extreme Programming and Flexible Processes in Software Engineering (XP2001), Italy, May 2001, pp. 14-17.
26. Höst M., Regnell B., and Wohlin C. Using students as subjects - a comparative study of students and professionals in lead-time impact assessment. Empirical Software Engineering 5 (2000) 201-214
27. Huang L., and Holcombe M. Empirical assessment of Test-First approach. Proceedings of Testing: Academic & Industrial Conference (2006), IEEE Computer Society pp. 197-200
28. Huang L., Thomson C., and Holcombe M. How good are you testers? An assessment of testing ability. Proceedings of Testing: Academic & Industrial Conference (2007), IEEE Computer Society pp.82-86
29. D. Janzen, Software architecture improvement through test-driven development, in: Proceedings of the Conference on Object Oriented Programming Systems Languages and Applications, ACM, 2005, pp. 222-223.
30. JUnit.org, Available from: .
31. J. Karn, Empirical Software Engineering: Developing Behavior and Pretence, Ph.D. thesis, the University of Sheffield, April 2006.
32. R. Kaufmann, D. Janzen, Implications of test-driven development: a pilot study, in: Proceedings of the 18th Annual ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, ACM, 2003, pp. 298-299.
33. Kitchenham B.A., Pfleeger S.L., Pickard L.M., Jones P.W., Hoaglin D.C., El-Emam K., and Rosenberg J. Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering 28 8 (2002) 721-734
34. Kolmogorov A.N. Sulla determinazione empirica di una leggi di distribuzione. Giornale dell'Istituta Italiano degli Attuari 4 (1933) 33
35. Larman C., and Basili V. A history of iterative and incremental development. IEEE Computer 36 (2003) 47-56
36. M. Pancur, M. Ciglaric, M. Trampus, T. Vidmar, Towards empirical evaluation of test-driven development in a university environment, in: Proceedings of EUROCON 2003, Computer as a Tool, The IEEE Region, 8(2) (2003), pp. 83-86.
37. Porter A., and Votta L. Comparing detection methods for software requirements inspection: a replication using professional subjects. Empirical Software Engineering 3 (1998) 355-380
38. F. Macias, M. Holcombe, M. Gheorghe, Design-led & design-less: one experiment and two approaches, in: Proceedings of 4th International Conference on Extreme Programming and Flexible Processes in Software Engineering (XP2003), 2003, pp. 394-401.
39. F. Macias, Empirical Assessment of Extreme Programming, Ph.D. thesis, University of Sheffield, UK, 2004.
40. Mann H.B., and Whitney D.R. On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematics and Statistics (1947)
41. E.M. Maximilien, L. Williams, Assessing test-driven development at IBM, in: Proceedings of International Conference of Software Engineering, Portland, 2003.
42. McCabe T.J. A complexity measure. IEEE Transactions on Software Engineering 2 (1976) 308-320
43. McConnell S. Rapid Development (1996), Microsoft Press, Redmond, WA
44. Müller M., and Hanger O. Experiment about Test-First programming. IEEE Proceedings on Software 149 5 (2002) 537-544 October
45. P. Runeson, Using students as experiment subjects - an analysis on graduate and freshmen student data, in: Proceedings of 7th International Conference on Empirical Assessment and Evaluation in Software Engineering (EASE 03), 2003.
46. Shadish W.R., Cook T.D., and Campbell D.T. Experimental and Quasi-Experimental Designs for General Causal Inference (2002), Houghton Mifflin, Boston
47. Sjøeberg D.I.K., Hannay J.E., Hansen O., Kampenes V.B., Karahasanovic A., Liborg N.K., and Rekdal A.C. A survey of controlled experiments in software engineering. IEEE Transactions on Software Engineering 31 9 (2005) 733-753
48. Smirnov N.V. Approximation of distribution laws of random variables on the basis of empirical data. Uspekhi Matematicheskikhi Nauka 10 (1944) 179-206
49. M. Stephens, D. Rosenberg, Extreme Programming Refactored: The Case Against XP, 2003.
50. S. Syed-Abdullah, Empirical Study on Extreme Programming, Ph.D. thesis, The University of Sheffield, UK, April 2005.
51. Tate R. Experimental design. In: Mitzel H.E., Hardin Best J.H., and Rabinowitz W. (Eds). Encyclopaedia of Education Research (1982), Collier Mac Millan, London 553-561
52. C. Thomson, Defining and Describing Change Events in Software Development Projects, Ph.D. Thesis, the University of Sheffield, UK, 2007.
53. C. Thomson, M. Holcombe, 20 Years of teaching and 7 years of research: research when you teach, in: Proceedings of 3rd South East European Formal Methods Workshop, 2007, pp. 141-153.
54. L. Williams, E. Maximilien, M. Vouk, Test driven development as a defect-reduction practice, in: Proceedings of the 14th International Symposium on Software Reliability Engineering, 2004, pp. 34-45.
55. Wohlin C., and Ahlgren M. Soft factors and their impact on time to market. Software Quality Journal 4 (1995) 189-205