Test-driven development: An empirical evaluation of agile practice

Test-Driven Development: An Empirical Evaluation of Agile Practice

Volumn , Issue , 2010, Pages 1-245

  Madeyski, Lech ^a  

^a WROCLAW UNIVERSITY OF TECHNOLOGY   (Poland)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84892346075     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-3-642-04288-1     Document Type: Book

References (273)

1. Activity Sensor project - Wroclaw University of Technology, Software Engineering Society (Cited 1 Apr 2009). URL http://www.e-informatyka.pl/sens/ Wiki.jsp?page=Projects.ActivitySensor.
2. Ant project (Cited 1 Apr 2009). URL http://ant.apache.org/.
3. Aopmetrics project - Wroclaw University of Technology, Software Engineering Society (http://www.e-informatyka.pl/sens/Wiki.jsp?page=Projects.
4. AOPMetrics (Cited 1 Apr 2009). URL http://aopmetrics.tigris.org.
5. AspectJ project (Cited 1 Apr 2009). URL http://www.eclipse.org/aspectj/.
6. Cactus project (Cited 1 Apr 2009). URL http://jakarta.apache.org/cactus/.
7. Wiki (Cited 1 Apr 2009). URL http://en.wikipedia.org/wiki/Wiki.
8. Aaron, B., Kromrey, J.D., Ferron, J.: Equating r-based and d-based effect size indices: Problems with a commonly recommended formula. In: FERA'98: 43rd Annual Meeting of the Florida Educational Research Association, pp. 1-13 (1998).
9. Abrahamsson, P., Hanhineva, A., Jaalinoja, J.: Improving business agility through technical solutions: A case study on test-driven development in mobile software development. In: R. Baskerville, L. Mathiassen, J. Pries-Heje, J.I. DeGross (eds.) IFIP TC8.
10. International Working Conference on Business Agility and Information Technology Diffusion, IFIP International Federation for Information Processing, vol. 180, pp. 1-17. Springer (2005). URL http://dx.doi.org/10.1007/0-387-25590- 7-14.
11. Al-Kilidar, H., Cox, K., Kitchenham, B.: The use and usefulness of the ISO/IEC 9126 quality standard. In: R. Jeffery, J. Verner, G.H. Travassos (eds.) ISESE'05: ACM/IEEE International Symposium on Empirical Software Engineering, pp. 126-132. IEEE (2005). URL http://dx.doi.org/10.1109/ISESE.2005.1541821.
12. Alshayeb, M., Li, W.: An Empirical Validation of Object-Oriented Metrics in Two Different Iterative Software Processes. IEEE Trans. Softw. Eng. 29(11), 1043-1049 (2003). URL http://dx.doi.org/10.1109/TSE.2003.1245305.
13. Ambler, S.: Introduction to Test Driven Design (TDD) (Cited 1 Apr 2009). URL http://www.agiledata.org/essays/tdd.html.
14. American Psychological Association: Publication manual of the American Psychological Association, 5th edn. American Psychological Association, Washington, DC, USA (2001).
15. Arisholm, E.: Empirical Assessment of Changeability in Object-Oriented Software. Ph.D. thesis, University of Oslo (2001). URLhttp://simula.no/research/ engineering/publications/SE.3.Arisholm.2001/simula-pdf-file.
16. Arisholm, E., Gallis, H., Dyba°, T., Sjøberg, D.I.K.: Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise. IEEE Trans. Softw. Eng. 33(2), 65-86 (2007). URL http://dx.doi.org/ 10.1109/TSE.2007.17.
17. Astels, D.: Test Driven Development: A Practical Guide. Prentice Hall (2003).
18. Atlassian Pty Ltd: Clover project (Cited 1 Apr 2009). URL http://www.atlassian.com/software/clover/.
19. Baheti, P., Gehringer, E.F., Stotts, P.D.: Exploring the efficacy of distributed pair programming. In: D.Wells, L.Williams (eds.) XP'02: Second XP Universe and First Agile Universe Conference on Extreme Programming and Agile Methods - XP/Agile Universe 2002, Lecture Notes in Computer Science, vol. 2418, pp. 208-220. Springer, Berlin Heidelberg, DE (2002). URL http://dx.doi.org/10. 1007/3-540-45672-4-20.
20. Bansiya, J., Davis, C.G.: A hierarchical model for object-oriented design quality assessment. IEEE Trans. Softw. Eng. 28(1), 4-17 (2002). URL http://dx.doi.org/10.1109/32.979986.
21. Basili, V.R., Briand, L.C., Melo, W.L.: A Validation of Object-Oriented Design Metrics as Quality Indicators. IEEE Trans. Softw. Eng. 22(10), 751-761 (1996). URL http://dx.doi.org/10.1109/32.544352.
22. Basili, V.R., Caldiera, G., Rombach, H.D.: Encyclopedia of Software Engineering, The Goal Question Metric Approach, pp. 528-532. Wiley-Interscience (1994).
23. Bayley, L., Eldredge, J.: The Structured Abstract: An Essential Tool for Researchers. Hypothesis 17(1), 11-13 (2003). URL http://gainweb.mercer.edu/mla/ research/hyp03v17n1.pdf.
24. Beck, K.: Test Driven Development: By Example. Addison-Wesley, Boston, MA, USA (2002).
25. Beck, K., Andres, C.: Extreme Programming Explained: Embrace Change, 2nd edn. Addison-Wesley, Boston, MA, USA (2004).
26. Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R., Kern, J., Marick, B., Martin, R.C., Mellor, S., Schwaber, K., Sutherland, J., Thomas, D.: Manifesto for Agile Software Development (Cited 1 Apr. 2009). URL http://www.agilemanifesto.org/.
27. Beck, K., Fowler, M.: Planning Extreme Programming. Addison-Wesley, Boston, MA, USA (2000).
28. Benbasat, I., Zmud, R.W.: Empirical research in information systems: The practice of relevance. MIS Q. pp. 3-16 (1999). URL http://dx.doi.org/10.2307/ 249403.
29. Berry, E., Coustere-Yakin, C., Grover, N.: The significance of nonsignificance. QJM: Int. J. Med. 91, 647-653 (1998). URL http://qjmed. oxfordjournals.org/cgi/reprint/91/9/647.pdf.
30. Bhat, T., Nagappan, N.: Evaluating the efficacy of test-driven development: industrial case studies. In: ISESE'06: ACM/IEEE International Symposium on Empirical Software Engineering, pp. 356-363. ACM Press, New York, NY, USA (2006). URL http://dx.doi.org/10.1145/1159733.1159787.
31. Bipp, T., Lepper, A., Schmedding, D.: Pair programming in software development teams - An empirical study of its benefits. Inf. Softw. Technol. 50(3), 231-240 (2008). URL http://dx.doi.org/10.1016/j.infsof.2007.05.006.
32. Bøegh, J.: A new standard for quality requirements. IEEE Softw. 25(2), 57-63 (2008). URL http://dx.doi.org/10.1109/MS.2008.30.
33. Briand, L.C., Melo, W.L., Wust, J.: Assessing the Applicability of Fault-Proneness Models Across Object-Oriented Software Projects. IEEE Trans. Softw. Eng. 28(7), 706-720 (2002). URL http://dx.doi.org/10.1109/TSE.2002. 1019484.
34. Briand, L.C., Wust, J.: Empirical Studies of Quality Models in Object-Oriented Systems. Adv. Comput. 59, 97-166 (2002). URLhttp://squall.sce. carleton.ca/pubs/journal/2002-Briand-Wuest.pdf.
35. Briand, L.C., Wust, J., Daly, J.W., Porter, D.V.: Exploring the relationships between design measures and software quality in object-oriented systems. J. Syst. Softw. 51(3), 245-273 (2000). URL http://dx.doi.org/10.1016/ S0164-1212(99)00102-8.
36. Briand, L.C., Wust, J., Ikonomovski, S.V., Lounis, H.: Investigating quality factors in object-oriented designs: An industrial case study. In: ICSE'99: International Conference on Software Engineering, pp. 345-354. IEEE Computer Society Press, Los Alamitos, CA, USA (1999). URL http://dx.doi.org/10. 1145/302405.302654.
37. Brown, S.: One-Tailed or Two-Tailed Hypothesis Test? (Cited 1 Apr 2009). URL http://www.tc3.edu/instruct/sbrown/stat/httails.htm.
38. Burke, E.M., Coyner, B.M.: JavaTM Extreme Programming Cookbook. O'Reilly (2003).
39. Cai, X., Lyu, M.R.: The Effect of Code Coverage on Fault Detection under Different Testing Profiles. SIGSOFT Softw. Eng. Notes 30(4), 1-7 (2005). URL http://dx.doi.org/10.1145/1082983.1083288.
40. Canfora, G., Cimitile, A., Carballeira, F.G., Piattini, M., Visaggio, C.A.: Productivity of test driven development: A controlled experiment with professionals. In: J.Munch, M. Vierimaa (eds.) PROFES'06: Product Focused Software Process Improvement, Lecture Notes in Computer Science, vol. 4034, pp. 383-388. Springer, Berlin, Heidelberg (2006). URL http://dx.doi.org/10.1007/ 11767718-32.
41. Canfora, G., Cimitile, A., Garcia, F., Piattini, M., Visaggio, C.A.: Confirming the influence of educational background in pair-design knowledge through experiments. In: SAC'05: ACM Symposium on Applied Computing, pp. 1478-1484. ACM Press, New York, NY, USA (2005). URL http://dx.doi.org/10.1145/ 1066677.1067013.
42. Canfora, G., Cimitile, A., Garcia, F., Piattini, M., Visaggio, C.A.: Evaluating advantages of test driven development: A controlled experiment with professionals. In: ISESE'06: ACM/IEEE International Symposium on Empirical Software Engineering, pp. 364-371. ACM Press, New York, NY, USA (2006). URL http://dx.doi.org/10.1145/1159733.1159788.
43. Carver, J., Jaccheri, L., Morasca, S., Shull, F.: Issues in using students in empirical studies in software engineering education. In: METRICS'03: 9th International Symposium on Software Metrics, pp. 239-249. IEEE Computer Society, Washington, DC, USA (2003). URL http://dx.doi.org/10.1109/METRIC.2003. 1232471.
44. Cazzola, W., Marchetto, A.: AOP HiddenMetrics: Separation, Extensibility and Adaptability in SW Measurement. J. Object Tech. 7(2), 53-68 (2008). URL http://www.jot.fm/issues/issue-2008-02/article3.pdf.
45. Chidamber, S.R., Kemerer, C.F.: A Metrics Suite for Object Oriented Design. IEEE Trans. Softw. Eng. 20(6), 476-493 (1994). URL http://dx.doi.org/10. 1109/32.295895.
46. Clare, J., Hamilton, H.: Writing Research: Transforming Data Into Text. Elsevier, Amsterdam, The Netherlands (2003).
47. Cohen, J.W.: Statistical Power Analysis for the Behavioral Sciences, 2nd edn. Lawrence Erlbaum Associates, Hillsdale, New York, USA (1988).
48. Cohen, J.W.: A Power Primer. Psychol. Bull. 112, 155-159 (1992).
49. Cohen, J.W.: A new goal: Preventing Disease, Not Infection. Science 262, 1820-1821 (1993). URL http://dx.doi.org/10.1126/science.8266069.
50. Cohn, M., Ford, D.: Introducing an Agile Process to an Organization. IEEE Comput. 36(6), 74-78 (2003). URL http://dx.doi.org/10.1109/MC.2003.1204378.
51. Coleman, D., Ash, D., Lowther, B., Oman, P.: Using Metrics to Evaluate Software System Maintainability. IEEE Comput. 27(8), 44-49 (1994). URL http://dx.doi.org/10.1109/2.303623.
52. Conover, W.J., Iman, R.L.: Rank Transformations as a Bridge Between Parametric and Nonparametric Statistics. Am. Stat. 35(3), 124-129 (1981).
53. Conover, W.J., Iman, R.L.: Analysis of Covariance Using the Rank Transformation. Biometrics 38(3), 715-724 (1982).
54. Cook, T.D., Campbell, D.T.: Quasi-Experimentation: Design and Analysis Issues. Houghton Mifflin Company, Boston, MA, USA (1979).
55. Cornett, S.: Code Coverage Analysis (Cited 1 Apr. 2009). URL http://www.bullseye.com/coverage.html.
56. Corporation, A.: (Cited 1 Apr. 2009). URL http://www.answers.com/topic/ rule-of-thumb.
57. Dallal, G.E.: Degrees of Freedom (Cited 1 Apr. 2009). URL http://www.tufts.edu/gdallal/dof.htm.
58. Damm, L.O., Lundberg, L.: Results from introducing component-level test automation and Test-Driven Development. J. Syst. Softw. 79(7), 1001-1014 (2006). URL http://dx.doi.org/10.1016/j.jss.2005.10.015.
59. Damm, L.O., Lundberg, L.: Quality impact of introducing component-level test automation and test-driven development. In: P. Abrahamsson, N. Baddoo, T. Margaria, R. Messnarz (eds.) EuroSPI'07: Software Process Improvement, Lecture Notes in Computer Science, vol. 4764, pp. 187-199. Springer (2007). URL http://dx.doi.org/10.1007/978-3-540-75381-0-17.
60. Davies, R.: Agile Requirements. Methods and Tools 13(3), 24-30 (2005). URL http://www.methodsandtools.com/PDF/mt200503.pdf.
61. De Marco, T.: Controlling Software Projects: Management, Measurement, and Estimation. Yourdon Press, New York, NY, USA (1982).
62. De Marco, T., Lister, T.: Peopleware: Productive Projects and Teams. Dorset House Publishing. New York, NY, USA (1999).
63. De Millo, R.A., Lipton, R.J., Sayward, F.G.: Hints on Test Data Selection: Help for the Practicing Programmer. IEEE Comput. 11(4), 34-41 (1978). URL http://dx.doi.org/10.1109/C-M.1978.218136.
64. Der Simonian, R., Laird, N.: Meta-Analysis in Clinical Trials. Control. Clin. Trials 7(3), 177-188 (1986).
65. Dmitrienko, A., Molenberghs, G., Chuang-Stein, C., Offen, W.: Analysis of Clinical Trials Using SAS: A Practical Guide. SAS Publishing (2005).
66. Dunlap, P.W., Cortina, J.M., Vaslow, J.B., Burke, M.J.: Meta-Analysis of Experiments with Matched Groups or Repeated Measures Designs. Psychol. Methods 1(2), 170-177 (1996).
67. Dyba, T., Arisholm, E., Sjøberg, D.I.K., Hannay, J.E., Shull, F.: Are two heads better than one? on the effectiveness of pair programming. IEEE Softw. 24(6), 12-15 (2007). URL http://dx.doi.org/10.1109/MS.2007.158.
68. Edwards, S.H.: Rethinking computer science education from a test-first perspective. In: OOPSLA'03: Companion of the 18th Annual ACM SIGPLAN Conference on Object- Oriented Programming, Systems, Languages, and Applications, pp. 148-155. ACM, New York, NY, USA (2003). URL http://dx.doi.org/10.1145/949344. 949390.
69. Edwards, S.H.: Teaching software testing: Automatic grading meets test-first coding. In: OOPSLA'03: Companion of the 18th Annual ACM SIGPLAN Conference on Object- Oriented Programming, Systems, Languages, and Applications, pp. 318-319. ACM, New York, NY, USA (2003). URL http://dx.doi.org/10.1145/949344.949431.
70. Eisenhauer, J.G.: Degrees of Freedom. Teach. Stat. 30(3), 75-78 (2008).
71. Elliott, A.C., Woodward, W.A.: Statistical Analysis Quick Reference Guidebook: With SPSS Examples. Sage Publications, Los Angeles, CA, USA (2006).
72. Emam, K.E., Benlarbi, S., Goel, N., Rai, S.N.: The confounding effect of class size on the validity of object-oriented metrics. IEEE Trans. Softw. Eng. 27(7), 630-650 (2001). URL http://dx.doi.org/10.1109/32.935855.
73. Emam, K.E., Melo, W.L., Machado, J.C.: The Prediction of Faulty Classes Using Object- Oriented Design Metrics. J. Syst. Softw. 56(1), 63-75 (2001). URL http://dx.doi.org/10.1016/S0164-1212(00)00086-8.
74. Erdogmus, H., Morisio, M., Torchiano, M.: On the Effectiveness of the Test-First Approach to Programming. IEEE Trans. Softw. Eng. 31(3), 226-237 (2005). URL http://dx.doi.org/10.1109/TSE.2005.37.
75. Evanco, W.M.: Comments on The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics. IEEE Trans. Softw. Eng. 29(7), 670-672 (2003). URL http://dx.doi.org/10.1109/TSE.2003.1214331.
76. Fenton, N., Pfleeger, S.L.: Software Metrics: A Rigorous and Practical Approach. PWS Publishing Co., Boston, MA, USA (1997).
77. Field, A.: Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed- and random-effects methods. Psychol. Methods 6(2), 161-180 (2001).
78. Field, A.: How to Design and Report Experiments. SAGE Publications, Los Angeles, CA, USA (2003).
79. Field, A.: Discovering Statistics Using SPSS. SAGE Publications, Los Angeles, CA, USA (2005).
80. Filho, F.C., Rubira, C.M.F., Ferreira, R., Garcia, A.: Aspectizing exception handling: A quantitative study. In: C. Dony, J.L. Knudsen, A.B. Romanovsky, A. Tripathi (eds.) ECOOP Workshops '06: Advanced Topics in Exception Handling Techniques, Lecture Notes in Computer Science, vol. 4119, pp. 255-274. Springer (2006). URL http://dx.doi.org/10.1007/11818502-14.
81. Flohr, T., Schneider, T.: Lessons Learned from an XP Experiment with Students: Test-First Need More Teachings. In: J. Munch, M. Vierimaa (eds.) PROFES'06: Product Focused Software Process Improvement, Lecture Notes in Computer Science, vol. 4034, pp. 305-318. Springer, Berlin, Heidelberg (2006). URL http://dx.doi.org/10.1007/11767718-26.
82. Fowler, M.: The New Methodology (Cited 1 Apr. 2009). URL http://www.martinfowler.com/articles/newMethodology.html.
83. Fowler, M., Beck, K., Brant, J., Opdyke, W., Roberts, D.: Refactoring: Improving the Design of Existing Code. Addison-Wesley, Boston, MA, USA (1999).
84. Frankl, P.G., Weiss, S.N., Hu, C.: All-Uses vs Mutation Testing: An Experimental Comparison of Effectiveness. J. Syst. Softw. 38(3), 235-253 (1997). URL http://dx.doi.org/10.1016/S0164-1212(96)00154-9.
85. Freeman, P.R.: The Performance of the Two-Stage Analysis of Two-Treatment, Two-Period Crossover Trials. Stat. Med. 8(12), 1421-1432 (1989).
86. Friedman, H.: Magnitude of Experimental Effect and a Table for its Rapid Estimation. Psychol. Bull. 70, 245-251 (1968).
87. Gamma, E., Beck, K.: JUnit (Cited 1 Apr. 2009). URL http://www.junit.org/ .
88. Garcia, F., Bertoa, M.F., Calero, C., Vallecillo, A., Ruiz-Sanchez, F., Piattini, M., Genero, M.: Towards a Consistent Terminology for Software Measurement. Inf. Softw. Technol. 48(8), 631-644 (2006). URL http://dx.doi.org/10.1016/j.infsof.2005.07.001.
89. George, B., Williams, L.: An initial investigation of test driven development in industry. In: SAC'03: ACM Symposium on Applied Computing, pp. 1135-1139. ACM, New York, NY, USA (2003). URL http://dx.doi.org/10.1145/952532. 952753.
90. George, B., Williams, L.A.: A Structured Experiment of Test-Driven Development. Inf. Softw. Technol. 46(5), 337-342 (2004). URL http://dx.doi.org/10.1016/j.infsof.2003.09.011.
91. Geras, A., Smith, M.R., Miller, J.: A Prototype Empirical Evaluation of Test Driven Development. In: IEEE METRICS'2004: IEEE International Software Metrics Symposium, pp. 405-416. IEEE Computer Society (2004). URL http://dx.doi.org/10.1109/METRICS.2004.2.
92. Glass, G.V., Mcgaw, B., Smith, M.L.: Meta-Analysis in Social Research. Sage Publications, Beverly Hills, CA, USA (1981).
93. Grigori Melnik and Frank Maurer: A cross-program investigation of students' perceptions of agile methods. In: ICSE'05: International Conference on Software Engineering, pp. 481- 488 (2005). URL http://dx.doi.org/10.1145/ 1062455.1062543.
94. Grissom, R.J., Kim, J.J.: Effect Sizes for Research: A Broad Practical Approach. Psychology Press, Indianapolis, Indiana, USA (2005).
95. Grizzle, J.E.: The Two-Period Change-Over Design and Its Use in Clinical Trials. Biometrics 21(2), 467-480 (1965).
96. Gupta, A., Jalote, P.: An experimental evaluation of the effectiveness and efficiency of the test driven development. In: ESEM'07: International Symposium on Empirical Software Engineering and Measurement, pp. 285-294. IEEE Computer Society, Washington, DC, USA (2007). URL http://dx.doi.org/10.1109/ ESEM.2007.20.
97. Gyimothy, T., Ferenc, R., Siket, I.: Empirical validation of object-oriented metrics on open source software for fault prediction. IEEE Trans. Softw. Eng. 31(10), 897-910 (2005). URL http://dx.doi.org/10.1109/TSE. 2005.112.
98. Hamlet, R.G.: Testing Programs with the Aid of a Compiler. IEEE Trans. Softw. Eng. 3(4), 279-290 (1977).
99. Hammell, T., Gold, R., Snyder, T.: Test-Driven Development: A J2EE Example. Apress, Berkeley, CA, USA (2004).
100. Hayward, R.S.A., Wilson, M.C., Tunis, S.R., Bass, E.B., Rubin, H.R., Haynes, R.B.: More Informative Abstracts of Articles Describing Clinical Practice Guidelines. Ann. Intern. Med. 118(9), 731-737 (1993). URL http://www.annals.org/cgi/content/abstract/118/9/731.
101. Hedges, L.V., Olkin, I.: Statistical Methods for Meta-Analysis. Academic Press, Orlando, Florida, USA (1985).
102. Hedges, L.V., Vevea, J.L.: Fixed- and Random-Effects Models in Meta-Analysis. Psychol. Methods 3(4), 486-504 (1998).
103. Heiberg, S., Puus, U., Salumaa, P., Seeba, A.: Pair-programming effect on developers productivity. In: M. Marchesi, G. Succi (eds.) XP'03: Extreme Programming and Agile Processes in Software Engineering, Lecture Notes in Computer Science, vol. 2675, pp. 215- 224. Springer (2003). URL http://dx.doi.org/10.1007/3-540-44870-5-27.
104. Hochberg, Y.: A Sharper Bonferroni Procedure for Multiple Tests of Significance. Biometrika 75, 800-802 (1988).
105. Hoffman, K., Eugster, P.: Towards reusable components with aspects: An empirical study on modularity and obliviousness. In: ICSE'08: International conference on Software engineering, pp. 91-100. ACM, New York, NY, USA (2008). URL http://dx.doi.org/10.1145/1368088.1368102.
106. Holm, S.: A Simple Sequential Rejective Multiple Test Procedure. Scand. J. Stat. 6, 65-70 (1979).
107. Hommel, G.: A Stagewise Rejective Multiple Test Procedure on a Modified Bonferroni Test. Biometrika 75, 383-386 (1988).
108. Host, M., Wohlin, C., Thelin, T.: Experimental context classification: Incentives and experience of subjects. In: ICSE'05: International Conference on Software Engineering, pp. 470-478. ACM Press, New York, NY, USA (2005). URL http://dx.doi.org/10.1145/1062455.1062539.
109. Host, M., Regnell, B., Wohlin, C.: Using Students as Subjects - A Comparative Study of Students and Professionals in Lead-Time Impact Assessment. Empir. Softw. Eng. 5(3), 201-214 (2000). URL http://dx.doi.org/10.1023/A: 1026586415054.
110. Huang, L., Holcombe, M.: Empirical Investigation Towards the Effectiveness of Test First Programming. Inf. Softw. Technol. 51(1), 182-194 (2009). URL http://dx.doi.org/10.1016/j.infsof.2008.03.007.
111. Huisman, M., Iivari, J.: Current Findings from Research on Structured Abstracts. J. Med. Libr. Assoc. 92(3), 368-371 (2004).
112. Huitema, B.E.: The Analysis of Covariance and Alternatives. Wiley, New York, NY, USA (1980).
113. Hulkko, H., Abrahamsson, P.: A multiple case study on the impact of pair programming on product quality. In: ICSE'05: International Conference on Software Engineering, pp. 495-504. ACM Press, New York, NY, USA (2005). URL http://dx.doi.org/10.1145/1062455.1062545.
114. Hunter, J.E., Schmidt, F.L.: Methods of Meta-analysis: Correcting Error and Bias in Research Findings, 2nd edn. SAGE Publications, Thousand Oaks, CA, USA (2004).
115. IEEE: IEEE Std 610.12-1990, IEEE standard glossary of software engineering terminology. Institute of Electrical and Electronics Engineers, Inc. (1990).
116. ISO: ISO/IEC 14598-5, Information Technology - Software product evaluation - Part 5: Process for evaluators. International Organization for Standardization (1998).
117. ISO: ISO/IEC 14598-1, Information Technology - Software Product Evaluation - Part 1: General Overview. International Organization for Standardization (1999).
118. ISO: ISO/IEC 14598-4, Software Engineering - Product Evaluation - Part 4: Process for Acquirers. International Organization for Standardization (1999).
119. ISO: ISO/IEC 14598, Software Engineering - Product Evaluation - Parts 1-5. International Organization for Standardization (1999).
120. ISO: ISO/IEC 14598-2, Software Engineering - Product Evaluation - Part 2: Planning and Management. International Organization for Standardization (2000).
121. ISO: ISO/IEC 14598-3, Software Engineering - Product Evaluation - Part 3: Process for Developers. International Organization for Standardization (2000).
122. ISO: ISO/IEC 14598-6, Software Engineering - Product Evaluation - Part 6: Documentation of Evaluation Modules. International Organization for Standardization (2001).
123. ISO: ISO/IEC 9126-1, Software Engineering - Product Quality - Part 1: Quality Model. International Organization for Standardization (2001).
124. ISO: ISO/IEC 9126-2, Software Engineering - Product Quality - Part2: External Metrics. International Organization for Standardization (2002).
125. ISO: ISO/IEC 9126-3, Software Engineering - Product Quality - Part3: Internal Metrics. International Organization for Standardization (2002).
126. ISO: ISO/IEC 9126-4, Software Engineering - Product Quality - Part4: Quality In Use metrics. International Organization for Standardization (2002).
127. ISO: ISO/IEC 25000, Software Engineering - Software product Quality Requirements and Evaluation (SQuaRE) - Guide to SQuaRE. International Organization for Standardization (2005).
128. Janzen, D., Saiedian, H.: Does Test-Driven Development Really Improve Software Design Quality? IEEE Softw. 25(2), 77-84 (Mar.-Apr. 2008). URL http://dx.doi.org/10.1109/MS.2008.34.
129. Jedlitschka, A., Pfahl, D.: Reporting guidelines for controlled experiments in software engineering. In: R. Jeffery, J. Verner, G.H. Travassos (eds.) ISESE'05: ACM/IEEE International Symposium on Empirical Software Engineering, pp. 95-104. IEEE (2005). URL http://dx.doi.org/10.1109/ISESE.2005. 1541818.
130. Jeffries, R., Melnik, G.: Guest Editors' Introduction: TDD - The Art of Fearless Programming. IEEE Software 24(3), 24-30 (2007). URL http://dx.doi.org/10.1109/MS.2007.75.
131. Jeffries, R.E., Anderson, A., Hendrickson, C.: Extreme Programming Installed. Addison- Wesley, Boston, MA, USA (2000).
132. Jørgensen, M.: Software Quality Measurement. Adv. Eng. Softw. 30(12), 907-912 (1999). URL http://dx.doi.org/10.1016/S0965-9978(99)00015-0.
133. Juristo, N., Moreno, A.M.: Basics of Software Engineering Experimentation. Kluwer, Dordrecht, The Netherlands (2001).
134. Juristo, N., Moreno, A.M. (eds.): Lecture Notes on Empirical Software Engineering. World Scientific Publishing Co., Inc., River Edge, NJ, USA (2003).
135. Kampenes, V.B., Dyba, T., Hannay, J.E., Sjøberg, D.I.K.: Systematic review: A systematic review of effect size in software engineering experiments. Inf. Softw. Technol. 49(11-12), 1073-1086 (2007). URLhttp://dx.doi.org/10.1016/j.infsof.2007.02.015.
136. Kampenes, V.B., Dyba, T., Hannay, J.E., Sjøberg, D.I.K.: Systematic review: A systematic review of effect size in software engineering experiments. Inf. Softw. Technol. 49(11-12), 1073-1086 (2007). URL http://dx.doi.org/10.1016/j.infsof.2007.02.015.
137. Kan, S.H.: Metrics and Models in Software Quality Engineering. Addison-Wesley, Boston, MA, USA (2002).
138. Kaner, C.: Software Negligence and Testing Coverage. In: STAR'96: International Conference, Software Testing, Analysis and Review, pp. 299-327. Orlando, USA (1996).
139. Kerievsky, J.: Refactoring to Patterns, 1st edn. Addison-Wesley, Westford, MA, USA (2004).
140. Kerr, A.W., Hall, H.K., Kozub, S.: Doing Statistics with SPSS, 1st edn. Sage Publications Ltd, London, UK (2002).
141. Kitchenham, B.: Software Metrics. Measurement for Software Process Improvement. Blackwell Publishers Inc. (1996).
142. Kitchenham, B.: Procedures for Performing Systematic Reviews. Tech. Rep. Technical Report TR/SE-0401, Keele University (2004).
143. Kitchenham, B., Brereton, P., Owen, S., Butcher, J., Jefferies, C.: Length and Readability of Structured Software Engineering Abstracts. IET Softw. 2(1), 37-45 (2008). URL http://dx.doi.org/10.1049/iet-sen:20070044.
144. Kitchenham, B., Pfleeger, S.L., Pickard, L., Jones, P., Hoaglin, D.C., Emam, K.E., Rosenberg, J.: Preliminary Guidelines for Empirical Research in Software Engineering. IEEE Trans. Softw. Eng. 28(8), 721-734 (2002). URL http://dx.doi.org/10.1109/TSE.2002.1027796.
145. Kitchenham, B.A., Dyba, T., Jørgensen, M.: Evidence-Based Software Engineering. In: ICSE'04: International Conference on Software Engineering, pp. 273-281. IEEE Computer Society (2004).
146. Koskela, L.: Test Driven: Practical TDD and Acceptance TDD for Java Developers. Manning Publications, Greenwich, CT, USA (2007).
147. Kramer, S.H., Rosenthal, R.: Effect sizes and significance levels in small-sample research. In: R.H. Hoyle (ed.) Statistical Strategies for Small Sample Research. SAGE Publications, Thousand Oaks, CA, USA (1999).
148. Laddad, R.: AspectJ in Action: Practical Aspect-Oriented Programming. Manning Publications, Greenwich, CT, USA (2003).
149. Langr, J.: Pair Programming Observations (Cited 1 Apr. 2009). URL http://www.langrsoft.com/articles/pairing.shtml.
150. Law, K.S., Schmidt, F.L., Hunter, J.E.: Nonlinearity of Range Corrections in Meta-Analysis: Test of an Improved Procedure. J. Appl. Psychol. 79(3), 425-438 (1994).
151. Levesque, R.: SPSS Programming And Data Management: A Guide for SPSS And SAS Users. SPSS Inc. (2005).
152. Li, W., Henry, S.: Object Oriented Metrics that Predict Maintainability. J. Syst. Softw. 23(2), 111-122 (1993). URLhttp://dx.doi.org/10.1016/0164- 1212(93)90077-B.
153. Lipsey, M.W., Wilson, D.B.: Practical Meta-Analysis. Sage Publications, California, USA (2001).
154. Littell, J.H., Coracoran, J., Pillai, V.: Systematic Reviews and Meta-Analysis. Oxford University Press, New York, USA (2008).
155. Loftus, C., Ratcliffe, M.: Extreme Programming Promotes Extreme Learning? SIGCSE Bull. 37(3), 311-315 (2005). URL http://dx.doi.org/10.1145/1151954. 1067531.
156. Ma, Y.S., Offutt, J., Kwon, Y.R.: MuJava: An Automated Class Mutation System. Softw. Test., Verif. Reliab. 15(2), 97-133 (2005). URL http://dx.doi.org/10.1002/stvr.v15:2.
157. MacCormack, A., Rusnak, J., Baldwin, C.Y.: The Impact of Component Modularity on Design Evolution: Evidence from the Software Industry. Research Paper No. 08-038, Harvard Business School - Technology & Operations Management Unit (2007). http://ssrn.com/abstract=1071720.
158. Maciaszek, L., Liong, B.L.: Practical Software Engineering. A Case-Study Approach. Addison-Wesley, Boston, MA, USA (2004).
159. Madeyski, L.: Preliminary Analysis of the Effects of Pair Programming and Test- Driven Development on the External Code Quality. In: K. Zielinski, T. Szmuc (eds.) Software Engineering: Evolution and Emerging Technologies, Frontiers in Artificial Intelligence and Applications, vol. 130, pp. 113-123. IOS Press (2005). URL http://madeyski.e-informatyka.pl/download/Madeyski05b.pdf.
160. Madeyski, L.: Is External code quality correlated with programming experience or feelgood factor? In: P. Abrahamsson, M. Marchesi, G. Succi (eds.) XP'06: Extreme Programming and Agile Processes in Software Engineering, Lecture Notes in Computer Science, vol. 4044, pp. 65-74. Springer (2006). URL http://dx.doi.org/10.1007/11774129-7.(http://madeyski.e-informatyka.pl/download/ Madeyski06b.pdf)
161. Madeyski, L.: The impact of pair programming and test-driven development on package dependencies in object-oriented design - an experiment. In: J. Munch, M. Vierimaa (eds.) PROFES'06: Product Focused Software Process Improvement, Lecture Notes in Computer Science, vol. 4034, pp. 278-289. Springer, Berlin, Heidelberg (2006). URL http://dx.doi.org/10.1007/11767718-24. (http://madeyski.einformatyka.pl/download/Madeyski06.pdf)
162. Madeyski, L.: On the Effects of pair programming on thoroughness and fault-finding effectiveness of unit tests. In: J. Munch, P. Abrahamsson (eds.) PROFES'07: Product Focused Software Process Improvement, Lecture Notes in Computer Science, vol. 4589, pp. 207-221. Springer (2007). URL http://dx.doi.org/10.1007/978-3-540-73460-4-20. (http://madeyski.einformatyka. pl/download/Madeyski07.pdf)
163. Madeyski, L.: The Impact of Pair Programming on Thoroughness and Fault Detection Effectiveness of Unit Tests Suites. Softw. Process Improv. Pract. 13(3), 281-295 (2008). URL http://dx.doi.org/10.1002/spip.382.(http://madeyski. einformatyka.pl/download/Madeyski08.pdf)
164. Madeyski, L.: The Impact of Test-First Programming on Branch Coverage and Mutation Score Indicator of Unit Tests: An experiment. Inf. Softw. Technol. 52(2), 169- 184 (2010). URL http://dx.doi.org/10.1016/j.infsof.2009.08.007. (http://madeyski.e-informatyka.pl/download/Madeyski10c.pdf)
165. Madeyski, L., Radyk, N.: Judy - A Mutation Testing Tool for Java. IET Softw. 4(1), (2010). (accepted) URL http://dx.doi.org/10.1049/iet-sen.2008.0038. (http://madeyski.e-informatyka.pl/download/Madeyski10b.pdf)
166. Madeyski, L., Radyk, N.: Judy - Mutation Testing Tool for Java (Cited 1. Apr 2009). URL http://www.e-informatyka.pl/sens/Wiki.jsp?page=Projects.Judy.
167. Madeyski, L., Stochmiałek, M.: Architectural Design of Modern Web Applications. J. Found. Comput. Decis. Sci. 30(1), 49-60 (2005). URL http://madeyski.einformatyka.pl/download/23.pdf.
168. Madeyski, L., Szała, Ł.: Impact of Aspect-Oriented Programming on Software Development Efficiency and Design Quality: An Empirical Study. IET Softw. 1(5), 180-187 (2007). URL http://dx.doi.org/10.1049/iet-sen: 20060071. (http://madeyski.einformatyka.pl/download/Madeyski07g.pdf)
169. Madeyski, L., Szała, Ł.: The impact of test-driven development on software development productivity - an empirical study. In: P. Abrahamsson, N. Baddoo, T. Margaria, R. Messnarz (eds.) EuroSPI'07: Software Process Improvement, Lecture Notes in Computer Science, vol. 4764, pp. 200-211. Springer (2007). URL http://dx.doi.org/10.1007/978-3-540-75381-0-18. (http://madeyski.einformatyka.pl/download/Madeyski07d.pdf)
170. Mantyla, M.: Bad Smells in Software - A Taxonomy and an Empirical Study (2003). MSc thesis, Helsinki University of Technology.
171. Mantyla, M., Vanhanen, J., Lassenius, C.: A taxonomy and an initial empirical study of bad smells in code. In: ICSM'03: International Conference on Software Maintenance, pp. 381-384. IEEE Computer Society (2003). URL http://dx.doi.org/10.1109/ICSM.2003.1235447.
172. Marick, B.: How to misuse code coverage. In: Proceedings of the 16th International Conference on Testing Computer Software. Washington, USA (1999). URL http://www.exampler.com/testing-com/writings/coverage.pdf.
173. Martin, R.C.: OO Design Quality Metrics: An Analysis of Dependencies (1994, Cited 1 Apr. 2009). URL http://www.objectmentor.com/resources/articles/ oodmetrc.pdf.
174. Martin, R.C.: Agile Software Development, Principles, Patterns, and Practices. Prentice Hall, Upper Saddle River, NJ, USA (2002).
175. Mattu, B., Shankar, R.: Test driven design methodology for component-based system. In: 1st Annual IEEE Systems Conference, pp. 1-7 (2007). URL http://dx.doi.org/10.1109/SYSTEMS.2007.374646.
176. Maximilien, E.M., Williams, L.A.: Assessing Test-Driven Development at IBM. In: ICSE'03: International Conference on Software Engineering, pp. 564-569. IEEE Computer Society (2003). URL http://dx.doi.org/10.1109/ICSE.2003.1201238.
177. Maxwell, J.A.: Qualitative Research Design : An Interactive Approach (Applied Social Research Methods). SAGE Publications, London, UK (2004).
178. Maxwell, K., Forselius, P.: Benchmarking Software-Development Productivity - Applied Research Results. IEEE Softw. 17(1), 80-88 (2000). URL http://dx.doi.org/10.1109/52.820015.
179. Maxwell, K.D.: Collecting Data for Comparability: Benchmarking Software Development Productivity. IEEE Software 18(5), 22-25 (2001). URL
180. Maxwell, S.E., Delaney, H.D.: Designing Experiments and Analyzing Data: A Model Comparison Perspective, 2nd edn. Lawrence Erlbaum, Mahwah (2004).
181. McBreen, P.: Questioning Extreme Programming. Addison-Wesley, Boston, MA, USA (2002).
182. Miller, R.G.: Beyond ANOVA: Basics of Applied Statistics. CRC Press, Boca Raton, FL, USA (1997).
183. Montgomery, D.C.: Design and Analysis of Experiments, 5th edn. Wiley, New York, NY, USA (2001).
184. Mudholkar, G.S., George, E.O.: The logit method for combining probabilities. In: Symposium on Optimizing Methods in Statistics, pp. 345-366 (1979).
185. Muller, M.M.: Are Reviews an Alternative to Pair Programming? In: EASE'03: Conference on Empirical Assessment In Software Engineering, pp. 3-12 (2003).
186. Muller, M.M.: Are Reviews an Alternative to Pair Programming? Empir. Softw. Eng. 9(4), 335-351 (2004). URL http://dx.doi.org/10.1023/B:EMSE. 0000039883.47173.39.
187. Muller, M.M.: Two Controlled Experiments Concerning the Comparison of Pair Programming to Peer Review. J. Syst. Softw. 78(2), 166-179 (2005). URL http://dx.doi.org/10.1016/j.jss.2004.12.019.
188. Muller, M.M.: The Effect of Test-Driven Development on Program Code. In: XP'06: Extreme Programming and Agile Processes in Software Engineering, 7th International Conference, XP 2006, Oulu, Finland, June 17-22, 2006, pp. 94-103. Springer (2006). URL http://dx.doi.org/10.1007/11774129-10.
189. Muller, M.M., Hagner, O.: Experiment About Test-First Programming. IEE Proc-Softw. 149(5), 131-136 (2002). URL http://dx.doi.org/10.1049/ip-sen: 20020540.
190. Muller, M.M., Hofer, A.: The Effect of Experience on the Test-Driven Development Process. Empir. Softw. Eng. 12(6), 593-615 (2007). URL http://dx.doi.org/10.1007/s10664-007-9048-2.
191. Munnelly, J., Fritsch, S., Clarke, S.: An aspect-oriented approach to the modularisation of context. In: PERCOM'07: IEEE International Conference on Pervasive Computing and Communications, pp. 114-124. IEEE Computer Society, Washington, DC, USA (2007). URL http://dx.doi.org/10.1109/PERCOM.2007.7.
192. Munson, J.C.: Software Engineering Measurement. Auerbach Publications, Boca Raton, FL, USA (2003).
193. Myers, G.J.: The art of software testing. Wiley, New York, NY, USA (1979).
194. Nagappan, N., Maximilien, E.M., Bhat, T., Williams, L.: Realizing quality improvement through test driven development: results and experiences of four industrial teams. Empir. Softw. Eng. 13(3) (2008). URL http://dx.doi.org/10. 1007/s10664-008-9062-z.
195. Nawrocki, J.R., Jasinski, M., Olek, L., Lange, B.: Pair Programming vs. Side-by-Side Programming. In: I. Richardson, P. Abrahamsson, R. Messnarz (eds.) EuroSPI'05: Software Process Improvement, Lecture Notes in Computer Science, vol. 3792, pp. 28-38. Springer (2005). URL http://dx.doi.org/10.1007/11586012-4.
196. Nawrocki, J.R., Wojciechowski, A.: Experimental evaluation of pair programming. In: ESCOM'01: European Software Control and Metrics, pp. 269-276. London, UK (2001). URL http://www.agilealliance.org/system/article/file/1215/ file.pdf.
197. Newkirk, J.W., Vorontsov, A.A.: Test-driven development in Microsoft. Microsoft Press, Redmond, WA, USA (2004).
198. Nosek, J.T.: The Case for Collaborative Programming. Commun. ACM 41(3), 105-108 (1998). URL http://dx.doi.org/10.1145/272287.272333.
199. Offutt, A.J., Pan, J., Tewary, K., Zhang, T.: An Experimental Evaluation of Data Flow and Mutation Testing. Softw., Pract. Exper. 26(2), 165-176 (1996).
200. Offutt, A.J., Untch, R.H.: Mutation Testing for the New Century, Mutation 2000: Uniting the Orthogonal, pp. 34-44. Kluwer Academic Publishers, Norwell, MA, USA (2001).
201. Olejnik, S.F., Algina, J.: Parametric ANCOVA vs. rank transform ANCOVA when assumptions of conditional normality and homoscedasticity are violated. In: Annual Meeting of the American Educational Research Association (67th, Montreal, Quebec, Apr. 11-15, 1983) (1983).
202. Pai, G.J., Dugan, J.B.: Empirical Analysis of Software Fault Content and Fault Proneness Using Bayesian Methods. IEEE Trans. Softw. Eng. 33(10), 675-686 (2007). URL http://dx.doi.org/10.1109/TSE.2007.70722.
203. Pancur, M., Ciglaric, M., Trampus, M., Vidmar, T.: Towards empirical evaluation of test-driven development in a university environment. In: EUROCON'03: International Conference on Computer as a Tool, pp. 83-86 (2003). URL http://dx.doi.org/10.1109/EURCON.2003.1248153.
204. Perneger, T.V.: What's wrong with Bonferroni adjustments. BMJ 27, 1236-1238 (1998).
205. Pfleeger, S.L., Fenton, N., Page, S.: Evaluating Software Engineering Standards. Computer 27(9), 71-79 (1994). URL http://dx.doi.org/10.1109/2.312041.
206. Piechowiak, A., Madeyski, L.: Project 2006 - Submission System: User Stories (2006). Power Media (Cited 1 Apr 2009). URL http://madeyski. einformatyka.pl/download/project2006/SUBMISSION.pdf.
207. Poppendieck, M.: Software Development Productivity. URL http://www.poppendieck.com/pdfs/SoftwareDevelopmentProductivity.pdf.
208. Poppendieck, M., Poppendieck, T.: Lean Software Development: An Agile Toolkit. Addison-Wesley, Boston, MA, USA (2003).
209. Power Media S.A.: Project 2007/P4: Requirements (2007). (Cited 1 Apr 2009). URL http://download.e-informatyka.pl/project2007/requirements.pdf.
210. Power Media S.A.: Project 2007/P4: User Stories (2007). (Cited 1 Apr 2009). URL http://download.e-informatyka.pl/project2007/project-4-us-1.pdf.
211. Raab, G.M., Day, S., Sales, J.: How to Select Covariates to Include in the Analysis of a Clinical Trial. Control. Clin. Trials 21, 330-342 (2000).
212. Robson, C.: Real World Research: A Resource for Social Scientists and Practitioner- Researchers. Blackwell Publishing Limited (2002).
213. Rosenthal, R.: An attempt at the experimental induction of the defence mechanism of projection. Ph.D. thesis, University of California, Los Angeles, CA, USA (1956).
214. Rosenthal, R.: Meta-analytic Procedures for Social Research, 2nd edn. SAGE Publications (1991).
215. Rosenthal, R., Rubin, D.B.: Equivalent: A Simple Effect Size Indicator. Psychol. Methods 8, 492-496 (2003).
216. Rosnow, R.L., Rosenthal, R.: Computing Contrasts, Effect Sizes, and Countermills on Other People's Published Data: General Procedures for Research Consumers. Psychol. Methods 1(4), 331-340 (1996).
217. Rubin, A., Babbie, E.R.: Research Methods for Social Work, 5th edn. Thomson, Belmont, CA, USA (2004).
218. Runeson, P.: Using students as Experiment subjects - An analysis of graduate and freshmen student data. In: EASE'03: Empirical Assessment and Evaluation in Software Engineering, pp. 95-102. British Computer Society (2003).
219. Sanchez, J.C., Williams, L., Maximilien, E.M.: On the sustained use of a testdriven development practice at ibm. In: AGILE'07: Conference on Agile Software Development, pp. 5-14. IEEE Computer Society, Washington, DC, USA (2007). URL http://dx.doi.org/10.1109/AGILE.2007.43.
220. Sarkar, S.K., Chang, C.K.: The Simes Method for Multiple Hypothesis Testing with Positively Dependent Test Statistics. J. Am. Stat. Assoc. 92(440), 1601-1608 (1997). URL http://dx.doi.org/10.1109/TSE.2007.4.
221. Scholfield, P.: Post Hoc Paired Comparisons after Kruskal-Wallis (Cited 1 Apr. 2009). URL http://privatewww.essex.ac.uk/scholp/kw-posthoc.htm.
222. Schwaber, K.: Agile Project Management with Scrum. Microsoft Press, Redmond, WA, USA (2004).
223. Schwaber, K., Beedle, M.: Agile Software Development with Scrum. Prentice Hall PTR, Upper Saddle River, NJ, USA (2001).
224. Senn, S.: Cross-over Trials in Clinical Research. Wiley, Indianapolis, Indiana, USA (2002).
225. Senn, S.: Within-patient studies: Cross-over Trials and n-of-1 Trials (Chap. 6). In: M.B. Max, J. Lynn (eds.) Symptom Research: Methods and Opportunities. National Institutes of Health, Bethesda, MD 20892-2190 (2008). URL http://symptomresearch.nih.gov/chapter-6/index.htm.
226. Seshadri, G.: Understanding JavaServer Pages Model 2 architecture - Exploring the MVC design pattern (1999). URL http://www.javaworld.com/javaworld/ jw-12-1999/jw-12-ssj-jspmvc.html.
227. Shadish, W.R., Cook, T.D., Campbell, D.T.: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin, Boston, MA, USA (2002).
228. Shirley, E.A.C.: A Distribution-Free Method for Analysis of Covariance Based on Ranked Data. Appl. Stat. 30(2), 158-162 (1981).
229. Shull, F., Singer, J., Sjøberg, D.I.K.: Guide to Advanced Empirical Software Engineering. Springer, London, UK (2008).
230. Simes, J.R.: An Improved Bonferroni Procedure for Multiple Tests of Significance. Biometrika 73, 751-754 (1986).
231. Simpson, E.H.: The Interpretation of Interaction in Contingency Tables. J. Royal Stat. Soc. B(13), 238-241 (1951).
232. Siniaalto, M., Abrahamsson, P.: A Comparative Case Study on the Impact of Test- Driven Development on Program Design and Test Coverage. In: ESEM'07: International Symposium on Empirical Software Engineering and Measurement, pp. 275-284. IEEE Computer Society (2007). URL http://dx.doi.org/10.1109/ESEM.2007. 2.
233. Slinger, S.: Code Smell Detection in Eclipse (2005). URL http://swerl.tudelft.nl/twiki/pub/Main/StefanSlinger/CodeNose-thesis.pdf. MSc thesis, Delft University of Technology.
234. Slyngstad, O.P.N., Li, J., Conradi, R., Rønneberg, H., Landre, E., Wesenberg, H.: The impact of test driven development on the evolution of a reusable framework of components - an industrial case study. In: ICSEA'08: International Conference on Software Engineering Advances, pp. 214-223. IEEE Computer Society, Washington, DC, USA (2008). URL http://dx.doi.org/10.1109/ ICSEA.2008.8.
235. van Solingen, R., Berghout, E.: The Goal/Question/Metric Method: A Practical Guide for Improvement of Software Development. McGraw-Hill, London, UK (1999).
236. Sørumgard, L.S.: Verification of Process Conformance in Empirical Studies of Software Development. Ph.D. thesis, The Norwegian University of Science and Technology (1997). URL http://www.idi.ntnu.no/grupper/su/publ/phd/ sorumgard-thesis.pdf.
237. Stevens, J.P.: Applied Multivariate Statistics for the Social Sciences, 4th edn. Lawrence Erlbaum, Mahwah, NJ, USA (2002).
238. Stouffer, S., Suchman, E.A., DeVinney, L.C., Star, S.A., Robin M. Williams, J.: The American soldier: Adjustment during army life. Princeton University Press, Princeton, NJ, USA (1949).
239. Stout, D.E., Ruble, T.L.: Assessing the Practical Significance of Empirical Results in Accounting Education Research: The Use of Effect Size Information. J. Account. Educ. 13(3), 281-298 (1995). URL http://dx.doi.org/10. 1016/0748-5751(95)00010-J.
240. Subramanyam, R., Krishnan, M.S.: Empirical Analysis of CK Metrics for Object-Oriented Design Complexity: Implications for Software Defects. IEEE Trans. Softw. Eng. 29(4), 297-310 (2003). URL http://dx.doi.org/10.1109/TSE. 2003.1191795.
241. Svahnberg, M., Aurum, A., Wohlin, C.: Using students as subjects - an empirical evaluation. In: ESEM'08: ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 288-290. ACM, New York, NY, USA (2008). URL http://doi.acm.org/10.1145/1414004.1414055.
242. Tabachnick, B.G., Fidell, L.S.: Using Multivariate Statistics, 5th edn. Allyn & Bacon, Inc., Needham Heights, MA, USA (2006).
243. Thode, H.C.: Testing for normality. Marcel Dekker, New York, NY, USA (2002).
244. Tichy, W.F.: Hints for Reviewing Empirical Work in Software Engineering. Empir. Softw. Eng. 5(4), 309-312 (2000). URL http://dx.doi.org/10.1023/A: 1009844119158.
245. Urdan, T.C.: Statistics in Plain English, 2th edn. Routledge, Oxon, UK (2005).
246. Van hanen, J., Lassenius, C.: Effects of pair programming at the development team level: An experiment. In: ISESE'05: ACM/IEEE International Symposium on Empirical Software Engineering, pp. 336-345. IEEE (2005). URL http://dx.doi.org/10.1109/ISESE.2005.1541842.
247. Venners, B.: Test-Driven Development. A Conversation with Martin Fowler, (Cited 1 Apr 2009). URL http://www.artima.com/intv/testdrivenP.html.
248. Vincent Massol and Ted Husted: JUnit in Action, 1st edn. Manning Publications, Greenwich, CT, USA (2003).
249. Wake, W.C.: Extreme Programming Explored, 1st edn. Addison-Wesley, Boston, MA, USA (2001).
250. Wake, W.C.: Overview of XP Explained, 2nd edn (Cited 1 Apr. 2009). URL http://xp123.com/xplor/xp0502/index.shtml.
251. Walker, H.: Degrees of Freedom. J. Educ. Psychol. 31(4), 253-269 (1940). URL http://courses.ncssm.edu/math/Stat-Inst/PDFS/DFWalker.pdf.
252. Walsh, P.J.: A Measure of Test Case Completeness. Ph.D. thesis, University of New York (1985).
253. Wang, Y., Erdogmus, H.: The role of process measurement in test-driven development. In: C. Zannier, H. Erdogmus, L. Lindstrom (eds.) XP/Agile Universe'04: Extreme Programming and Agile Methods, Lecture Notes in Computer Science, vol. 3134, pp. 32-42. Springer (2004). URL http://dx.doi.org/10.1007/ b99820.
254. Williams, L.: The Collaborative Software Process. Ph.D. thesis, University of Utah (2000). URL http://collaboration.csc.ncsu.edu/laurie/Papers/ dissertation.pdf.
255. Williams, L., Kessler, R.: Pair Programming Illuminated. Addison-Wesley, Boston, MA, USA (2002).
256. Williams, L., Kessler, R.R., Cunningham, W., Jeffries, R.: Strengthening the Case for Pair Programming. IEEE Softw. 17(4), 19-25 (2000).
257. Williams, L., Maximilien, E.M., Vouk, M.: Test-driven development as a defectreduction practice. In: ISSRE'03: International Symposium on Software Reliability Engineering, pp. 34-48. IEEE Computer Society, Washington, DC, USA (2003). URL http://dx.doi.org/10.1109/ISSRE.2003.1251029.
258. Williams, L., Shukla, A., Anton, A.I.: An initial exploration of the relationship between pair programming and Brooks' Law. In: ADC'04: Agile Development Conference, pp. 11-20. IEEE Computer Society, Washington, DC, USA (2004). URL http://dx.doi.org/10.1109/ADEVC.2004.6.
259. Williams, L., Upchurch, R.L.: In support of student pair-programming. In: SIGCSE'01: SIGCSE Technical Symposium on Computer Science Education, pp. 327-331. ACM Press, New York, NY, USA (2001). URL http://dx.doi.org/10.1145/ 364447.364614.
260. Williams, L.A., Kessler, R.R.: All I really need to know about pair programming I learned in kindergarten. Commun. ACM 43(5), 108-114 (2000). URL http://dx.doi.org/10.1145/332833.332848.
261. Wohlin, C., Runeson, P., Host, M., Ohlsson, M.C., Regnell, B., Wesslen, A.: Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers, Norwell, MA, USA (2000).
262. Wojcicki, P.: Test-Driven Development (in Polish) (2008). Wroclaw University of Technology, MSc Thesis supervised by Lech Madeyski.
263. Wojcicki, P.: Project 2007/P1: Eclipse plug-in which supports the developer during the process of creating unit tests (2007). Wroclaw University of Technology (Cited 1 Apr. 2009). URL http://download.e-informatyka.pl/ project2007/project-1-requirements.pdf.
264. Wojcicki, P.: Project 2007/P2: Transforming HTML into XHTML (2007). Wroclaw University of Technology (Cited 1 Apr. 2009). URL http://download.e- informatyka.pl/project2007/project-html-requirements.pdf.
265. Wojcicki, P.: Project 2007/P3: Bad Smells Specification (2007). Wroclaw University of Technology (Cited 1 Apr. 2009). URL http://download.einformatyka. pl/project2007/project-3-code-metrics-and-bad-smells.pdf.
266. Wojcicki, P., Madeyski, L.: Project 2007/P3: Bad Smells Requirements and User Stories (2007). Wroclaw University of Technology (Cited 1 Apr. 2009). URL http://download.e-informatyka.pl/project2007/project-badsmells-requirements.pdf.
267. Wojcicki, P.: Project 2007/P4: User Stories Set 2 (2007). Wroclaw University of Technology (Cited 1 Apr. 2009). URL http://download.e-informatyka. pl/project2007/project-4-us-2.pdf.
268. Xu, S., Rajlich, V.: Empirical validation of test-driven pair programming in game development. In: ICIS-COMSAR'06: 5th IEEE/ACIS International Conference on Computer and Information Science and 1st IEEE/ACIS International Workshop on Component-Based Software Engineering, Software Architecture and Reuse, pp. 500-505. IEEE Computer Society, Washington, DC, USA (2006). URL http://dx.doi.org/10.1109/ICIS-COMSAR.2006.34.
269. Yin, R.K.: Case Study Research: Design and Methods, 3rd edn. SAGE Publications, Thousand Oaks, CA, USA (2003).
270. Ynchausti, R.A.: Integrating unit testing into A software development team's process. In: M. Marchesi, G. Succi (eds.) XP'01: Extreme Programming and Flexible Processes in Software Engineering, pp. 84-87. Sardinia, Italy (2001). URL http://www.agilealliance.org/system/article/file/1072/file.pdf.
271. Yule, G.U.: Notes on the Theory of Association of Attributes in Statistics. Biometrika 2, 121-134 (1903). URL http://dx.doi.org/10.1093/biomet/ 2.2.121.
272. Zhou, Y., Leung, H.: Empirical Analysis of Object-Oriented Design Metrics for Predicting High and Low Severity Faults. IEEE Trans. Softw. Eng. 32(10), 771-789 (2006). URL http://dx.doi.org/10.1109/TSE.2006.102.
273. Zhu, H., Hall, P.A.V., May, J.H.R.: Software Unit Test Coverage and Adequacy. ACM Comput. Surv. 29(4), 366-427 (1997). URL http://dx.doi.org/10. 1145/267580.267590.