Empirical studies on quality in agile practices: A systematic literature review

Proceedings - 7th International Conference on the Quality of Information and Communications Technology, QUATIC 2010

Volumn , Issue , 2010, Pages 44-53

  Sfetsos, Panagiotis ^a   Stamelos, Ioannis ^b  

^a Alexander Technological Educational Institution   (Greece)

^b ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

Author keywords

Agile methods; Agile practices; Empirical studies; ISO IEC 12207; ISO IEC 9126; Pair programming; Quality; Systematic literature review; Test driven development; Test first development

Indexed keywords

AGILE METHODS; AGILE PRACTICES; EMPIRICAL STUDIES; ISO/IEC; ISO/IEC 9126; PAIR-PROGRAMMING; QUALITY; SYSTEMATIC LITERATURE REVIEW; TEST DRIVEN DEVELOPMENT; TEST-FIRST DEVELOPMENT;

INFORMATION TECHNOLOGY; TESTING;

COMPUTER SOFTWARE SELECTION AND EVALUATION;

EID: 78751496927     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/QUATIC.2010.17     Document Type: Conference Paper

References (68)

1. B. Kitchenham. "Guidelines for performing Systematic Literature Reviews in Software Engineering", Version 2.3, Keele University and University of Durham, EBSE Technical Report, 2007.
2. I. Stamelos and P. Sfetsos, "Agile Software Development Quality Assurance", IGI Publishing, 2007, ISBN: 978-159904216-9.
3. K. Beck. "Extreme programming explained: embrace change", Addison-Wesley, Boston, USA, 2000.
4. A. Cockburn and L. Williams, "The Costs and Benefits of Pair Programming, " in Extreme Programming examined, G. Succi and M. Marchesi, Eds. Boston: Addison-Wesley, 2001, pp. xv, 569 p.
5. D. Astels. "TestDriven Development: A Practical Guide". Upper Saddle River, New Jersey, USA, Prentice Hall, 2003.
6. K. Beck. "Test-driven development: By example". Boston: Addison-Wesley, 2003.
7. P. Sfetsos, I. Stamelos, L. Angelis, I. Deligiannis. "An Experimental Investigation of Personality Types Impact on Pair Effectiveness in Pair Programming". Empirical Software Engineering, Volume 14, Number 21/April, 2009 p.p. 187-226.
8. M. Fowler. "Refactoring: Improving the design of existing code". Menlo Park, Calif.: Addison-Wesley Longman, Inc, 1999.
9. J.P.T. Higgins, S. Green (Eds.), Cochrane Handbook for Systematic Reviews of Interventions, Version 5.0.0 (updated February 2008), The Cochrane Collaboration, 2008. Available from: .
10. P. D. Leedy, J. E. Ormrod, Practical Research Planning and Design, Pearson Merril Prentice Hall, 2005.S.
11. L. Spencer, J. Ritchie, J. Lewis, L. Dillon, Quality in qualitative evaluation: A framework for assessing research evidence. London: Government Chief Social Researcher's Office, 2003.
12. Critical Appraisal Skills Programme (CASP), Available from: www.phru.nhs.uk/Pages/PHD/CASP.htm.
13. G.W. Noblit, R.D. Hare, Meta-Ethnography: Synthesizing Qualitative Studies, Sage Publications, London, 1988.
14. N. Nagappan, E. M. Maximilien, T. Bhat and L. Williams. "Realizing quality improvement through test driven development: results and experiences of four industrial teams", Empirical Software Engineering, Volume 13, Number 3, June, 2008.
15. C. Sanchez, L. Williams, E.M. Maximilien. "On the Sustained Use of a Test-Driven Development Practice at IBM", Proceedings of the AGILE 2007, pp. 5-14, 2007.
16. T. Bhat, and N. Nagappan. "Evaluating the efficacy of test-driven development: industrial case studies". Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering, ACM: 356-363, 2006.
17. L.Huang and M. Holcombe. "Empirical investigation towards the effectiveness of Test First programming". Information and Software Technology, 51(1):182-194, 2009.
18. D. Janzen and H. Saiedian. Does Test-Driven Development Really Improve Software Design Quality? EEE Software, 25(2):77-84, March- April 2008.
19. A. Gupta and P. Jalote. An experimental evaluation of the effectiveness and efficiency of the test driven development. In ESEM'07: Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, pages 285-294, Washington, DC, USA, 2007.
20. C.Desai, D. S. Janzen. Implications of integrating test-driven development into CS1/CS2 curricula. Proceedings of the 40th ACM technical symposium on Computer science education. Chattanooga, TN, USA, ACM: 148-152, 2009.
21. L.O. Damm and L. Lundberg. Results from introducing component-level test automation and Test-Driven Development. Journal of Systems and Software, 79(7):1001-1014, 2006.
22. L.O. Damm and L. Lundberg. Quality impact of introducing componentlevel test automation and test-driven development, in: P. Abrahamsson, N. Baddoo, T. Margaria, R. Messnarz (Eds.), Software Process Improvement, Vol. 4764 of Lecture Notes in Computer Science, Springer, 2007, pp. 187-199.
23. G. Melnik and F. Maurer. A cross-program investigation of students' perceptions of agile methods. In ICSE'05: Proceedings of the 27th International Conference on Software Engineering, pages 481-488, 2005.
24. R. A. Ynchausti, Integrating Unit Testing Into A Software Development Team's Process, in: M. Marchesi, G. Succi (Eds.), XP 2001: Proceedings of the 2nd International Conference on Extreme Programming and Flexible Processes in Software Engineering, Sardinia, Italy, 2001, pp. 84-87.
25. M. Pancur, M. Ciglaric, M. Trampus, and T. Vidmar. Towards empirical evaluation of test-driven development in a university environment. In EUROCON'03: Proceedings of the International Conference on Computer as a Tool, pages 83-86, 2003.
26. L. Williams, E. M. Maximilien, and M. Vouk. Test-Driven Development as a Defect-Reduction Practice. In ISSRE'03: Proceedings of the 14th International Symposium on Software Reliability Engineering, pages 34-48, Washington, DC, USA, 2003.
27. E.M. Maximilien, and L. Williams. Assessing test-driven development at IBM. Proceedings of the 25th International Conference on Software Engineering. Portland, Oregon, IEEE Computer Society: 564-569, 2003.
28. B. George and L. A. Williams. A structured experiment of test-driven development. Information and Software Technology, 46(5):337-342, 2004.
29. M. Müller, O. Hanger, Experiment about Test-First programming, IEEE Proceedings on Software 149 (5) pp. 537-544, 2002.
30. S. H. Edwards. Using TestDriven Development in the Classroom: Providing Students with Automatic, Concrete Feedback on Performance. International Conference on Education and Information Systems: Technologies and Applications, USA, 2003.
31. H. Erdogmus, M. Morisio and M. Torchiano. "On the effectiveness of the testfirst approach to programming." IEEE Transactions on Software Engineering 31(3): 226 - 237, 2005.
32. E. Arisholm, Gallis, H., Dyba, T., Sjoberg, D., "Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise", IEEE Transactions in Software Engineering, 33(2), pp. 65 - 86, 2007.
33. A. Begel and N. Nagappan. "Pair programming: what's in it for me?", ESEM'08: Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measuremen, pp. 120-128, 2008.
34. T. Bipp, A. Lepper, and D. Schmedding. "Pair programming in software development teams - An empirical study of its benefits." Information and Software Technology 50(3): 231-240, 2008.
35. G. Canfora, A. Cimitile, F. Garcia, M. Piattini, C.A. Visaggio. "Evaluating performances of pair designing in industry", Journal of Systems and Software, Volume 80, Issue 8 pp: 1317-1327, 2007.
36. C. McDowell, L. Werner, H. Bullock, J. Fernald. "Pair programming improves student retention, confidence, and program quality." Commun. ACM 49(8): 90-95, 2006.
37. J. Chao, and G. Atli, Critical Personality Traits in Successful Pair Programming. AGILE'06, IEEE Computer Society, 2006.
38. J.T. Nosek, "The Case for Collaborative Programming, " Comm. ACM, Vol. 41, No. 3, pp. 105-108, 1998.
39. J. Nawrocki and A. Wojciechowski, "Experimental Evaluation of Pair Programming", proc. of European Software Control and Metrics (Escom), 2001.
40. L. Williams, R. R. Kessler, W. Cunningham, and R. Jeffries, "Strengthening the Case for Pair Programming, " IEEE Software, vol. 17, no. 4, pp. 19-25, 2000.
41. C. McDowell, L. Werner, H. Bullock, and J. Fernald, "The Effects of Pair-Programming on Performance in an Introductory Programming Course, " proc. Proceedings of the 33rd SIGCSE technical symposium on Computer science education, pp. 38-42, 2002.
42. L. Madeyski. "Is External Code Quality Correlated with Programming Experience or Feelgood Factor?", In proc. XP 2006, LNCS 4044, pp 65- 74, 2006.
43. H. Hulkko, and P. Abrahamsson. "A Multiple Case Study on the Impact of Pair Programming on Product Quality". In: ICSE'05: Proceedings of the 27th International Conference on Software Engineering, New York, NY, USA, ACM Press 495-504, 2005.
44. K.M.Lui and K.C.C Chan. "When Does a Pair Outperform Two Individuals?", LNCS, volume 2675/2003, XP 2003, pp. 225-233, 2003.
45. P. Sfetsos, I. Stamelos, L. Angelis, I. Deligiannis. "An Experimental Investigation of Personality Types Impact on Pair Effectiveness in Pair Programming". Empirical Software Engineering, Volume 14, Number 21/April, p.p. 187-226, 2009.
46. M.M. Müller. "Two controlled experiments concerning the comparison of pair programming to peer review." Journal of Systems and Software, 78(2): pp. 166-179, 2005.
47. M.M. Müller. "Do programmer pairs make different mistakes than solo programmers?" Journal of Systems and Software 80(9): pp. 1460-1471, 2007.
48. K.M. Lui and K.C.C. Chan. Pair programming productivity: novice-novice vs. expert-expert. Int. J. Human-Comput Studies 64(9):pp. 915-925, 2006.
49. T.H. DeClue, Pair programming and pair trading: effects on learning and motivation in a CS2 course, Journal of Computing Sciences in Colleges, May 2003, 18(5), 2003.
50. B. Hanks, C. McDowell, D. Draper, M. Krnjajic. Program quality with pair programming in CS1, Proceedings of the 9th annual SIGCSE conference on Innovation and technology in computer science education, pp. 176-180, 2004.
51. S. Xu and V. Rajlich. "Empirical Validation of Test-Driven Pair Programming in Game Development". Proceedings of the 5th IEEE/ACIS International Conference on Computer and Information Science, 2006.
52. L. Layman, L. Williams, L. Cunningham, Exploring extreme programming in context: an industrial case study, Agile Development Conference, 2004.
53. S. Ilieva, P. Ivanov, E. Stefanova, Analyses of an agile methodology implementation, in: Proceedings 30th Euromicro Conference, IEEE Computer Society Press, 2004, pp. 326-333.
54. C.A. Wellington, T. Briggs, C.D. Girard, Comparison of student experiences with plan-driven and agile methodologies, in: Proceeedings of the 35th ASEE/IEEE Frontiers in Education Conference, 2005.
55. B. Tessem, Experiences in learning xp practices: a qualitative study, in: XP 2003, vol. 2675, Springer Verlag, Berlin, pp. 131-137, 2003.
56. G. Melnik, and F. Maurer, A cross-program investigation of student's perceptions of agile methods, in: International Conference on Software Engineering (ICSE), St. Louis, MI, USA, 2005.
57. F. Macias, M. Holcombe, M. Gheorghe, A formal experiment comparing extreme programming with traditional software construction, in: Proceedings of the Fourth Mexican International Conference on Computer Science (ENC 2003), 2003.
58. S. Lee and H.S. Yong, Distributed agile: project management in a global environment, Empirical Software Engineering, Volume 15, Nubr. 2, pp. 204-217, 2010.
59. R. Moser, P. Abrahamsson, W. Pedrycz, A.Sillitti and G. Succi, A Case Study on the Impact of Refactoring on Quality and Productivity in an Agile Team, LNCS, Volume 5082/2008, pp. 252-266, 2008.
60. Y.S.Lincoln, E.G.Guba. Naturalistic inquiry. Sage, Thousand Oaks, 1985.
61. B. Kitchenham and J. Walker, A quantitative approach to monitoring software development, Software Engineering Journal, pp. 1-13, 1989.
62. ISO, "ISO 8402 Quality Vocabulary, " in International Organization for Standardization. Geneva, 1986.
63. IEEE, "IEEE Std 1074 -1997 - Standard for Software Life Cycle Processes, " 1998.
64. ISO/IEC, "ISO/IEC 9126-1 Software engineering- Product quality- Part 1: Quality model, " 2001.
65. ISO/IEC, "ISO/IEC 9126-2 Software engineering -Product quality- part2: External metrics, " 2002.
66. ISO/IEC, "ISO/IEC 9126-3 Software engineering -Product quality- part3: Internal metrics, " 2002.
67. ISO/IEC, "ISO/IEC 9126-4 Software engineering -Product quality- part4: Quality In Use metrics, " 2002.
68. ISO/IEC, "ISO/IEC ISO/IEC 12207: Information Technology - Software Life Cycle Processes, 1995.