Studying the co-evolution of production and test code in open source and industrial developer test processes through repository mining

Empirical Software Engineering

Volumn 16, Issue 3, 2011, Pages 325-364

  Zaidman, Andy ^a   Van Rompaey, Bart ^b   Van Deursen, Arie ^a   Demeyer, Serge ^b  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b UNIVERSITY OF ANTWERP   (Belgium)

Author keywords

Co evolution; Software evolution; Software repository mining; Software testing; Test coverage

Indexed keywords

CO-EVOLUTION; CODE INSPECTIONS; DEVELOPMENT TESTING; FUNCTIONAL CODES; GAIN INSIGHT; INDUSTRIAL SOFTWARE; OPEN SOURCES; REPOSITORY MINING; SOFTWARE EVOLUTION; SOFTWARE PRODUCTION; SOFTWARE PROJECT; SOFTWARE REPOSITORY MINING; SOFTWARE SYSTEMS; TEST CODE; TEST COVERAGE; TEST ENGINEERS; TEST PROCESS; VERSIONING SYSTEMS;

COMPUTER SOFTWARE SELECTION AND EVALUATION; SOFTWARE TESTING;

OPEN SYSTEMS;

EID: 80052498571     PISSN: 13823256     EISSN: 15737616     Source Type: Journal    
DOI: 10.1007/s10664-010-9143-7     Document Type: Article

References (80)

1. Adams B, Jiang ZM, Hassan AE (2010) Identifying crosscutting concerns using historical code changes. In: Proceedings of the international conference on software engineering (ICSE). ACM, New York, pp 305-314
2. Ant (2010) http://ant.apache.org. Website last visited: 4 August 2010
3. Bachmann A, Bernstein A (2010) When process data quality affects the number of bugs: correlations in software engineering datasets. In: Proceedings of the 7th international working conference on mining software repositories (MSR). IEEE Computer Society, Washington, pp 62-71
4. Ball T, Eick SG (1996) Software visualization in the large. IEEE Comput 29 (4):33-43 (Pubitemid 126504807)
5. Ball T, Kim J, Porter A, Siy H (1997) If your version control system could talk. In: ICSE workshop on process modelling and empirical studies of software engineering. Boston, USA
6. Beck K (1999) Extreme programming explained: embrace change. Addison Wesley, Reading, MA
7. Beck K (2003) Test-driven development: by example. Addison-Wesley, Reading, MA
8. Berner S, Weber R, Keller RK (2007) Enhancing software testing by judicious use of code coverage information. In: Proceedings of the international conference on software engineering (ICSE). IEEE Computer Society, Los Alamitos, pp 612-620 (Pubitemid 47430695)
9. Beyer D, Hassan AE (2006) Animated visualization of software history using evolution storyboards. In: Proceedings of the working conference on reverse engineering (WCRE). IEEE Computer Society, Washington, pp 199-210
10. Beyer D, Noack A (2005) Clustering software artifacts based on frequent common changes. In: Proceedings of the international workshop on program comprehension (IWPC). IEEE Computer Society, Washington, pp 259-268 (Pubitemid 41624387)
11. Bhat T, Nagappan N (2006) Evaluating the efficacy of test-driven development: industrial case studies. In: Proceedings of the international symposium on empirical software engineering (ISESE). ACM, New York, pp 356-363 (Pubitemid 46634470)
12. Bible J, Rothermel G, Rosenblum D (2001) A comparative study of coarse-and fine-grained safe regression test selection. ACM Trans Softw Eng Methodol 10 (2):149-183 (Pubitemid 33602333)
13. Binder R (2000) Testing object-oriented systems: models, patterns, and tools. Addison-Wesley, Reading, MA
14. Bouktif S, Guéhéneuc YG, Antoniol G (2006) Extracting change-patterns from cvs repositories. In: Proceedings of the working conference on reverse engineering (WCRE). IEEE Computer Society, Washington, pp 221-230 (Pubitemid 47514644)
15. Breu S, Zimmermann T (2006) Mining aspects from version history. In: Proceedings of the international conference on automated software engineering (ASE). IEEE Computer Society, Washington, pp 221-230 (Pubitemid 47159349)
16. Bruntink M, van Deursen A (2006) An empirical study into class testability. J Syst Softw 79 (9):1219-1232 (Pubitemid 44314121)
17. Clover (2010) http://www.atlassian.com/software/clover/. Website last visited: 4 August 2010
18. Cornelissen B, Zaidman A, van Deursen A, Van Rompaey B (2009) Trace visualization for program comprehension: a controlled experiment. In: Proceedings of the international conference on program comprehension (ICPC). IEEE Computer Society, Washington, pp 100-109
19. D'Ambros M, Gall H, Lanza M, Pinzger M (2008) Analyzing software repositories to understand software evolution. In: Mens T, Demeyer S (eds) Software evolution. Springer, Berlin, pp 39-70
20. D'Ambros M, Lanza M, Lungu M (2009) Visualizing co-change information with the evolution radar. IEEE Trans Softw Eng 35 (5):720-735
21. Demeyer S, Ducasse S, Nierstrasz O (2002) Object-oriented reengineering patterns. Morgan Kaufmann, San Francisco
22. Elbaum S, Gable D, Rothermel G (2001) The impact of software evolution on code coverage information. In: Proceedings of the international conference on software maintenance (ICSM). IEEE Computer Society, Washington, pp 170-179
23. Emma (2010) http://emma.sourceforge.net/. Website last visited: 4 August 2010
24. Feathers M (2005) Working effectively with legacy code. Prentice Hall
25. Fewster M, Graham D (1999) Software Test Automation: effective use of test execution tools. ACM Press/Addison-Wesley Publishing Co, New York
26. Fluri B, Würsch M, Gall H (2007) Do code and comments co-evolve? On the relation between source code and comment changes. In: Proceedings of the working conference on reverse engineering (WCRE). IEEE Computer Society, Washington, pp 70-79
27. Gaelli M, Lanza M, Nierstrasz O, Wuyts R (2004) Ordering broken unit tests for focused debugging. In: Proceedings of the international conference on software maintenance (ICSM). IEEE Computer Society, Washington, pp 114-123 (Pubitemid 40601016)
28. Gall H, Hajek K, Jazayeri M (1998) Detection of logical coupling based on product release history. In: Proceedings of the international conference on software maintenance (ICSM). IEEE Computer Society, Washington, pp 190-197
29. Gamma E, Beck K (1998) Test infected: programmers love writing tests. Java Rep 3 (7):37-50
30. Gîrba T, Ducasse S (2006) Modeling history to analyze software evolution. J Softw Maint 18 (3):207-236 (Pubitemid 43944236)
31. Glover A (2006) In pursuit of code quality: don't be fooled by the coverage report. http://www.ibm.com/developerworks/java/library/j-cq01316/. Last visited: 4 August 2010
32. gnuplot (2010) http://www.gnuplot.info. Website last visited: 4 August 2010
33. Godfrey M, Tu Q (2000) Evolution in open source software: a case study. In: Proceedings of the international conference on software maintenance (ICSM). IEEE Computer Society, Washington, pp 131-142 (Pubitemid 32072790)
34. Hassan AE, Mockus A, Holt RC, Johnson PM (2005) Special issue on mining software repositories. IEEE Trans Softw Eng 31 (6):426-428
35. Hindle A, Godfrey M, Holt R (2007a) Release pattern discovery: a case study of database systems. In: Proceedings of the international conference on software maintenance (ICSM). IEEE Computer Society, Washington, pp 285-294
36. Hindle A, Godfrey M, Holt R (2007b) Release pattern discovery via partitioning: methodology and case study. In: Proceedings of the international workshop on mining software repositories (MSR). IEEE Computer Society, Washington, pp 19-26
37. JFreeChart (2010) http://www.jfree.org/jfreechart/. Website last visited: 4 August 2010
38. Kagdi H, Collard ML, Maletic JI (2007) A survey and taxonomy of approaches for mining software repositories in the context of software evolution. J Softw Maint 19 (2):77-131 (Pubitemid 46752051)
39. Kagdi H, Maletic J, Sharif B (2007) Mining software repositories for traceability links. In: Proceedings of the international conference on program comprehension (ICPC). IEEE Computer Society, Washington, pp 145-154 (Pubitemid 47515018)
40. Kanstrén T (2008) Towards a deeper understanding of test coverage. J Softw Maint 20 (1):59-76 (Pubitemid 351321942)
41. Lanza M (2001) The evolution matrix: recovering software evolution using software visualization techniques. In: Proceedings of the international workshop on principles of software evolution (IWPSE). ACM, pp 37-42
42. Lanza M (2010) The visual terminator. In: Proceedings of the 7th international working conference on mining software repositories (MSR). IEEE Computer Society, http://www.slideshare. net/michele.lanza/the-visual-terminator
43. Lanza M, Ducasse S (2002) Understanding software evolution using a combination of software visualization and software metrics. In: Proceedings of LMO 2002 (Languages et Modèles à Objets). Hermes Publications, pp 135-149
44. Lanza M, Ducasse S (2003) Polymetric views-a lightweight visual approach to reverse engineering. IEEE Trans Softw Eng 29 (9):782-795
45. Lehman M (1980) On understanding laws, evolution and conservation in the large program life cycle. J Syst Softw 1 (3):213-221 (Pubitemid 12514021)
46. LOCC (2010) http://csdl.ics.hawaii.edu/Plone/research/locc/. Website last visited: 4 August 2010
47. Lubsen Z, Zaidman A, Pinzger M (2009) Using association rules to study the co-evolution of production & test code. In: Proceedings of the 6th international working conference on mining software repositories (MSR). IEEE Computer Society, Washington, pp 151-154
48. Luijten B, Visser J, Zaidman A (2010) Assessment of issue handling efficiency. In: Proceedings of the international working conference on mining software repositories (MSR). IEEE Computer Society, Washington, pp 94-97
49. Mathur AP (2008) Foundations of software testing. Pearson Education
50. Maven (2010) http://maven. apache.org. Website last visited: 4 August 2010
51. Maximilien E, Williams L (2003) Assessing test-driven development at IBM. In: Proceedings of the international conference on software engineering (ICSE). IEEE Computer Society, Washington, pp 564-569
52. Mens T, Wermelinger M, Ducasse S, Demeyer S, Hirschfeld R, Jazayeri M (2005) Challenges in software evolution. In: Proceedings of the international workshop on principles of software evolution (IWPSE). IEEE Computer Society, Washington, pp 13-22 (Pubitemid 46151614)
53. Meszaros G (2007) xUnit test patterns: refactoring test code. Addison-Wesley, Reading, MA
54. Moonen L, van Deursen A, Zaidman A, Bruntink M (2008) The interplay between software testing and software evolution. In: Mens T, Demeyer S (eds) Software evolution. Springer, Berlin, pp 173-202
55. Mulder F, Zaidman A (2010) Identifying cross-cutting concerns using software repository mining. In: Proceedings of the 4th international joint ERCIM/IWPSE symposium on software evolution (IWPSE-EVOL). ACM (To appear)
56. Prevalence and incidence of color blindness (2010) http://www. wrongdiagnosis.com/c/color-blindness/prevalence.htm. Website last visited: 4 August 2010
57. Ratzinger J, Fischer M, Gall H (2005) Evolens: lens-view visualizations of evolution data. In: Proceedings of the international workshop on principles of software evolution (IWPSE). IEEE Computer Society, Washington, pp 103-112 (Pubitemid 46151626)
58. Runeson P (2006) A survey of unit testing practices. IEEE Softw 25 (4):22-29
59. Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering. Empir Software Eng 14 (2):131-164
60. Sangwan R, Laplante P (2006) Test-driven development in large projects. IT Pro 8 (5):25-29 (Pubitemid 44711766)
61. Schwaber K, Beedle M (2002) Agile software development with Scrum. Prentice Hall, Upper Saddle River
62. Siniaalto M, Abrahamsson P (2007) A comparative case study on the impact of test-driven development on program design and test coverage. In: Proceedings of the first international symposium on empirical software engineering and measurement (ESEM). IEEE Computer Society, Washington, pp 275-284
63. Slaughter SA, Harter DE, Krishnan MS (1998) Evaluating the cost of software quality. Commun ACM 41 (8):67-73 (Pubitemid 128445855)
64. Storey MA, Čubranić D, German D (2005) On the use of visualization to support awareness of human activities in software development: a survey and a framework. In: Proceedings of the symposium on software visualization. ACM, New York, pp 193-202 (Pubitemid 43111320)
65. SVNKit (2010) http://svnkit.com/. Website last visited: 4 August 2010
66. Tassey G (2002) Economic impacts of inadequate infrastructure for software testing. Planning Report 02-3, National Institute of Standards and Technology (NIST)
67. Telea A, Auber D (2008) Code flows: visualizing structural evolution of source code. Comput Graph Forum 27 (3):831-838
68. Van Rompaey B, Demeyer S (2008) Estimation of test code changes using historical release data. In: Proceedings of the working conference on reverse engineering (WCRE). IEEE Computer Society, Washington, pp 269-278
69. Van Rompaey B, Du Bois B, Demeyer S, Rieger M (2007) On the detection of test smells: a metrics-based approach for general fixture and eager test. IEEE Trans Softw Eng 33 (12):800-817
70. Van Rysselberghe F, Demeyer S (2004) Studying software evolution information by visualizing the change history. In: Proceedings of the international conference on software maintenance (ICSM). IEEE Computer Society, Washington, pp 328-337 (Pubitemid 40601035)
71. Voinea L, Telea A (2006) An open framework for cvs repository querying, analysis and visualization. In: Proceedings of the international workshop on mining software repositories (MSR). ACM, New York, pp 33-39
72. Voinea L, Telea A (2007) Visual data mining and analysis of software repositories. Comput Graph 31 (3):410-428 (Pubitemid 46726593)
73. Voinea L, Telea A, van Wijk JJ (2005) Cvsscan: visualization of code evolution. In: Proceedings of the ACM symposium on software visualization (SOFTVIS). ACM, pp 47-56 (Pubitemid 43111305)
74. Wu J, Holt RC, Hassan AE (2004) Exploring software evolution using spectographs. In: Proceedings of the working conference on reverse engineering (WCRE). IEEE Computer Society, Washington, pp 80-89 (Pubitemid 40610325)
75. XQuery (2010) http://www.w3.org/TR/xquery/. Website last visited: 4 August 2010
76. Yamaura T (1998) How to design practical test cases. IEEE Softw 15 (6):30-36 (Pubitemid 128568662)
77. Yin RK (2002) Case study research: design and methods, 3rd edn. Sage Publications
78. Zaidman A, Van Rompaey B, Demeyer S, van Deursen A (2008) Mining software repositories to study co-evolution of production & test code. In: Proceedings of the international conference on software testing, verification, and validation (ICST). IEEE Computer Society, Washington, pp 220-229
79. Zhu H, Hall PA, May J (1997) Software unit test coverage and adequacy. ACM Comput Surv 29 (4):366-427 (Pubitemid 127583222)
80. Zimmermann T, Kim S, Whitehead J, Zeller A (2006) Mining version archives for co-changed lines. In: Proceedings of the international workshop on mining software repositories (MSR). ACM, New York, pp 72-75