A systematic review on regression test selection techniques

Information and Software Technology

Volumn 52, Issue 1, 2010, Pages 14-30

  Engström, Emelie ^a   Runeson, Per ^a   Skoglund, Mats ^a  

^a LUND UNIVERSITY   (Sweden)

Author keywords

Empirical studies; Regression testing; Systematic review; Test selection

Indexed keywords

EMPIRICAL EVALUATIONS; EMPIRICAL EVIDENCE; EMPIRICAL STUDIES; QUALITATIVE ANALYSIS; REGRESSION TEST SELECTION; REGRESSION TESTING; SMALL VARIATIONS; SYSTEMATIC REVIEW; TECHNICAL IMPLEMENTATION; TEST SELECTION;

REGRESSION ANALYSIS; RESEARCH; TESTING;

QUALITY CONTROL;

EID: 70350570383     PISSN: 09505849     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.infsof.2009.07.001     Document Type: Review

References (73)

1. Agrawal H., Horgan J.R., Krauser E.W., and London S.A. Incremental regression testing. Proceedings of the Conference on Software Maintenance 1993. CSM-93 (Cat. No. 93CH3360-5) (1993), IEEE Comput. Soc. Press 348-357
2. G. Baradhi, N. Mansour, A comparative study of five regression testing algorithms, in: Proceedings of the Software Engineering Conference, 1997, Australian, 1997, pp. 174-182.
3. Bible J., Rothermel G., and Rosenblum D.S. A comparative study of coarse- and fine-grained safe regression test-selection techniques. ACM Transactions on Software Engineering and Methodology 10 2 (2001) 149-183
4. Binkley D. The application of program slicing to regression testing. Information and Software Technology 40 11-12 (1998) 583-594
5. Brereton P., Kitchenham B.A., Budgen D., Turner M., and Khalil M. Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software 80 4 (2007) 571-583
6. Chen Y.-F., Rosenblum D.S., and Vo K.-P. Test tube: a system for selective regression testing. Proceedings of the International Conference on Software Engineering (1994), IEEE, Los Alamitos, CA, USA 211-220
7. Dieste O., Grimán A., and Juristo N. Developing search strategies for detecting relevant experiments. Empirical Software Engineering (2008)
8. Do H., Elbaum S., and Rothermel G. Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empirical Software Engineering, An International Journal 10 4 (2005)
9. H. Do, G. Rothermel, An empirical study of regression testing techniques incorporating context and lifecycle factors and improved cost-benefit models, in: Proceedings of the ACM SIGSOFT Symposium on Foundations of Software Engineering, November 2006, pp. 141-151.
10. T. Dybå, T. Dingsöyr, G.K. Hanssen, Applying systematic reviews to diverse study types: an experience report, in: First International Symposium on Empirical Software Engineering and Measurement, 2007 (ESEM 2007), 2007, pp. 225-234.
11. Elbaum S., Kallakuri P., Malishevsky A., Rothermel G., and Kanduri S. Understanding the effects of changes on the cost-effectiveness of regression testing techniques. Software Testing, Verification and Reliability 13 2 (2003) 65-83
12. E. Engström, Mats Skoglund, Per Runeson, Empirical evaluations of regression test selection techniques: a systematic review, in: ESEM 08, 2008.
13. K. Fischer, F. Raji, A. Chruscicki, A methodology for retesting modified software, in: NTC'81, IEEE 1981 National Telecommunications Conference, Innovative Telecommunications - Key to the Future, IEEE, 1981, p. 6-3.
14. P.G. Frankl, G. Rothermel, K. Sayre, F.I. Vokolos, An empirical comparison of two safe regression test selection techniques, in: Proceedings of the ISESE International Symposium on Empirical Software Engineering 2003, 2003, pp. 195-204.
15. Graves T.L., Harrold M.J., Kim J.M., Porter A., and Rothermel G. An empirical study of regression test selection techniques. ACM Transactions on Software Engineering and Methodology 10 2 (2001) 184-208
16. Gupta R., Harrold M.J., and Soffa M.L. An approach to regression testing using slicing. Conference on Software Maintenance 1992 (Cat. No. 92CH3206-0) (1992), IEEE Comput. Soc. Press 299-308
17. Gupta R., Harrold M.J., and Soffa M.L. Program slicing-based regression testing techniques. Software Testing, Verification and Reliability 6 2 (1996) 83-111
18. Haftmann F., Kossmann D., and Lo E. A framework for efficient regression tests on database applications. VLDB Journal 16 1 (2007) 145-164
19. Harrold M.J., Jones J.A., Tongyu L., Donglin L., Orso A., Pennings M., Sinha S., Spoon S.A., and Gujarathi A. Regression test selection for Java software. SIGPLAN Not. (USA) (2001), ACM 312-326
20. Harrold M.J., and Souffa M.L. An incremental approach to unit testing during maintenance. Proceedings of the Conference on Software Maintenance - 1988 (IEEE Cat. No. 88CH2615-3) (1988), IEEE Comput. Soc. Press 362-367
21. Hartmann J., and Robson D.J. Approaches to regression testing. Proceedings of the Conference on Software Maintenance - 1988 (IEEE Cat. No. 88CH2615-3) (1988), IEEE Comput. Soc. Press 368-372
22. Hartmann J., and Robson D.J. Techniques for selective revalidation. IEEE Software 7 1 (1990) 31-36
23. Hutchins M., Foster H., Goradia T., and Ostrand T. Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria. ICSE-16, 16th International Conference on Software Engineering (Cat. No. 94CH3409-0) (1994), IEEE Comput. Soc. Press 191-200
24. A. Jedlitschka, D. Pfahl, Reporting guidelines for controlled experiments in software engineering, in: Proceedings of ACM/ IEEE International Symposium on Empirical Software Engineering, 2005, pp. 95-104.
25. Juristo N., Moreno A.M., Vegas S., and Solari M. In search of what we experimentally know about unit testing [software testing]. IEEE Software 23 6 (2006) 72-80
26. Kampenes Vigdis B., Dybå T., Hannay Jo E., and Sjöberg Dag I.K. A systematic review of effect size in software engineering experiments. Information and Software Technology 49 11-12 (2007) 1073
27. Kim J.-M., Porter A., and Rothermel G. An empirical study of regression test application frequency. Software Testing, Verification, and Reliability 15 4 (2005) 257-279
28. Kitchenham B., Brereton O.P., Budgen D., Turner M., Bailey J., and Linkman S. Systematic literature reviews in software engineering - a systematic literature review. Information and Software Technology 51 1 (2009) 7-15
29. B.A. Kitchenham, Guidelines for performing Systematic Literature reviews in Software Engineering Version 2.3. Technical Report S.o.C.S.a.M. Software Engineering Group, Keele University and Department of Computer Science University of Durham, 2007.
30. Kitchenham B.A., Mendes E., and Travassos G.H. Cross versus within-company cost estimation studies: a systematic review. IEEE Transactions on Software Engineering 33 5 (2007) 316-329
31. Klosch R.R., Glaser P.W., and Truschnegg R.J. A testing approach for large system portfolios in industrial environments. Journal of Systems and Software 62 1 (2002) 11-20
32. Koju T., Takada S., and Doi N. Regression test selection based on intermediate code for virtual machines. Conference on Software Maintenance (2003), Institute of Electrical and Electronics Engineers Inc. 420-429
33. Landis J.R., and Gary G.K. The measurement of observer agreement for categorical data. Biometrics 33 1 (1977) 159-174
34. Leung H.K.N., and White L. Insights into testing and regression testing global variables. Journal of Software Maintenance. Research and Practice 2 4 (1990) 209-222
35. Leung H.K.N., and White L. A study of integration testing and software regression at the integration level. In Proceedings of the Conference on Software Maintenance 1990 (Cat. No.90CH2921-5) (1990), IEEE Comput. Soc. Press 290-301
36. Mansour N., Bahsoon R., and Baradhi G. Empirical comparison of regression test selection algorithms. The Journal of Systems and Software 57 1 (2001) 79-90
37. Mansour N., and El-Fakih K. Natural optimization algorithms for optimal regression testing. Proceedings of the IEEE Computer Society's International Computer Software and Applications Conference (1997), IEEE, Los Alamitos, CA, USA 511-514
38. Mao C., and Lu Y. Regression testing for component-based software systems by enhancing change information. Proceedings of the 12th Asia-Pacific Software Engineering Conference (2005), IEEE Computer Society 8 pp
39. Memon A.M. Using tasks to automate regression testing of GUIs. IASTED International Conference on Artificial Intelligence and Applications (AIA 2004) (2004), ACTA Press 477-482
40. Orso A., Harrold M.J., Rosenblum D., Rothermel G., Soffa M.L., and Do H. Using component metacontent to support the regression testing of component-based software. Proceedings of the IEEE International Conference on Software Maintenance (ICSM 2001) (2001), IEEE Comput. Soc. 716-725
41. Orso A., Nanjuan S., and Harrold M.J. Scaling regression testing to large software systems. Softw. Eng. Notes (USA) (2004), ACM 241-251
42. Pasala A., and Bhowmick A. An approach for test suite selection to validate applications on deployment of COTS upgrades. Proceedings of the Asia-Pacific Software Engineering Conference, APSEC (2005), IEEE Computer Society, Los Alamitos, CA 90720-1314, United States 401-407
43. Pei H., Xiaolin L., Kung D.C., Chih-Tung H., Liang L., Toyoshima Y., and Chen C. A technique for the selective revalidation of OO software. Journal of Software Maintenance. Research and Practice 9 4 (1997) 217-233
44. Ren X., Shah F., Tip F., Ryder B.G., and Chesley O. Chianti: a tool for change impact analysis of java programs. 19th Annual ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA'04 (2004), Association for Computing Machinery, New York, NY 10036-5701, United States 432-448
45. Rothermel G., Elbaum S., Malishevsky A., Kallakuri P., and Davia B. The impact of test suite granularity on the cost-effectiveness of regression testing. Proceedings of the International Conference on Software Engineering (2002), Institute of Electrical and Electronics Engineers Computer Society 130-140
46. Rothermel G., Elbaum S., Malishevsky A.G., Kallakuri P., and Xuemei Q. On test suite composition and cost-effective regression testing. ACM Transactions on Software Engineering and Methodology 13 3 (2004) 227-331
47. G. Rothermel, M.J. Harrold, A safe, efficient algorithm for regression test selection, Proceedings of the Conference on Software Maintenance, 1993 (CSM-93), 1993, pp. 358-367.
48. Rothermel G., and Harrold M.J. Analyzing regression test selection techniques. IEEE Transactions on Software Engineering 22 8 (1996) 529-551
49. Rothermel G., and Harrold M.J. A safe, efficient regression test selection technique. ACM Transactions on Software Engineering and Methodology 6 2 (1997) 173-210
50. Rothermel G., and Harrold M.J. Empirical studies of a safe regression test selection technique. IEEE Transactions on Software Engineering 24 6 (1998) 401-419
51. Rothermel G., Harrold M.J., and Dedhia J. Regression test selection for C++ software. Journal of Software Testing Verification and Reliability 10 2 (2000) 77-109
52. Runeson P., and Höst M. Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering 14 2 (2009) 131-164
53. P. Runeson, M. Skoglund, E. Engström, Test Benchmarks - what is the question? in: TestBench Workshop at International Conference on Software Testing, Verification and Validation, Lillehammer, Norway, April 2008.
54. Sajeev A.S.M., and Wibowo B. Regression test selection based on version changes of components. Tenth Asia-Pacific Software Engineering Conference (2003), IEEE Comput. Soc. 78-85
55. Shadish T., Cook T., and Campbell D. Experimental and Quasi-Experimental Designs - for Generalized Causal Inference. second ed. (2002), Houghton Mifflin Company, Boston p. 623
56. Skoglund M., and Runeson P. A case study of the class firewall regression test selection technique on a large scale distributed software system. 2005 International Symposium on Empirical Software Engineering (IEEE Cat. No. 05EX1213) (2005), IEEE 10 pp
57. Staples M., and Niazi M. Experiences using systematic review guidelines. The Journal of Systems and Software 80 9 (2007) 1425-1437
58. Toshihiko K., Shingo T., and Norihisa D. Regression test selection based on intermediate code for virtual machines. Proceedings International Conference on Software Maintenance ICSM 2003 (2003), IEEE Comput. Soc. 420-429
59. White L., and Abdullah K. A firewall approach for the regression testing of object-oriented software. Software Quality Week (1997)
60. White L., Jaber K., and Robinson B. Utilization of extended firewall for object-oriented regression testing. IEEE International Conference on Software Maintenance, ICSM (2005), IEEE Computer Society, Los Alamitos, CA 90720-1314, United States 695-698
61. L. White, B. Robinson, Industrial real-time regression testing and analysis using firewalls, in: Proceedings of the 20th IEEE International Conference on Software Maintenance, 2004, pp. 18-27.
62. White L.J., and Leung H.K.N. A firewall concept for both control-flow and data-flow in regression integration testing. Conference on Software Maintenance 1992 (Cat. No. 92CH3206-0) (1992), IEEE Comput. Soc. Press 262-271
63. Willmor D., and Embury S.M. A safe regression test selection technique for database-driven applications. Proceedings of the 21st IEEE International Conference on Software Maintenance (2005), IEEE Comput. Soc. Press 421-430
64. Vokolos F.I., and Frankl P.G. Pythia: a regression test selection tool based on textual differencing. 3rd International Conference on Reliability, Quality and Safety of Software-Intensive Systems, IFIP TC5 WG5.4 (1997), Chapman & Hall 3-21
65. Vokolos F.I., and Frankl P.G. Empirical evaluation of the textual differencing regression testing technique. Proceedings of the International Conference on Software Maintenance (Cat. No. 98CB36272) (1998), IEEE Comput. Soc. 44-53
66. Wong W.E., Horgan J.R., London S., and Agrawal H. A study of effective regression testing in practice. Proceedings of the Eighth International Symposium on Software Reliability Engineering (Cat. No. 97TB100170) (1997), IEEE Comput. Soc. 264-274
67. Wu Y., Chen M.-H., and Kao H.M. Regression testing on object-oriented programs. Proceedings 10th International Symposium on Software Reliability Engineering (Cat. No. PR00443) (1999), IEEE Comput. Soc. 270-279
68. Yanping C., Robert L.P., and Sims D.P. Specification-based regression test selection with risk analysis. Proceedings of the 2002 Conference of the Centre for Advanced Studies on Collaborative research (2002), IBM Press
69. Yin R.K. In: Leonard Bickman D.J.R. (Ed). Case Study Research - Design and Methods Applied Social Research Methods Series vol. 5 (2003), Sage Publications, London
70. Zheng J. In regression testing selection when source code is not available. Proceedings of the 20th IEEE/ACM international Conference on Automated Software Engineering (2005), ACM
71. Zheng J., Robinson B., Williams L., and Smiley K. An initial study of a lightweight process for change identification and regression test selection when source code is not available. Proceedings of the International Symposium on Software Reliability Engineering, ISSRE (2005), IEEE Computer Society 225-234
72. Zheng J., Robinson B., Williams L., and Smiley K. Applying regression test selection for COTS-based applications. Proceedings of the International Conference on Software Engineering (2006), Institute of Electrical and Electronics Engineers Computer Society, Piscataway, NJ 08855-1331, United States 512-521
73. Zheng J., Robinson B., Williams L., and Smiley K. A lightweight process for change identification and regression test selection in using COTS components. Proceedings of the Fifth International Conference on Commercial-off-the-Shelf (COTS)-Based Software Systems (2006), Institute of Electrical and Electronics Engineers Computer Society, Piscataway, NJ 08855-1331, United States 137-143