Input domain reduction through irrelevant variable removal and its effect on local, global, and hybrid search-based structural test data generation

IEEE Transactions on Software Engineering

Volumn 38, Issue 2, 2012, Pages 453-477

  McMinn, Phil ^a   Harman, Mark ^b   Lakhotia, Kiran ^b   Hassoun, Youssef ^c   Wegener, Joachim ^d  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

^c KING'S COLLEGE LONDON   (United Kingdom)

^d Berner & Mattner Systemtechnik GmbH   (Germany)

Author keywords

automated test data generation; evolutionary testing; input domain reduction; Search based software testing

Indexed keywords

AUTOMATED TEST DATA GENERATION; DEPENDENCE ANALYSIS; EMPIRICAL ANALYSIS; EVOLUTIONARY TESTING; FITNESS FUNCTIONS; HYBRID SEARCH; INDUSTRIAL PRODUCTION; INPUT DOMAIN REDUCTION; IRRELEVANT VARIABLES; OPEN SOURCES; OPTIMIZATION ALGORITHMS; PERFORMANCE IMPACT; PROGRAM SLICING; RANDOM SEARCHES; SEARCH SPACE REDUCTION; SEARCH SPACES; SEARCH-BASED; SMALL SYSTEMS; STRUCTURAL TESTS; TEST DATA GENERATION; TEST INPUTS;

ALGORITHMS; DATA COMMUNICATION SYSTEMS; SOFTWARE TESTING;

OPEN SYSTEMS;

EID: 84859749195     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2011.18     Document Type: Article

References (52)

1. The Software-Artifact Infrastructure Repository, http://sir.unl. edu/portal/index.html, 2011.
2. W. Afzal, R. Torkar, and R. Feldt, "A Systematic Review of Search-Based Testing for Non-Functional System Properties, " Information and Software Technology, vol. 51, no. 6, pp. 957-976, 2009.
3. S. Ali, L. C. Briand, H. Hemmati, and R. K. Panesar-Walawege, "A Systematic Review of the Application and Empirical Investigation of Search-Based Test-Case Generation, " IEEE Trans. Software Eng., vol. 36, no. 6, pp. 742-762, Nov./Dec. 2010.
4. L. O. Andersen, "Program Analysis and Specialization for the C Programming Language, " PhD thesis, DIKU, Univ. of Copenhagen, May 1994.
5. A. Arcuri, "Theoretical Analysis of Local Search in Software Testing, " Proc. Symp. Stochastic Algorithms, Foundations and Applications, pp. 156-168, 2009.
6. A. Arcuri, "It Does Matter How You Normalise the Branch Distance in Search Based Software Testing, " Proc. Int'l Conf. Software Testing, Verification and Validation, pp. 205-214, 2010.
7. A. Arcuri, M. Z. Iqbal, and L. Briand, "Formal Analysis of the Effectiveness and Predictability of Random Testing, " Proc. Int'l Symp. Software Testing and Analysis, pp. 219-230, 2010.
8. J. E. Baker, "Reducing Bias and Inefficiency in the Selection Algorithm, " Proc. Second Int'l Conf. Genetic Algorithms and Their Application, 1987.
9. D. W. Binkley, N. Gold, and M. Harman, "An Empirical Study of Static Program Slice Size, " ACM Trans. Software Eng. and Methodology, vol. 16, no. 2, pp. 1-32, 2007.
10. L. C. Briand, J. Feng, and Y. Labiche, "Using Genetic Algorithms and Coupling Measures to Devise Optimal Integration Test Orders, " Proc. 14th Int'l Conf. Software Eng. and Knowledge Eng., pp. 43-50, 2002.
11. O. Buehler and J. Wegener, "Evolutionary Functional Testing of an Automated Parking System, " Proc. Int'l Conf. Computer, Comm. and Control Technologies and Ninth Int'l Conf. Information Systems Analysis and Synthesis, 2003.
12. S. Chen and S. Smith, "Improving Genetic Algorithms by Search Space Reductions, " Proc. Genetic and Evolutionary Computation Conf., pp. 135-140, 1999.
13. P. Cousot and R. Cousot, "Abstract Interpretation Frameworks, " J. Logic and Computation, vol. 2, no. 4, pp. 511-547, 1992.
14. H. Do, S. Elbaum, and G. Rothermel, "Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and Its Potential Impact, " Empirical Software Eng., vol. 10, no. 4, pp. 405-435, 2005. (Pubitemid 41408554)
15. J. W. Duran and S. C. Ntafos, "An Evaluation of Random Testing, " IEEE Trans. Software Eng., vol. 10, no. 4, pp. 438-444, July 1984.
16. Grammatech Inc. "The Codesurfer Slicing System, " 2002.
17. M. Harman, "The Current State and Future of Search Based Software Engineering, " Proc. Future of Software Eng., L. Briand and A. Wolf, eds., pp. 342-357, 2007. (Pubitemid 47484936)
18. M. Harman, C. Fox, R. M. Hierons, L. Hu, S. Danicic, and J. Wegener, "Vada: A Transformation-Based System for Variable Dependence Analysis, " Proc. IEEE Int'l Workshop Source Code Analysis and Manipulation, pp. 55-64, Oct. 2002.
19. M. Harman, Y. Hassoun, K. Lakhotia, P. McMinn, and J. Wegener, "The Impact of Input Domain Reduction on Search-Based Test Data Generation, " Proc. ACM SIGSOFT Symp. Foundations of Software Eng., pp. 155-164, 2007.
20. M. Harman, F. Islam, T. Xie, and S. Wappler, "Automated Test Data Generation for Aspect-Oriented Programs, " Proc. Eighth Int'l Conf. Aspect-Oriented Software Development, pp. 185-196, 2009.
21. M. Harman, S. G. Kim, K. Lakhotia, P. McMinn, and S. Yoo, "Optimizing for the Number of Tests Generated in Search Based Test Data Generation with an Application to the Oracle Cost Problem, " Proc. Third Int'l Workshop Search-Based Testing, 2010.
22. M. Harman, A. Mansouri, and Y. Zhang, "Search Based Software Engineering: A Comprehensive Analysis and Review of Trends Techniques and Applications, " Technical Report TR-09-03, Dept. of Computer Science, King's College London, Apr. 2009.
23. M. Harman and P. McMinn, "A Theoretical and Empirical Study of Search-Based Testing: Local, Global and Hybrid Search, " IEEE Trans. Software Eng., vol. 36, no. 2, pp. 226-247, Mar./Apr. 2010.
24. S. Horwitz, T. Reps, and D. W. Binkley, "Interprocedural Slicing Using Dependence Graphs, " Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, pp. 35-46, June 1988.
25. J. King, "Symbolic Execution and Program Testing, " Comm. ACM, vol. 19, no. 7, pp. 385-394, 1976.
26. B. Korel, "Automated Software Test Data Generation, " IEEE Trans. Software Eng., vol. 16, no. 8, pp. 870-879, Aug. 1990. (Pubitemid 20738724)
27. J. Krinke and G. Snelting, "Validation of Measurement Software as an Application of Slicing and Constraint Solving, " Information and Software Technology, special issue on program slicing, vol. 40, nos. 11/12, pp. 661-675, 1998. (Pubitemid 128411138)
28. K. Lakhotia, P. McMinn, and M. Harman, "An Empirical Investigation into Branch Coverage for C Programs Using CUTE and AUSTIN, " J. Systems and Software, vol. 83, pp. 2379-2391, 2010.
29. F. Lammermann, A. Baresel, and J. Wegener, "Evaluating Evolutionary Testability with Software-Measurements, " Proc. Genetic and Evolutionary Computation Conf., pp. 1350-1362, 2004. (Pubitemid 39740198)
30. P. Larrañaga and J. A. Lozano, Estimation of Distribution Algorithms. Kluwer, 2002.
31. N. Mansour and M. Salame, "Data Generation for Path Testing, " Software Quality J., vol. 12, no. 2, pp. 121-134, 2004.
32. G. McGraw, C. Michael, and M. Schatz, "Generating Software Test Data by Evolution, " IEEE Trans. Software Eng., vol. 27, no. 12, pp. 1085-1110, Dec. 2001. (Pubitemid 34035564)
33. P. McMinn, "Search-Based Software Test Data Generation: A Survey, " Software Testing, Verification and Reliability, vol. 14, no. 2, pp. 105-156, 2004.
34. P. McMinn, M. Harman, D. Binkley, and P. Tonella, "The Species per Path Approach to Search-Based Test Data Generation, " Proc. Int'l Symp. Software Testing and Analysis, pp. 13-24, 2006.
35. P. McMinn, M. Stevenson, and M. Harman, "Reducing Qualitative Human Oracle Costs Associated with Automatically Generated Test Data, " Proc. First Int'l Workshop Software Test Output Validation, pp. 1-4, 2010.
36. W. Miller and D. Spooner, "Automatic Generation of FloatingPoint Test Data, " IEEE Trans. Software Eng., vol. 2, no. 3, pp. 223-226, Sept. 1976.
37. H. Mühlenbein and D. Schlierkamp-Voosen, "Predictive Models for the Breeder Genetic Algorithm: I. Continuous Parameter Optimization, " Evolutionary Computation, vol. 1, no. 1, pp. 25-49, 1993.
38. R. Pargas, M. Harrold, and R. Peck, "Test-Data Generation Using Genetic Algorithms, " Software Testing, Verification and Reliability, vol. 9, no. 4, pp. 263-282, 1999. (Pubitemid 30546633)
39. R Development Core Team, R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2009.
40. O. Räihä, "A Survey on Search Based Software Design, " Technical Report D-2009-1, Dept. of Computer Sciences, Univ. of Tampere, 2009.
41. J. Ribeiro, M. Zenha-Rela, and F. de Vega, "Test Case Evaluation and Input Domain Reduction Strategies for the Evolutionary Testing of Object-Oriented Software, " Information and Software Technology, vol. 51, pp. 1534-1548, 2009.
42. R. Sagarna and J. Lozano, "Dynamic Search Space Transformations for Software Test Data Generation, " Computational Intelligence, vol. 24, no. 1, pp. 23-61, 2008. (Pubitemid 351169465)
43. F. Schmiedle, R. Drechsler, and B. Becker, "Exact Routing with Search Space Reduction, " IEEE Trans. Computers, vol. 52, no. 6, pp. 815-825, June 2003.
44. B. Steensgaard, "Points-to Analysis in Almost Linear Time, " Proc. Conf. Record of the 23rd ACM SIGPLAN-SIGACT Symp. Principles of Programming Languages, pp. 32-41, Jan. 1996.
45. N. Tracey, J. Clark, K. Mander, and J. McDermid, "An Automated Framework for Structural Test-Data Generation, " Proc. Int'l Conf. Automated Software Eng., pp. 285-288, 1998.
46. J. Voas and K. Miller, "Software Testability: The New Verification, " IEEE Software, vol. 12, no. 3, pp. 17-28, May 1995.
47. H.-C. Wang and B. Jeng, "Structural Testing Using Memetic Algorithm, " Proc. Second Taiwan Conf. Software Eng., 2006.
48. J. Wegener, A. Baresel, and H. Sthamer, "Evolutionary Test Environment for Automatic Structural Testing, " Information and Software Technology, vol. 43, no. 14, pp. 841-854, 2001. (Pubitemid 33050964)
49. J. Wegener and M. Grochtmann, "Verifying Timing Constraints of Real-Time Systems by Means of Evolutionary Testing, " Real-Time Systems, vol. 15, no. 3, pp. 275-298, 1998. (Pubitemid 128512931)
50. M. Weiser, "Program Slicing, " IEEE Trans. Software Eng., vol. 10, no. 4, pp. 352-357, July 1984.
51. D. Whitley, "The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials Is Best, " Proc. Int'l Conf. Genetic Algorithms, J. D. Schaffer, ed., pp. 116-121, 1989.
52. S. Xanthakis, C. Ellis, C. Skourlas, A. Le Gall, S. Katsikas, and K. Karapoulios, "Application of Genetic Algorithms to Software Testing (Application des Algorithmes Génétiques au Test des Logiciels), " Proc. Fifth Int'l Conf. Software Eng. and Its Applications, pp. 625-636, 1992.