Search-based software testing: Past, present and future

Proceedings - 4th IEEE International Conference on Software Testing, Verification, and Validation Workshops, ICSTW 2011

Volumn , Issue , 2011, Pages 153-163

  McMinn, Phil ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATIC GENERATION; FITNESS FUNCTIONS; METAHEURISTIC; OPEN PROBLEMS; OPTIMIZATION PROCESS; RESEARCH AREAS; SEARCH SPACES; SEARCH TECHNIQUE; SEARCH-BASED; STATE OF THE ART; TEST DATA;

COMPUTER SOFTWARE SELECTION AND EVALUATION; ENGINEERING RESEARCH; OPTIMIZATION; VERIFICATION;

SOFTWARE TESTING;

EID: 80051619581     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICSTW.2011.100     Document Type: Conference Paper

References (58)

1. W. Miller and D. Spooner, "Automatic generation of floatingpoint test data, " IEEE Transactions on Software Engineering, vol. 2, no. 3, pp. 223-226, 1976.
2. http://www.time.com.
3. B. Korel, "Automated software test data generation, " IEEE Transactions on Software Engineering, vol. 16, no. 8, pp. 870-879, 1990. (Pubitemid 20738724)
4. B. Korel, "Dynamic method for software test data generation, " Software Testing, Verification and Reliability, vol. 2, no. 4, pp. 203-213, 1992.
5. S. Xanthakis, C. Ellis, C. Skourlas, A. Le Gall, S. Katsikas, and K. Karapoulios, "Application of genetic algorithms to software testing (Application des algorithmes ǵeńetiques au test des logiciels), " in 5th International Conference on Software Engineering and its Applications, Toulouse, France, 1992, pp. 625-636.
6. O. Buehler and J. Wegener, "Evolutionary functional testing of an automated parking system, " in International Conference on Computer, Communication and Control Technologies and The 9th International Conference on Information Systems Analysis and Synthesis, Orlando, Florida, USA, 2003.
7. O. Buehler and J. Wegener, "Evolutionary functional testing, " Computers & Operations Research, vol. 35, pp. 3144-3160, 2008. (Pubitemid 351335139)
8. P. Puschner and R. Nossal, "Testing the results of static worstcase execution-time analysis, " in Proceedings of the 19th IEEE Real-Time Systems Symposium. Madrid, Spain: IEEE Computer Society Press, 1998, pp. 134-143.
9. J. Wegener, H. Sthamer, B. F. Jones, and D. E. Eyres, "Testing real-time systems using genetic algorithms, " Software Quality Journal, vol. 6, no. 2, pp. 127-135, 1997. (Pubitemid 127479282)
10. J. Wegener and M. Grochtmann, "Verifying timing constraints of real-time systems by means of evolutionary testing, " Real-Time Systems, vol. 15, no. 3, pp. 275-298, 1998. (Pubitemid 128512931)
11. L. C. Briand, J. Feng, and Y. Labiche, "Using genetic algorithms and coupling measures to devise optimal integration test orders, " in 14th IEEE Software Engineering and Knowledge Engineering (SEKE), Ischia, Italy, 2002, pp. 43-50.
12. Z. Li, M. Harman, and R. M. Hierons, "Search algorithms for regression test case prioritization, " vol. 33, no. 4, pp. 225- 237, 2007.
13. L. C. Briand, Y. Labiche, and M. Shousha, "Stress testing real-time systems with genetic algorithms, " in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2005). Washington DC, USA: ACM Press, 2005, pp. 1021-1028. (Pubitemid 43226410)
14. Y. Jia and M. Harman, "Constructing subtle faults using higher order mutation testing, " in 8th International Working Conference on Source Code Analysis and Manipulation (SCAM 2008. Beijing, China: IEEE Computer Society, 2008, to appear.
15. K. R. Walcott, M. L. Soffa, G. M. Kapfhammer, and R. S. Roos, "Time aware test suite prioritization, " in International Symposium on Software Testing and Analysis (ISSTA 06). Portland, Maine, USA: ACM Press, 2006, pp. 1-12.
16. S. Yoo and M. Harman, "Pareto efficient multi-objective test case selection, " in International Symposium on Software Testing and Analysis (ISSTA'07). ACM Press, July 2007, pp. 140-150. (Pubitemid 47317588)
17. M. B. Cohen, P. B. Gibbons, W. B. Mugridge, and C. J. Colbourn, "Constructing test suites for interaction testing, " in Proceedings of the 25th International Conference on Software Engineering (ICSE-03). IEEE Computer Society, 2003, pp. 38-48.
18. K. Derderian, R. Hierons, M. Harman, and Q. Guo, "Automated unique input output sequence generation for conformance testing of FSMs, " The Computer Journal, vol. 39, pp. 331-344, 2006.
19. N. Tracey, J. Clark, K. Mander, and J. McDermid, "Automated test data generation for exception conditions, " Software - Practice and Experience, vol. 30, no. 1, pp. 61-79, 2000. (Pubitemid 30555918)
20. M. Harman and B. Jones, "Search-based software engineering, " Information and Software Technology, vol. 43, no. 14, pp. 833-839, 2001. (Pubitemid 33050963)
21. P. McMinn, "Search-based software test data generation: A survey, " Software Testing, Verification and Reliability, vol. 14, no. 2, pp. 105-156, 2004.
22. S. Kirkpatrick, C. D. Gellat, and M. P. Vecchi, "Optimization by simulated annealing, " Science, vol. 220, no. 4598, pp. 671- 680, 1983.
23. D. Whitley, "A genetic algorithm tutorial, " Statistics and Computing, vol. 4, pp. 65-85, 1994.
24. D. Whitley, "An overview of evolutionary algorithms: Practical issues and common pitfalls, " Information and Software Technology, vol. 43, no. 14, pp. 817-831, 2001. (Pubitemid 33050962)
25. M. Harman, "The current state and future of search based software engineering, " in Future of Software Engineering 2007 (FOSE 2007). IEEE Computer Society, 2007, pp. 342- 357. (Pubitemid 47484936)
26. M. Harman and J. Clark, "Metrics are fitness functions too, " in International Software Metrics Symposium (METRICS 2004). IEEE Computer Society, 2004, pp. 58-69. (Pubitemid 40338149)
27. J. Wegener and F. Mueller, "A comparison of static analysis and evolutionary testing for the verification of timing constraints, " Real-Time Systems, vol. 21, no. 3, pp. 241-268, 2001. (Pubitemid 32886810)
28. J. Wegener, A. Baresel, and H. Sthamer, "Evolutionary test environment for automatic structural testing, " Information and Software Technology, vol. 43, no. 14, pp. 841-854, 2001. (Pubitemid 33050964)
29. N. Tracey, J. Clark, K. Mander, and J. McDermid, "An automated framework for structural test-data generation, " in Proceedings of the International Conference on Automated Software Engineering. Hawaii, USA: IEEE Computer Society Press, 1998, pp. 285-288.
30. A. Arcuri, "It does matter how you normalise the branch distance in search based software testing, " in Proceedings of the International Conference on Software Testing, Verification and Validation. IEEE, to appear, 2010.
31. K. Lakhotia, P. McMinn, and M. Harman, "Automated test data generation for coverage: Haven't we solved this problem yet?" in Proceedings of the Testing: Academic & Industrial Conference - Practice And Research Techniques (TAIC PART 2009). IEEE Computer Society, 2009, pp. 95-104.
32. K. Lakhotia, P. McMinn, and M. Harman, "An empirical investigation into branch coverage for C programs using CUTE and AUSTIN, " Journal of Systems and Software, vol. 83, pp. 2379-2391, 2010.
33. K. Lakhotia, M. Harman, and P. McMinn, "Handling dynamic data structures in search based testing, " in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2008). Atlanta, USA: ACM Press, 2008, pp. 1759-1766.
34. P. Tonella, "Evolutionary testing of classes, " in Proceedings of the International Symposium on Software Testing and Analysis. Boston, USA: ACM Press, 2004, pp. 119-128. (Pubitemid 41121383)
35. P. McMinn, D. Binkley, and M. Harman, "Empirical evaluation of a nesting testability transformation for evolutionary testing, " ACM Transactions on Software Engineering Methodology, vol. 3, 2009.
36. L. Bottaci, "Instrumenting programs with flag variables for test data search by genetic algorithm, " in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2002). New York, USA: Morgan Kaufmann, 2002, pp. 1337 - 1342.
37. M. Harman, L. Hu, R. Hierons, A. Baresel, and H. Sthamer, "Improving evolutionary testing by flag removal, " in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2002). New York, USA: Morgan Kaufmann, 2002, pp. 1359-1366.
38. A. Baresel and H. Sthamer, "Evolutionary testing of flag conditions, " in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2003), Lecture Notes in Computer Science vol. 2724. Chicago, USA: Springer-Verlag, 2003, pp. 2442 - 2454.
39. A. Baresel, D. Binkley, M. Harman, and B. Korel, "Evolutionary testing in the presence of loop-assigned flags: A testability transformation approach, " in Proceedings of the International Symposium on Software Testing and Analysis (ISSTA 2004). Boston, Massachusetts, USA: ACM, 2004, pp. 43-52.
40. M. Harman, L. Hu, R. Hierons, J. Wegener, H. Sthamer, A. Baresel, and M. Roper, "Testability transformation, " IEEE Transactions on Software Engineering, vol. 30, no. 1, pp. 3- 16, 2004.
41. P. McMinn, D. Binkley, and M. Harman, "Testability transformation for efficient automated test data search in the presence of nesting, " in Proceedings of the UK Software Testing Workshop (UKTest 2005). University of Sheffield Computer Science Technical Report CS-05-07, 2005, pp. 165-182.
42. B. Korel, M. Harman, S. Chung, P. Apirukvorapinit, and G. R., "Data dependence based testability transformation in automated test generation, " in 16th International Symposium on Software Reliability Engineering (ISSRE 05), Chicago, Illinios, USA, 2005, pp. 245-254. (Pubitemid 44733907)
43. P. McMinn, "Co-testability transformation, " in Proceedings of the Testing: Academic & Industrial Conference: Practice And Research Techniques (TAIC PART 2008), Fast Abstract, 2008.
44. P. McMinn, M. Harman, D. Binkley, and P. Tonella, "The species per path approach to search-based test data generation, " in Proceedings of the International Symposium on Software Testing and Analysis (ISSTA 2006). Portland, Maine, USA: ACM, 2006, pp. 13-24.
45. P. McMinn, "Search-based failure discovery using testability transformations to generate pseudo-oracles, " in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2009). Montreal, Canada: ACM Press, 2009, pp. 1689-1696.
46. A. Avizienis and L. Chen, "On the implementation of Nversion programming for software fault-tolerance during execution, " in Proceedings of the First International Computer Software and Application Conference (COMPSAC '77), 1977, pp. 149-155.
47. A. Avizienis, "The N-version approach to fault-tolerant software, " IEEE Transactions on Software Engineering, vol. 11, pp. 1491-1501, 1985.
48. D. Goldberg, "What every computer scientist should know about floating-point arithmetic, " ACM Computing Surveys, vol. 21, no. 1, pp. 5-48, 1991. (Pubitemid 21723680)
49. M. Davies and E. Weyuker, "Pseudo-oracles for non-testable programs, " in Proceedings of the ACM '81 Conference, 1981, pp. 254-257.
50. M. Harman, S. G. Kim, K. Lakhotia, P. McMinn, and S. Yoo, "Optimizing for the number of tests generated in search based test data generation with an application to the oracle cost problem, " in Proceedings of the 3rd International Workshop on Search-Based Testing. IEEE digital library, 2010.
51. G. Fraser and A. Arcuri, "Whole suite test data generation, " in International Conference On Quality Software (QSIC 2011), to appear, 2011.
52. G. Fraser and A. Arcuri, "It is not the length that matters, it is how you control it, " in IEEE International Conference on Software Testing, Verification and Validation (ICST 2011), 2011.
53. A. Leitner, M. Oriol, A. Zeller, I. Ciupa, and B. Meyer, "Efficient unit test case minimization, " in Automated Software Engineering (ASE 2007). Atlanta, Georgia, USA: ACM Press, 2007, pp. 417-420.
54. A. Zeller and R. Hildebrandt, "Simplifying and isolating failure-inducing input, " IEEE Transactions on Software Engineering, vol. 28, pp. 183-200, 2002. (Pubitemid 34198322)
55. P. McMinn, M. Stevenson, and M. Harman, "Reducing qualitative human oracle costs associated with automatically generated test data, " in Proceedings of the 1st International Workshop on Software Test Output Validation (STOV 2010). Trento, Italy: ACM, 2010, pp. 1-4.
56. D. Lawrie, D. Binkley, and C. Morrell, "Normalizing source code vocabulary, " in International Working Conference on Reverse Engineering (WCRE 2010). IEEE Computer Society, 2010, pp. 3-12.
57. K. Deb, "Multi-objective evolutionary optimization: Past, present and future, " in Proceedings of the Fourth International Conference on Adaptive Computing in Design and Manufacture (ACDM'2000). University of Plymouth, UK: Springer, London, 2000, pp. 225-236.
58. M. Harman, K. Lakhotia, and P. McMinn, "A multi-objective approach to search-based test data generation, " in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007). London, UK: ACM Press, 2007, pp. 1098- 1105. (Pubitemid 47291668)