Search-based failure discovery using testability transformations to generate pseudo-oracles

Proceedings of the 11th Annual Genetic and Evolutionary Computation Conference, GECCO-2009

Volumn , Issue , 2009, Pages 1689-1696

  McMinn, Phil ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

Non testable program; Oracle; Program transformation; Pseudo oracle; Search based software testing; Testability transformation

Indexed keywords

EXPERIMENTAL STUDIES; MULTITHREADED; NON-TESTABLE PROGRAM; PROGRAM TRANSFORMATIONS; RACE CONDITION; RANDOM TESTING; ROUND-OFF ERRORS; TEST CASE; TESTABILITY; TESTABILITY TRANSFORMATIONS;

COMPUTER SOFTWARE SELECTION AND EVALUATION;

SOFTWARE TESTING;

EID: 72749091690     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1569901.1570127     Document Type: Conference Paper

References (46)

1. A. Avizienis. The N-version approach to fault-tolerant software. IEEE Trans. Software Engineering, 11:1491-1501, 1985.
2. A. Avizienis and L. Chen. On the implementation of N-version programming for software fault-tolerance during execution. First International Computer Software and Application Conference (COMPSAC '77), pp. 149-155, 1977.
3. B. Baudry, F. Fleurey, J.-M. Jézéquel, and Y. Le Traon. From genetic to bacteriological algorithms for mutation-based testing. Software Testing, Verification & Reliability, 15:73-96, 2005.
4. L. Briand, Y. Labiche, and M. Shousha. Stress testing real-time systems with genetic algorithms. Genetic and Evolutionary Computation Conference (GECCO '05), pp. 1021-1028, Washington DC, USA, 2005. ACM Press.
5. O. Buehler and J. Wegener. Evolutionary functional testing of an automated parking system. International Conference on Computer, Communication and Control Technologies and The 9th International Conference on Information Systems Analysis and Synthesis, Orlando, Florida, USA, 2003.
6. T. Chen, S. Cheung, and S. Yiu. Metamorphic testing: a new approach for generating next test cases. Technical Report HKUST-CS98-01, Department of Computer Science, Hong Kong University of Science and Technology, 1998.
7. J. Clark and R. Hierons. Search-based metamorphic testing. Technical report, University of York, unpublished, 2003.
8. M. Cohen, P. Gibbons, W. Mugridge, and C. Colbourn. Constructing test suites for interaction testing. 25th International Conference on Software Engineering (ICSE '03), pp. 38-48. IEEE Computer Society, 2003.
9. B. Daniel, D. Dig, K. Garcia, and D. Marinov. Automated testing of refactoring engines. 6th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE '07), pp. 185-194, 2007.
10. M. Davies and E. Weyuker. Pseudo-oracles for non-testable programs. ACM '81 Conference, pp. 254-257, 1981.
11. K. Derderian, R. Hierons, M. Harman, and Q. Guo. Automated unique input output sequence generation for conformance testing of FSMs. The Computer Journal, 39:331-344, 2006.
12. M. Dowson. The Ariane 5 software failure. Software Engineering Notes, 22:84, 1997.
13. D. Goldberg. What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys, 21(1):5-48, 1991.
14. M. Harman. Open problems in testability transformation. 1st International Workshop on Search-Based Testing. IEEE digital library, 2008.
15. M. Harman, A. Baresel, D. Binkley, R. Hierons, L. Hu, B. Korel, P. McMinn, and M. Roper. Testability transformation - program transformation to improve testability. Formal Methods and Testing, Lecture Notes in Computer Science, volume 4949, pp. 320-344. Springer-Verlag, 2008.
16. M. Harman, L. Hu, R. Hierons, A. Baresel, and H. Sthamer. Improving evolutionary testing by flag removal. Genetic and Evolutionary Computation Conference (GECCO '02), pp. 1359-1366, New York, USA, 2002. Morgan Kaufmann.
17. M. Harman, L. Hu, R. Hierons, J. Wegener, H. Sthamer, A. Baresel, and M. Roper. Testability transformation. IEEE Trans. Software Engineering, 30(1):3-16, 2004.
18. M. Harman and P. McMinn. A theoretical & empirical analysis of evolutionary testing and hill climbing for structural test data generation. International Symposium on Software Testing and Analysis (ISSTA '07), pp. 73-83, London, UK, 2007. ACM Press.
19. M. Harman and P. McMinn. A theoretical and empirical study of search based testing: local, global and hybrid search. IEEE Trans. Software Engineering, to appear, 2009.
20. Bug ID: 4466549 Add saturation/overflow integer arithmetic. http://bugs.sun.com/bugdatabase/view-bug.do?bug-id=4466549.
21. Y. Jia and M. Harman. Constructing subtle faults using higher order mutation testing. 8th International Working Conference on Source Code Analysis and Manipulation (SCAM '08), pp. 249-258, Beijing, China, 2008. IEEE Computer Society.
22. M. E. Kabay. Salami fraud. Network World Security Newsletter, http://www.networkworld.com/newsletters/sec/2002/01467137.html, 2002.
23. B. Korel. Automated software test data generation. IEEE Trans. Software Engineering, 16(8):870-879, 1990.
24. B. Korel, M. Harman, S. Chung, P. Apirukvorapinit, R. Gupta and Q. Zhang. Data dependence based testability transformation in automated test generation. 16th International Symposium on Software Reliability Engineering (ISSRE '05), pp. 245-254, Chicago, Illinios, USA, 2005.
25. Z. Li, M. Harman, and R. Hierons. Search algorithms for regression test case prioritization. IEEE Trans. Software Engineering, 33:225-237, 2007.
26. G. McGraw, C. Michael, and M. Schatz. Generating software test data by evolution. IEEE Trans. Software Engineering, 27(12):1085-1110, 2001.
27. P. McMinn. Search-based software test data generation: A survey. Software Testing, Verification & Reliability, 14(2):105-156, 2004.
28. P. McMinn. Co-testability transformation. Testing: Academic & Industrial Conference - Practice And Research Techniques (TAIC PART '08), Fast Abstract, 2008.
29. P. McMinn, D. Binkley, and M. Harman. Testability transformation for efficient automated test data search in the presence of nesting. UK Software Testing Workshop (UKTest '05), pp. 165-182. University of Sheffield Computer Science Technical Report CS-05-07, 2005.
30. P. McMinn, D. Binkley, and M. Harman. Empirical evaluation of a nesting testability transformation for evolutionary testing. ACM Transactions on Software Engineering Methodology, 18(3), 2009.
31. P. McMinn, M. Harman, D. Binkley, and P. Tonella. The species per path approach to search-based test data generation. International Symposium on Software Testing and Analysis (ISSTA '06), pp. 13-24, Portland, Maine, USA, 2006. ACM Press.
32. W. Miller and D. Spooner. Automatic generation of floating-point test data. IEEE Trans. Software Engineering, 2(3):223-226, 1976.
33. S. Oaks and H. Wong. Java Threads. O'Reilly, 2nd edition, 1999.
34. A. J. Offutt. A practical system for mutation testing: Help for the common programmer. International Test Conference, pp. 824-830, 1994.
35. A. J Offutt and J. Pan. Automatically detecting equivalent mutants and infeasible paths. Software Testing, Verification & Reliability, 7:137-194, 1997.
36. R. Pargas, M. Harrold, and R. Peck. Test-data generation using genetic algorithms. Software Testing, Verification & Reliability, 9(4):263-282, 1999.
37. R. Sedgewick and K. Wayne. Introduction to Programming in Java: An Interdisciplinary Approach. Addison Wesley, 1st edition, 2007.
38. P. Tonella. Evolutionary testing of classes. International Symposium on Software Testing and Analysis, pp. 119-128, Boston, USA, 2004. ACM Press.
39. N. Tracey, J. Clark, K. Mander, and J. McDermid. Automated test data generation for exception conditions. Software - Practice and Experience, 30(1):61-79, 2000.
40. K. Walcott, M. Soffa, G. Kapfhammer, and R. Roos. Time aware test suite prioritization. International Symposium on Software Testing and Analysis (ISSTA '06), pp. 1-12. ACM Press, 2006.
41. J. Wegener, A. Baresel, and H. Sthamer. Evolutionary test environment for automatic structural testing. Information and Software Technology, 43(14):841-854, 2001.
42. J. Wegener and F. Mueller. A comparison of static analysis and evolutionary testing for the verification of timing constraints. Real-Time Systems, 21(3):241-268, 2001.
43. E. Weyuker. On testing non-testable programs. The Computer Journal, 25, 1982.
44. S. Yoo and M. Harman. Pareto efficient multi-objective test case selection. International Symposium on Software Testing and Analysis (ISSTA '07), pp. 140-150, New York, NY, USA, 2007. ACM Press.
45. Z. Zhou, D. Huang, T. Tse, Z. Yang, H. Huang, and T. Chen. Metamorphic testing and its applications. 8th International Symposium on Future Software Technology (ISFST '04), 2004.
46. J. Zukowski. The need for BigDecimal. Core Java Technologies Tech. Tips, http://blogs.sun.com/CoreJavaTechTips/entry/the-need-for-bigdecimal, 2007.