Automatically performing weak mutation with the aid of symbolic execution, concolic testing and search-based testing

Software Quality Journal

Volumn 19, Issue 4, 2011, Pages 691-723

  Papadakis, Mike ^a   Malevris, Nicos ^a  

^a ATHENS UNIVERSITY OF ECONOMICS AND BUSINESS   (Greece)

Author keywords

Automated test case generation; Concolic execution; Mutant schemata; Mutation testing; Search based testing; Symbolic execution; Weak mutation

Indexed keywords

EID: 80052260419     PISSN: 09639314     EISSN: 15731367     Source Type: Journal    
DOI: 10.1007/s11219-011-9142-y     Document Type: Article

References (43)

1. Ammann, P., & Offutt, J. (2008). Introduction to software testing. Cambridge: Cambridge University Press.
2. Andrews, J. H., Briand, L. C., & Labiche, Y. (2005). Is mutation an appropriate tool for testing experiments? In Proceedings of the 27th international conference on software engineering, St. Louis, MO, USA (pp. 402-411). 1062530: ACM. doi: 10. 1145/1062455. 1062530.
3. Andrews, J. H., Briand, L. C., Labiche, Y., & Namin, A. S. (2006). Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Transaction on Software Engineering, 32(8), 608-624. doi: 10. 1109/TSE. 2006. 83.
4. Ayari, K., Bouktif, S., & Antoniol, G. (2007). Automatic mutation test input data generation via ant colony. In Proceedings of the 9th annual conference on Genetic and evolutionary computation, London, England (pp. 1074-1081). 1277172: ACM. doi: 10. 1145/1276958. 1277172.
5. Baluda, M., Braione, P., Denaro, G., & Pezzè, M. (2010). Structural coverage of feasible code. In Proceedings of the 5th workshop on automation of software test, Cape Town, South Africa (pp. 59-66). 1808275: ACM. doi: 10. 1145/1808266. 1808275.
6. Baresi, L., Lanzi, P. L., & Miraz, M. (2010). Testful: An evolutionary test approach for java. In Software testing, verification and validation (ICST), 2010 third international conference on, 6-10 April 2010 (pp. 185-194). doi: 10. 1109/ICST. 2010. 54.
7. Bottaci, L. (2001). A genetic algorithm fitness function for mutation testing. In SEMINAL: Software engineering using metaheuristic inovative algorithms, workshop (pp. 3-7).
8. Burnim, J., & Sen, K. (2008). Heuristics for scalable dynamic test generation. In Proceedings of the 2008 23rd IEEE/ACM international conference on automated software engineering (pp. 443-446). 1642995: IEEE Computer Society. doi: 10. 1109/ASE. 2008. 69.
9. DeMillo, R. A., Lipton, R. J., & Sayward, F. G. (1978). Hints on test data selection: Help for the practicing programmer. Computer, 11(4), 34-41. doi: 10. 1109/C-M. 1978. 218136.
10. DeMillo, R. A., & Offutt, A. J. (1991). Constraint-based automatic test data generation. IEEE Transaction on Software Engineering, 17(9), 900-910. doi: 10. 1109/32. 92910.
11. Fraser, G., & Zeller, A. (2010). Mutation-driven generation of unit tests and oracles. In Proceedings of the 19th international symposium on software testing and analysis, Trento, Italy (pp. 147-158). 1831728: ACM. doi: 10. 1145/1831708. 1831728.
12. Hamlet, R. G. (1977). Testing programs with the aid of a compiler. IEEE Transaction on Software Engineering, 3(4), 279-290. doi: 10. 1109/TSE. 1977. 231145.
13. Harman, M., & McMinn, P. (2007). A theoretical & empirical analysis of evolutionary testing and hill climbing for structural test data generation. In Proceedings of the 2007 international symposium on Software testing and analysis, London, United Kingdom (pp. 73-83). 1273475: ACM. doi: 10. 1145/1273463. 1273475.
14. Harrold, J., & Rothermel, G. (2010). Siemens programs, hr variants. http://www. cc. gatech. edu/aristotle/Tools/subjects/, May 2010.
15. Howden, W. E. (1982). Weak mutation testing and completeness of test sets. IEEE Transaction on Software Engineering, 8(4), 371-379. doi: 10. 1109/TSE. 1982. 235571.
16. Jia, Y., & Harman, M. (2010). An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 99(PrePrints), doi: 10. 1109/TSE. 2010. 62.
17. King, J. C. (1976). Symbolic execution and program testing. Communication ACM, 19(7), 385-394. doi: 10. 1145/360248. 360252.
18. King, K. N., & Offutt, A. J. (1991). A fortran language system for mutation-based software testing. Software Practical Experiment, 21(7), 685-718. doi: 10. 1002/spe. 4380210704.
19. Lakhotia, K., Harman, M., & Gross, H. (2010). Austin: A tool for search based software testing for the c language and its evaluation on deployed automotive systems. In 2nd International symposium on search based software engineering. doi: 10. 1109/SSBSE. 2010. 21.
20. Li, N., Praphamontripong, U., & Offutt, A. J. (2009). An experimental comparison of four unit test criteria: Mutation, edge-pair, all-uses and prime path coverage. In Proceedings of the 4th international workshop on mutation analysis (MUTATION'09), Denver, Colorado, 1-4 April 2009 (pp. 220-229). doi: 10. 1109/ICSTW. 2009. 30.
21. Ma, Y.-S., Offutt, J., & Kwon, Y. R. (2005). Mujava: An automated class mutation system: Research articles. Software Test Verification Reliability, 15(2), 97-133. doi: 10. 1002/stvr. v15: 2.
22. Maldonado, J. C., Delamaro, M. E., Fabbri, S. C. P. F., Simão, A. D. S., Sugeta, T., Vincenzi, A. M. R., et al. (2001). Proteum: A family of tools to support specification and program testing based on mutation. In W. E. Wong (Ed.), Mutation testing for the new century (pp. 113-116). Kluwer.
23. Mastorantonakis, M., & Malevris, N. (2003). An effective metrod for mutating java programs. In Proceedings of the IASTED international conference on software engineering and applications (Vol. 7, pp. 252-257).
24. Offutt, A. J. (1992). Investigations of the software testing coupling effect. ACM Transaction on Software Engineering Methodology, 1(1), 5-20. doi: 10. 1145/125489. 125473.
25. Offutt, A. J., Jin, Z., & Pan, J. (1999). The dynamic domain reduction procedure for test data generation. Software Practical Experiment, 29(2), 167-193. doi: 10. 1002/(SICI)1097-024X(199902)29: 2< 167:: AID-SPE225> 3. 3. CO; 2-M.
26. Offutt, A. J., & Lee, S. D. (1994). An empirical evaluation of weak mutation. IEEE Transaction Software Engineering, 20(5), 337-344. doi: 10. 1109/32. 286422.
27. Offutt, A. J., Lee, A., Rothermel, G., Untch, R. H., & Zapf, C. (1996). An experimental determination of sufficient mutant operators. ACM Transaction Software Engineering Methodology, 5(2), 99-118. doi: 10. 1145/227607. 227610.
28. Offutt, A. J., & Pan, J. (1997). Automatically detecting equivalent mutants and infeasible paths. Software Testing, Verification and Reliability, 7, 165-192, doi: 10. 1002/(SICI)1099-1689(199709)7: 3 < 165:: AID-STVR143> 3. 0. CO; 2-U.
29. Offutt, A. J., & Untch, R. H. (2001). Mutation 2000: Uniting the orthogonal. In Mutation testing for the new century (pp. 34-44). Kluwer.
30. Offutt, J., & Voas, J. (1996). Subsumption of condition coverage techniques by mutation testing. Dept. of Information and Software Systems Engineering, George Mason Univ., Fairfax, VA.
31. Papadakis, M., & Malevris, N. (2009). An effective path selection strategy for mutation testing. In Proceedings of the 16th Asia-Pacific software engineering conference, 2009 (pp. 422-429). IEEE Computer Society. doi: 10. 1109/APSEC. 2009. 68.
32. Papadakis, M., & Malevris, N. (2010a). Automatic mutation test case generation via dynamic symbolic execution. In Software reliability engineering (ISSRE), 2010 IEEE 21st international symposium on, 1-4 Nov. 2010 (pp. 121-130). doi: 10. 1109/ISSRE. 2010. 38.
33. Papadakis, M., & Malevris, N. (2010b). An empirical evaluation of the first and second order mutation testing strategies. In Software testing, verification, and validation workshops (ICSTW), 2010 third international conference on, 6-10 April 2010 (pp. 90-99). doi: 10. 1109/ICSTW. 2010. 50.
34. Papadakis, M., Malevris, N., & Kallia, M. (2010). Towards automating the generation of mutation tests. In Proceedings of the 5th workshop on automation of software test, Cape Town, South Africa, 2010 (pp. 111-118). 1808283: ACM. doi: 10. 1145/1808266. 1808283.
35. Pǎsǎreanu, C. S., Mehlitz, P. C., Bushnell, D. H., Gundy-Burlet, K., Lowry, M., Person, S., et al. (2008). Combining unit-level symbolic execution and system-level concrete execution for testing nasa software. In Proceedings of the 2008 international symposium on software testing and analysis, Seattle, WA, USA (pp. 15-26). 1390635: ACM. doi: 10. 1145/1390630. 1390635.
36. Polo, M., Piattini, M., & García-Rodríguez, I. (2009). Decreasing the cost of mutation testing with second-order mutants. Software Testing, Verification and Reliability, 19(2), 111-131. doi: 10. 1002/stvr. 392.
37. Schuler, D., & Zeller, A. (2010). (un-)covering equivalent mutants. In Software testing, verification and validation (ICST), 2010 third international conference on, 6-10 April 2010 (pp. 45-54). doi: 10. 1109/ICST. 2010. 30.
38. Sen, K., Marinov, D., & Agha, G. (2005). Cute: A concolic unit testing engine for c. In Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, Lisbon, Portugal (pp. 263-272). 1081750: ACM. doi: 10. 1145/1081706. 1081750.
39. Smith, B. H., & Williams, L. (2009). On guiding the augmentation of an automated test suite via mutation analysis. Empirical Software Engineering, 14(3), 341-369. doi: 10. 1007/s10664-008-9083-7.
40. Tonella, P. (2004). Evolutionary testing of classes. SIGSOFT Software Engineering Notes, 29(4), 119-128. doi: 10. 1145/1013886. 1007528.
41. Untch, R. H., Offutt, A. J., & Harrold, M. J. (1993). Mutation analysis using mutant schemata. In Proceedings of the 1993 ACM SIGSOFT international symposium on software testing and analysis, Cambridge, Massachusetts, United States (pp. 139-148). 154265: ACM. doi: 10. 1145/154183. 154265.
42. Visser, W., Pǎsǎreanu, C. S., & Khurshid, S. (2004). Test input generation with java pathfinder. In Proceedings of the 2004 ACM SIGSOFT international symposium on software testing and analysis, Boston, Massachusetts, USA (pp. 97-107). 1007526: ACM. doi: 10. 1145/1007512. 1007526.
43. Yates, D., & Malevris, N. (1989). Reducing the effects of infeasible paths in branch testing. SIGSOFT Software Engineering Notes 14(8), 48-54. doi: 10. 1145/75309. 75315.