Efficient multi-objective higher order mutation testing with genetic programming

Journal of Systems and Software

Volumn 83, Issue 12, 2010, Pages 2416-2430

  Langdon, William B ^a   Harman, Mark ^a   Jia, Yue ^a  

^a KING'S COLLEGE LONDON   (United Kingdom)

Author keywords

Genetic algorithm; Genetic programming; Grammar based GP; Gzip; Higher order mutation; Monte Carlo; Mutation testing; Non determinism; NSGA II; Pareto optimality; SBSE; Schedule; Strongly typed GP; tcas; Triangle

Indexed keywords

GRAMMAR BASED GP; GZIP; HIGHER ORDER; MONTE CARLO; MUTATION TESTING; NON-DETERMINISM; NSGA-II; PARETO-OPTIMALITY; SBSE; SCHEDULE; STRONGLY TYPED GP; TCAS; TRIANGLE;

COMPUTER PROGRAMMING; COMPUTER SOFTWARE SELECTION AND EVALUATION; GENETIC ALGORITHMS; MONTE CARLO METHODS; MULTIOBJECTIVE OPTIMIZATION; PARETO PRINCIPLE;

SOFTWARE TESTING;

EID: 78049330669     PISSN: 01641212     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jss.2010.07.027     Document Type: Conference Paper

References (33)

1. Adamopoulos, K., Harman, M., Hierons, R.M., 2004. How to overcome the equivalent mutant problem and achieve tailored selective mutation using co-evolution. In: Deb, K., et al. (Eds.), Genetic and Evolutionary Computation - GECCO-2004, Part II. LNCS 3103. Springer, Seattle, WA, USA, pp. 1338-1349. http://link.springer.de/link/service/series/0558/bibs/3103/31031338.htm.
2. Ayari, K., Bouktif, S., Antoniol, G., 2007. Automatic mutation test input data generation via ant colony. In: Thierens, D., et al. (Eds.), GECCO '07: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, vol. 1. ACM Press, London, pp. 1074-1081. http://www.cs.bham.ac.uk/ wbl/biblio/gecco2007/docs/p1074.pdf.
3. J.C. Bongard, and H. Lipson Nonlinear system identification using coevolution of models and tests IEEE Transaction on Evolutionary Computation 9 4 2005 361 384
4. K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan A fast and elitist multiobjective genetic algorithm: NSGA-II IEEE Transactions on Evolutionary Computation 6 2 2002 182 197
5. R.A. DeMillo, R.J. Lipton, and F.G. Sayward Hints on test data selection: help for the practicing programmer Computer 11 4 1978 34 41
6. R.A. DeMillo, and A.J. Offutt Constraint-based automatic test data generation IEEE Transactions on Software Engineering 17 9 1991 900 910
7. M.C.F.P. Emer, and S.R. Vergilio Selection and evaluation of test data based on genetic programming Software Quality Journal 11 2 2003 167 186
8. F.C. Ferrari, J.C. Maldonado, and A. Rashid Mutation testing for aspect-oriented programs ICST '08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation 2008 52 61
9. Fonseca, C.M., Fleming, P.J., 1993. Genetic algorithms for multiobjective optimization: formulation, discussion and generalization. In: Forrest, S. (Ed.), Proceedings of the 5th International Conference on Genetic Algorithms, ICGA-93. Morgan Kaufmann, University of Illinois at Urbana-Champaign, pp. 416-423.
10. Harman, M., Jia, Y., Langdon, W.B., 2010. A manifesto for higher order mutation testing. In: du Bousquet, L., Bradbury, J., Fraser, G. (Eds.), Mutation 2010. IEEE Computer Society, Paris, pp. 80-89, keynote. http://www.dcs.kcl.ac. uk/pg/jiayue/publications/papers/HarmanJL10.pdf.
11. M. Hutchins, H. Foster, T. Goradia, and T. Ostrand Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria Proceedings of 16th International Conference on Software Engineering, ICSE-16 May 1994 191 200
12. Jia, Y., Harman, M., 2009a. An Analysis and Survey of the Development of Mutation Testing. Technical Report TR-09-06. CREST Centre, King's College London, London, UK.
13. Y. Jia, and M. Harman Higher order mutation testing Journal of Information and Software Technology 51 10 2009 1379 1393
14. Kim, S., Clark, J.A., Mcdermid, J.A., 2001. Investigating the effectiveness of object-oriented testing strategies using the mutation method. In: Mutation Testing for the New, Century, vol. 11., pp. 207-225.
15. Lakhotia, K., Harman, M., McMinn, P., 2007. A multi-objective approach to search-based test data generation. In: Thierens, et al. (Eds.), GECCO '07: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, vol. 1. ACM Press, London, pp. 1098-1105. http://www.cs.bham.ac.uk/ wbl/biblio/gecco2007/docs/p1098.pdf.
16. Langdon, W.B., 1998. Genetic Programming and Data Structures. Kluwer, Boston. http://www.cs.ucl.ac.uk/staff/W.Langdon/gpdata.
17. Langdon, W.B., Harman, M., Jia, Y., 2009. Multi objective mutation testing with genetic programming. In: Bottaci, L., Kapfhammer, G., Walkinshaw, N. (Eds.), TAIC-PART. IEEE, Windsor, UK, pp. 21-29. http://www.cs.ucl.ac.uk/ staff/W.Langdon/ftp/papers/langdon-2009-TAICPART.pdf.
18. Langdon, W.B., Harrison, A.P., 2008. Evolving regular expressions for GeneChip probe performance prediction. Tech. Rep. CES-483, Computing and Electronic Systems. University of Essex, Wivenhoe Park, Colchester CO4 3SQ, UK. http://www.essex.ac.uk/dces/research/publications/technicalreports/2008/CES-483. pdf
19. Langdon, W.B., Harrison, A.P., 2009. Evolving DNA motifs to predict GeneChip probe performance. Algorithms in Molecular Biology 4(6). http://www.cs.ucl.ac.uk/staff/W.Langdon/ftp/papers/langdon-amb.pdf.
20. Y.-S. Ma, J. Offutt, and Y.R. Kwon MuJava: an automated class mutation system Software Testing, Verification & Reliability 15 2 2005 97 133
21. A. Offutt, and S. Lee An empirical evaluation of weak mutation IEEE Transactions on Software Engineering 20 5 1994 337 344
22. A.J. Offutt Investigations of the software testing coupling effect ACM Transactions on Software Engineering and Methodology 1 1 1992 5 20
23. A.J. Offutt, A. Lee, G. Rothermel, R.H. Untch, and C. Zapf An experimental determination of sufficient mutant operators ACM Transactions on Software Engineering and Methodology 5 2 1996 99 118
24. A.J. Offutt, J. Pan, K. Tewary, and T. Zhang An experimental evaluation of data flow and mutation testing Software - Practice and Experience 26 2 1996 165 176
25. Offutt, A.J., Untch, R.H., 2001. Mutation 2000: Uniting the Orthogonal. In: Proceedings of the 1st Workshop on Mutation Analysis (MUTATION'00), San Jose, California, pp. 34-44, published in book form, as Mutation Testing for the New Century.
26. Poli, R., Langdon, W.B., McPhee, N.F., 2008. A field guide to genetic programming. Published via http://lulu.com and freely available at http://www.gp-field-guide.org.uk (with contributions by J.R. Koza). http://www.gp-field-guide.org.uk.
27. R. Purushothaman, and D.E. Perry Toward understanding the rhetoric of small source code changes IEEE Transactions on Software Engineering 31 2005 511 526
28. Sen, K., Marinov, D., Agha, G., 2005. CUTE: a concolic unit testing engine for C. In: ESEC/FSE-13: Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, New York, NY, USA, pp. 263-272.
29. Tuya, J., Suarez-Cabal, M.J., de la Riva, C., 2006. SQLMutation: A tool to generate mutants of SQL database queries. In: Proceedings of the Second Workshop on Mutation Analysis, November. IEEE Computer Society, Raleigh, North Carolina, p. 1.
30. R.H. Untch, A.J. Offutt, and M.J. Harrold Mutation analysis using mutant schemata Proceedings of the 1993 ACM SIGSOFT international symposium on Software testing and analysis Cambridge, Massachusetts 1993 139 148
31. Weimer, W., Nguyen, T., Le Goues, C., Forrest, S., 2009. Automatically finding patches using genetic programming. In: Fickas, S. (Ed.), International Conference on Software Engineering (ICSE) 2009. Vancouver, 16-24 May, pp. 364-374. http://www.cs.unm.edu/tnguyen/papers/genprog.pdf.
32. S. Yoo, and M. Harman Pareto efficient multi-objective test case selection ISSTA '07: Proceedings of the 2007 international symposium on Software testing and analysis 2007 New York, NY, USA 140 150
33. H. Zhong, L. Zhang, and H. Mei An experimental study of four typical test suite reduction techniques Information and Software Technology 50 6 2008 534 546 http://www.sciencedirect.com/science/article/B6V0B-4P2J035-1/2/ d5014b52df76dbb547b94b0cf387eab5