Validating second-order mutation at system level

IEEE Transactions on Software Engineering

Volumn 39, Issue 4, 2013, Pages 570-587

  Reales Mateo, Pedro ^a   Polo Usaola, MacArio ^a   Fernández Alemán, José Luis ^b  

^a UNIVERSITY OF CASTILLA LA MANCHA   (Spain)

^b UNIVERSITY OF MURCIA   (Spain)

Author keywords

Empirical evaluation; high order mutation; mutation testing

Indexed keywords

COMBINATION STRATEGIES; EMPIRICAL EVALUATIONS; EMPIRICAL STUDIES; HIGH-ORDER; MUTATION OPERATORS; MUTATION TESTING; PROGRAM STATEMENTS; SOFTWARE TESTING TECHNIQUES;

COST REDUCTION; RISK PERCEPTION; SOFTWARE TESTING; TESTING;

COST BENEFIT ANALYSIS;

EID: 84875720693     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2012.39     Document Type: Article

References (53)

1. Y. Jia and M. Harman, "Higher Order Mutation Testing," Information and Software Technology, vol. 51, no. 10, pp. 1379-1393, 2009.
2. P. Ammann and J. Offutt, Introduction to Software Testing. Cambridge Univ. Press, 2008.
3. M. Polo, M. Piattini, and I. Garca-Rodrguez, "Decreasing the Cost of Mutation Testing with Second-Order Mutants," Software Testing, Verification, and Reliability, vol. 19, no. 2, pp. 111-131, 2008.
4. A.J. Offutt and J. Pan, "Automatically Detecting Equivalent Mutants and Infeasible Paths," Software Testing, Verification, and Reliability, vol. 7, no. 3, pp. 165-192, 1997. (Pubitemid 127661584)
5. D. Schuler and A. Zeller, "(Un-)Covering Equivalent Mutants," Proc. Third Int'l Conf. Software Testing Verification and Validation, Apr. 2010.
6. B.J.M. Grü n, D. Schuler, and A. Zeller, "The Impact of Equivalent Mutants," Proc. IEEE Int'l Conf. Software Testing, Verification, and Validation Workshops, pp. 192-199, Apr. 2009.
7. M. Polo and P. Reales, "Mutation Testing Cost Reduction Techniques: A Survey," IEEE Software, vol. 27, no. 3, pp. 80-86, May/June 2010.
8. Y. Jia and M. Harman, "Constructing Subtle Faults Using Higher Order Mutation Testing," Proc. Eighth Int'l Working Conf. Source Code Analysis and Manipulation, pp. 28-29, Sept. 2008.
9. M. Papadakis, N. Malevris, and M. Kintis, "Mutation Testing Strategies: A Collateral Approach," Proc. Fifth Int'l Conf. Software and Data Technologies, pp. 325-328, June 2011.
10. M. Harman, Y. Jia, and W. Langdon, "A Manifesto for Higher Order Mutation Testing," Proc. Third Int'l Conf. Software Testing, Verification, and Validation Workshops, pp. 80-89, 2010.
11. M. Papadakis and N. Malevris, "An Empirical Evaluation of the First and Second Order Mutation Testing Strategies," Proc. Software Testing, Verification, and Validation Workshops, pp. 90-99, Apr. 2010.
12. P. Reales, M. Polo, and J. Offutt, "Mutation at System and Functional Levels," Proc. Third Int'l Conf. Software Testing, Verification, and Validation Workshops, pp. 110-119, Apr. 2010.
13. T.A. Budd and D. Angluin, "Two Notions of Correctness and Their Relation to Testing," Acta Informatica, vol. 18, no. 1, pp. 31-45, Mar. 1982. (Pubitemid 13474053)
14. D. Baldwin and F.G. Sayward, "Heuristics for Determining Equivalence of Program Mutations," report, 1979.
15. A. Offutt and J. Pan, "Detecting Equivalent Mutants and the Feasible Path Problem," Wiley's Software Testing, Verification, and Reliability, vol. 7, no. 3, pp. 165-192, Sept. 1997. (Pubitemid 127661584)
16. R. Hierons, M. Harman, and S. Danicic, "Using Program Slicing to Assist in the Detection of Equivalent Mutants," Software Testing, Verification, and Reliability, vol. 9, no. 4, pp. 233-262, Dec. 1999. (Pubitemid 30546632)
17. E. Mresa and L. Bottaci, "Efficiency of Mutation Operators and Selective Mutation Strategies: An Empirical Study," Software Testing, Verification, and Reliability, vol. 9, no. 4, pp. 205-232, Dec. 1999. (Pubitemid 30546631)
18. A.J. Offutt, G. Rothermel, R.H. Untch, and C. Zapf, "An Experimental Determination of Sufficient Mutant Operators," ACM Trans. Software Eng. and Methodology, vol. 5, no. 2, pp. 99-118, 1996. (Pubitemid 126418670)
19. E.F. Barbosa, J.C. Maldonado, and A.M.R. Vincenzi, "Toward the Determination of Sufficient Mutant Operators for C," Software Testing, Verification, and Reliability, vol. 11, no. 2, pp. 113-136, 2001. (Pubitemid 32636646)
20. R.A. DeMillo and E.H. Spafford, "The Mothra Software Testing Environment," Proc. 11th NASA Software Eng. Laboratory Workshop, 1986.
21. A.T. Acree, "On Mutation," thesis, School of Information and Computer Science, Georgia Inst. of Technology, 1980.
22. K.N. King and A.J. Offutt, "A Fortran Language System for Mutation Based Software Testing," Software: Practice and Experience, vol. 21, no. 7, pp. 685-718, 1991. (Pubitemid 21666424)
23. R. Untch, A. Offutt, and M. Harrold, "Mutation Analysis Using Program Schemata," Proc. Int'l Symp. Software Testing and Analysis, pp. 139-148, June 1993.
24. Y.-S. Ma, J. Offutt, and Y.R. Kwon, "MuJava: An Automated Class Mutation System," Software Testing, Verification, and Reliability, vol. 15, no. 2, pp. 97-133, 2005. (Pubitemid 40858060)
25. M. Grindal, A.J. Offutt, and S.F. Andler, "Combination Testing Strategies: A Survey," Software Testing, Verification, and Reliability, vol. 15, pp. 167-199, 2005. (Pubitemid 41320808)
26. K.-C. Tai and Y. Lei, "A Test Generation Strategy for Pairwise Testing," IEEE Trans. Software Eng., vol. 28, no. 1, pp. 109-111, Jan. 2002. (Pubitemid 34129838)
27. A.J. Offutt, "Investigations of the Software Testing Coupling Effect," ACM Trans. Software Eng. and Methodology, vol. 1, no. 1, pp. 15-20, 1992.
28. Y. Jia and M. Harman, "MILU: A Customizable, Runtime-Optimized Higher Order Mutation Testing Tool for the Full C Language," Proc. Third Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 94-98, Aug. 2008.
29. M. Polo, M. Piattini, and S. Tendero, "Integrating Techniques and Tools for Testing Automation," Software Testing, Verification, and Reliability, vol. 17, no. 1, pp. 3-39, 2007. (Pubitemid 46336330)
30. H. Do, S. Elbaum, and G. Rothermel, "Infrastructure Support for Controlled Experimentation with Software Testing and Regression Testing Techniques," Proc. Int'l Symp. Empirical Software Eng., pp. 60-70, Aug. 2004. (Pubitemid 40062819)
31. M.E. Delamaro, J.C. Maldonado, and A. Vincenzi, "Proteum/IM 2.0: An Integrated Mutation Testing Environment," Proc. First Workshop Mutation Analysis, pp. 91-101, Oct. 2000.
32. M. Kintis, M. Papadakis, and N. Malevris, "Evaluating Mutation Testing Alternatives: A Collateral Experiment," Proc. Asia Pacific Eng. Conf., Nov./Dec. 2010.
33. H. Agrawal, "Dominators, Super Blocks, and Program Coverage," Proc. 21st ACM SIGPLAN-SIGACT Symp. Principles of Programming Languages, pp. 25-34, Jan. 1994.
34. W. Langdon, M. Harman, and Y. Jia, "Efficient Multi-Objective Higher Order Mutation Testing with Genetic Programming," J. Systems and Software, vol. 83, no. 12, pp. 2416-2430, 2010.
35. W. Langdon, M. Harman, and Y. Jia, "Multi Objective Higher Order Mutation Testing with GP," Proc. 11th Ann. Genetic and Evolutionary Computation Conf., pp. 1945-1946, 2009.
36. W. Langdon, M. Harman, and Y. Jia, "Multi Objective Higher Order Mutation Testing with Genetic Programming," Proc. Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 21-29, 2009.
37. M. Hutchins, H. Foster, T. Goradia, and T. Ostrand, "Experiments of the Effectiveness of Dataflow-and Controlflow-Based Test Adequacy Criteria," Proc. 16th Int'l Conf. Software Eng., 1994.
38. R. DeMillo, R.J. Lipton, and F.G. Sayward, "Hints on Test Data Selection: Help for the Practicing Programmer," Computer, vol. 11, no. 4, pp. 34-41, April 1978.
39. G. Kaminski, U. Praphamontripong, P. Ammann, and J. Offutt, "A Logic Mutation Approach to Selective Mutation for Programs and Queries," Information and Software Technology, vol. 53, pp. 1137-1152, 2011.
40. G. Kaminski, P. Amman, and A.J. Offut, "Better Predicate Testing," Proc. Sixth Int'l Workshop Automation of Software Test, pp. 57-63, 2011.
41. G. Kaminski and P. Amman, "Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection," Proc. Int'l Conf. Software Testing, Apr. 2009.
42. R. Just, G.M. Kapfhammer, and F. Schweiggert, "Using Conditional Mutation to Increase the Efficiency of Mutation Analysis," Proc. Sixth Int'l Workshop Automation of Software Test, pp. 50-56, 2010, 2011.
43. R. Purushothaman and D.E. Perry, "Toward Understanding the Rhetoric of Small Source Code Changes," IEEE Trans. Software Eng., vol. 31, no. 6 pp. 511-526, June 2005. (Pubitemid 41245072)
44. V.R. Basili and H.D. Rombach, "The TAME Project: Towards Improvement-Oriented Softwareenvironments," IEEE Trans. Software Eng., vol. 14, no. 6, pp. 758-773, June 1988. (Pubitemid 18640017)
45. V.R. Basili, F. Shull, and F. Lanubile, "Building Knowledge through Families of Experiments," IEEE Trans. Software Eng., vol. 25, no. 4, pp. 456-473, July/Aug. 1999. (Pubitemid 129304112)
46. M. Harrold, R. Gupta, and M. Soffa, "A Methodology for Controlling the Size of a Test Suite," ACM Trans. Software Eng. and Methodology, vol. 2, no. 3, pp. 270-285, 1993. (Pubitemid 23704558)
47. D. Jeffrey and N. Gupta, "Test Suite Reduction with Selective Redundancy," Proc. Int'l Conf. Software Maintenance, pp. 549-558, 2005. (Pubitemid 43790967)
48. S. Tallam and N. Gupta, "A Concept Analysis Inspired Greedy Algorithm for Test Suite Minimization," Proc. Sixth ACM SIGPLAN-SIGSOFT Workshop Program Analysis for Software Tools and Eng., pp. 35-42, 2005.
49. "Eclipse Framework," http://www.eclipse.org/, Feb. 2011.
50. L. Walton, "Eclipse Metrics Plugin," http://eclipse-metrics. sourceforge.net/, Feb. 2011.
51. "SPSS Statistics," http://www.spss.com/, Feb. 2011.
52. S. Ghosh and A.P. Mathur, "Interface Mutation," Software Testing, Verification, and Reliability, no. 11, pp. 227-247, 2001. (Pubitemid 34012880)
53. C. Wohlin, P. Runeson, M. Hö st, M. Ohlsson, B. Regnell, and A. Wesslén, Experimentation in Software Engineering: An Introduction. Kluwer Academic, 2000.