Mutation-driven generation of unit tests and oracles

IEEE Transactions on Software Engineering

Volumn 38, Issue 2, 2012, Pages 278-292

  Fraser, Gordon ^a   Zeller, Andreas ^a  

^a SAARLAND UNIVERSITY   (Germany)

Author keywords

assertions; Mutation analysis; search based testing; test case generation; test oracles; unit testing

Indexed keywords

ASSERTIONS; MUTATION ANALYSIS; SEARCH-BASED; TEST CASE GENERATION; TEST ORACLES; UNIT TESTING;

DEFECTS; SOFTWARE TESTING;

TESTING;

EID: 84859707325     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2011.93     Document Type: Article

References (52)

1. S. Ali, L.C. Briand, H. Hemmati, and R.K. Panesar-Walawege, "A Systematic Review of the Application and Empirical Investigation of Search-Based Test-Case Generation," IEEE Trans. Software Eng., vol. 36, no. 6, pp. 742-762, Nov./Dec. 2010
2. J.H. Andrews, L.C. Briand, and Y. Labiche, "Is Mutation an Appropriate Tool for Testing Experiments?" Proc. 27th Int'l Conf. Software Eng., pp. 402-411, 2005
3. J.H. Andrews, A. Groce, M. Weston, and R.G. Xu, "Random Test Run Length and Effectiveness," Proc. IEEE/ACM 23rd Int'l Conf. Automated Software Eng., pp. 19-28, 2008
4. J.H. Andrews, S. Haldar, Y. Lei, and F.C.H. Li, "Tool Support for Randomized Unit Testing," Proc. First Int'l Workshop Random Testing, pp. 36-45, 2006 (Pubitemid 46645424)
5. A. Arcuri, "It Does Matter How You Normalise the Branch Distance in Search Based Software Testing," Proc. Third Int'l Conf. Software Testing, Verification and Validation, pp. 205-214, 2010
6. A. Arcuri, "Longer Is Better: On the Role of Test Sequence Length in Software Testing," Proc. Third Int'l Conf. Software Testing, Verification and Validation, pp. 469-478, 2010
7. A. Arcuri and L. Briand, "A Practical Guide for Using Statistical Tests to Assess Randomized Algorithms in Software Engineering," Proc. IEEE Int'l Conf. Software Eng., pp. 1-10, 2011
8. A. Arcuri and X. Yao, "Search Based Software Testing of Object-Oriented Containers," Information Sciences, vol. 178, no. 15, pp. 3075-3095, 2008
9. K. Ayari, S. Bouktif, and G. Antoniol, "Automatic Mutation Test Input Data Generation via Ant Colony," Proc. Ninth Ann. Conf. Genetic and Evolutionary Computation, pp. 1074-1081, 2007 (Pubitemid 47291665)
10. M. Boshernitsan, R. Doong, and A. Savoia, "From Daikon to Agitator: Lessons and Challenges in Building a Commercial Tool for Developer Testing," Proc. Int'l Symp. Software Testing and Analysis, pp. 169-180, 2006
11. L. Bottaci, "A Genetic Algorithm Fitness Function for Mutation Testing," Proc. Int'l Workshop Software Eng. Using Metaheuristic Inovative Algorithms, a Workshop at 23rd Int'l Conf. Software Eng., pp. 3-7, 2001
12. V. Chvatal, "A Greedy Heuristic for the Set-Covering Problem," Math. Operations Research, vol. 4, no. 3, pp. 233-235, 1979
13. I. Ciupa, A. Leitner, M. Oriol, and B. Meyer, "Artoo: Adaptive Random Testing for Object-Oriented Software," Proc. 30th Int'l Conf. Software Eng., pp. 71-80, 2008
14. C. Csallner and Y. Smaragdakis, "JCrasher: An Automatic Robustness Tester for Java," Software Practice and Experience, vol. 34, no. 11, pp. 1025-1050, 2004
15. R.A. DeMillo, R.J. Lipton, and F.G. Sayward, "Hints on Test Data Selection: Help for the Practicing Programmer," Computer, vol. 11, no. 4, pp. 34-41, Apr. 1978
16. R.A. DeMillo and A.J. Offutt, "Constraint-Based Automatic Test Data Generation," IEEE Trans. Software Eng., vol. 17, no. 9, pp. 900-910, Sept. 1991 (Pubitemid 21717628)
17. R.B. Evans and A. Savoia, "Differential Testing: A New Approach to Change Detection," Proc. Sixth Joint Meeting of the European Software Eng. Conf. and the ACM SIGSOFT Symp. Foundations of Software Eng., pp. 549-552, 2007 (Pubitemid 351438718)
18. G. Fraser and A. Arcuri, "It Is Not the Length that Matters, It Is How You Control It," Proc. IEEE Fourth Int'l Conf. Software Testing, Verification and Validation, 2011
19. G. Fraser and A. Zeller, "Mutation-Driven Generation of Unit Tests and Oracles," Proc. 19th Int'l Symp. Software Testing and Analysis, pp. 147-158, 2010
20. P. Godefroid, N. Klarlund, and K. Sen, "DART: Directed Automated Random Testing," Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, pp. 213-223, 2005
21. M. Harman, L. Hu, R. Hierons, J.Wegener, H. Sthamer, A. Baresel, and M. Roper, "Testability Transformation," IEEE Trans. Software Eng., vol. 30, no. 1, pp. 3-16, Jan. 2004
22. M. Harman and P. McMinn, "A Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search," IEEE Trans. Software Eng., vol. 36, pp. 226-247, Mar./Apr. 2010
23. M.J. Harrold, R. Gupta, and M.L. Soffa, "A Methodology for Controlling the Size of a Test Suite," ACM Trans. Software Eng. Methodology, vol. 2, pp. 270-285, July 1993 (Pubitemid 23704558)
24. K. Inkumsah and T. Xie, "Improving Structural Testing of Object-Oriented Programs via Integrating Evolutionary Testing and Symbolic Execution," Proc. IEEE/ACM 23rd Int'l Conf. Automated Software Eng., pp. 297-306, 2008
25. Y. Jia and M. Harman, "An Analysis and Survey of the Development of Mutation Testing," IEEE Trans. Software Eng., vol. 37, no. 5, pp. 649-678, Sept./Oct. 2011
26. B.F. Jones, D.E. Eyres, and H.-H. Sthamer, "A Strategy for Using Genetic Algorithms to Automate Branch and Fault-Based Testing," The Computer J., vol. 41, no. 2, pp. 98-107, 1998 (Pubitemid 128485593)
27. Y. Lei and J.H. Andrews, "Minimization of Randomized Unit Test Cases," Proc. IEEE 16th Int'l Symp. Software Reliability Eng., pp. 267-276, 2005 (Pubitemid 44733909)
28. A. Leitner, M. Oriol, A. Zeller, I. Ciupa, and B. Meyer, "Efficient Unit Test Case Minimization," Proc. IEEE/ACM 22nd Int'l Conf. Automated Software Eng., pp. 417-420, 2007
29. P. McMinn, "Search-Based Software Test Data Generation: A Survey: Research Articles," Software Testing Verification Reliability, vol. 14, no. 2, pp. 105-156, 2004
30. A.J. Offutt, "Investigations of the Software Testing Coupling Effect," ACM Trans. Software Eng. Methodology, vol. 1, pp. 5-20, Jan. 1992
31. A.J. Offutt, Z. Jin, and J. Pan, "The Dynamic Domain Reduction Procedure for Test Data Generation," Software Practice and Experience, vol. 29, pp. 167-193, Feb. 1999
32. A.J. Offutt, A. Lee, G. Rothermel, R.H. Untch, and C. Zapf, "An Experimental Determination of Sufficient Mutant Operators," ACM Trans. Software Eng. and Methodology, vol. 5, pp. 99-118, Apr. 1996 (Pubitemid 126418670)
33. A. Orso and T. Xie, "BERT: BEhavioral Regression Testing," Proc. Int'l Workshop Dynamic Analysis, pp. 36-42, July 2008
34. C. Pacheco and M.D. Ernst, "Eclat: Automatic Generation and Classification of Test Inputs," Proc. 19th European Conf. Object-Oriented Programming, pp. 504-527, July 2005 (Pubitemid 41435995)
35. C. Pacheco and M.D. Ernst, "Randoop: Feedback-Directed Random Testing for Java," Companion to the 22nd ACM SIGPLAN Conf. Object-Oriented Programming Systems and Applications Companion, pp. 815-816, 2007
36. G. Rothermel, M.J. Harrold, J. Ostrin, and C. Hong, "An Empirical Study of the Effects of Minimization on the Fault Detection Capabilities of Test Suites," Proc. Int'l Conf. Software Maintenance, pp. 34-43, 1998
37. A. Salcianu and M.C. Rinard, "Purity and Side Effect Analysis for Java Programs," Proc. Sixth Int'l Conf. Verification, Model Checking, and Abstract Interpretation, pp. 199-215, 2005 (Pubitemid 41231362)
38. D. Schuler, V. Dallmeier, and A. Zeller, "Efficient Mutation Testing by Checking Invariant Violations," Proc. 18th Int'l Symp. Software Testing and Analysis, pp. 69-80, 2009
39. D. Schuler and A. Zeller, "Javalanche: Efficient Mutation Testing for Java," Proc. Seventh Joint Meeting of the European Software Eng. Conf. and the ACM SIGSOFT Symp. Foundations of Software Eng., pp. 297-298, 2009
40. D. Schuler and A. Zeller, "(Un-)Covering Equivalent Mutants," Proc. Third Int'l Conf. Software Testing, Verification, and Validation, pp. 45-54, 2010
41. K. Sen, D. Marinov, and G. Agha, "CUTE: A Concolic Unit Testing Engine for C," Proc. 10th European Software Eng. Conf. Held Jointly with 13th ACM SIGSOFT Int'l Symp. Foundations of Software Eng., pp. 263-272, 2005 (Pubitemid 43218836)
42. R. Sharma, M. Gligoric, A. Arcuri, G. Fraser, and D. Marinov"- Testing Container Classes: Random or Systematic?" Proc. 14th Int'l Conf. Fundamental Approaches to Software Eng., D. Giannakopoulou and F. Orejas, eds., pp. 262-277, 2011
43. A. Siami Namin, J.H. Andrews, and D.J. Murdoch, "Sufficient Mutation Operators for Measuring Test Effectiveness," Proc. 30th Int'l Conf. Software Eng., pp. 351-360, 2008
44. K. Taneja and T. Xie, "DiffGen: Automated Regression Unit-Test Generation," Proc. IEEE/ACM 23rd Int'l Conf. Automated Software Eng., pp. 407-410, 2008
45. N. Tillmann and J.N. de, "Halleux, Pex-White Box Test Generation for .NET," Proc. Int'l Conf. Tests and Proofs, pp. 134-253, 2008
46. P. Tonella, "Evolutionary Testing of Classes," Proc. ACM SIGSOFT Int'l Symp. Software Testing and Analysis, pp. 119-128, 2004 (Pubitemid 41121383)
47. A. Vargha and H.D. Delaney, "A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong," J. Educational and Behavioral Statistics, vol. 25, no. 2, pp. 101-132, 2000
48. S. Wappler and F. Lammermann, "Using Evolutionary Algorithms for the Unit Testing of Object-Oriented Software," Proc. Conf. Genetic and Evolutionary Computation, pp. 1053-1060, 2005 (Pubitemid 43226414)
49. N. Williams, B. Marre, P. Mouy, and M. Roger, "PathCrawler: Automatic Generation of Path Tests by Combining Static and Dynamic Analysis," Proc. Fifth European Dependable Computing Conf., pp. 281-292, 2005 (Pubitemid 41312765)
50. T. Xie, "Augmenting Automatically Generated Unit-Test Suites with Regression Oracle Checking," Proc. 20th European Conf. Object-Oriented Programming, pp. 380-403, July 2006 (Pubitemid 44153420)
51. Z. Xu, M.B. Cohen, and G. Rothermel, "Factors Affecting the Use of Genetic Algorithms in Test Suite Augmentation," Proc. 12th Ann. Conf. Genetic and Evolutionary Computation, pp. 1365-1372, 2010
52. L. Zhang, T. Xie, L. Zhang, N. Tillmann, J. de Halleux, and H. Mei, "Test Generation via Dynamic Symbolic Execution for Mutation Testing," Proc. IEEE 26th Int'l Conf. Software Maintenance, Sept. 2010.