An analysis and survey of the development of mutation testing

IEEE Transactions on Software Engineering

Volumn 37, Issue 5, 2011, Pages 649-678

  Jia, Yue ^a   Harman, Mark ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Mutation testing; survey

Indexed keywords

COMPREHENSIVE ANALYSIS; DEVELOPMENT TRENDS; EMPIRICAL RESULTS; FAULT-BASED; MUTATION TESTING; SOFTWARE TESTING TECHNIQUES;

SURVEYS; TESTING;

SOFTWARE TESTING;

EID: 80053606092     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2010.62     Document Type: Review

References (264)

1. R. Abraham and M. Erwig, "Mutation Operators for Spreadsheets," IEEE Trans. Software Eng., vol. 35, no. 1, pp. 94-108, Jan./ Feb. 2009.
2. A.T. Acree, "On Mutation," PhD thesis, Georgia Inst. of Technology, 1980.
3. A.T. Acree, T.A. Budd, R.A. DeMillo, R.J. Lipton, and F.G. Sayward, "Mutation Analysis," Technical Report GIT-ICS-79/08, Georgia Inst. of Technology, 1979.
4. K. Adamopoulos, "Search Based Test Selection and Tailored Mutation," master's thesis, King's College London, 2009.
5. K. Adamopoulos, M. Harman, and R.M. Hierons, "How to Overcome the Equivalent Mutant Problem and Achieve Tailored Selective Mutation Using Co-Evolution," Proc. Genetic and Evolutionary Computation Conf., pp. 1338-1349, June 2004. (Pubitemid 39740197)
6. H. Agrawal, R.A. DeMillo, B. Hathaway, W. Hsu, W. Hsu, E.W. Krauser, R.J. Martin, A.P. Mathur, and E. Spafford, "Design of Mutant Operators for the C Programming Language," Technical Report SERC-TR-41-P, Purdue Univ., Mar. 1989.
7. B.K. Aichernig, "Mutation Testing in the Refinement Calculus," Formal Aspects of Computing, vol. 15, nos. 2-3, pp. 280-295, Nov. 2003.
8. B.K. Aichernig and C.C. Delgado, "From Faults via Test Purposes to Test Cases: On the Fault-Based Testing of Concurrent Systems," Proc. Ninth Int'l Conf. Fundamental Approaches to Software Eng., pp. 324-338, Mar. 2006. (Pubitemid 44019317)
9. R.T. Alexander, J.M. Bieman, S. Ghosh, and B. Ji, "Mutation of Java Objects," Proc. 13th Int'l Symp. Software Reliability Eng., pp. 341-351, Nov. 2002.
10. S. Ali, L.C. Briand, H. Hemmati, and R.K. Panesar-Walawege, "A Systematic Review of the Application and Empirical Investigation of Search-Based Test-Case Generation," IEEE Trans. Software Eng., vol. 36, no. 6, pp. 742-762, Nov./Dec. 2010.
11. P. Ammann and J. Offutt, Introduction to Software Testing. Cambridge Univ. Press, 2008.
12. P. Anbalagan and T. Xie, "Efficient Mutant Generation for Mutation Testing of Pointcuts in Aspect-Oriented Programs," Proc. Second Workshop Mutation Analysis, pp. 51-56, Nov. 2006.
13. P. Anbalagan and T. Xie, "Automated Generation of Pointcut Mutants for Testing Pointcuts in AspectJ Programs," Proc. 19th Int'l Symp. Software Reliability Eng., pp. 239-248, Nov. 2008.
14. J.H. Andrews, L.C. Briand, and Y. Labiche, "Is Mutation an Appropriate Tool for Testing Experiments?" Proc. 27th Int'l Conf. Software Eng., pp. 402-411, May 2005.
15. J.H. Andrews, L.C. Briand, Y. Labiche, and A.S. Namin, "Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria," IEEE Trans. Software Eng., vol. 32, no. 8, pp. 608-624, Aug. 2006. (Pubitemid 46405276)
16. K. Ayari, S. Bouktif, and G. Antoniol, "Automatic Mutation Test Input Data Generation via Ant Colony," Proc. Genetic and Evolutionary Computation Conf., pp. 1074-1081, July 2007. (Pubitemid 47291665)
17. J.S. Baekken and R.T. Alexander, "A Candidate Fault Model for AspectJ Pointcuts," Proc. 17th Int'l Symp. Software Reliability Eng., pp. 169-178, Nov. 2006. (Pubitemid 47213953)
18. D. Baldwin and F.G. Sayward, "Heuristics for Determining Equivalence of Program Mutations," Research Report 276, Yale Univ., 1979.
19. E.F. Barbosa, J.C. Maldonado, and A.M.R. Vincenzi, "Toward the Determination of Sufficient Mutant Operators for C," Software Testing, Verification, and Reliability, vol. 11, no. 2, pp. 113-136, May 2001. (Pubitemid 32636646)
20. S.S. Batth, E.R. Vieira, A.R. Cavalli, and M.U. Uyar, "Specification of Timed EFSM Fault Models in SDL," Proc. 27th IFIP WG 6.1 Int'l Conf. Formal Techniques for Networked and Distributed Systems, pp. 50-65, June 2007.
21. B. Baudry, F. Fleurey, J.-M. Jezequel, and Y. Le Traon, "Genes and Bacteria for Automatic Test Cases Optimization in the .NET Environment," Proc. 13th Int'l Symp. Software Reliability Eng., pp. 195-206, Nov. 2002.
22. B. Baudry, V. Le Hanh, J.-M. Jézéquel, and Y. Le Traon, "Trustable Components: Yet Another Mutation-Based Approach," Proc. First Workshop Mutation Analysis, pp. 47-54, Oct. 2000.
23. F. Belli, C.J. Budnik, and W.E. Wong, "Basic Operations for Generating Behavioral Mutants," Proc. Second Workshop Mutation Analysis, p. 9, 2006.
24. J. Bieman, S. Ghosh, and R.T. Alexander, "A Technique for Mutation of Java Objects," Proc. 16th IEEE Int'l Conf. Automated Software Eng., p. 337, Nov. 2001.
25. P.E. Black, V. Okun, and Y. Yesha, "Mutation of Model Checker Specifications for Test Generation and Evaluation," Proc. First Workshop Mutation Analysis, pp.14-20, Oct. 2000.
26. B. Bogacki and B. Walter, "Evaluation of Test Code Quality with Aspect-Oriented Mutations," Proc. Seventh Int'l Conf. eXtreme Programming and Agile Processes in Software Eng., pp. 202-204, June 2006. (Pubitemid 44098523)
27. B. Bogacki and B. Walter, "Aspect-Oriented Response Injection: An Alternative to Classical Mutation Testing," Software Eng. Techniques: Design for Quality, pp. 273-282, Springer, 2007.
28. N. Bombieri, F. Fummi, and G. Pravadelli, "A Mutation Model for the SystemC TLM2.0 Communication Interfaces," Proc. Conf. Design, Automation and Test in Europe, pp. 396-401, Mar. 2008.
29. J.H. Bowser, "Reference Manual for Ada Mutant Operators," Technical Report GIT-SERC-88/02, Georgia Inst. of Technology, 1988.
30. J.S. Bradbury, J.R. Cordy, and J. Dingel, "ExMAn: A Generic and Customizable Framework for Experimental Mutation Analysis," Proc. Second Workshop Mutation Analysis, pp. 57-62, Nov. 2006.
31. J.S. Bradbury, J.R. Cordy, and J. Dingel, "Mutation Operators for Concurrent Java (J2SE 5.0)," Proc. Second Workshop Mutation Analysis, pp. 83-92, Nov. 2006.
32. J.S. Bradbury, J.R. Cordy, and J. Dingel, "Comparative Assessment of Testing and Model Checking Using Program Mutation," Proc. Third Workshop Mutation Analysis, published with Proc. Second Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 210-222, 2007.
33. P. Brady, "MutateMe," http://github.com/padraic/mutateme/ tree/master, 2007.
34. T.A. Budd, "Mutation Analysis of Program Test Data," PhD thesis, Yale Univ., 1980.
35. T.A. Budd and D. Angluin, "Two Notions of Correctness and Their Relation to Testing," Acta Informatica, vol. 18, no. 1, pp. 31- 45, Mar. 1982. (Pubitemid 13474053)
36. T.A. Budd, R.A. DeMillo, R.J. Lipton, and F.G. Sayward, "The Design of a Prototype Mutation System for Program Testing," Proc. Am. Fed. of Information Processing Soc. Nat'l Computer Conf., vol. 74, pp. 623-627, June 1978.
37. T.A. Budd, R.A. DeMillo, R.J. Lipton, and F.G. Sayward, "Theoretical and Empirical Studies on Using Program Mutation to Test the Functional Correctness of Programs," Proc. Seventh ACM SIGPLAN-SIGACT Symp. Principles of Programming Languages, pp. 220-233, Jan. 1980.
38. T.A. Budd and A.S. Gopal, "Program Testing by Specification Mutation," Computer Languages, vol. 10, no. 1, pp. 63-73, 1985.
39. T.A. Budd, R. Hess, and F.G. Sayward, "EXPER Implementor's Guide," technical report, Yale Univ., 1980.
40. T.A. Budd and F.G. Sayward, "Users Guide to the Pilot Mutation System," Technical Report 114, Yale Univ., 1977.
41. R.H. Carver, "Mutation-Based Testing of Concurrent Programs," Proc. IEEE Int'l Test Conf. Designing, Testing, and Diagnostics, pp. 845-853, Oct. 1993.
42. Cetress, "Certitude," http://www.certess.com/product/, 2006.
43. W.K. Chan, S.C. Cheung, and T.H. Tse, "Fault-Based Testing of Database Application Programs with Conceptual Data Model," Proc. Fifth Int'l Conf. Quality Software, pp. 187-196, Sept. 2005. (Pubitemid 46442558)
44. P. Chevalley, "Applying Mutation Analysis for Object-Oriented Programs Using a Reflective Approach," Proc. Eighth Asia-Pacific Software Eng. Conf., p. 267, Dec. 2001.
45. P. Chevalley and P. Thévenod-Fosse, "A Mutation Analysis Tool for Java Programs," Int'l J. Software Tools for Technology Transfer, vol. 5, no. 1, pp. 90-103, Nov. 2002.
46. B. Choi, "Software Testing Using High Performance Computers," PhD thesis, Purdue Univ., July 1991.
47. B. Choi and A.P. Mathur, "High-Performance Mutation Testing," J. Systems and Software, vol. 20, no. 2, pp. 135-152, Feb. 1993. (Pubitemid 23641583)
48. W.M. Craft, "Detecting Equivalent Mutants Using Compiler Optimization Techniques," master's thesis, Clemson Univ., Sept. 1989.
49. Y. Crouzet, H. Waeselynck, B. Lussier, and D. Powell, "The SESAME Experience: From Assembly Languages to Declarative Models," Proc. Second Workshop Mutation Analysis, p. 7, Nov. 2006.
50. M. Daran and P. Thévenod-Fosse, "Software Error Analysis: A Real Case Study Involving Real Faults and Mutations," ACM SIGSOFT Software Eng. Notes, vol. 21, no. 3, pp. 158-177, May 1996.
51. R. Delamare, B. Baudry, and Y. Le Traon, "AjMutator: A Tool for the Mutation Analysis of AspectJ Pointcut Descriptors," Proc. Fourth Int'l Workshop Mutation Analysis, published with Proc. Second Int'l Conf. Software Testing, Verification, and Validation Workshops, pp. 200-204, Apr. 2009.
52. M.E. Delamaro, "Proteum-A Mutation Analysis Based Testing Environmen," master's thesis, Univ. of São Paulo, 1993.
53. M.E. Delamaro and J.C. Maldonado, "Proteum-A Tool for the Assessment of Test Adequacy for C Programs," Proc. Conf. Performability in Computing Systems, pp. 79-95, July 1996.
54. M.E. Delamaro and J.C. Maldonado, "Interface Mutation: Assessing Testing Quality at Interprocedural Level," Proc. 19th Int'l Conf. Chilean Computer Science Soc., pp. 78-86, Nov. 1999.
55. M.E. Delamaro, J.C. Maldonado, and A.P. Mathur, "Integration Testing Using Interface Mutation," Proc. Seventh Int'l Symp. Software Reliability Eng., pp. 112-121, Oct.-Nov. 1996.
56. M.E. Delamaro, J.C. Maldonado, and A.P. Mathur, "Interface Mutation: An Approach for Integration Testing," IEEE Trans. Software Eng., vol. 27, no. 3, pp. 228-247, Mar. 2001. (Pubitemid 32297403)
57. M.E. Delamaro, J.C. Maldonado, A. Pasquini, and A.P. Mathur, "Interface Mutation Test Adequacy Criterion: An Empirical Evaluation," technical report, State Univ. of Maringá, 2000.
58. M.E. Delamaro, J.C. Maldonado, A. Pasquini, and A.P. Mathur, "Interface Mutation Test Adequacy Criterion: An Empirical Evaluation," Empirical Software Eng., vol. 6, no. 2, pp. 111-142, June 2001. (Pubitemid 32583598)
59. M.E. Delamaro, J.C. Maldonado, and A. Vincenzi, "Proteum/IM 2.0: An Integrated Mutation Testing Environment," Proc. First Workshop Mutation Analysis, Oct. 2000.
60. R.A. DeMillo, "Program Mutation: An Approach to Software Testing," technical report, Georgia Inst. of Technology, 1983.
61. R.A. DeMillo and A.P. Mathur, "On the Use of Software Artifacts to Evaluate the Effectiveness of Mutation Analysis in Detecting Errors in Production Software," Technical Report SERC-TR-92-P, Purdue Univ., 1992.
62. R.A. DeMillo, "Test Adequacy and Program Mutation," Proc. 11th Int'l Conf. Software Eng., pp. 355-356, May 1989.
63. R.A. DeMillo, D.S. Guindi, K.N. King, and W.M. McCracken, "An Overview of the Mothra Software Testing Environment," Technical Report SERC-TR-3-P, Purdue Univ., 1987.
64. R.A. DeMillo, D.S. Guindi, K.N. King, W.M. McCracken, and A.J. Offutt, "An Extended Overview of the Mothra Software Testing Environment," Proc. Second Workshop Software Testing, Verification, and Analysis, pp. 142-151, July 1988.
65. R.A. DeMillo, E.W. Krauser, and A.P. Mathur, "Compiler- Integrated Program Mutation," Proc. Fifth Ann. Computer Software and Applications Conf., pp. 351-356, Sept. 1991.
66. R.A. DeMillo, R.J. Lipton, and F.G. Sayward, "Hints on Test Data Selection: Help for the Practicing Programmer," Computer, vol. 11, no. 4, pp. 34-41, Apr. 1978.
67. R.A. DeMillo and A.J. Offutt, "Constraint-Based Automatic Test Data Generation," IEEE Trans. Software Eng., vol. 17, no. 9, pp. 900- 910, Sept. 1991. (Pubitemid 21717628)
68. R.A. DeMillo and A.J. Offutt, "Experimental Results from an Automatic Test Case Generator," ACM Trans. Software Eng. and Methodology, vol. 2, no. 2, pp. 109-127, Apr. 1993. (Pubitemid 23639972)
69. A. Derezińska, "Object-Oriented Mutation to Assess the Quality of Tests," Proc. 29th Euromicro Conf., pp. 417-420, Sept. 2003.
70. A. Derezińska, "Advanced Mutation Operators Applicable in C# Programs," technical report, Warsaw Univ. of Technology, 2005.
71. A. Derezińska, "Quality Assessment of Mutation Operators Dedicated for C# Programs," Proc. Sixth Int'l Conf. Quality Software, Oct. 2006.
72. A. Derezińska and A. Szustek, "CREAM-A System for Object- Oriented Mutation of C# Programs," technical report, Warsaw Univ. of Technology, 2007.
73. A. Derezińska and A. Szustek, "Tool-Supported Advanced Mutation Approach for Verification of C# Programs," Proc. Third Int'l Conf. Dependability of Computer Systems, pp. 261-268, June 2008.
74. W. Ding, "Using Mutation to Generate Tests from Specifications, " master's thesis, George Mason Univ., 2000.
75. H. Do and G. Rothermel, "A Controlled Experiment Assessing Test Case Prioritization Techniques via Mutation Faults," Proc. 21st IEEE Int'l Conf. Software Maintenance, pp. 411-420, Sept. 2005. (Pubitemid 43790954)
76. H. Do and G. Rothermel, "On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques," IEEE Trans. Software Eng., vol. 32, no. 9, pp. 733-752, Sept. 2006. (Pubitemid 44604891)
77. J.J. Domínguez-Jiménez, A. Estero-Botaro, and I. Medina-Bulo, "A Framework for Mutant Genetic Generation for WS-BPEL," Proc. 35th Conf. Current Trends in Theory and Practice of Computer Science, pp. 229-240, Jan. 2009.
78. W. Du and A.P. Mathur, "Vulnerability Testing of Software System Using Fault Injection," Technical Report COAST TR 98-02, Purdue Univ., 1998.
79. W. Du and A.P. Mathur, "Testing for Software Vulnerability Using Environment Perturbation," Proc. Int'l Conf. Dependable Systems and Networks, pp. 603-612, June 2000.
80. L.du Bousquet and M. Delaunay, "Mutation Analysis for Lustre Programs: Fault Model Description and Validation," Proc. Third Workshop Mutation Analysis, published with Proc. Second Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 176-184, Sept. 2007.
81. L. du Bousquet and M. Delaunay, "Using Mutation Analysis to Evaluate Test Generation Strategies in a Synchronous Context," Proc. Second Int'l Conf. Software Eng. Advances, p. 40, Aug. 2007.
82. Ellims, "Csaw," http://www.skicambridge.com/papers/ Csaw-v1-files.html, 2007.
83. M. Ellims, D.C. Ince, and M. Petre, "The Csaw C Mutation Tool: Initial Results," Proc. Third Workshop Mutation Analysis, published with Proc. Second Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 185-192, Sept. 2007.
84. A. Estero-Botaro, F. Palomo-Lozano, and I. Medina-Bulo, "Mutation Operators for WS-BPEL 2.0," Proc. 21st Int'l Conf. Software and Systems Eng. and Their Applications, Dec. 2008.
85. S.C.P.F. Fabbri, J.C. Maldonado, P.C. Masiero, and M.E. Delamaro, "Proteum/FSM: A Tool to Support Finite State Machine Validation Based on Mutation Testing," Proc. 19th Int'l Conf. Chilean Computer Science Soc., pp. 96-104, Nov. 1999.
86. S.C.P.F. Fabbri, J.C. Maldonado, P.C. Masiero, M.E. Delamaro, and W.E. Wong, "Mutation Testing Applied to Validate Specifications Based on Petri Nets," Proc. IFIP TC6 Eighth Int'l Conf. Formal Description Techniques VIII, vol. 43, pp. 329-337, 1995.
87. S.C.P.F. Fabbri, J.C. Maldonado, T. Sugeta, and P.C. Masiero, "Mutation Testing Applied to Validate Specifications Based on Statecharts," Proc. 10th Int'l Symp. Software Reliability Eng., pp. 210-219, Nov. 1999. (Pubitemid 30516667)
88. S.P.F. Fabbri, M.E. Delamaro, J.C. Maldonado, and P. Masiero, "Mutation Analysis Testing for Finite State Machines," Proc. Fifth Int'l Symp. Software Reliability Eng., pp. 220-229, Nov. 1994.
89. X. Feng, S. Marr, and T. O'Callaghan, "ESTP: An Experimental Software Testing Platform," Proc. Third Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 59-63, Aug. 2008.
90. F.C. Ferrari, J.C. Maldonado, and A. Rashid, "Mutation Testing for Aspect-Oriented Programs," Proc. First Int'l Conf. Software Testing, Verification, and Validation, pp. 52-61, Apr. 2008.
91. S. Fichter, "Parallelizing Mutation on a Hypercube," master's thesis, Clemson Univ., 1991.
92. V.N. Fleyshgakker and S.N. Weiss, "Efficient Mutation Analysis: A New Approach," Proc. Int'l Symp. Software Testing and Analysis, pp. 185-195, Aug. 1994.
93. P.G. Frankl, S.N. Weiss, and C. Hu, "All-Uses versus Mutation Testing: An Experimental Comparison of Effectiveness," technical report, Polytechnic Univ., 1994.
94. P.G. Frankl, S.N. Weiss, and C. Hu, "All-Uses vs Mutation Testing: An Experimental Comparison of Effectiveness," J. Systems and Software, vol. 38, no. 3, pp. 235-253, Sept. 1997. (Pubitemid 127384432)
95. G. Fraser and F. Wotawa, "Mutant Minimization for Model- Checker Based Test-Case Generation," Proc. Third Workshop Mutation Analysis, published with Proc. Second Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 161-168, Sept. 2007.
96. R. Geist, A.J. Offutt, and F.C. Harris, "Estimation and Enhancement of Real-Time Software Reliability through Mutation Analysis," IEEE Trans. Computers, vol. 41, no. 5, pp. 550-558, May 1992.
97. A.K. Ghosh, T. O'Connor, and G. McGraw, "An Automated Approach for Identifying Potential Vulnerabilities in Software," Proc. IEEE Symp. Security and Privacy, pp. 104-114, May 1998.
98. S. Ghosh, "Testing Component-Based Distributed Applications," PhD thesis, Purdue Univ., 2000.
99. S. Ghosh, "Towards Measurement of Testability of Concurrent Object-Oriented Programs Using Fault Insertion: A Preliminary Investigation," Proc. Second IEEE Int'l Workshop Source Code Analysis and Manipulation, pp. 17-25, 2002.
100. S. Ghosh, P. Govindarajan, and A.P. Mathur, "TDS: A Tool for Testing Distributed Component-Based Applications," Proc. First Workshop Mutation Analysis, pp. 103-112, Oct. 2000.
101. S. Ghosh and A.P. Mathur, "Interface Mutation to Assess the Adequacy of Tests for Components and Systems," Proc. 34th Int'l Conf. Technology of Object-Oriented Languages and Systems, pp. 37-46, July-Aug. 2000.
102. S. Ghosh and A.P. Mathur, "Interface Mutation," Software Testing, Verification and Reliability, vol. 11, no. 3, pp. 227-247, Mar. 2001. (Pubitemid 34012880)
103. M.R. Girgis and M.R. Woodward, "An Integrated System for Program Testing Using Weak Mutation and Data Flow Analysis," Proc. Eighth Int'l Conf. Software Eng., pp. 313-319, Aug. 1985. (Pubitemid 16597557)
104. P. Godefroid, N. Klarlund, and K. Sen, "DART: Directed Automated Random Testing," Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, pp. 213-223, June 2005.
105. A.S. Gopal and T.A. Budd, "Program Testing by Specification Mutation," Technical Report TR 83-17, Univ. of Arizona, 1983.
106. B.J.M. Grün, D. Schuler, and A. Zeller, "The Impact of Equivalent Mutants," Proc. Fourth Int'l Workshop Mutation Analysis, published with Proc. Second Int'l Conf. Software Testing, Verification, and Validation Workshops, pp. 192-199, Apr. 2009.
107. R.G. Hamlet, "Testing Programs with the Aid of a Compiler," IEEE Trans. Software Eng., vol. 3, no. 4, pp. 279-290, July 1977.
108. J.M. Hanks, "Testing Cobol Programs by Mutation," PhD thesis, Georgia Inst. of Technology, 1980.
109. M. Harman, R. Hierons, and S. Danicic, "The Relationship between Program Dependence and Mutation Analysis," Proc. First Workshop Mutation Analysis, pp. 5-13, Oct. 2000.
110. R.M. Hierons, M. Harman, and S. Danicic, "Using Program Slicing to Assist in the Detection of Equivalent Mutants," Software Testing, Verification, and Reliability, vol. 9, no. 4, pp. 233-262, Dec. 1999. (Pubitemid 30546632)
111. R.M. Hierons and M.G. Merayo, "Mutation Testing from Probabilistic Finite State Machines," Proc. Third Workshop Mutation Analysis, published with Proc. Second Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 141-150, Sept. 2007.
112. R.M. Hierons and M.G. Merayo, "Mutation Testing from Probabilistic and Stochastic Finite State Machines," J. Systems and Software, vol. 82, no. 11, pp. 1804-1818, Nov. 2009.
113. J.R. Horgan and A.P. Mathur, "Weak Mutation is Probably Strong Mutation," Technical Report SERC-TR-83-P, Purdue Univ., 1990.
114. S.-S. Hou, L. Zhang, T. Xie, H. Mei, and J.-S. Sun, "Applying Interface-Contract Mutation in Regression Testing of Component- Based Software," Proc. 23rd Int'l Conf. Software Maintenance, pp. 174-183, Oct. 2007.
115. W.E. Howden, "Weak Mutation Testing and Completeness of Test Sets," IEEE Trans. Software Eng., vol. 8, no. 4, pp. 371-379, July 1982.
116. S. Hussain, "Mutation Clustering," master's thesis, King's College London, 2008.
117. J. Hwang, T. Xie, F. Chen, and A.X. Liu, "Systematic Structural Testing of Firewall Policies," Proc. IEEE Symp. Reliable Distributed Systems, pp. 105-114, Oct. 2008.
118. Itregister, "Plextest," http://www.itregister.com.au/products/ plextest.htm, 2007.
119. D. Jackson and M.R. Woodward, "Parallel Firm Mutation of Java Programs," Proc. First Workshop Mutation Analysis, pp. 55-61, Oct. 2000.
120. C. Ji, Z. Chen, B. Xu, and Z. Zhao, "A Novel Method of Mutation Clustering Based on Domain Analysis," Proc. 21st Int'l Conf. Software Eng. and Knowledge Eng., July 2009.
121. Y. Jia, "Mutation Testing Repository," http://www.dcs.kcl. ac.uk/pg/jiayue/repository/, 2009.
122. Y. Jia and M. Harman, "Constructing Subtle Faults Using Higher Order Mutation Testing," Proc. Eighth Int'l Working Conf. Source Code Analysis and Manipulation, pp. 249-258, Sept. 2008.
123. Y. Jia and M. Harman, "MILU: A Customizable, Runtime- Optimized Higher Order Mutation Testing Tool for the Full C Language," Proc. Third Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 94-98, Aug. 2008.
124. C. Jing, Z. Wang, X. Shi, X. Yin, and J. Wu, "Mutation Testing of Protocol Messages Based on Extended TTCN-3," Proc. 22nd Int'l Conf. Advanced Information Networking and Applications, pp. 667- 674, Mar. 2008.
125. K. Kapoor, "Formal Analysis of Coupling Hypothesis for Logical Faults," Innovations in Systems and Software Eng., vol. 2, no. 2, pp. 80-87, July 2006. (Pubitemid 44178081)
126. S.-W. Kim, M.J. Harrold, and Y.-R. Kwon, "MUGAMMA: Mutation Analysis of Deployed Software to Increase Confidence Assist Evolution," Proc. Second Workshop Mutation Analysis, p. 10, Nov. 2006.
127. S. Kim, J.A. Clark, and J.A. McDermid, "Assessing Test Set Adequacy for Object Oriented Programs Using Class Mutation," Proc. Third Symp. Software Technology, Sept. 1999.
128. S. Kim, J.A. Clark, and J.A. McDermid, "The Rigorous Generation of Java Mutation Operators Using HAZOP," Proc. 12th Int'l Conf. Software and Systems Eng. and Their Applications, Nov.-Dec. 1999.
129. S. Kim, J.A. Clark, and J.A. McDermid, "Class Mutation: Mutation Testing for Object-Oriented Programs," Proc. Net.ObjectDays Conf. Object-Oriented Software Systems, 2000.
130. S. Kim, J.A. Clark, and J.A. McDermid, "Investigating the Effectiveness of Object-Oriented Testing Strategies Using the Mutation Method," Proc. First Workshop Mutation Analysis, pp. 207-225, Oct. 2000. (Pubitemid 34012879)
131. K.N. King and A.J. Offutt, "A Fortran Language System for Mutation-Based Software Testing," Software: Practice and Experience, vol. 21, no. 7, pp. 685-718, Oct. 1991. (Pubitemid 21666424)
132. E.W. Krauser, "Compiler-Integrated Software Testing," PhD thesis, Purdue Univ., 1991.
133. E.W. Krauser, A.P. Mathur, and V.J. Rego, "High Performance Software Testing on SIMD Machines," IEEE Trans. Software Eng., vol. 17, no. 5, pp. 403-423, May 1991.
134. E.W. Krauser, A.P. Mathur, and V.J. Rego, "High Performance Software Testing on SIMD Machines," Proc. Second Workshop Software Testing, Verification, and Analysis, pp. 171-177, July 1988.
135. K. Lakhotia, P. McMinn, and M. Harman, "Automated Test Data Generation for Coverage: Haven't We Solved This Problem Yet?" Proc. Testing: Academia & Industry Conf.-Practice And Research Techniques, pp. 95-104, Sept. 2009.
136. W.B. Langdon, M. Harman, and Y. Jia, "Multi Objective Higher Order Mutation Testing with Genetic Programming," Proc. Fourth Testing: Academic and Industrial Conf.-Practice and Research, Sept. 2009.
137. W.B. Langdon, M. Harman, and Y. Jia, "Multi Objective Mutation Testing with Genetic Programming," Proc. Genetic and Evolutionary Computation Conf., July 2009.
138. Y. Le Traon, B. Baudry, and J.-M. Jézéquel, "Design by Contract to Improve Software Vigilance," IEEE Trans. Software Eng., vol. 32, no. 8, pp. 571-586, Aug. 2006. (Pubitemid 46405274)
139. Y.L. Traon, T. Mouelhi, and B. Baudry, "Testing Security Policies: Going Beyond Functional Testing," Proc. 18th IEEE Int'l Symp. Software Reliability, pp. 93-102, Nov. 2007.
140. S. Lee, X. Bai, and Y. Chen, "Automatic Mutation Testing and Simulation on OWL-S Specified Web Services," Proc. 41st Ann. Simulation Symp., pp. 149-156, Apr. 2008.
141. S.D. Lee, "Weak versus Strong: An Empirical Comparison of Mutation Variants," master's thesis, Clemson Univ., 1991.
142. S.C. Lee and A.J. Offutt, "Generating Test Cases for XML-Based Web Component Interactions Using Mutation Analysis," Proc. 12th Int'l Symp. Software Reliability Eng., pp. 200-209, Nov. 2001. (Pubitemid 34082599)
143. J.B. Li and J. Miller, "Testing the Semantics of W3C XML Schema," Proc. 29th Ann. Int'l Computer Software and Applications Conf., pp. 443-448, July 2005. (Pubitemid 46744922)
144. R. Lipton, "Fault Diagnosis of Computer Programs," student report, Carnegie Mellon Univ., 1971.
145. R.J. Lipton and F.G. Sayward, "The Status of Research on Program Mutation," Proc. Workshop Software Testing and Test Documentation, pp. 355-373, Dec. 1978.
146. M.-H. Liu, Y.-F. Gao, J.-H. Shan, J.-H. Liu, L. Zhang, and J.-S. Sun, "An Approach to Test Data Generation for Killing Multiple Mutants," Proc. 22nd IEEE Int'l Conf. Software Maintenance, pp. 113- 122, Sept. 2006. (Pubitemid 47133336)
147. B. Long, R. Duke, D. Goldson, P. Strooper, and L. Wildman, "Mutation-Based Exploration of a Method for Verifying Concurrent Java Components," Proc. 18th Int'l Parallel and Distributed Processing Symp., p. 265, Apr. 2004.
148. Y.-S. Ma, "Object-Oriented Mutation Testing for Java," PhD thesis, KAIST Univ. in Korea, 2005.
149. Y.-S. Ma, M.J. Harrold, and Y.-R. Kwon, "Evaluation of Mutation Testing for Object-Oriented Programs," Proc. 28th Int'l Conf. Software Eng., pp. 869-872, May 2006. (Pubitemid 46601007)
150. Y.-S. Ma, Y.-R. Kwon, and A.J. Offutt, "Inter-Class Mutation Operators for Java," Proc. 13th Int'l Symp. Software Reliability Eng., pp. 352-363, Nov. 2002.
151. Y.-S. Ma, A.J. Offutt, and Y.-R. Kwon, "MuJava: An Automated Class Mutation System," Software Testing, Verification, and Reliability, vol. 15, no. 2, pp. 97-133, June 2005. (Pubitemid 40858060)
152. Y.-S. Ma, A.J. Offutt, and Y.-R. Kwon, "MuJava: A Mutation System for Java," Proc. 28th Int'l Conf. Software Eng., pp. 827-830, May 2006. (Pubitemid 46600997)
153. B. Marick, "The Weak Mutation Hypothesis," Proc. Fourth Symp. Software Testing, Analysis, and Verification pp. 190-199, Oct. 1991.
154. E.E. Martin and T. Xie, "A Fault Model and Mutation Testing of Access Control Policies," Proc. 16th Int'l Conf. World Wide Web, pp. 667-676, May 2007. (Pubitemid 47582296)
155. A.P. Mathur, Foundations of Software Testing. Pearson Education, 2008.
156. A.P. Mathur, "Performance, Effectiveness, and Reliability Issues in Software Testing," Proc. Fifth Int'l Computer Software and Applications Conf., pp. 604-605, Sept. 1991.
157. A.P. Mathur, "CS 406 Software Engineering I," Course Project Handout, Aug. 1992.
158. A.P. Mathur and E.W. Krauser, "Mutant Unification for Improved Vectorization," Technical Report SERC-TR-14-P, Purdue Univ., 1988.
159. A.P. Mathur and W.E. Wong, "An Empirical Comparison of Mutation and Data Flow Based Test Adequacy Criteria," technical report, Purdue Univ., 1993.
160. A.P. Mathur and W.E. Wong, "An Empirical Comparison of Data Flow and Mutation-Based Test Adequacy Criteria," Software Testing, Verification, and Reliability, vol. 4, no. 1, pp. 9-31, 1994.
161. P.S. May, "Test Data Generation: Two Evolutionary Approaches to Mutation Testing," PhD thesis, Univ. of Kent, 2007.
162. P. McMinn, "Search-Based Software Test Data Generation: A Survey," Software Testing, Verification, and Reliability, vol. 14, no. 2, pp. 105-156, 2004.
163. I. Moore, "Jester and Pester," http://jester.sourceforge.net/, 2001.
164. L.J. Morell, "A Theory of Error-Based Testing," PhD thesis, Univ. of Maryland at College Park, 1984.
165. T. Mouelhi, F. Fleurey, and B. Baudry, "A Generic Metamodel For Security Policies Mutation," Proc. IEEE Int'l Conf. Software Testing, Verification, and Validation Workshop, pp. 278-286, Apr. 2008.
166. T. Mouelhi, Y. Le Traon, and B. Baudry, "Mutation Analysis for Security Tests Qualification," Proc. Third Workshop Mutation Analysis, published with Proc. Second Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 233-242, Sept. 2007.
167. E.S. Mresa and L. Bottaci, "Efficiency of Mutation Operators and Selective Mutation Strategies: An Empirical Study," Software Testing, Verification, and Reliability, vol. 9, no. 4, pp. 205-232, Dec. 1999. (Pubitemid 30546631)
168. A.S. Namin and J.H. Andrews, "Finding Sufficient Mutation Operators via Variable Reduction," Proc. Second Workshop Mutation Analysis, p. 5, Nov. 2006.
169. A.S. Namin and J.H. Andrews, "On Sufficiency of Mutants," Proc. 29th Int'l Conf. Software Eng., pp. 73-74, May 2007. (Pubitemid 47430747)
170. A.S. Namin, J.H. Andrews, and D.J. Murdoch, "Sufficient Mutation Operators for Measuring Test Effectiveness," Proc. 30th Int'l Conf. Software Eng., pp. 351-360, May 2008.
171. R. Nilsson, A.J. Offutt, and S.F. Andler, "Mutation-Based Testing Criteria for Timeliness," Proc. 28th Ann. Int'l Computer Software and Applications Conf., pp. 306-311, Sept. 2004. (Pubitemid 40673869)
172. R. Nilsson, A.J. Offutt, and J. Mellin, "Test Case Generation for Mutation-Based Testing of Timeliness," Proc. Second Workshop Model Based Testing, pp. 97-114, Mar. 2006. (Pubitemid 44584838)
173. A.J. Offutt, J. Pan, and J.M. Voas, "Procedures for Reducing the Size of Coverage-Based Test Sets," Proc. 12th Int'l Conf. Testing Computer Software, pp. 111-123, June 1995.
174. A.J. Offutt, "The Coupling Effect: Fact or Fiction," ACM SIGSOFT Software Eng. Notes, vol. 14, no. 8, pp. 131-140, Dec. 1989.
175. A.J. Offutt, "Investigations of the Software Testing Coupling Effect," ACM Trans. Software Eng. and Methodology, vol. 1, no. 1, pp. 5-20, Jan. 1992.
176. A.J. Offutt, private communication, July 2008.
177. A.J. Offutt, P. Ammann, and L.L. Liu, "Mutation Testing Implements Grammar-Based Testing ," Proc. Second Workshop Mutation Analysis, p. 12, Nov. 2006.
178. A.J. Offutt and W.M. Craft, "Using Compiler Optimization Techniques to Detect Equivalent Mutants," Software Testing, Verification, and Reliability, vol. 4, no. 3, pp. 131-154, Sept. 1994.
179. A.J. Offutt, Z. Jin, and J. Pan, "The Dynamic Domain Reduction Approach for Test Data Generation: Design and Algorithms," Technical Report ISSE-TR-94-110, George Mason Univ., 1994.
180. A.J. Offutt, Z. Jin, and J. Pan, "The Dynamic Domain Reduction Procedure for Test Data Generation," Software: Practice and Experience, vol. 29, no. 2, pp. 167-193, Feb. 1999.
181. A.J. Offutt and K.N. King, "A Fortran 77 Interpreter for Mutation Analysis," ACM SIGPLAN Notices, vol. 22, no. 7, pp. 177-188, July 1987.
182. A.J. Offutt, A. Lee, G. Rothermel, R.H. Untch, and C. Zapf, "An Experimental Determination of Sufficient Mutant Operators," ACM Trans. Software Eng. and Methodology, vol. 5, no. 2, pp. 99- 118, Apr. 1996. (Pubitemid 126418670)
183. A.J. Offutt and S. Lee, "An Empirical Evaluation of Weak Mutation," IEEE Trans. Software Eng., vol. 20, no. 5, pp. 337-344, May 1994.
184. A.J. Offutt and S.D. Lee, "How Strong is Weak Mutation?" Proc. Fourth Symp. Software Testing, Analysis, and Verification, pp. 200- 213, Oct. 1991.
185. A.J. Offutt, Y.-S. Ma, and Y.-R. Kwon, "An Experimental Mutation System for Java," ACM SIGSOFT Software Eng. Notes, vol. 29, no. 5, pp. 1-4, Sept. 2004.
186. A.J. Offutt and J. Pan, "Detecting Equivalent Mutants and the Feasible Path Problem," Proc. Ann. Conf. Computer Assurance, pp. 224-236, June 1996.
187. A.J. Offutt and J. Pan, "Automatically Detecting Equivalent Mutants and Infeasible Paths," Software Testing, Verification, and Reliability, vol. 7, no. 3, pp. 165-192, Sept. 1997. (Pubitemid 127661584)
188. A.J. Offutt, J. Pan, K. Tewary, and T. Zhang, "An Experimental Evaluation of Data Flow and Mutation Testing," Software: Practice and Experience, vol. 26, no. 2, pp. 165-176, Feb. 1996. (Pubitemid 126594500)
189. A.J. Offutt, R.P. Pargas, S.V. Fichter, and P.K. Khambekar, "Mutation Testing of Software Using a MIMD Computer," Proc. Int'l Conf. Parallel Processing, pp. 255-266, Aug. 1992.
190. A.J. Offutt, G. Rothermel, and C. Zapf, "An Experimental Evaluation of Selective Mutation," Proc. 15th Int'l Conf. Software Eng., pp. 100-107, May 1993.
191. A.J. Offutt and R.H. Untch, "Mutation 2000: Uniting the Orthogonal," Proc. First Workshop Mutation Analysis, pp. 34-44, Oct. 2000.
192. A.J. Offutt, J. Voas, and J. Payn, "Mutation Operators for Ada," Technical Report ISSE-TR-96-09, George Mason Univ., 1996.
193. A.J. Offutt and W. Xu, "Generating Test Cases for Web Services Using Data Perturbation," Proc. Workshop Testing, Analysis and Verification of Web Services, pp. 1-10, July 2004.
194. A.J. Offutt, "Automatic Test Data Generation," PhD thesis, Georgia Inst. of Technology, 1988.
195. V. Okun, "Specification Mutation for Test Generation and Analysis," PhD thesis, Univ. of Maryland, 2004.
196. T. Olsson and P. Runeson, "System Level Mutation Analysis Applied to a State-Based Language," Proc. Eighth Ann. IEEE Int'l Conf. and Workshop Eng. of Computer Based Systems, pp. 222-228, Apr. 2001. (Pubitemid 32542136)
197. J. Pan, "Using Constraints to Detect Equivalent Mutants," master's thesis, George Mason Univ., 1994.
198. Parasoft, "Parasoft Insure++," http://www.parasoft.com/jsp/ products/home.jsp?product=Insure, 2006.
199. M. Polo, M. Piattini, and I. Garcia-Rodriguez, "Decreasing the Cost of Mutation Testing with Second-Order Mutants," Software Testing, Verification, and Reliability, vol. 19, no. 2, pp. 111-131, June 2008.
200. M. Polo, S. Tendero, and M. Piattini, "Integrating Techniques and Tools for Testing Automation: Research Articles," Software Testing, Verification, and Reliability, vol. 17, no. 1, pp. 3-39, Mar. 2007. (Pubitemid 46336330)
201. A. Pretschner, T. Mouelhi, and Y.L. Traon, "Model-Based Tests for Access Control Policies," Proc. First Int'l Conf. Software Testing, Verification, and Validation, pp. 338-347, Apr. 2008.
202. R. Probert and F. Guo, "Mutation Testing of Protocols: Principles and Preliminary Experimental Results," Proc. Workshop Protocol Test Systems, pp. 57-76, Oct. 1991.
203. S.T. Redwine and W.E. Riddle, "Software Technology Maturation," Proc. Eighth Int'l Conf. Software Eng., pp. 189-200, 1985.
204. C.K. Roy and J.R. Cordy, "Towards a Mutation-Based Automatic Framework for Evaluating Code Clone Detection Tools ," Proc. Canadian Conf. Computer Science and Software Eng., pp. 137-140, May 2008.
205. C.K. Roy and J.R. Cordy, "A Mutation/Injection-Based Automatic Framework for Evaluating Code Clone Detection Tools," Proc. Fourth Int'l Workshop Mutation Analysis, published with Proc. Second Int'l Conf. Software Testing, Verification, and Validation Workshops, pp. 157-166, Apr. 2009.
206. Rubyforge, "Heckle," http://seattlerb.rubyforge.org/heckle/, 2007.
207. M. Sahinoglu and E.H. Spafford, "A Bayes Sequential Statistical Procedure for Approving Software Products," Proc. IFIP Conf. Approving Software Products, pp. 43-56, Sept. 1990.
208. D. Schuler, V. Dallmeier, and A. Zeller, "Efficient Mutation Testing by Checking Invariant Violations," Proc. Int'l Symp. Software Testing and Analysis, July 2009.
209. K. Sen, D. Marinov, and G. Agha, "CUTE: A Concolic Unit Testing Engine for C," Proc. 13th ACM SIGSOFT Int'l Symp. Foundations of Software Eng., pp. 263-272, 2005. (Pubitemid 43218836)
210. Y. Serrestou, V. Beroulle, and C. Robach, "Functional Verification of RTL Designs Driven by Mutation Testing Metrics," Proc. 10th Euromicro Conf. Digital System Design Architectures, Methods and Tools, pp. 222-227, Aug. 2007.
211. Y. Serrestou, V. Beroulle, and C. Robach, "Impact of Hardware Emulation on the Verification Quality Improvement," Proc. IFIP WG 10.5 Int'l Conf. Very Large Scale Integration of System-on-Chip, pp. 218-223, Oct. 2007.
212. H. Shahriar and M. Zulkernine, "MUSIC: Mutation-Based SQL Injection Vulnerability Checking," Proc. Eighth Int'l Conf. Quality Software, pp. 77-86, Aug. 2008.
213. H. Shahriar and M. Zulkernine, "Mutation-Based Testing of Buffer Overflow Vulnerabilities," Proc. Second Ann. IEEE Int'l Workshop Security in Software Eng., pp. 979-984, July-Aug. 2008.
214. H. Shahriar and M. Zulkernine, "Mutation-Based Testing of Format String Bugs," Proc. 11th IEEE High Assurance Systems Eng. Symp., pp. 229-238, Dec. 2008.
215. H. Shahriar and M. Zulkernine, "MUTEC: Mutation-Based Testing of Cross Site Scripting," Proc. Fifth Int'l Workshop Software Eng. for Secure Systems, pp. 47-53, May 2009.
216. D.P. Sidhu and T.K. Leung, "Fault Coverage of Protocol Test Methods," Proc. IEEE INFOCOM, pp. 80-85, Mar. 1988.
217. A. Simao, J.C. Maldonado, and R. da Silva Bigonha, "A Transformational Language for Mutant Description," Computer Languages, Systems, and Structures, vol. 35, no. 3, pp. 322-339, Oct. 2009.
218. B.H. Smith and L. Williams, "An Empirical Evaluation of the MuJava Mutation Operators," Proc. Third Workshop Mutation Analysis, published with Proc. Second Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 193-202, Sept. 2007.
219. B.H. Smith and L. Williams, "On Guiding the Augmentation of an Automated Test Suite via Mutation Analysis," Empirical Software Eng., vol. 14, no. 3, pp. 341-369, 2009.
220. SourceForge, "Nester," http://nester.sourceforge.net/, 2002.
221. SourceForge, "Jumble," http://jumble.sourceforge.net/, 2007.
222. S.D.R.S.D. Souza, J.C. Maldonado, S.C.P.F. Fabbri, and W.L.D. Souza, "Mutation Testing Applied to Estelle Specifications," Software Quality Control, vol. 8, no. 4, pp. 285-301, Dec. 1999.
223. S.D.R.S.D. Souza, J.C. Maldonado, S.C.P.F. Fabbri, and W.L.D. Souza, "Mutation Testing Applied to Estelle Specifications," Proc. 33rd Hawaii Int'l Conf. System Sciences, vol. 8, p. 8011, Jan. 2000.
224. E.H. Spafford, "Extending Mutation Testing to Find Environmental Bugs," Software: Practice and Experience, vol. 20, no. 2, pp. 181-189, Feb. 1990.
225. T. Srivatanakul, J.A. Clark, S. Stepney, and F. Polack, "Challenging Formal Specifications by Mutation: A CSP Security Example," Proc. 10th Asia-Pacific Software Eng. Conf., pp. 340-350, Dec. 2003.
226. T. Sugeta, J.C. Maldonado, and W.E. Wong, "Mutation Testing Applied to Validate SDL Specifications," Proc. 16th IFIP Int'l Conf. Testing of Comm. Systems, p. 2741, Mar. 2004.
227. A. Sung, J. Jang, and B. Choi, "Fault-Based Interface Testing between Real-Time Operating System and Application," Proc. Second Workshop Mutation Analysis, p. 8, Nov. 2006.
228. A. Tanaka, "Equivalence Testing for Fortran Mutation System Using Data Flow Analysis," master's thesis, Georgia Inst. of Technology, 1981.
229. P. Thévenod-Fosse, H. Waeselynck, and Y. Crouzet, "An Experimental Study on Software Structural Testing: Deterministic versus Random Input Generation," Proc. 25th Int'l Symp. Fault-Tolerant Computing, pp. 410-417, June 1991. (Pubitemid 21704479)
230. N. Tillmann and J. de Halleux, "Pex-White Box Test Generation for .NET," Proc. Second Int'l Conf. Tests and Proofs, pp. 134-153, Apr. 2008. (Pubitemid 351642008)
231. M. Trakhtenbrot, "New Mutations for Evaluation of Specification and Implementation Levels of Adequacy in Testing of Statecharts Models," Proc. Third Workshop Mutation Analysis, published with Proc. Second Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 151-160, Sept. 2007.
232. U. Trier, "DBLP," http://www.informatik.uni-trier.de/ley/db/, 2009.
233. J. Tuya, M.J.S. Cabal, and C. de la Riva, "SQLMutation: A Tool to Generate Mutants of SQL Database Queries," Proc. Second Workshop Mutation Analysis, p. 1, Nov. 2006.
234. J. Tuya, M.J.S. Cabal, and C. de la Riva, "Mutating Database Queries," Information and Software Technology, vol. 49, no. 4, pp. 398-417, Apr. 2007. (Pubitemid 46164882)
235. R.H. Untch, "Mutation-Based Software Testing Using Program Schemata," Proc. 30th Ann. Southeast Regional Conf., pp. 285-291, 1992.
236. R.H. Untch, "Schema-Based Mutation Analysis: A New Test Data Adequacy Assessment Method," PhD thesis, Clemson Univ., Dec. 1995.
237. R.H. Untch, A.J. Offutt, and M.J. Harrold, "Mutation Analysis Using Mutant Schemata," Proc. Int'l Symp. Software Testing and Analysis, pp. 139-148, 1993.
238. G. Vigna, W. Robertson, and D. Balzarotti, "Testing Network- Based Intrusion Detection Signatures Using Mutant Exploits," Proc. 11th ACM Conf. Computer and Comm. Security, pp. 21-30, 2004. (Pubitemid 40338186)
239. P. Vilela, M. Machado, and W.E. Wong, "Testing for Security Vulnerabilities in Software," Proc. Conf. Software Eng. and Applications, 2002.
240. A.M.R. Vincenzi, J.C. Maldonado, E.F. Barbosa, and M.E. Delamaro, "Unit and Integration Testing Strategies for C Programs Using Mutation," Software Testing, Verification, and Reliability, vol. 11, no. 4, pp. 249-268, Nov. 2001. (Pubitemid 34012881)
241. J. Voas and G. McGraw, Software Fault Injection: Inoculating Programs against Errors. John Wiley & Sons, 1997.
242. K.S.H.T. Wah, "Fault Coupling in Finite Bijective Functions," Software Testing, Verification, and Reliability, vol. 5, no. 1, pp. 3-47, 1995.
243. K.S.H.T. Wah, "A Theoretical Study of Fault Coupling," Software Testing, Verification, and Reliability, vol. 10, no. 1, pp. 3-46, Apr. 2000.
244. K.S.H.T. Wah, "An Analysis of the Coupling Effect I: Single Test Data," Science of Computer Programming, vol. 48, nos. 2-3, pp. 119- 161, Aug.-Sept. 2003.
245. R. Wang and N. Huang, "Requirement Model-Based Mutation Testing for Web Service," Proc. Fourth Int'l Conf. Next Generation Web Services Practices, pp. 71-76, Oct. 2008.
246. S.N. Weiss and V.N. Fleyshgakker, "Improved Serial Algorithms for Mutation Analysis," ACM SIGSOFT Software Eng. Notes, vol. 18, no. 3, pp. 149-158, July 1993.
247. E.J. Weyuker, "On Testing Non-Testable Programs," The Computer J., vol. 25, pp. 456-470, 1982.
248. W.E. Wong, "On Mutation and Data Flow," PhD thesis, Purdue Univ., 1993.
249. W.E. Wong, J.R. Horgan, S. London, and A.P. Mathur, "Effect of Test Set Minimization on Fault Detection Effectiveness," Software: Practice and Experience, vol. 28, pp. 347-369, 1998. (Pubitemid 128614804)
250. W.E. Wong, J.R. Horgan, A.P. Mathur, and A. Pasquini, "Test Set Size Minimization and Fault Detection Effectiveness: A Case Study in a Space Application," J. Systems and Software, vol. 48, no. 2, pp. 79-89, Oct. 1999. (Pubitemid 30509010)
251. W.E. Wong and A.P. Mathur, "Fault Detection Effectiveness of Mutation and Data Flow Testing," Software Quality J., vol. 4, no. 1, pp. 69-83, Mar. 1995.
252. W.E. Wong and A.P. Mathur, "Reducing the Cost of Mutation Testing: An Empirical Study," J. Systems and Software, vol. 31, no. 3, pp. 185-196, Dec. 1995.
253. M.R. Woodward, "Mutation Testing-An Evolving Technique," Proc. IEE Colloquium on Software Testing for Critical Systems, pp. 3/1-3/6, June 1990.
254. M.R. Woodward, "OBJTEST: An Experimental Testing Tool for Algebraic Specifications," Proc. IEE Colloquium on Automating Formal Methods for Computer Assisted Prototying, p. 2, Jan. 1990.
255. M.R. Woodward, "Errors in Algebraic Specifications and an Experimental Mutation Testing Tool," Software Eng. J., vol. 8, no. 4, pp. 221-224, July 1993.
256. M.R. Woodward, "Mutation Testing-Its Origin and Evolution," J. Information and Software Technology, vol. 35, no. 3, pp. 163-169, Mar. 1993.
257. M.R. Woodward and K. Halewood, "From Weak to Strong Dead or Alive? An Analysis of Some Mutationtesting Issues," Proc. Second Workshop Software Testing, Verification, and Analysis, pp. 152- 158, July 1988.
258. T. Xie, N. Tillmann, J. de Halleux, and W. Schulte, "Mutation Analysis of Parameterized Unit Tests," Proc. Fourth Int'l Workshop Mutation Analysis, published with Proc. Second Int'l Conf. Software Testing, Verification, and Validation Workshops, pp. 177-181, Apr. 2009.
259. W. Xu, A.J. Offutt, and J. Luo, "Testing Web Services by XML Perturbation," Proc. 16th IEEE Int'l Symp. Software Reliability Eng., pp. 257-266, July 2005. (Pubitemid 44733908)
260. H. Yoon, B. Choi, and J.O. Jeon, "Mutation-Based Inter-Class Testing," Proc. Fifth Asia Pacific Software Eng. Conf., pp. 174-181, Dec. 1998.
261. C.N. Zapf, "A Distributed Interpreter for the Mothra Mutation Testing System," master's thesis, Clemson Univ., 1993.
262. Y. Zhan and J.A. Clark, "Search-Based Mutation Testing for Simulink Models," Proc. Conf. Genetic and Evolutionary Computation, pp. 1061-1068, June 2005. (Pubitemid 43226415)
263. S. Zhang, T.R. Dean, and G.S. Knight, "Lightweight State Based Mutation Testing for Security," Proc. Third Workshop Mutation Analysis, published with Proc. Second Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 223-232, Sept. 2007.
264. C. Zhou and P. Frankl, "Mutation Testing for Java Database Applications," Proc. Second Int'l Conf. Software Testing Verification and Validation, pp. 396-405, Apr. 2009.