All-uses vs mutation testing: An experimental comparison of effectiveness

Journal of Systems and Software

Volumn 38, Issue 3, 1997, Pages 235-253

  Frankl, Phyllis G ^a   Weiss, Stewart N ^b   Hu, Cang ^c  

^a POLYTECHNIC UNIVERSITY   (United States)

^b CITY UNIVERSITY OF NEW YORK   (United States)

^c NONE

Author keywords

[No Author keywords available]

Indexed keywords

DATA PROCESSING; PROBABILITY; RANDOM PROCESSES;

ALL USES TEST DATA ADEQUACY CRITERIA; MUTATION TESTING;

COMPUTER SOFTWARE SELECTION AND EVALUATION;

EID: 0031235549     PISSN: 01641212     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0164-1212(96)00154-9     Document Type: Article

References (31)

1. Boyer, R. S., Elspas, B., and Levitt, K. N., SELECT - a formal system for testing and debugging programs by symbolic execution. SIGPLAN Notices, 10(6):234-245 (June 1975). International Conference on Reliable Software.
2. Bhattacharyya, G. K., and Johnson, R. A., Statistical Concepts and Methods, John Wiley & Sons, Inc., New York, 1977.
3. Budd, T. A., Mutation analysis: Ideas, examples, problems and prospects. In: (B. Chandrasekaran and S. Radicchi, eds.), Computer Program Testing, North Holland, July 1981, pp. 129-148.
4. DeMillo, R. A., Guindi, D. S., McCracken, W. M., Offutt, A. J., and King, K. N., An extended overview of the Mothra software testing environment. In: Proceedings Second Workshop on Software Testing Verification and Analysis, IEEE Computer Society Press, July 1988, pp. 142-151.
5. DeMillo, R. A., Lipton, R. J., and Sayward, F. G., Hints on Test Data Selection: Help for the Practicing Programmer. Computer, 11(4):34-41 (April 1978).
6. DeMillo, R. A., and Offutt, A. J., Experimental Results From an Automatic Test Case Generator. ACM Transactions on Software Engineering Methodology, 2(2): 109-127 (April 1993).
7. Duran, J. W., and Ntafos, S. C., An Evaluation of Random Testing. IEEE Transactions on Software Engineering, SE-10(7):438-444 (July 1984).
8. Frankl, P. G., The Use of Data Flow Information for the Selection and Evaluation of Software Test Data. PhD thesis, Courant Institute of Mathematical Sciences, New York University, October 1987.
9. Frankl, P. G., and Weyuker, E. J., An Applicable Family of Data Flow Testing Criteria. IEEE Transactions on Software Engineering, SE-14(10):1483-1498 (October 1988).
10. Frankl, P. G., and Weiss, S. N., An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing. IEEE Transactions on Software Engineering, 19(8):774-787 (August 1993).
11. Frankl, P. G., and Weyuker, E. J., A Formal Analysis of the Fault Detecting Ability of Testing Methods. IEEE Transactions on Software Engineering, 202-213 (March 1993).
12. Frankl, P. G., and Weyuker, E. J., Provable Improvements on Branch Testing. IEEE Transactions on Software Engineering, 19(10):962-975 (October 1993).
13. Frankl, P. G., Weiss, S. N., and Hu, C., All-uses versus mutation: an experimental comparison of effectiveness. Technical Report PUCS-94-100, Polytechnic University Computer Science Dept., February 1994.
14. Frankl, P. G., Weiss, S. N., and Weyuker, E. J. ASSET - a system to select and evaluate tests. In: Proceedings of the IEEE Conference on Software Tools, pp. 72-79, April 1985.
15. Goodenough, J. B., and Gerhart, S. L., Toward a Theory of Test Data Selection. IEEE Transactions on Software Engineering, SE-1(2):156-173 (June 1975).
16. Gourlay, J. S., A Mathematical Framework for the Investigation of Testing. IEEE Transactions on Software Engineering, SE-9(6):686-709 (November 1983).
17. Gustavson, F., Remark on Algorithm 408. ACM Transactions on Mathematical Software, 4:295 (1978).
18. Hamlet, D., Theoretical comparison of testing methods. In: Proceedings ACM SIGSOFT Third Symposium on Software Testing, Analysis, and Verification, pages 28-37. ACM Press, December 1989.
19. Hamlet, D., and Taylor, R., Partition Testing Does Not Inspire Confidence. IEEE Transactions on Software Engineering, 16(12):1402-1411 (December 1990).
20. McNamee, J. M., Algorithm 408: A Sparse Matrix Package (Part I) [F4]. Communications of the ACM, 14(4) (April 1971).
21. Mathur, A. P., and Wong, W. E., Comparing the fault detection effectiveness of mutation and data flow testing: An empirical study. Technical Report SERC-TR-146-P, Software Engineering Research Center, Purdue University, West Lafayette, Indiana, March 1993.
22. Offutt, A. J., Investigations of the Software Testing Coupling Effect. ACM Transactions on Software Engineering Methodology, 1(1):5-20 (January 1992).
23. Offutt, A. J., Pan, J., Tewary, K., and Zhang, T., Experiments with data flow and mutation testing. Technical Report ISSE-TR-94-105, Department of Information and Software Systems Engineering, George Mason University, Fairfax, Virginia, 1994.
24. Press, W. H., Flannery, B. P., Teukolsky, S. A., and Vetterling, W. T., Numerical Recipes: The Art of Scientific Computing, Cambridge University Press, Cambridge, UK, 1986.
25. Rapps, S., and Weyuker, E. J., Selecting Software Test Data Using Data Flow Information, IEEE Transactions on Software Engineering, SE-14(4):367-375, April 1985.
26. Weiss, S. N., Comparing Test Data Adequacy Criteria. Software Engineering Notes, 14(6):42-49 (October 1989).
27. Weiss, S. N., Methods of Comparing Test Data Adequacy Criteria. In: COMPSAC 90, pages 1-6, October 1990.
28. Weiss, S. N., and Frankl, P. G., Comparison of all-uses and all-edges: Design, data, and analysis. Technical Report CS-91-03, Hunter College Computer Science Dept., March 1991.
29. Wong, W. E., Horgan, J. R., London, S., and Mathur, A. P., Effect of test set minimization on the fault detection effectiveness of the all-uses criterion. Technical Report SERC-TR-152-P, Software Engineering Research Center, Purdue University, April 1994.
30. Weyuker, E. J., and Jeng, B., Analyzing Partition Testing Strategies. IEEE Transactions on Software Engineering, 17(7):703-711 (July 1991).
31. Weyuker, E. J., Weiss, S. N., and Hamlet, R., Comparison of program testing strategies. In: Proceedings Fourth Symposium on Software Testing, Analysis, and Verification, pages 1-10. ACM Press, October 1991.