Experimental evaluation of the tolerance for control-flow test criteria

Software Testing Verification and Reliability

Volumn 14, Issue 3, 2004, Pages 167-187

  Kapoor, Kalpesh ^a   Bowen, Jonathan P ^a  

^a LONDON SOUTH BANK UNIVERSITY   (United Kingdom)

Author keywords

Boolean specification; CC; Control flow testing criteria; DCC; Effectiveness; Empirical evaluation; Fault based testing; FPC; MCDC; RCDC; Tolerance

Indexed keywords

BOOLEAN FUNCTIONS; COMPUTER SOFTWARE; COMPUTER TESTING; FAULT TOLERANT COMPUTER SYSTEMS; SOFTWARE ENGINEERING;

BOOLEAN SPECIFICATION; CONDITION COVERAGE; CONTROL FLOW TESTING CRITERIA; DECISION CONDITION COVERAGE; EMPIRICAL EVALUATION; FAULT BASED TESTING; FAULT DETECTION EFFECTIVENESS; FULL PREDICATE COVERAGE; MODIFIED CONDITION DECISION COVERAGE; REINFORCED CONDITION DECISION COVERAGE;

COMPUTER SOFTWARE SELECTION AND EVALUATION;

EID: 4644317037     PISSN: 09600833     EISSN: None     Source Type: Journal    
DOI: 10.1002/stvr.303     Document Type: Article

References (44)

1. Zhu H, Hall P, May HR. Software unit test coverage and adequacy. ACM Computing Surveys 1997; 29(4):336-427.
2. Zhu H. Axiomatic assessment of control flow-based software test adequacy criteria. Software Engineering Journal 1995; 10(9):194-204.
3. Frankl PG, Iakounenko O. Further empirical studies of test effectiveness. Proceedings of the Sixth International Symposium on Foundations of Software Engineering, vol. 23. ACM Press: New York, 1998; 153-162.
4. Frankl PG, Weiss S. An experimental comparison of the effectiveness of branch testing and data flow testing. IEEE Transactions on Software Engineering 1993; 19(8):774-787.
5. Frankl PG, Weyuker E. A formal analysis of the fault-detecting ability of testing methods. IEEE Transactions on Software Engineering 1993; 19(3 ):202-213.
6. Frankl PG, Weyuker E. Provable improvements on branch testing. IEEE Transactions on Software Engineering 1993; 19(10):962-975.
7. Frankl PG, Weyuker E. Testing software to detect and reduce risk. Journal of Systems and Software 2000; 53(3): 275-286.
8. Hutchins M, Foster H, Goradia T, Ostrand T. Experiments on the effectiveness of dataflow- and controlflow-based test adequacy criteria. Proceedings of the 16th International Conference on Software Engineering (ICSE). IEEE Computer Society Press: Los Alamitos, CA, 1994; 191-200.
9. Weyuker E. Can we measure software testing effectiveness? Proceedings of the First International Software Metrics Symposium, Baltimore, U.S.A. IEEE Computer Society Press: Los Alamitos, CA, 1993; 100-107.
10. Zhu H. A formal analysis of the subsume relation between software test adequacy criteria. IEEE Transactions on Software Engineering 1996; 22(4 ):248-255.
11. Vilkomir SA, Kapoor K, Bowen JP. Tolerance of control-flow testing criteria. Proceedings of the 27th International Computer Software and Applications Conference (COMPSAC). IEEE Computer Society Press: Los Alamitos, CA, 2003; 182-187.
12. RTCA/DO-178B. Software Considerations in Airborne Systems and Equipment Certification. Washington DC, U.S.A. 1992.
13. Paradkar A, Tai KC. Test-generation for Boolean expressions. Proceedings of the Sixth International Symposium on Software Reliability Engineering (ISSRE). IEEE Computer Society Press: Los Alamitos, CA, 1995; 106-115.
14. Paradkar A, Tai KC, Vouk MA. Automatic test-generation for predicates. IEEE Transactions on Reliability 1996; 45(4):515-530.
15. Vouk MA, Tai KC, Paradkar A. Empirical studies of predicate-based software testing. Proceedings of the 5th International Symposium on Software Reliability Engineering. IEEE Computer Society Press: Los Alamitos, CA, 1994; 55-64.
16. Weyuker E, Gorodia T, Singh A. Automatically generating test data from a Boolean specification. IEEE Transactions on Software Engineering 1994; 20(5):353-363.
17. Offutt AJ. Investigations of the software testing coupling effect. ACM Transactions on Software Engineering and Methodology 1992; 1(1):5-20.
18. Voas JM. PIE: A dynamic failure-based technique. IEEE Transactions on Software Engineering 1992; 18(2):717-727.
19. Richardson DJ, Thompson MC. An analysis of test data selection criteria using the RELAY model of fault detection. IEEE Transactions on Software Engineering 1993; 19(6):533-553.
20. Vilkomir SA, Bowen JP. Reinforced condition/decision coverage (RC/DC): A new criterion for software testing. Proceedings of the 2nd International Conference, Formal Specification and Development in Z and B (Lecture Notes in Computer Science, vol. 2272), Bert D, Bowen JP, Henson M, Robinson K (eds.). Springer: Berlin, 2002; 295-313.
21. Briand LC, Pfahl D. Using simulation for assessing the real impact of test-coverage on defect-coverage. IEEE Transactions on Reliability 2000; 49(1):60-70.
22. Harman M, Hierons R, Holcombe M, Jones B, Reid S, Roper M, Woodward M. Towards a maturity model for empirical studies of software testing. Proceedings of the 5th Workshop on Empirical Studies of Software Maintenance (WESS), 1999.
23. Chilenski J, Miller S. Applicability of modified condition/decision coverage to software testing. Software Engineering Journal 1994; 9(5 ):193-200.
24. Offutt AJ, Xiong Y, Liu S. Criteria for generating specification-based tests. Proceedings of the 5th International Conference on Engineering of Complex Computer Systems (ICECCS). IEEE Computer Society Press: Los Alamitos, CA, 1999; 119-129.
25. Vilkomir SA, Bowen JP. Formalization of software testing criteria using the Z notation. Proceedings of the International Computer Software and Applications Conference (COMPSAC'O1). IEEE Computer Society Press: Los Alamitos, CA, 2001; 351-356.
26. Dupuy A, Leveson N. An empirical evaluation of the MC/DC coverage criterion on the HETE-2 satellite software. DASC: Digital Aviation Systems Conference, Philadelphia, PA, 2000. IEEE: Piscataway, NJ, 2000; 1B6/1-1B6/7.
27. Jasper R, Brennan M, Williamson K, Currier B, Zimmerman D. Test data generation and feasible path analysis. Proceedings of the International Symposium on Software Testing and Analysis. ACM Press: New York, 1994; 95-107.
28. Jones JA, Harrold MJ. Test-suite reduction and prioritization for modified condition/decision coverage. Proceedings of the International Conference on Software Maintenance (ICSM). IEEE Computer Society Press: Los Alamitos, CA, 2001; 92-101.
29. White A. Comments on modified condition/decision coverage for software testing. Proceedings of the IEEE Aerospace Conference, Big Sky, Montana, USA, March 2001, vol. 6. IEEE: Piscataway, NJ, 2001; 2821-2828.
30. Chilenski JJ. An investigation of three forms of the modified condition decision coverage (MCDC) criterion. Technical Report DOT/FAA/AR-O1/18, U.S. Department of Transportation, Federal Aviation Administration, 2001.
31. . Morell LJ. A theory of fault-based testing. IEEE Transactions on Software Engineering 1990; 16(8):844-857.
32. Tai K-C. Theory of fault-based predicate testing for computer programs. IEEE Transactions on Software Engineering 1996; 22(8):552-562.
33. Kuhn DR. Fault classes and error detection capability of specification-based testing. ACM Transactions on Software Engineering and Methodology 1999; 8(4):411-424.
34. Tsuchiya T, Kikuno T. On fault classes and error detection capability of specification-based testing. ACM Transactions on Software Engineering and Methodology 2002; 11(1):58-62.
35. Goodenough JB, Gerhart SL. Toward a theory of test data selection. IEEE Transactions on Software Engineering 1975; 1(2):156-173.
36. Howden WE. Reliability of the path analysis testing strategy. IEEE Transactions on Software Engineering 1976; 2(3):208-215.
37. Hamlet D. What can we learn by testing a program? Proceedings of the International Symposium on Software Testing and Analysis. ACM Press: New York, 1998; 50-52.
38. Kapoor K, Bowen JP. Experimental evaluation of the variation in effectiveness for DC, FPC and MC/DC test criteria. Proceedings of the International Symposium on Empirical Software Engineering. IEEE Computer Society Press: Los Alamitos, CA, 2003; 185-194.
39. Weyuker E. Thinking formally about testing without a formal specification. Proceedings of Formal Approaches to Testing of Software (FATES). A Satellite Workshop of CONCUR'02, August 2002; 1-10.
40. Wong WE, Horgan JR, Mathur AP, Pasquini A. Test set size minimization and fault detection effectiveness: A case study in a space application. Proceedings of the 21st Annual International Computer Software and Applications Conference (COMPSAC), August 1997. IEEE Computer Society Press: Los Alamitos, CA, 1997; 522-528.
41. Rothermel G, Harrold MJ, Ronne JV, Hong C. Empirical studies of test-suite reduction. Software Testing. Verification and Reliability 2002; 12(4):219-249.
42. Chen W, Untch RH, Rothermel G, Elbaum S, Ronne JV. Can fault-exposure-potential estimates improve the fault detection abilities of test suites? Software Testing, Vertfication and Reliability 2002; 12(4):197-218.
43. Kapoor K. Stability of test criteria and fault hierarchies in software testing. PhD Thesis, London South Bank University, June 2004; (submitted ).
44. Bowen JP, Bogdanov K, Clark J, Harman M, Hierons R, Krause P. FORTEST: Formal methods and testing. Proceedings of the 26th International Computer Software and Applications Conference (COMPSAC). IEEE Computer