A specification-based coverage metric to evaluate test sets

International Journal of Reliability, Quality and Safety Engineering

Volumn 8, Issue 4, 2001, Pages 275-299

  Ammann, Paul E ^a   Black, Paul E ^b  

^a Krasnow Institute for Advanced Study   (United States)

^b NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

Author keywords

Formal methods; Model checking; Mutation analysis; Software testing

Indexed keywords

EID: 0006589580     PISSN: 02185393     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218539301000530     Document Type: Article

References (39)

1. S. B. Akers, "On a theory of boolean functions", SIAM Journal 7 (4) (1959).
2. P. Aramann and P. E. Black, "Abstracting formal specifications to generate software tests via model checking", NIST IR 6405 (1999).
3. P. E. Ammann, P. E. Black and W. Majurski, "Using model checking to generate tests from specifications", in Proc. Second IEEE Int. Conference on Formal Eng. Methods (IEEE Computer Society, 1998), pp. 46-54.
4. J. M. Atlee, "Native model-checking of SCR requirements", in Proc. Fourth Int. SCR Workshop (November, 1994).
5. J. M. Atlee and M. A. Buckley, "A logic-model semantics for SCR software requirements", in Proc. 1996 Int. Symp. on Software Testing and Analysis (January, 1996), pp. 280-292.
6. J. M. Atlee and J. Gannon, "State-based model checking of event-driven system requirements", IEEE Tran. on Software Eng. 19 (1) (1993), pp. 24-40.
7. J. Beskin, "Human error simulation in test case generation", Technical report, Reliable Software Technologies Corp. (Sterling, Virginia, 1998).
8. R. Bharadwaj and C. Heitmeyer, "Verifying SCR requirements specifications using state exploration", in Proc. First ACM SIGPLAN Workshop on Automatic Analysis of Software (January, 1997).
9. R. Bharadwaj and C. L. Heitmeyer, "Model checking complete requirements specifications using abstraction", Memorandum Report NRL/MR/5540-97-7999, U. S. Naval Research Laboratory, Washington, DC (November, 1997).
10. P. E. Black, V. Okun and Y. Yesha, "Mutation of model checker specifications for test generation and evaluation", in Proc. Mutation 2000 (October, 2000).
11. 20 states and beyond", Information and Computation 98 (2) (1992), pp. 142-170.
12. W. Chan, R. J. Anderson, P. Beame, S. Burns, F. Modugno, D. Notkin and J. D. Reese, "Model checking large software specifications", IEEE Tran. on Software Eng. 24 (7) (1998), pp. 498-520.
13. T. S. Chow, "Testing software design modeled by finite-state machines", IEEE Tran. on Software Eng. SE-4 (3) (1978), pp. 178-187.
14. E. M. Clarke, Jr., O. Grumberg and D. E. Long, "Verification tools for finite-state concurrent systems", in A Decade of Concurrency - Reflections and Perspectives, Vol. 803 of Lecture Notes in Computer Science (Springer-Verlag, 1994).
15. R. A. De Millo, R. J. Lipton and F. G. Sayward, "Hints on test data selection: Help for the practicing programmer", IEEE Computer 11 (4) (1978), pp. 34-41.
16. R. A. De Millo and A. J. Offutt, "Constraint-based automatic test data generation", IEEE Tran. on Software Eng. 17 (9) (1991), pp. 900-910.
17. A. Engels, L. Feijs and S. Mauw, "Test generation for intelligent networks using model checking", in Proc. Third Int. Workshop on Tools and Algorithms for the Construction and Analysis of Systems, ed. E. Brinksma, Vol. 1217 of Lecture Notes in Computer Science (Springer-Verlag, 1997), pp. 384-398.
18. S. R. Faulk, L. Finneran, J. Kirby, Jr., S. Shah and J. Sutton, "Experience applying the CoRE method to the Lockheed C-130J", in Proc. 9th Annual Conference on Computer Assurance (June, 1994), pp. 3-8.
19. P. G. Frankl and E. J. Weyuker, "An applicable family of data flow testing criteria", IEEE Tran. on Software Eng. 14 (10) (1988), pp. 1483-1498.
20. A. Gargantini and C. Heitmeyer, "Using model checking to generate tests from requirements specifications", in Proc. Joint 7th European Software Eng. Conference and 7th ACM SIGSOFT Int. Symp. on Foundations of Software Eng. (September, 1999).
21. C. Hagwood and J. Yen, Personal Communication (June, 1999).
22. C. L. Heitmeyer, R. D. Jeffords and B. G. Labaw, "Automated consistency checking of requirements specifications", ACM Tran. on Software Eng. and Methodology 5 (3) (1996), pp. 231-261.
23. C. L. Heitmeyer and J. Mclean, "Abstract requirements specifications: A new approach and its application", IEEE Tran. on Software Eng. SE-9 (5) (1983), pp. 580-589.
24. K. L. Heninger, "Specifying software requirements for complex systems", IEEE Tran. on Software Eng. SE-6 (1) (1980), pp. 2-13.
25. D. Hoffman, P. Strooper and L. White, "Boundary values and automated component testing", Software Testing, Verification & Reliability 9 (1) (1999).
26. G. J. Holzmann, "The model checker SPIN", IEEE Tran. on Software Eng. 23 (5) (1997), pp. 279-295.
27. W. E. Howden, "Weak mutation testing and completeness of test sets", IEEE Tran. on Software Eng. 8 (4) (1982), pp. 371-379.
28. D. R. Kuhn, "Fault classes and error detection in specification based testing", ACM Tran. on Software Eng. Methodology 8 (4) (1999).
29. P. M. Maurer, "Generating test data with enhanced context-free grammars", IEEE Software (July, 1990), pp. 50-55.
30. S. P. Miller, "Specifying the mode logic of a flight guidance system in CoRE and SCR", in Second Workshop on Formal Methods in Software Practice (March, 1998).
31. S. Owre, J. M. Rushby and N. Shankar, "Analyzing tabular and state-transition requirements specifications in PVS", Technical Report CSL-95-12 (Computer Science Laboratory SRI International, June, 1995), (Revised April, 1996).
32. C. R. Ramakrishnan and R. Sekar, "Model-based vulnerability analysis of computer systems", in 2nd Int. Workshop on Verification, Model Checking and Abstract Interpretation (September, 1998).
33. R. W. Ritchey and P. Ammann, "Using model checking to analyze network vulnerabilities", in Proc. 2000 IEEE Computer Society Symp. on Security and Privacy (May, 2000).
34. A. J. van Schouwen, D. L. Parnas and J. Madey, "Documentation of requirements for computer systems", in Proc. IEEE Int. Symp. on Requirements Eng. (IEEE Computer Society Press, 1993), pp. 198-207.
35. J. M. Voas, "PIE: A dynamic failure-based technique", IEEE Tran. on Software Eng. 18 (8) (1992).
36. E. Weyuker, T. Goradia and A. Singh, "Automatically generating test data from a boolean specification", IEEE Tran. on Software Eng. 20 (5) (1994), pp. 353-363.
37. M. R. Woodward, "Errors in algebraic specifications and an experimental mutation testing tool", Software Eng. Journal (July, 1993), pp. 211-224.
38. D. Wu, M. A. Hennell, D. Hedley and I. J. Riddell, "A practical method for software quality control via program mutation", in Proc. 2nd Workshop on Software Testing, Verification and Analysis (July, 1988), pp. 159-170.
39. H. Zhu, P. A. V. Hall and J. H. R. May, "Software unit test coverage and adequacy", ACM Computing Surveys 29 (4) (1997), pp. 366-427.