On the order of test goals in specification-based testing

Journal of Logic and Algebraic Programming

Volumn 78, Issue 6, 2009, Pages 472-490

  Fraser, Gordon ^a   Gargantini, Angelo ^b   Wotawa, Franz ^a  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

^b UNIVERSITY OF BERGAMO   (Italy)

Author keywords

Automated software testing; Automated test case generation; Minimization; Monitoring; Performance; Testing with model checkers

Indexed keywords

AUTOMATED SOFTWARE TESTING; AUTOMATED TEST CASE GENERATION; COVERAGE CRITERIA; MINIMIZATION; MODEL BASED TESTING; MODEL CHECKER; PERFORMANCE; SPECIFICATION BASED TESTING; TEST CASE; TEST CASE GENERATION; TEST GENERATIONS;

AUTOMATION; COMPUTER SOFTWARE SELECTION AND EVALUATION; EXPERIMENTS; HEURISTIC METHODS; MODEL CHECKING; OPTIMIZATION; SOFTWARE TESTING;

TESTING;

EID: 67649208337     PISSN: 15678326     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jlap.2009.01.004     Document Type: Article

References (49)

1. T.A. Alspaugh, S.R. Faulk, K.H. Britton, R.A. Parker, D.L. Parnas, J.E. Shore, Software requirements for the A-7E aircraft, Tech. Rep. NRL-9194, Naval Research Lab., Washington, DC, 1992.
2. P. Ammann, P.E. Black, A specification-based coverage metric to evaluate test sets, in: HASE '99: The 4th IEEE International Symposium on High-Assurance Systems Engineering, IEEE Computer Society, Washington, DC, USA, 1999, pp. 239-248.
3. P. Ammann, W. Ding, D. Xu, Using a model checker to test safety properties, in: Proceedings of the 7th International Conference on Engineering of Complex Computer Systems (ICECCS, 2001), IEEE, 2001, pp. 212-221.
4. Ammann P., and Offutt J. Introduction to Software Testing (2008), Cambridge University Press, New York, NY, USA
5. P.E. Ammann, P.E. Black, W. Majurski, Using model checking to generate tests from specifications, in: Proceedings of the Second IEEE International Conference on Formal Engineering Methods (ICFEM'98), IEEE Computer Society, 1998, pp. 46-54.
6. Bharadwaj R., and Heitmeyer C.L. Model checking complete requirements specifications using abstraction. Automat. Software Eng. 6 1 (1999) 37-68
7. Bharadwaj R., and Heitmeyer C.L. Model checking complete requirements specifications using abstraction. Automat. Software Eng.: Int. J. 6 1 (1999) 37-68
8. A. Biere, A. Cimatti, E.M. Clarke, Y. Zhu, Symbolic model checking without BDDs, in: TACAS '99: Proceedings of the 5th International Conference on Tools and Algorithms for Construction and Analysis of Systems, Springer-Verlag, London, UK, 1999, pp. 193-207.
9. Bryant R.E. Graph-based algorithms for boolean function manipulation. IEEE Trans. Comput. 35 8 (1986) 677-691
10. J. Chilenski, L.A., Richey, Definition for a masking form of modified condition decision coverage (MCDC), Tech. rep., Boeing, 1997.
11. Chilenski J.J., and Miller S.P. Applicability of modified condition/decision coverage to software testing. Software Eng. J. (1994) 193-200
12. Chvatal V. A greedy heuristic for the set-covering problem. Math. Oper. Res. 4 3 (1979)
13. A. Cimatti, E.M. Clarke, F. Giunchiglia, M. Roveri, NUSMV: a new symbolic model verifier, in: CAV '99: Proceedings of the 11th International Conference on Computer Aided Verification, Springer-Verlag, London, UK, 1999, pp. 495-499.
14. Clarke E.M., and Emerson E.A. Design and synthesis of synchronization skeletons using branching-time temporal logic. Logic of Programs, Workshop (1982), Springer-Verlag, London, UK 52-71
15. E.M. Clarke, E.A. Emerson, A.P. Sistla, Automatic verification of finite state concurrent system using temporal logic specifications: a practical approach, in: POPL '83: Proceedings of the 10th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages, ACM Press, New York, NY, USA, 1983, pp. 117-126.
16. E.M. Clarke, O. Grumberg, K.L. McMillan, X. Zhao, Efficient generation of counterexamples and witnesses in symbolic model checking, in: Proceedings of the 32st Conference on Design Automation (DAC), ACM Press, 1995, pp. 427-432.
17. P.-J. Courtois, D.L. Parnas, Documentation for safety critical software, in: Proceedings of the 15th International Conference on Software Engineering (ICSE '93), Baltimore, MD, 1993, pp. 315-323.
18. L. de Moura, S. Owre, H. Rueß, J. Rushby, N. Shankar, M. Sorea, A. Tiwari, SAL 2, in: R. Alur, D. Peled (Eds.), Computer-Aided Verification, CAV 2004, Lecture Notes in Computer Science, vol. 3114, Springer-Verlag, Boston, MA, 2004, pp. 496-500.
19. A.S. Kalaji, R. Hierons, S. Swift, Generating feasible transition paths for testing from an extended finite state machine (EFSM), in: Proceedings of the 2nd International Conference on Software Testing, Verification, and Validation, (ICST 2009), 2009, in press.
20. S. Edelkamp, A.L. Lafuente, S. Leue, Directed explicit model checking with HSF-SPIN, in: SPIN '01: Proceedings of the 8th International SPIN Workshop on Model Checking of Software, Springer-Verlag New York, Inc., New York, NY, USA, 2001, pp. 57-79.
21. N. Eén, N. Sörensson, An extensible SAT-solver, in: Theory and Applications of Satisfiability Testing, 6th International Conference, SAT 2003. Santa Margherita Ligure, Italy, May 5-8, 2003 Selected Revised Papers, Lecture Notes in Computer Science, vol. 2919, Springer, 2003, pp. 502-518.
22. E.A. Emerson, J.Y. Halpern, Decision procedures and expressiveness in the temporal logic of branching time, in: STOC '82: Proceedings of the Fourteenth Annual ACM Symposium on Theory of Computing, ACM Press, New York, NY, USA, 1982, pp. 169-180.
23. G. Fraser, F. Wotawa, Using LTL rewriting to improve the performance of model-checker based test-case generation, in: A-MOST '07: Proceedings of the 3rd International Workshop on Advances in Model-based Testing, ACM Press, New York, NY, USA, 2007, pp. 64-74.
24. G. Fraser, F. Wotawa, Ordering coverage goals in model checker based testing, in: A-MOST '08: Proceedings of the 4th International Workshop on Advances in Model-based Testing, IEEE Computer Society, 2008, pp. 31-40.
25. Garey M.R., and Johnson D.S. Computers and Intractability: A Guide to the Theory of NP-Completeness (1979), W.H. Freeman & Co., New York, NY, USA
26. A. Gargantini, Using model checking to generate fault detecting tests, in: Proceedings of the International Conference on Tests And Proofs (TAP), Lecture Notes in Computer Science, vol. 4454, Zurich, Switzerland, 2007, pp. 189-206.
27. A. Gargantini, C. Heitmeyer, Using model checking to generate tests from requirements specifications, in: ESEC/FSE'99: 7th European Software Engineering Conference, Held Jointly with the 7th ACM SIGSOFT Symposium on the Foundations of Software Engineering, vol. 1687, Springer, 1999, pp. 146-162.
28. Rothermel G., Harrold M.J., and Christie Hong J.v.R. Empirical studies of test-suite reduction. Software Test. Verification Reliab. 12 4 (2002) 219-249
29. G. Hamon, de L. Moura, J. Rushby, Generating efficient test sets with a model checker in: Proceedings of the Second International Conference on Software Engineering and Formal Methods (SEFM'04), 2004, pp. 261-270.
30. Harrold M.J., Gupta R., and Soffa M.L. A methodology for controlling the size of a test suite. ACM Trans. Software Eng. Methodol. 2 3 (1993) 270-285
31. M.P.E. Heimdahl, G. Devaraj, Test-suite reduction for model based tests: effects on test quality and implications for testing. in: ASE, IEEE Computer Society, 2004, pp. 176-185.
32. M.P.E., Heimdahl, S. Rayadurgam, W. Visser, Specification centered testing, in: Proceedings of the Second International Workshop on Automated Program Analysis, Testing and Verification (ICSE 2001), 2001.
33. Heitmeyer C.L. Encyclopedia of Software Engineering, Ch. Software Cost Reduction vol. 2 (2002), John Wiley & Sons
34. Holzmann G.J. The Model Checker SPIN. IEEE Trans. Software Eng. 23 5 (1997) 279-295
35. H.S. Hong, H. Ural, Using model checking for reducing the cost of test generation, in: Formal Approaches to Software Testing, Lecture Notes in Computer Science, vol. 3395, Springer-Verlag, GmbH, 2005, pp. 110-124.
36. R.D. Jeffords, C.L. Heitmeyer, An algorithm for strengthening state invariants generated from requirements specifications, in: 5th IEEE International Symposium on Requirements Engineering (RE 2001), 27-31 August 2001, Toronto, Canada, IEEE Computer Society, 2001, pp. 182-193.
37. Jones J.A., and Harrold M.J. Test-suite reduction and prioritization for modified condition/decision coverage. IEEE Trans. Software Eng. 29 3 (2003) 195-209
38. J. Kirby, Example NRL/SCR software requirements for an automobile cruise control and monitoring system, Tech. Rep. TR-87-07, Wang Institute of Graduate Studies, 1987.
39. O. Lichtenstein, A. Pnueli, Checking that finite state concurrent programs satisfy their linear specification, in: POPL '85: Proceedings of the 12th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages, ACM Press, New York, NY, USA, 1985, pp. 97-107.
40. Marré M., and Bertolino A. Using spanning sets for coverage testing. IEEE Trans. Software Eng. 29 11 (2003) 974-984
41. McMillan K.L. Symbolic Model Checking (1993), Kluwer Academic Publishers, Norwell, MA,USA
42. A. Pnueli, The temporal logic of programs, in: 18th Annual Symposium on Foundations of Computer Science, 31 October-2 November, Providence, Rhode Island, USA, IEEE, 1977, pp. 46-57.
43. J.-P. Queille, J. Sifakis, Specification and verification of concurrent systems in CESAR, in: Proceedings of the 5th Colloquium on International Symposium on Programming, Springer-Verlag, London, UK, 1982, pp. 337-351.
44. G. Rothermel, M.J. Harrold, J. Ostrin, C. Hong, An empirical study of the effects of minimization on the fault detection capabilities of test suites, in: ICSM '98: Proceedings of the International Conference on Software Maintenance, IEEE Computer Society, 1998, p. 34.
45. S. Tallam, N. Gupta, A concept analysis inspired greedy algorithm for test suite minimization, in: PASTE '05: Proceedings of the 6th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, ACM, New York, NY, USA, 2005, pp. 35-42.
46. L. Tan, O. Sokolsky, I. Lee, Specification-based testing with linear temporal logic, in: Proceedings of IEEE International Conference on Information Reuse and Integration (IRI'04), 2004, pp. 493-498.
47. M.Y. Vardi, P. Wolper, An Automata-theoretic approach to automatic program verification (preliminary report), in: Proceedings of the 1st IEEE Symposium on Logic in Computer Science (LICS'86), IEEE Computer Society, 1986, pp. 332-344.
48. W.E. Wong, J.R., Horgan, S. London, A.P. Mathur, Effect of test set minimization on fault detection effectiveness, in: ICSE '95: Proceedings of the 17th International Conference on Software Engineering, ACM Press, 1995, pp. 41-50.
49. H. Zhong, L. Zhang, H. Mei, An experimental comparison of four test suite reduction techniques, in: ICSE '06: Proceeding of the 28th International Conference on Software Engineering, ACM Press, 2006, pp. 636-640.