A framework for specification-based testing

IEEE Transactions on Software Engineering

Volumn 22, Issue 11, 1996, Pages 777-793

  Stocks, Phil ^a   Carrington, David ^b  

^a RUTGERS UNIVERSITY   (United States)

^b UNIVERSITY OF QUEENSLAND   (Australia)

Author keywords

Specification based testing; Test data; Test oracles; Testing strategies; Z notation

Indexed keywords

COMPUTER HARDWARE DESCRIPTION LANGUAGES; COMPUTER SIMULATION; COMPUTER SOFTWARE; REGRESSION ANALYSIS;

SPECIFICATION BASED TESTING; TEST ORACLES; TEST TEMPLATE FRAMEWORK;

SOFTWARE ENGINEERING;

EID: 0030285205     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/32.553698     Document Type: Article

References (41)

1. O.-J. Dahl, E.W. Dijkstra, and C.A.R. Hoare, Structured Programming, APIC Studies in Data Processing, no. 8. Academic Press, 1972.
2. A.S. Tanenbaum, "In Defense of Program Testing or Correctness Proofs Considered Harmful," ACM SIGPLAN Notices, vol. 11, no. 5, pp. 64-68, May 1976.
3. D. Carrington and P. Stocks, "A Tale of Two Paradigms: Formal Methods and Software Testing," Proc. Eighth Z User Meeting, Springer-Verlag, pp. 51-68, 1994.
4. J.B. Goodenough and S.L. Gerhart, "Toward a Theory of Test Data Selection," IEEE Trans. Software Eng., vol. 1, no. 2, pp. 156-173, June 1975.
5. D.J. Richardson, S.L. Aha, and T.O. O'Malley, "Specification-Based Test Oracles for Reactive Systems," Proc. 14th Int'l Conf. Software Eng., pp. 105-118, May 1992.
6. B. Hetzel, The Complete Guide to Software Testing, QED Information Sciences, Wellesley, Mass., second edition, 1988.
7. C.B. Jones, Systematic Software Development Using VDM, Series in Computer Science. Prentice Hall Int'l, second edition, 1990.
8. Specification Case Studies, Series in Computer Science. I. Hayes, ed., Prentice Hall Int'l, second edition, 1993.
9. J.M. Spivey, The Z Notation: A Reference Manual, Series in Computer Science. Prentice Hall Int'l, second edition, 1992.
10. P.A.V. Hall, "Towards Testing with Respect to Formal Specifications," Second IEE/BCS Conf. Software Eng. 88. pp. 159-163, July 1988.
11. P.A.V. Hall, "Relationship Between Specifications and Testing," Information and Software Technology, vol. 33, no. 1, pp. 47-52, 1991.
12. J. Dick and A. Faivre, "Automating the Generating and Sequencing of Test Cases from Model-Based Specification," FME '93: Industrial Strength Formal Methods, Fifth Int'l Symp. Formal Methods Europe, J.C.P. Woodcock and P.G. Larsen, eds., Odense, Denmark, vol. 670 of Lecture Notes in Computer Science, pp. 268-284. Springer-Verlag, Apr. 1992.
13. H.-M. Hörcher, "Improving Software Tests Using Z Specifications," Bowen and Hinchey [41], pp. 152-166, Proc. Ninth Int'l Conf. Z Users.
14. H.-M. Hörcher and J. Peleska, "Using Formal Specifications to Support Software Testing," Software Quality J., vol. 4, no. 4, pp. 309-327, Dec. 1995.
15. S. Stepney, "Testing as Abstraction," Bowen and Hinchey [41], pp. 137-151, Proc. Ninth Int'l Conf. Z Users.
16. P. Stocks and D.A. Carrington, "Test Templates: A Specification-Based Testing Framework," Proc. 15th Int'l Conf. Software Eng., pp. 405-414, May 1993.
17. P. Stocks, "Applying Formal Methods to Software Testing," PhD thesis, the Univ. of Queensland, Australia, 1993.
18. T.J. Ostrand and M.J. Balcer, "The Category-Partition Method for Specifying and Generating Functional Tests," Comm. ACM., vol. 31, no. 6, pp. 676-686, June 1988.
19. M.J. Balcer, W.M. Hasling, and T.J. Ostrand, "Automatic Generation of Test Scripts from Formal Test Specifications," Software Eng. Notes, vol. 14, no. 8, pp. 210-218, Proc. ACM SIGSOFT '89 Third Symp. Software Testing, Analysis, and Verification, Dec. 1989.
20. J. Pachl, "A Notation for Specifying Test Selection Criteria," Protocol Specification, Testing, and Verification, X, L. Logrippo, R.L. Probert, and H. Ural, eds., pp. 71-84. North-Holland, 1990.
21. T.J. Ostrand, R. Sigal, and E.J. Weyuker, "Design for a Tool to Manage Specification-Based Testing," Workshop on 'Software Testing, Banff, Canada , pp. 41-50, IEEE CS Press, July 1986.
22. R.A. Kemmerer, "Testing Formal Specifications to Detect Design Errors," IEEE Trans. Software Eng., vol. 11, no. 1, pp. 32-43, Jan. 1985.
23. R. Kneuper, "Symbolic Execution as a Tool for Validation of Specifications," PhD dissertation, Technical Report UMCS-89-7-1, Dept. of Computer Science, Univ. of Manchester, 1989.
24. E.J. Weyuker and T.J. Ostrand, "Theories of Program Testing and the Application of Revealing Subdomains," IEEE Trans. Software Eng., vol. 6, no. 3, pp. 236-246, May 1980.
25. D.J. Richardson and L.A. Clarke, "Partition Analysis: A Method Combining Testing and Verification," IEEE Trans. Software Eng., vol. 11, no. 12, pp. 1,477-1,490, Dec. 1985.
26. J.W. Duran and S.C. Ntafos, "An Evaluation of Random Testing," IEEE Trans. Software Eng., vol. 10, no. 4, pp. 438-444, July 1984.
27. P. Stocks and D.A. Carrington, "Test Template Framework: A Specification-Based Testing Case Study," Proc. Int'l Symp. Software Testing and Analysis, ISSTA '93, pp. 11-18, June 1993.
28. I. MacColl, D. Carrington, and P. Stocks, "An Experiment in Specification-Based Testing," Proc. 19th Australasian Computer Science Conf., ACSC'96, K. Ramamohanarao, ed., vol. 18, pp. 159-168, Feb. 1996.
29. The RAISE Language Group, The RAISE Specification Language, 1992.
30. E.J. Weyuker and B. Jeng, "Analyzing Partition Testing Strategies," IEEE Trans. Software Eng., vol. 17, no. 7, pp. 703-711, July 1991.
31. A. Parrish and S.H. Zweben, "Analysis and Refinement of Software Test Data Adequacy Properties," IEEE Trans. Software Eng., vol. 17, no. 6, pp. 565-581, June 1991.
32. E. Weyuker, T. Goradia, and A. Singh, "Automatically Generating Test Data from a Boolean Specification," IEEE Trans. Software Eng., vol. 20, no. 5, pp. 353-363, May 1994.
33. P. Stocks and D.A. Carrington, "Deriving Software Test Cases from Formal Specifications," Sixth Australian Software Eng. Conf., pp. 327-340, July 1991.
34. R. Hamlet, "Theoretical Comparison of Testing Methods," Software Eng. Notes, vol. 14/no. 8, pp. 28-37, Proc. ACM SIGSOFT '89 Third Symp. Software Testing, Analysis, and Verification, Dec. 1989.
35. R. Hamlet and J. Voas, "Faults on Its Sleeve: Amplifying Software Reliability Testing," Proc. Int'l Symp. Software Testing and Analysis, ISSTA '93, pp. 89-98, June 1993.
36. I.J. Hayes, "Specification Directed Module Testing," IEEE Trans. Software Eng., vol. 12, no. 1, pp. 124-133, Jan. 1986.
37. W.E. Howden, "A Functional Approach to Program Testing and Analysis," IEEE Trans. Software Eng., vol. 12, no. 10, pp. 997-1,005, Oct. 1986.
38. W.E. Howden, Functional Program Testing and Analysis. McGraw-Hill, 1986.
39. D. Carrington, D. Duke, I. Hayes, and J. Welsh, "Deriving Modular Designs from Formal Specifications," Software Eng. Notes, vol. 18, no. 5, pp. 89-98, Proc. First ACM SIGSOFT Symp. Foundations of Software Eng., Dec. 1993.
40. J. Cannon, P. McMullin, and R. Hamlet, "Data-Abstraction Implementation, Specification, and Testing," ACM Trans. Programming Languages and Systems, vol. 3, no. 2, pp. 211-223, July 1981.
41. "ZUM '95: The Z Formal Specification Notation," J.P. Bowen and M.G. Hinchey, eds., vol. 967, Lecture Notes in Computer Science, Limerick, Ireland, Proc. Ninth Int'l Conf. of Z Users, Springer-Verlag, June 1995.