Using test oracles generated from program documentation

IEEE Transactions on Software Engineering

Volumn 24, Issue 3, 1998, Pages 161-173

  Peters, Dennis K ^a   Parnas, David Lorge ^a  

^a MCMASTER UNIVERSITY   (Canada)

Author keywords

Finite state machine; Formal specification; Program testing; Test oracle

Indexed keywords

ALGORITHMS; CODES (SYMBOLS); COMPUTER AIDED SOFTWARE ENGINEERING; ERROR DETECTION; FINITE AUTOMATA; FORMAL LOGIC; PROGRAM DOCUMENTATION; SEQUENTIAL MACHINES;

PROGRAM TESTING; TEST ORACLES;

COMPUTER SOFTWARE SELECTION AND EVALUATION;

EID: 0032022243     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/32.667877     Document Type: Article

References (37)

1. R.F. Abraham, "Evaluating Generalized Tabular Expressions in Software Documentation," MEng thesis, Dept. of Electrical and Computer Engineering, McMaster Univ., Hamilton, Ontario, Canada, also printed as CRL346, Telecommunications Research Inst. of Ontario, Feb. 1997.
2. ADL Language Reference Manual for ANSI C Programmers, Release 1.1. Sun Microsystems Inc., Document Reference MITI/0002/D/R1.1, Dec. 1996.
3. S. Antoy and R. Hamlet, "Objects that Check Themselves Against Formal Specifications," TR 91-1, Dept. of Computer Science, Portland State Univ. School of Engineering and Applied Sciences, Oregon, 1991.
4. B.J. Bauer, "Documenting Complicated Programs," MEng thesis, Dept. of Electrical and Computer Engineering, McMaster Univ., Ontario, also printed as CRL316, Telecommunications Research Inst. of Ontario, Dec. 1995.
5. G. Bernot, M.C. Gaudel, and B. Marre, "Software Testing Based on Formal Specifications: A Theory and A Tool," Software Eng. J., vol. 6, pp. 387-405, June 1990.
6. D. Chapman, "A Program Testing Assistant," Comm. ACM, vol. 25, no. 9, pp. 625-634, Sept. 1982.
7. S.R. Faulk, J. Brackett, P. Ward, and J. Kirby Jr., "The CORE Method for Real-Time Requirements," IEEE Software, vol. 9, no. 6, pp. 22-33, Sept. 1992.
8. J. Gannon, P. McMullin, and R. Hamlet, "Data-Abstraction Implementation, Specification, and Testing," ACM Trans. Programming Languages and Systems, vol. 3, no. 3, pp. 211-223, July 1981.
9. R. Hamlet, "Testing Programs with the Aid of a Compiler," IEEE Trans. Software Eng., vol. 3, no. 4, pp. 279-290, July 1977.
10. E.C.R. Hehner, "Predicative Programming Part 1," Comm. ACM, vol. 27, no. 2, pp. 134-143, Feb. 1984.
11. K. Heninger, "Specifying Software Requirements for Complex Systems: New Techniques and their Applications," IEEE Trans. Software Eng, vol. 6, no. 1, pp. 2-13, Jan. 1980.
12. D.N. Hoover and Z. Chen, "Tablewise, A Decision Table Tool," Proc. Ninth Conf. Computer Assurance, COMPASS '95, June 1995.
13. W.E. Howden, Functional Program Testing and Analysis. McGraw-Hill, 1987.
14. H.M. Hörcher, "Improving Software Tests Using Z Specifications," ZUM '95: The Z Formal Specification Notation, Lecture Notes in Computer Science 967, J.P. Bowen and M.G. Hinchey, eds., pp. 152-166. Springer-Verlag, 1995.
15. M.S. Jaffe, N.G. Leveson, M.P.E. Heimdahl, and B.E. Melhart, "Software Requirements Analysis for Real-Time Process-Control Systems," IEEE Trans. Software Eng., vol. 17, no. 3, pp. 241-258, Mar. 1991.
16. R. Janicki, "Towards a Formal Semantics of Parnas Tables," Proc. 17th Int'l Conf. Software Eng., ICSE, pp. 231-240, Apr. 1995.
17. R. Janicki, D.L. Parnas, and J. Zucker, "Tabular Representations in Relational Documents," Relational Methods in Computer Science-Advances in Computing Science, C. Brink, W. Kahl, and G. Schmidt, eds., pp. 184-196. New York: Springer Wien, 1997.
18. D.C. Luckham, F.W. von Henke, B. Krieg-Brückner, and O. Owe, ANNA A Language for Annotating Ada Programs Reference Manual, Lecture Notes in Computer Science 260, G. Goos and J. Hartmanis, eds. Springer-Verlag, 1987.
19. E. Mikk, "Compilation of Z Specifications into C for Automatic Test Result Evaluation," ZUM '95: The Z Formal Specification Notation, Lecture Notes in Computer Science 967, J.P. Bowen and M.G. Hinchey, eds., pp. 167-180. Springer-Verlag, 1995.
20. D.J. Panzl, "Automatic Software Test Drivers," Computer, pp. 44-50, Apr. 1978.
21. D.L. Parnas, "A Generalized Control Structure and Its Formal Definition," Comm. ACM, vol. 26, no. 8, pp. 572-581, Aug. 1983.
22. D.L. Parnas and Y. Wang, "The Trace Assertion Method of Module Interface Specification," TR89-261, Queen's Univ., Telecommunications Research Inst. of Ontario, TRIO, Oct. 1989.
23. D.L. Parnas and J. Madey, "Functional Documentation for Computer Systems," Science of Computer Programming. Elsevier-Science, vol. 25, no. 1, pp. 41-61, Oct. 1995.
24. D.L. Parnas, J. Madey, and M. Iglewski, "Precise Documentation of Well-Structured Programs, IEEE Trans. Software Eng., vol. 20, no. 12, pp. 948-976, Dec. 1994.
25. D.L. Parnas, "Tabular Representation of Relations," CRL260, Telecommunications Research Inst. of Ontario, Nov. 1992.
26. D.L. Parnas, "Predicate Logic for Software Engineering," IEEE Trans. Software Eng., vol. 19, no. 9, pp. 856-862, Sept. 1993.
27. D.L. Parnas, "Mathematical Description and Specification of Software," Proc. IFIP Congress, vol. I, pp. 354-359, North Holland, Aug. 1994.
28. D.L. Parnas, "A Logic for Describing, not Verifying, Software," Erkenntnis, vol. 43, no. 3, pp. 321-338, Kluwer, Nov. 1995.
29. D.K. Peters, "Shortest Path Algorithm: Formal Program Documentation," CRL280, Telecommunications Research Inst. of Ontario, Feb. 1996, to appear.
30. D.K. Peters, "Generating a Test Oracle from Program Documentation," MEng thesis, Dept. of Electrical and Computer Eng., McMaster Univ., Hamilton, Ontario, also printed as CRL302, Telecommunications Research Inst. of Ontario, Apr. 1995.
31. D.J. Richardson, S.L. Aha, and T.O. O'Malley, "Specification-Based Test Oracles for Reactive Systems," Proc. Int'l Conf. Software Eng., ICSE, pp. 105-118, May 1992.
32. D.S. Rosenblum, "A Practical Approach to Programming with Assertions," IEEE Trans. Software Eng., vol. 21, no. 1, pp. 19-31, Jan. 1995.
33. S. Sankar and R. Hayes, "Specifying and Testing Software Components Using ADL," SMLI TR-94-23. Sun Microsystems Laboratories Inc., Mountain View, Calif., Apr. 1994.
34. Software Engineering Research Group, "Table Tool System Developer's Guide," CRL339, Telecommunications Research Inst. Of Ontario, Jan. 1997.
35. P. Stocks and D. Carrington, "Test Template Framework: A Specification-Based Testing Case Study," Proc. Int'l Symp. Software Testing and Analysis, ISSTA, pp. 11-18, June 1993.
36. Y. Wang, "Specifying and Simulating the Externally Observable Behavior of Modules," PhD thesis, Dept. of Computing and Information Science, Queen's Univ., Kingston, Ontario, Canada, 1994.
37. E.J. Weyuker, "On Testing Nontestable Programs," The Computer J., vol. 25, no. 4, pp. 465-470, 1982.