Automated oracle creation support, or: How I learned to stop worrying about fault propagation and love mutation testing

Proceedings - International Conference on Software Engineering

Volumn , Issue , 2012, Pages 870-880

  Staats, Matt ^a   Gay, Gregory ^b   Heimdahl, Mats P E ^b  

^a Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

^b UNIVERSITY OF MINNESOTA   (United States)

Author keywords

oracle data; oracle selection; test oracles; testing; verification

Indexed keywords

APPLICATION UNDER TESTS; COST-EFFECTIVE APPROACH; EXPECTED VALUES; FAULT FINDING; FAULT PROPAGATION; MUTATION ANALYSIS; MUTATION TESTING; ORACLE DATA; ORACLE SELECTION; OVER CURRENT; TEST INPUTS; TEST ORACLES; TESTING PROCESS;

SOFTWARE TESTING; TESTING; VERIFICATION;

SOFTWARE ENGINEERING;

EID: 84864241599     PISSN: 02705257     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICSE.2012.6227132     Document Type: Conference Paper

References (32)

1. J. Andrews, L. Briand, and Y. Labiche. Is mutation an appropriate tool for testing experiments? Proc of the 27th Int'l Conf on Software Engineering (ICSE), pages 402-411, 2005.
2. L. Baresi and M. Young. Test oracles. In Technical Report CIS-TR-01-02, Dept. of Computer and Information Science, Univ. of Oregon.
3. L. Briand, M. DiPenta, and Y. Labiche. Assessing and improving state-based class testing: A series of experiments. IEEE Trans. on Software Engineering, 30 (11), 2004.
4. J. Chilenski. An investigation of three forms of the modified condition decision coverage (MCDC) criterion. Technical Report DOT/FAA/AR-01/18, Office of Aviation Research, Washington, D.C., April 2001.
5. V. Chvatal. A greedy heuristic for the set-covering problem. Mathematics of operations research, 4(3):233-235, 1979.
6. T. Cormen, C. Leiserson, R. Rivest, and C. Stein. Introduction to Algorithms. MIT Press and McGraw-Hill, 2001.
7. R. DeMillo, R. Lipton, and F. Sayward. Hints on test data selection: Help for the practicing programmer. IEEE computer, 11(4):34-41, 1978.
8. R. Evans and A. Savoia. Differential testing: a new approach to change detection. In Proc. of the 6th Joint European Software Engineering Conference and Foundations of Software Engineering, pages 549-552. ACM, 2007.
9. R. Fisher. The Design of Experiment. New York: Hafner, 1935.
10. G. Fraser and A. Zeller. Mutation-driven generation of unit tests and oracles. In Prc. of the 19th Int'l Symp. on Software Testing and Analysis, pages 147-158. ACM, 2010.
11. G. Fraser and A. Zeller. Generating parameterized unit tests. In Prc. of the 20th Int'l Symp. on Software Testing and Analysis, pages 147-158. ACM, 2011.
12. M. Garey and M. Johnson. Computers and Intractability. Freeman, New York, 1979.
13. A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. Software Engineering Notes, 24(6):146-162, November 1999.
14. N. Halbwachs. Synchronous Programming of Reactive Systems. Klower Academic Press, 1993.
15. M. P. Heimdahl, G. Devaraj, and R. J. Weber. Specification test coverage adequacy criteria = specification test generation inadequacy criteria? In Proc. of the Eighth IEEE Int'l Symp. on High Assurance Systems Engineering (HASE), Tampa, Florida, March 2004.
16. Y. Jia and M. Harman. An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, (99):1, 2010.
17. P. Kvam and B. Vidakovic. Nonparametric Statistics with Applications to Science and Engineering. Wiley-Interscience, 2007.
18. Mathworks Inc. Simulink product web site. http://www.mathworks.com/ products/simulink.
19. The NuSMV Toolset: Users Manual, 2005. Available at http://nusmv.fbk.eu/.
20. C. Pacheco and M. Ernst. Eclat: Automatic generation and classification of test inputs. ECOOP 2005-Object-Oriented Programming, pages 504-527, 2005.
21. A. Rajan, M. Whalen, and M. Heimdahl. The Effect of Program and Model Structure on MC/DC Test Adequacy Coverage. In Proceedings of 30th International Conference on Software Engineering (ICSE), 2008. Available at http://crisys.cs.umn.edu/ICSE08.pdf.
22. A. Rajan, M. Whalen, and M. Heimdahl. The effect of program and model structure on MC/DC test adequacy coverage. In Proc. of the 30th Int'l Conference on Software engineering, pages 161-170. ACM New York, NY, USA, 2008.
23. A. Rajan, M. Whalen, M. Staats, and M. Heimdahl. Requirements coverage as an adequacy measure for conformance testing. In Proc. of the 10th Int'l Conf. on Formal Methods and Software Engineering, pages 86-104. Springer, 2008.
24. S. Rayadurgam and M. P. Heimdahl. Coverage based test-case generation using model checkers. In Proc. of the 8th IEEE Int'l. Conf. and Workshop on the Engineering of Computer Based Systems, pages 83-91. IEEE Computer Society, April 2001.
25. D. J. Richardson, S. L. Aha, and T. O'Malley. Specification-based test oracles for reactive systems. In Proc. of the 14th Int'l Conference on Software Engineering, pages 105-118. Springer, May 1992.
26. RTCA. DO-178B: Software Considerations In Airborne Systems and Equipment Certification. RTCA, 1992.
27. M. Staats, M. Whalen, and M. Heimdahl. New ideas and emerging results track: Better testing through oracle selection. In Proc. of NIER Workshop, Int'l. Conf. on Software Engineering (ICSE) 2011. ACM New York, NY, USA, 2011.
28. M. Staats, M. Whalen, and M. Heimdahl. Programs, testing, and oracles: The foundations of testing revisited. In Proc. of Int'l. Conf. on Software Engineering (ICSE) 2011. ACM New York, NY, USA, 2011.
29. K. Taneja and T. Xie. Diffgen: Automated regression unit-test generation. In Automated Software Engineering, 2008, pages 407-410. IEEE, 2008.
30. C. Van Eijk. Sequential equivalence checking based on structural similarities. Computer-Aided Design of Integrated Circuits and Systems, IEEE Trans. on, 19(7):814-819, 2002.
31. J. Voas and K. Miller. Putting assertions in their place. In Software Reliability Engineering, 1994., 5th Int'l Symposium on, pages 152-157, 1994.
32. Q. Xie and A. Memon. Designing and comparing automated test oracles for gui-based software applications. ACM Trans. on Software Engineering and Methodology (TOSEM), 16(1):4, 2007.