Cross-checking oracles from intrinsic software redundancy

Proceedings - International Conference on Software Engineering

Volumn , Issue 1, 2014, Pages 931-942

  Carzaniga, Antonio ^a   Goffi, Alberto ^a   Gorla, Alessandra ^b   Mattavelli, Andrea ^a   Pezzè, Mauro ^a,c  

^a UNIVERSITY OF LUGANO   (Switzerland)

^b SAARLAND UNIVERSITY   (Germany)

^c UNIVERSITY OF MILANO BICOCCA   (Italy)

Author keywords

oracle generation; Redundancy; test oracles

Indexed keywords

FORMAL SPECIFICATION; SOFTWARE ENGINEERING; SOFTWARE TESTING; SPECIFICATIONS;

AUTOMATIC TEST GENERATION; DECISION PROCEDURE; EXPERIMENTAL EVALUATION; ORACLE GENERATION; SYSTEM UNDER TEST; TEST ORACLES; UNIT TESTS;

REDUNDANCY;

EID: 84993660551     PISSN: 02705257     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2568225.2568287     Document Type: Conference Paper

References (48)

1. P. E. Ammann and J. C. Knight. Data diversity: An approach to software fault tolerance. IEEE Transactions on Computers, 37 (4): 418-425, 1988.
2. J. H. Andrews, L. C. Briand, and Y. Labiche. Is mutation an appropriate tool for testing experiments In ICSE'05: Proceedings of the 27th International Conference on Software Engineering, pages 402-411, 2005.
3. S. Antoy and D. Hamlet. Automatically checking an implementation against its formal specification. IEEE Transaction on Software Engineering, 26 (1): 55-69, 2000.
4. W. Araujo, L. C. Briand, and Y. Labiche. Enabling the runtime assertion checking of concurrent contracts for the Java modeling language. In ICSE'11: Proceedings of the 33rd International Conference on Software Engineering, pages 786-795, 2011.
5. A. Avizienis. The N-version approach to fault-tolerant software. IEEE Transactions on Software Engineering, 11 (12): 1491-1501, 1985.
6. L. Baresi, P. L. Lanzi, and M. Miraz. TestFul: an evolutionary test approach for Java. In ICST'10: Proceedings of the 2010 International Conference on Software Testing Verification and Validation, pages 185-194, 2010.
7. L. Baresi and M. Young. Test oracles. Technical Report CIS-TR-01-02, University of Oregon, Department of Computer and Information Science, 2001.
8. A. Carzaniga, A. Gorla, A. Mattavelli, N. Perino, and M. Pezze. Automatic recovery from runtime failures. In ICSE'13: Proceedings of the 35th International Conference on Software Engineering, pages 782-791, 2013.
9. A. Carzaniga, A. Gorla, N. Perino, and M. Pezze. Automatic workarounds for Web applications. In FSE'10: Proceedings of the 2010 Foundations of Software Engineering conference, pages 237-246, 2010.
10. A. Carzaniga, A. Gorla, and M. Pezze. Healing Web applications through automatic workarounds. International Journal on Software Tools for Technology Transfer, 10 (6): 492-502, 2008.
11. A. Carzaniga, A. Gorla, and M. Pezze. Handling software faults with redundancy. In R. de Lemos, J.-C. Fabre, C. Gacek, F. Gadducci, and M. H. ter Beek, editors, Architecting Dependable Systems VI, LNCS, pages 148-171. Springer, 2009.
12. T. Y. Chen, S. C. Cheung, and S. Yiu. Metamorphic testing: a new approach for generating next test cases. Technical Report HKUST-CS98-01, Departiment of Computer Science, Hong Kong University of Science and Technology, 1998.
13. T. Y. Chen, F.-C. Kuo, T. H. Tse, and Z. Q. Zhou. Metamorphic testing and beyond. In STEP'03: Proceedings of the 11th International Workshop on Software Technology and Engineering Practice, pages 94-100, 2003.
14. Y. Cheon. Abstraction in assertion-based test oracles. In QSIC'07: Proceedings of the 7th International Conference on Quality Software, pages 410-414, 2007.
15. Y. Cheon and G. T. Leavens. A simple and practical approach to unit testing: The JML and JUnit way. In ECOOP'02: Proceedings of the 16th European Conference on Object-Oriented Programming, pages 231-255, 2002.
16. I. Ciupa, A. Leitner, M. Oriol, and B. Meyer. ARTOO: Adaptive random testing for object-oriented software. In ICSE '08: Proceedings of the 30th International Conference on Software Engineering, pages 71-80, 2008.
17. M. D. Davis and E. J. Weyuker. Pseudo-oracles for non-testable programs. In Proceedings of the ACM '81 Conference, pages 254-257, 1981.
18. J. D. Day and J. D. Gannon. A test oracle based on formal specifications. In SOFTAIR'85: Proceedings of the 2nd Conference on Software development tools, techniques, and alternatives, pages 126-130, 1985.
19. R.-K. Doong and P. G. Frankl. The ASTOOT approach to testing object-oriented programs. ACM Transactions on Software Engineering and Methodology, 3: 101-130, 1994.
20. G. Fraser and A. Zeller. Mutation-driven generation of unit tests and oracles. IEEE Transactions on Software Engineering, 38 (2): 278-292, 2012.
21. S. Fujiwara, G. von Bochmann, F. Khendek, M. Amalou, and A. Ghedamsi. Test selection based on -nite state models. IEEE Transaction on Software Engineering, 17 (6): 591-603, 1991.
22. M. Gabel, L. Jiang, and Z. Su. Scalable detection of semantic clones. In ICSE'08: Proceedings of the 30th international conference on Software engineering, pages 321-330, 2008.
23. J. Gannon, P. McMullin, and R. Hamlet. Data abstraction, implementation, specification, and testing. ACM Transactions on Programming Languages and Systems, 3 (3): 211-223, 1981.
24. A. Gorla. Automatic Workarounds: Exploiting the Intrinsic Redundancy of Software Systems. PhD thesis, University of Lugano, Switzerland, 2011.
25. A. Gotlieb. Exploiting symmetries to test programs. In ISSRE'03: Proceedings of the 14th International Symposium on Software Reliability Engineering, pages 365-375, 2003.
26. F. Gross, G. Fraser, and A. Zeller. EXSYST: search-based GUI testing. In ICSE'12: Proceedings of the 34th International Conference on Software Engineering, pages 1423-1426, 2012.
27. M. Harman, P. McMinn, M. Shahbaz, and S. Yoo. A comprehensive survey of trends in oracles for software testing. Technical Report CS-13-01, University of Sheeld, Department of Computer Science, 2013.
28. M. Hennessy and R. Milner. On observing nondeterminism and concurrency. In Proceedings of the 7th Colloquium on Automata, Languages and Programming, pages 299-309, 1980.
29. L. Jiang and Z. Su. Automatic mining of functionally equivalent code fragments via random testing. In ISSTA'09: Proceedings of the 18th International Symposium on Software Testing and Analysis, 2009.
30. R. Just, F. Schweiggert, and G. M. Kapfhammer. MAJOR: An eficient and extensible tool for mutation analysis in a Java compiler. In ASE'11: Proceedings of the 26th International Conference on Automated Software Engineering, pages 612-615, 2011.
31. K. Kawachiya, K. Ogata, D. Silva, T. Onodera, H. Komatsu, and T. Nakatani. Cloneable JVM: a new approach to start isolated Java applications faster. In VEE'07: Proceedings of the 3rd International Conference on Virtual Execution Environments, pages 1-11, 2007.
32. J. Mcdonald. Translating object-z specifications to passive test oracles. In ICFEM'98: Proceedings of the 1998 International Conference on Formal Engineering Methods, pages 165-174, 1998.
33. B. Meyer. Object-Oriented Software Construction. Prentice Hall, 1st edition, 1988.
34. B. Meyer, I. Ciupa, A. Leitner, and L. L. Liu. Automatic testing of object-oriented software. In SOFSEM'07: Proceedings of the 33rd Conference on Current Trends in Theory and Practice of Computer Science, pages 114-129, 2007.
35. E. Mikk. Compilation of Z specifications into C for automatic test result evaluation. In ZUM'95: Proceedings of the 9th International Conference of Z Users, pages 167-180, 1995.
36. C. Murphy, K. Shen, and G. Kaiser. Automatic system testing of programs without test oracles. In ISSTA'09: Proceedings of the 18th International Symposium on Software Testing and Analysis, pages 189-200, 2009.
37. C. Pacheco and M. D. Ernst. Eclat: Automatic generation and classification of test inputs. In ECOOP'05: Proceedings of the 19th European Conference on Object-Oriented Programming, pages 504-527, 2005.
38. C. Pacheco and M. D. Ernst. Randoop: feedback-directed random testing for Java. In OOPSLA'07: Proceedings of the 22nd Conference on Object-Oriented Programming Systems and Applications, pages 815-816, 2007.
39. C. Pacheco, S. K. Lahiri, M. D. Ernst, and T. Ball. Feedback-directed random test generation. In ICSE'07: Proceedings of the 29th International Conference on Software Engineering, pages 75-84, 2007.
40. F. Pastore, L. Mariani, and G. Fraser. CrowdOracles: Can the crowd solve the oracle problem In ICST'13: Proceedings of the 2013 International Conference on Software Testing Verification and Validation, pages 342-351, 2013.
41. M. Pezze, K. Rubinov, and J. Wuttke. Generating effective integration test cases from unit ones. In ICST'03: Proceedings of the 2013 International Conference on Software Testing, Verification and Validation, 2013.
42. D. S. Rosenblum. A practical approach to programming with assertions. IEEE Transaction on Software Engineering, 21 (1): 19-31, 1995.
43. M. Staats, G. Gay, and M. P. E. Heimdahl. Automated oracle creation support, or: how i learned to stop worrying about fault propagation and love mutation testing. In ICSE'12: Proceedings of the 34th International Conference on Software Engineering, pages 870-880, 2012.
44. R. N. Taylor. An integrated verification and testing environment. Software: Practice and Experience, 13 (8): 697-713, 1983.
45. M. Utting, A. Pretschner, and B. Legeard. A taxonomy of model-based testing approaches. Software Testing, Verification and Reliability, 22 (5): 297-312, 2012.
46. E. J. Weyuker. On testing non-testable programs. The Computer Journal, 25 (4): 465-470, 1982.
47. S. Zhang, D. Sa, Y. Bu, and M. D. Ernst. Combined static and dynamic automated test generation. In ISSTA'11: Proceedings of the 21th International Symposium on Software Testing and Analysis, pages 353-363, 2011.
48. Z. Q. Zhou, S. Zhang, M. Hagenbuchner, T. H. Tse, F.-C. Kuo, and T. Y. Chen. Automated functional testing of online search services. Software Testing, Verification and Reliability, 22 (4): 221-243, 2012.