Improving the accuracy of oracle verdicts through automated model steering

ASE 2014 - Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering

Volumn , Issue , 2014, Pages 527-537

  Gay, Gregory ^a   Rayadurgam, Sanjai ^a   Heimdahl, Mats P E ^a  

^a UNIVERSITY OF MINNESOTA   (United States)

Author keywords

Model based testing; Software testing; Test oracles

Indexed keywords

AUTOMATION; EMBEDDED SYSTEMS; MODEL CHECKING; REAL TIME SYSTEMS; SOFTWARE ENGINEERING;

ANALYSIS AND SIMULATION; AUTOMATED MODELING; DEVELOPMENT COSTS; MODEL BASED TESTING; REAL-TIME EMBEDDED SYSTEMS; SYSTEM UNDER TEST; TEST ORACLES; TESTING PROCESS;

SOFTWARE TESTING;

EID: 84908635180     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2642937.2642989     Document Type: Conference Paper

References (32)

1. IBM Rational Rhapsody. http://www.ibm.com/developerworks/rational products/rhapsody/, 2014.
2. J. Andrews, L. Briand, and Y. Labiche. Is mutation an appropriate tool for testing experiments? Proc of the 27th Int'l Conf on Software Engineering (ICSE), pages 402-411, 2005.
3. J. Andrews, L. Briand, Y. Labiche, and A. Namin. Using mutation analysis for assessing and comparing testing coverage criteria. Software Engineering, IEEE Transactions on, 32(8):608-624, aug. 2006.
4. A. Arcuri, M. Z. Iqbal, and L. Briand. Black-box system testing of real-time embedded systems using random and search-based testing. In Proceedings of the 22nd IFIP WG 6.1 Int'l Conf. on Testing software and systems, pages 95-110. Springer-Verlag, 2010.
5. L. B. Briones and E. Brinksma. A test generation framework for quiescent real-time systems. In IN FATES'04, pages 64-78. Springer-Verlag GmbH, 2004.
6. S.-H. Cha. Comprehensive survey on distance/similarity measures between probability density functions. International Journal of Mathematical Models and Methods in Applied Sciences, 1(4):300-307, 2007.
7. D. Coppit and J. Haddox-Schatz. On the use of specification-based assertions as test oracles. In Proceedings of the 29th Annual IEEE/NASA on Software Engineering Workshop, SEW'05, pages 305-314, Washington, DC, USA, 2005. IEEE Computer Society.
8. L. De Moura and N. Bjørner. Z3: An efficient smt solver. In Tools and Algorithms for the Construction and Analysis of Systems, pages 337-340. Springer, 2008.
9. A. En-Nouaary, R. Dssouli, and F. Khendek. Timed wp-method: testing real-time systems. Software Engineering, IEEE Transactions on, 28(11):1023-1038, Nov.
10. G. Gay, S. Rayadurgam, and M. P. Heimdahl. Steering model-based oracles to admit real program behaviors. In Proceedings of the 36th International Conference on Software Engineering - NIER Track, ICSE'14, New York, NY, USA, 2014. ACM.
11. A. Gomes, A. Mota, A. Sampaio, F. Ferri, and E. Watanabe. Constructive model-based analysis for safety assessment. Int'l Journal on Software Tools for Technology Transfer, 14(6):673-702, 2012.
12. W. Gu, J. Vetter, and K. Schwan. An annotated bibliography of interactive program steering. ACM SIGPLAN Notices, 29, 1994.
13. G. Hagen. Verifying safety properties of Lustre programs: an SMT-based approach. PhD thesis, University of Iowa, December 2008.
14. N. Halbwachs. Synchronous Programming of Reactive Systems. Klower Academic Press, 1993.
15. D. Harel, H. Lachover, A. Naamad, A. Pnueli, M. Politi, R. Sherman, A. Shtull-Trauring, and M. Trakhtenbrot. Statemate: A working environment for the development of complex reactive systems. IEEE Transactions on Software Engineering, 16(4):403-414, April 1990.
16. M. Harman, P. McMinn, M. Shahbaz, and S. Yoo. A comprehensive survey of trends in oracles for software testing. Technical Report CS-13-01, University of Sheffield, Department of Computer Science, 2013.
17. W. Howden. Theoretical and empirical studies of program testing. IEEE Transactions on Software Engineering, 4(4):293-298, 1978.
18. M. S. Jaffe, N. G. Leveson, M. P. Heimdahl, and B. E. Melhart. Software requirements analysis for real-time process-control systems. IEEE Transactions on Software Engineering, 17(3):241-258, March 1991.
19. S. Kannan, M. Kim, I. Lee, O. Sokolsky, and M. Viswanathan. Run-time monitoring and steering based on formal specifications. In Workshop on Modeling Software System Structures in a Fastly Moving Scenario, 2000.
20. K. G. Larsen, M. Mikucionis, and B. Nielsen. Online testing of real-time systems using UPPAAL. In Int'l workshop on formal approaches to testing of software (FATES 04). Springer, 2004.
21. D. Lee and M. Yannakakis. Principles and methods of testing finite state machines-a survey. Proceedings of the IEEE, 84(8):1090-1123, 1996.
22. MathWorks Inc. Stateflow. http://www.mathworks.com/stateflow.
23. D. Miller, J. Guo, E. Kraemer, and Y. Xiong. On-the-fly calculation and verification of consistent steering transactions. In Supercomputing, ACM/IEEE 2001 Conf., pages 8-8, 2001.
24. S. P. Miller, A. C. Tribble, M. W. Whalen, and M. P. E. Heimdahl. Proving the shalls: Early validation of requirements through formal methods. Int. J. Softw. Tools Technol. Transf., 8(4):303-319, 2006.
25. A. Murugesan, S. Rayadurgam, and M. Heimdahl. Modes, features, and state-based modeling for clarity and flexibility. In Proceedings of the 2013 Workshop on Modeling in Software Engineering, 2013.
26. M. Pezze and M. Young. Software Test and Analysis: Process, Principles, and Techniques. John Wiley and Sons, October 2006.
27. A. Rajan, M. Whalen, and M. Heimdahl. The effect of program and model structure on MC/DC test adequacy coverage. In Proc. of the 30th Int'l Conf. on Software engineering, pages 161-170. ACM, 2008.
28. A. Rajan, M. Whalen, M. Staats, and M. Heimdahl. Requirements coverage as an adequacy measure for conformance testing, 2008.
29. T. Savor and R. Seviora. An approach to automatic detection of software failures in real-time systems. In Real-Time Technology and Applications Symposium, 1997. Proceedings., Third IEEE, pages 136-146, 1997.
30. M. Staats, G. Gay, and M. Heimdahl. Automated oracle creation support, or: how I learned to stop worrying about fault propagation and love mutation testing. In Proceedings of the 2012 Int'l Conf. on Software Engineering, pages 870-880. IEEE Press, 2012.
31. M. Veanes, C. Campbell, W. Grieskamp, W. Schulte, N. Tillmann, and L. Nachmanson. Model-based testing of object-oriented reactive systems with spec explorer. In R. M. Hierons, J. P. Bowen, and M. Harman, editors, Formal Methods and Testing, volume 4949 of Lecture Notes in Computer Science, pages 39-76. Springer, 2008.
32. E. Weyuker. The oracle assumption of program testing.