Efficiency and early fault detection with lower and higher strength combinatorial interaction testing

2013 9th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC/FSE 2013 - Proceedings

Volumn , Issue , 2013, Pages 26-36

  Petke, Justyna ^a   Yoo, Shin ^a   Cohen, Myra B ^b   Harman, Mark ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF NEBRASKA LINCOLN   (United States)

Author keywords

Combinatorial Interaction Testing; Empirical studies; Prioritisation; Software testing

Indexed keywords

COMBINATORIAL INTERACTION TESTING; CONFIGURATION SPACE; EARLY FAULT; EMPIRICAL STUDIES; PRIORITISATION; PRIORITY ORDER; REAL-WORLD; SOFTWARE SYSTEMS;

FAULT DETECTION; SOFTWARE TESTING;

SOFTWARE ENGINEERING;

EID: 84883698011     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2491411.2491436     Document Type: Conference Paper

References (30)

1. R. C. Bryce and C. J. Colbourn. Test prioritization for pairwise interaction coverage. ACM SIGSOFT Software Engineering Notes, 30(4):1-7, 2005.
2. R. C. Bryce and C. J. Colbourn. Prioritized interaction testing for pair-wise coverage with seeding and constraints. Information & Software Technology, 48(10):960-970, 2006.
3. R. C. Bryce and A. M. Memon. Test suite prioritization by interaction coverage. In A. Hartman, M. Katara, and A. M. Paradkar, editors, DOSTA, pages 1-7. ACM, 2007.
4. R. C. Bryce, S. Sampath, and A. M. Memon. Developing a single model and test prioritization strategies for event-driven software. IEEE Trans. Software Eng., 37(1):48-64, 2011.
5. R. C. Bryce, S. Sampath, J. B. Pedersen, and S. Manchester. Test suite prioritization by cost-based combinatorial interaction coverage. Int. J. Systems Assurance Engineering and Management, 2(2):126-134, 2011.
6. D. M. Cohen, S. R. Dalal, M. L. Fredman, and G. C. Patton. The AETG system: an approach to testing based on combinatorial design. IEEE Transactions on Software Engineering, 23(7):437-444, 1997.
7. M. B. Cohen, C. J. Colbourn, P. B. Gibbons, and W. B. Mugridge. Constructing test suites for interaction testing. In Proceedings of the International Conference on Software Engineering, pages 38-48, May 2003.
8. M. B. Cohen, M. B. Dwyer, and J. Shi. Interaction testing of highly-configurable systems in the presence of constraints. In D. S. Rosenblum and S. G. Elbaum, editors, ISSTA, pages 129-139. ACM, 2007.
9. M. B. Cohen, M. B. Dwyer, and J. Shi. Constructing interaction test suites for highly-configurable systems in the presence of constraints: A greedy approach. IEEE Trans. Software Eng., 34(5):633-650, 2008.
10. H. Do, S. G. Elbaum, and G. Rothermel. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering, 10(4):405-435, 2005.
11. S. G. Elbaum, A. G. Malishevsky, and G. Rothermel. Prioritizing test cases for regression testing. In ISSTA, pages 102-112, 2000.
12. B. J. Garvin, M. B. Cohen, and M. B. Dwyer. Evaluating improvements to a meta-heuristic search for constrained interaction testing. Empirical Software Engineering, 16(1):61-102, 2011.
13. M. Grindal, J. Offutt, and J. Mellin. Handling constraints in the input space when using combination strategies for software testing. Technical Report TR-06-001, University of Skövde, 2006.
14. B. Hnich, S. D. Prestwich, E. Selensky, and B. M. Smith. Constraint models for the covering test problem. Constraints, 11(2-3):199-219, 2006.
15. D. Kuhn, R. Kacker, and Y. Lei. Automated combinatorial test methods: Beyond pairwise testing. Crosstalk, Journal of Defense Software Engineering, 21(6), 2008.
16. D. R. Kuhn and V. Okun. Pseudo-exhaustive testing for software. In SEW, pages 153-158. IEEE Computer Society, 2006.
17. D. R. Kuhn, D. R. Wallace, and A. M. Gallo. Software fault interactions and implications for software testing. IEEE Trans. Software Eng., 30(6):418-421, 2004.
18. Y. Lei, R. Kacker, D. R. Kuhn, V. Okun, and J. Lawrence. IPOG/IPOG-D: efficient test generation for multi-way combinatorial testing. Softw. Test., Verif. Reliab., 18(3):125-148, 2008.
19. S. Manchester, N. Samant, R. Bryce, S. Sampath, D. R. Kuhn, and R. Kacker. Applying higher strength combinatorial criteria to test prioritization: a case study. Journal of Combinatorial Mathematics and Combinatorial Computing. To appear.
20. T. Nanba, T. Tsuchiya, and T. Kikuno. Using satisfiability solving for pairwise testing in the presence of constraints. IEICE Transactions, 95-A(9):1501-1505, 2012.
21. C. Nie and H. Leung. A survey of combinatorial testing. ACM Comput. Surv., 43(2):11, 2011.
22. T. J. Ostrand and M. J. Balcer. The category-partition method for specifying and generating functional tests. Commun. ACM, 31(6):676-686, 1988.
23. X. Qu and M. B. Cohen. A study in prioritization for higher strength combinatorial testing. The 2nd International Workshop on Combinatorial Testing, 2013.
24. X. Qu, M. B. Cohen, and G. Rothermel. Configuration-aware regression testing: an empirical study of sampling and prioritization. In Proceedings of the International Symposium On Software Testing and Analysis, pages 75-86, 2008.
25. X. Qu, M. B. Cohen, and K. M. Woolf. Combinatorial interaction regression testing: A study of test case generation and prioritization. In ICSM, pages 255-264. IEEE, 2007.
26. S. Sampath, R. C. Bryce, G. Viswanath, V. Kandimalla, and A. G. Koru. Prioritizing user-session-based test cases for web applications testing. In ICST, pages 141-150. IEEE Computer Society, 2008.
27. P. Schroeder, P. Bolaki, and V. Gopu. Comparing the fault detection effectiveness of n-way and random test suites. In Proceedings of Empirical Software Engineering, pages 49-59, August 2004.
28. L. Shi, C. Nie, and B. Xu. A software debugging method based on pairwise testing. In International Conference on Computational Science (3), pages 1088-1091, 2005.
29. C. Yilmaz, M. B. Cohen, and A. Porter. Covering arrays for efficient fault characterization in complex configuration spaces. IEEE Transactions on Software Engineering, 31(1):20-34, Jan 2006.
30. S. Yoo and M. Harman. Regression testing minimisation, selection and prioritisation: A survey. Software Testing, Verification, and Reliability, 22(2):67-120, March 2012.