The density algorithm for pairwise interaction testing

Software Testing Verification and Reliability

Volumn 17, Issue 3, 2007, Pages 159-182

  Bryce, Renée C ^a   Colbourn, Charles J ^b  

^a University of Nevada Las Vegas   (United States)

^b Arizona State University   (United States)

Author keywords

Covering array; Mixed level covering array; One row at a time greedy algorithm; Pairwise coverage; Software interaction testing

Indexed keywords

ALGORITHMS; SOFTWARE RELIABILITY; STATISTICAL METHODS; VERIFICATION;

DENSITY ALGORITHM; MIXED-LEVEL COVERING ARRAY; ONE-ROW-AT-A-TIME GREEDY ALGORITHM; PAIRWISE COVERAGE; SOFTWARE INTERACTION TESTING;

SOFTWARE TESTING;

EID: 34548434817     PISSN: 09600833     EISSN: 10991689     Source Type: Journal    
DOI: 10.1002/stvr.365     Document Type: Article

References (44)

1. Cohen DM, Dalal SR, Fredman ML, Patton GC. The AETG system: An approach to testing based on combinatorial design. IEEE Transactions on Software Engineering 1997; 23(7):437-444.
2. Dunietz IS, Ehrlich WK, Szablak BD, Mallows CL, Iannino A. Applying design of experiments to software testing. Proceedings of the International Conference on Software Engineering (ICSE '97), Boston, MA, 1997. ACM Press: New York, 1997; 205-215.
3. Williams AW, Probert RL. A practical strategy for testing pair-wise coverage of network interfaces. Proceedings of the the 7th International Symposium on Software Reliability Engineering, San Jose, CA, 2000. IEEE Computer Society Press: Piscataway, NJ, 2000; 246-254.
4. Yu L, Tai KC. In-parameter-order: A test generation strategy for pairwise testing. Proceedings of the 3rd IEEE International High-Assurance Systems Engineering Symposium, Washington, DC, 1998. IEEE Computer Society Press: Piscataway, NJ, 1998; 254-261.
5. Dalal SR, Karunanithi AJN, Leaton JML, Patton GCP, Horowitz BM. Model-based testing in practice. Proceedings of the International Conference on Software Engineering (ICSE '99), Los Angeles, CA, 1999. ACM Press: New York, 1999; 285-294.
6. Berling T, Runeson P. Efficient evaluation of multifactor dependent system performance using fractional factorial design. IEEE Transactions on Software Engineering 2003; 29(9):769-781.
7. Burr K, Young W. Combinatorial test techniques: Table-based automation, test generation and code coverage. Proceedings of the International Conference on Software Testing Analysis and Review, Orlando, FL, 1998; 503-513.
8. Lazić LJ, Velgevid D. Applying simulation and design of experiments to the embedded software testing process. Software Testing, Verification and Reliability 2004; 14:257-282.
9. Kuhn D, Reilly M. An investigation of the applicability of design of experiments to software testing. Proceedings of the 27th Annual NASA Goddard/IEEE Software Engineering Workshop, Greenbelt, MD, 2002. IEEE Computer Society Press: Piscataway, NJ, 2002; 91-95.
10. Kuhn R, Wallace D, Gallo A. Software fault interactions and implications for software testing. IEEE Transactions of Software Engineering 2004; 30(6):418-421.
11. Wong WE, Horgan JR, London S, Mathur AP. Effect of test set minimization on fault detection effectiveness. Proceedings of the 17th IEEE International Conference on Software Engineering, Limerick, Ireland, 2000. ACM Press: New York, 2000; 41-50.
12. Memon AM, Soffa ML. Regression testing of GUIs. Proceedings of the 9th European Software Engineering Conference (ESEC) and 11th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE-11), 1-5 September 2003. ACM Press: New York, 2003; 118-127.
13. White L. Regressing testing of GUI event interaction. Proceedings of the International Conference on Software Maintenance, Monterey, CA, 1996. IEEE Computer Society Press: Piscataway, NJ, 1996; 350-358.
14. White L, Almezen H. Generating test cases for GUI responsibilities using complete interaction sequences. Proceedings of the Interactional Symposium on Software Reliability Engineering, San Jose, CA, 2000. IEEE Computer Society Press: Piscataway, NJ, 2000.
15. Yilmaz C, Cohen MB, Porter A. Covering arrays for efficient fault characterization in complex configuration spaces. IEEE Transactions on Software Engineering 2006; 31(1):20-34.
16. Mandl R. Orthogonal Latin squares an application of experiment design to compiler testing. Communications of the ACM 1985; 28(10):1054-1058.
17. Berry RF. Computer bench mark evaluation and design of experiments a case study. Proceedings of the IEEE Wireless Communications Networking Conference (WCNC03). IEEE Transactions on Computers 1992; 41(10):1279-1289.
18. Chateauneuf MA, Kreher DL. On the state of strength-three covering arrays. Journal of Combinatorial Designs 2002; 10(4):217-238.
19. Moura L, Stardom J, Stevens B, Williams A. Covering arrays with mixed alphabet sizes. Journal of Combinatorial Designs 2003; 11(6):113-132.
20. Stardom J. Metaheuristics and the search for covering and packing arrays. Master's Thesis, Simon Fraser University, 2001.
21. Williams AW, Probert RL. A measure for component interaction test coverage. Proceedings of the ACS/IEEE International Conference on Computer Systems and Applications, Beirut, Lebanon, 2000. ACS/IEEE, 2000; 301-311.
22. Bryce R, Colbourn CJ. Prioritized interaction testing for pairwise coverage with seeding and avoids. Information and Software Technology Journal 2006; 48(10):960-970.
23. Cohen MB, Colbourn CJ, Ling ACH. Augmented simulated annealing to build interaction test suites. Proceedings of the IEEE International Symposium on Software Reliability Engineering (ISSRE 2003), White Plains, NJ, 2003. Computer Society Press: Piscataway, NJ, 2003; 394-405.
24. Cohen MB, Colbourn CJ, Ling ACH. Constructing strength 3 covering arrays with augmented annealing. Discrete Mathematics, to appear.
25. Hartman A. Software and hardware testing using combinatorial covering suites. Interdisciplinary Applications of Graph Theory, Combinatorics, and Algorithms, Golumbic M (ed.). Springer: Berlin, 2005; 237-266.
26. Hartman A, Raskin L. Problems and algorithms for covering arrays. Discrete Mathematics 2004; 284:149-156.
27. Hedayat A, Sloane N, Stufken J. Orthogonal Arrays. Springer: New York, 1999.
28. Sloane N. Covering arrays and intersecting codes. Journal of Combinatorial Designs 1993; 1(1):51-63.
29. Cohen MB, Colbourn CJ, Collofello JS, Gibbons PB, Mugridge WB. Variable Strength Interaction Testing of Components. Proceedings of the 27th International Computer Software and Applications Conference (COMPSAC 2003), Dallas, TX, 2003. IEEE Computer Society Press: Piscataway, NJ, 2003; 413-418.
30. Cohen MB, Colbourn CJ, Gibbons PB, Mugridge WB. Constructing test suites for interaction testing. Proceedings of the International Conference on Software Engineering (ICSE 2003), Portland, OR, 2003. ACM Press: New York, 2003; 38-48.
31. Nurmela K. Upper bounds for covering arrays by tabu search. Discrete Applied Mathematics 2004; 138:143-152.
32. Cheng C, Dumitrescu A, Schroeder P. Generating small combinatorial test suites to cover input-output relationships. Proceedings of the 3rd International Conference on Quality Software (QSIC '03), Dallas, TX, 2003. IEEE Computer Society Press: Piscataway, NJ, 2003; 76-82.
33. Dumitrescu A. Efficient algorithms for generation of combinatorial covering suites. Proceedings of the 14th Annual International Symposium on Algorithms and Computation (ISAAC '03), 2003 (Lecture Notes in Computer Science, vol. 2906). Springer: Berlin, 2003; 300-308.
34. Kobayashi N, Tsuchiya T, Kikuno T. A new method for constructing pair-wise covering designs for software testing. Information Processing Letters 2002; 8:85-91.
35. Bryce RC, Colbourn CJ, Cohen MB. A framework of greedy methods for constructing interaction tests. Proceedings of the 27th International Conference on Software Engineering (ICSE), St. Louis, MO, 2005. ACM Press: New York, 2005; 146-155.
36. Colbourn CJ, Cohen MB, Turban RC. A deterministic density algorithm for pairwise interaction coverage. Proceedings of the IASTED International Conference on Software Engineering, Vienna, Austria, 2004. IASTED, 2004; 245-252.
37. Tung YW, Aldiwan WS. Automating test case generation for the new generation mission software system. Proceedings of the IEEE Aerospace Conference, Big Sky, MT, 2000. IEEE Computer Society Press: Piscataway, NJ, 2000; 431-437.
38. Cohen DM, Dalal SR, Parelius J, Patton GC. The combinatorial design approach to automatic test generation. IEEE Software 1996; 13(5):83-88.
39. Tai KC, Yu L. A test generation strategy for pairwise testing. IEEE Transactions on Software Engineering 2002; 28(1):109-111.
40. Williams AW. Determination of test configurations for pair-wise interaction coverage. Proceedings of Testing of Communicating Systems: Tools and Techniques, IFIP TC6/WG6.1 13th International Conference on Testing Communicating Systems (TestCom 2000), Ottawa, Canada, 2000; 59-74.
41. Colbourn CJ. Combinatorial aspects of covering arrays. Le Matematiche (Catania) 2004; 58:121-167.
42. Seroussi G, Bsbouty N. Vector sets for exhaustive testing of logic circuits. IEEE Transactions on Information Theory 1988; 34:513-522.
43. Cohen DM, Dalal SR, Fredman ML, Patton GC. Method and system for automatically generating efficient test cases for systems having interacting elements. United States Patent Number 5,542,043, 1996.
44. Montgomery DC. Design and Analysis of Experiments (5th edn). Wiley: New York, 2001.