Design and implementation of a t-way test data generation strategy with automated execution tool support

Information Sciences

Volumn 181, Issue 9, 2011, Pages 1741-1758

  Zamli, Kamal Z ^a   Klaib, Mohammad F J ^a   Younis, Mohammed I ^a   Isa, Nor Ashidi Mat ^a   Abdullah, Rusli ^b  

^a SCHOOL OF PHYSICS   (Malaysia)

^b UNIVERSITY PUTRA MALAYSIA   (Malaysia)

Author keywords

Combinatorial testing; GTWay; Software testing; t way testing

Indexed keywords

COMBINATORIAL EXPLOSION; COMBINATORIAL TESTING; COST FACTORS; EFFICIENT STRATEGY; EXHAUSTIVE TESTING; EXHAUSTIVE TESTS; GTWAY; INPUT DATAS; NP COMPLETE PROBLEMS; OPTIMAL TEST SET; PARAMETER INTERACTIONS; PLANNING PROCESS; RESOURCE CONSTRAINT; SAMPLING STRATEGIES; SOFTWARE PRODUCTS; T-WAY TESTING; TEST DATA; TEST DATA GENERATION; TEST PLANNING; TEST SETS; TEST SIZE; TIME-TO-MARKET; TOOL SUPPORT; VARIOUS CONFIGURATION;

AUTOMATION; COMPUTATIONAL COMPLEXITY; COMPUTER SOFTWARE SELECTION AND EVALUATION; DATA COMMUNICATION SYSTEMS; OPTIMIZATION; SOFTWARE RELIABILITY; SOFTWARE TESTING;

TESTING;

EID: 79952310321     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2011.01.002     Document Type: Article

References (62)

1. J. Arshem, TVG. (last accessed 24.10.10).
2. J. Bach, AllPairs Test Generation Tool, Version 1.2.1. (last accessed 24.10.10).
3. Brigham-Young-University, Logic Design. (last accessed 24.10.10).
4. R. Bryce, and C.J. Colbourn Prioritized interaction testing for pairwise coverage with seeding and avoids Information and Software Technology Journal 48 10 2006 960 970 (Pubitemid 44354089)
5. R. Bryce, C.J. Colbourn, M.B. Cohen, A framework of greedy methods for constructing interaction tests, in: Proceedings of the 27th International Conference on Software Engineering, St. Louis, MO, USA, 2005, pp. 146-155.
6. R.C. Bryce, and C.J. Colbourn A density-based greedy algorithm for higher strength covering arrays Software Testing, Verification, and Reliability 19 1 2009 37 53
7. K.A. Bush Orthogonal Arrays of Index Unity The Annals of Mathematical Statistics 23 3 1952 426 434
8. K.-Y. Cai, Z. Dong, and K. Liu Software testing processes as a linear dynamic system Information Sciences 178 6 2008 1558 1597
9. K.-Y. Cai, and B.-B. Yin Software execution processes as an evolving complex network Information Sciences 179 12 2009 1903 1928
10. C. Catal, and B. Diri Investigating the effect of dataset size, metrics sets, and feature selection techniques on software fault prediction problem Information Sciences 179 8 2009 1040 1058
11. D.M. Cohen, S.R. Dalal, M.L. Fredman, and G.C. Patton The AETG system: an approach to testing based on combinatorial design IEEE Transactions on Software Engineering 23 7 1997 437 444 (Pubitemid 127828305)
12. D.M. Cohen, S.R. Dalal, A. Kajla, and G.C. Patton The automatic efficient test generator (AETG) system Proceedings of the 5th International Symposium on Software Reliability Engineering 1994 IEEE Computer Society Monterey, CA, USA 303 309
13. D.M. Cohen, S.R. Dalal, J. Parelius, and G.C. Patton The combinatorial design approach to automatic test generation IEEE Software 13 5 1996 83 88
14. M.B. Cohen, Designing Test Suites for Software Interaction Testing, Ph.D. Thesis, Computer Science, University of Auckland, 2004.
15. M.B. Cohen, C.J. Colbourn, and A.C.H. Ling Constructing strength three covering arrays with augmented annealing Discrete Mathematics 308 13 2008 2709 2722 (Pubitemid 351474499)
16. M.B. Cohen, M.B. Dwyer, and J. Shi Coverage and adequacy in software product line testing Proceedings of the ISSTA Workshop Role of Software Architecture for Testing and Analysis 2006 ACM Portland, Maine, USA 53 63 (Pubitemid 46645442)
17. M.B. Cohen, M.B. Dwyer, and J. Shi Exploiting Constraint Solving History to Construct Interaction Test Suites Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques (MUTATION 2007) 2007 IEEE Computer Society UK 121 132
18. M.B. Cohen, M.B. Dwyer, and J. Shi Interaction testing of highly-configurable systems in the presence of constraints Proceedings of the 2007 International Symposium on Software Testing and Analysis 2007 ACM London, UK 129 139 (Pubitemid 47317587)
19. M.B. Cohen, P.B. Gibbons, W.B. Mugridge, C.J. Colbourn, Constructing test suites for interaction testing, in: Proceedings of the 25th International Conference on Software Engineering, Portland, Oregon USA, 2003, pp. 38-48.
20. C.J. Colbourn, M.B. Cohen, R.C. Turban, A deterministic density algorithm for pairwise interaction coverage, in: Proceedings of the IASTED International Conference on Software Engineering, vol. 17, Innsbruck, Austria, 2004, pp. 345-352.
21. G.T. Daich, Testing combinations of parameters made easy [software testing], in: Proceedings of the IEEE Systems Readiness Technology Conference (AUTOTESTCON 2003), 2003, pp. 379-384.
22. Z. Ding, K. Zhang, and J. Hu A rigorous approach towards test case generation Information Sciences 178 21 2008 4057 4079
23. I.S. Dunietz, W.K. Ehrlich, B.D. Szablak, C.L. Mallows, and A. Iannino Applying design of experiments to software testing Proceedings of the International Conference on Software Engineering (ICSE '97) 1997 ACM Press New York, Boston, MA 205 215
24. M. Forbes, J. Lawrence, Y. Lei, R.N. Kacker, and D.R. Kuhn Refining the in-parameter-order strategy for constructing covering arrays NIST Journal of Research 113 5 2008 287 297
25. M. Grindal, J. Offutt, J. Mellin, Conflict Management when using combination strategies for software testing, in: Proceedings of 18th Australian Software Engineering Conference, Melbourne, Australia, 2007, pp. 255-264. (Pubitemid 47296931)
26. A. Hartman, T. Klinger, L. Raskin, IBM Intelligent Test Case Handler. (last accessed 24.10.10.
27. B. Hnich, S.D. Prestwich, E. Selensky, and B.M. Smith Constraint models for the covering test problem Constraints 11 2-3 2006 199 219
28. B. Jenkins, Jenny. (last accessed 24.10.10.
29. M.F.J. Klaib, K.Z. Zamli, N.A.M. Isa, M.I. Younis, R. Abdullah, G2Way - a backtracking strategy for pairwise test data generation, in: Proceedings of the 15th IEEE Asia-Pacific Software Engineering Conference, Beijing, China, 2008, pp. 463-470.
30. R. Krishnan, S.M. Krishna, and P.S. Nandhan Combinatorial testing: learnings from our experience ACM SIGSOFT Software Engineering Notes 32 3 2007 1 8
31. D.R. Kuhn, Y. Lei, and R. Kacker Practical combinatorial testing: beyond pairwise, IT professional IEEE Computer Society 10 3 2008 19 23 (Pubitemid 351715255)
32. D.R. Kuhn, V. Okun, Pseudo exhaustive testing for software, in: Proceeding of the 30th NASA/IEEE Software Engineering Workshop, 2006, pp. 25-27.
33. R. Kuhn, Y. Lei, and R. Kacker Practical combinatorial testing: beyond pairwise IEEE IT Professional 10 3 2008 19 23 (Pubitemid 351715255)
34. E. Lehmann, J. Wegener, Test case design by means of the CTE XL, in: Proceedings of the 8th European International Conference on Software Testing, Analysis & Review (EuroSTAR 2000), Copenhagen, Denmark, 2000, pp. 1-10.
35. Y. Lei, R. Kacker, D. Kuhn, V. Okun, and J. Lawrence IPOG/IPOD: efficient test generation for multi-way software testing Journal of Software Testing, Verification, and Reliability 18 2008 125 148
36. Y. Lei, R. Kacker, D.R. Kuhn, V. Okun, J. Lawrence, IPOG: a general strategy for t-way software testing, in: Proceedings of the 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems, Tucson, AZ, USA, 2007, pp. 549-556. (Pubitemid 46900712)
37. Y. Lei, K.C. Tai, In-parameter-order: a test generation strategy for pairwise testing, in: Proceedings of the 3rd IEEE International High-Assurance Systems Engineering Symposium, Washington, DC, USA, 1998, pp. 254-261.
38. Y. Ma, J. Offutt, and Y. Kwon MuJava: An Automated Class Mutation System Journal of Software Testing, Verification and Reliability 15 2 2005 97 133 (Pubitemid 40858060)
39. R. Mandl, Orthogonal Latin Squares: An Application of Experiment Design to Compiler Testing, Communications of the ACM, vol. 28, New York, NY, USA, 1985, pp. 1054-1058. (Pubitemid 15587965)
40. M.M. Mano Digital Design third ed. 2002 Prentice Hall Inc.
41. C. Nie, B. Xu, L. Shi, and G. Dong Quality of software architectures and software quality-automatic test generation for N-way combinatorial testing Lecture Notes in Computer Science 3712 2005 203 211
42. P.J. Schroeder, P. Faherty, B. Korel, Generating expected results for automated black-box testing, in: Proceedings of the 17th IEEE International Conference on Automated Software Engineering (ASE'02), Edinburgh, UK, 2002, pp. 139-148.
43. P.J. Schroeder, E. Kim, J. Arshem, P. Bolaki, Combining behavior and data modeling in automated test case generation, in: Proceedings of the 3rd International Conference on Quality Software (Qsic 2003), 2003, pp. 247-254.
44. P.J. Schroeder, B. Korel, Black-box test reduction using input-output analysis, in: Proceedings of the International Symposium on Software Testing and Analysis (ISSTA 2000), Portland, OR, USA, 2000, pp. 21-24.
45. T. Shiba, T. Tsuchiya, T. Kikuno, Using artificial life techniques to generate test cases for combinatorial testing, in: Proceedings of the 28th Annual International Computer Software and Applications Conference (COMPSAC'04), IEEE Computer Society, Hong Kong, 2004, pp. 72-77. (Pubitemid 40673832)
46. N.J.A. Sloane, Home page. (last accessed 24.10.10).
47. Smartware, SmartTest - Pairwise Testing tool. (last accessed 24.10.10).
48. B. Stevens, E. Mendelsohn, Efficient software testing protocols, in: Proceedings of the 8th IBM Centre for Advanced Studies Conference (CASCON '98), Toronto, ON, 1998, pp. 279-293.
49. K.C. Tai, and Y. Lei A test generation strategy for pairwise testing IEEE Transactions on Software Engineering 28 1 2002 109 111 (Pubitemid 34129838)
50. Y.-W. Tung, W.S. Aldiwan, Automating test case generation for the new generation mission software system, in: Proceedings of the Aerospace Conference, vol. 1, Big Sky, MT, USA, 2000, pp. 431-437. (Pubitemid 32838334)
51. Vlad-Roubtsov, EMMA: a free Java code coverage tool. (last accessed 24.10.10.
52. A.W. Williams, TConfig. (last accessed 24.10.10).
53. A.W. Williams, Determination of test configurations for pair-wise interaction coverage, in: Proceedings of the 13th International Conference on Testing of Communicating Systems, 2000, pp. 57-74.
54. A.W. Williams, Software Component Interaction Testing: Coverage Measurement and Generation of Configurations. Ph.D. Thesis, School of Information Technology and Engineering, University of Ottawa, 2002.
55. A.W. Williams, R.L. Probert, A measure for component interaction test coverage, in: Proceedings of the ACSI/IEEE International Conference on Computer Systems and Applications (AICCSA 2001), IEEE Computer Society, Beirut, Lebanon, 2001, pp. 304-311.
56. A.W. Williams, R.L. Probert, A practical strategy for testing pair-wise coverage of network interfaces, in: Proceedings of the 7th International Symposium on Software Reliability Engineering, 1996, pp. 246-254.
57. J. Yan, J. Zhang, Backtracking algorithms and search heuristics to generate test suites for combinatorial testing, in: Proceedings of the 30th Annual International Computer Software and Applications Conference (COMPSAC'06), vol. 1, Chicago, 2006, pp. 385-394.
58. J. Yan, and J. Zhang A backtracking search tool for constructing combinatorial test suites Journal of Systems and Software 81 10 2008 1681 1693
59. M.I. Younis, K.Z. Zamli, Assessing combinatorial interaction strategy for reverse engineering of combinational circuits, in: Proceedings of the IEEE Symposium on Industrial Electronics and Applications (ISIEA2009), Kuala Lumpur, Malaysia, 2009, pp. 473-478.
60. M.I. Younis, K.Z. Zamli, and N.A.M. Isa IRPS - an efficient test data generation strategy for pairwise testing Proceedings of the 12th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems (KES2008) 2008 Springer Verlag, Zagreb, Croatia 493 500
61. M.I. Younis, K.Z. Zamli, M.F.J. Klaib, Z.C. Soh, S.C. Abdullah, and N.A.M. Isa Assessing IRPS as an efficient pairwise test data generation strategy International Journal of Advanced Intelligence Paradigms (IJAIP) 2 1 2010 90 104
62. K.Z. Zamli, N.A.M. Isa, M.F.J. Klaib, and S.N. Azizan A tool for automated test data generation (and execution) based on combinatorial approach International Journal of Software Engineering and Its Applications 1 1 2007 19 34