Evaluation of t-wise approach for testing logical expressions in software

Proceedings - IEEE 6th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2013

Volumn , Issue , 2013, Pages 249-256

  Vilkomir, Sergiy ^a   Starov, Oleksii ^a   Bhambroo, Ranjan ^a  

^a EAST CAROLINA UNIVERSITY   (United States)

Author keywords

effectiveness; experimental evaluation; logical expressions; pair wise; t wise; testing

Indexed keywords

EFFECTIVENESS; EXPERIMENTAL EVALUATION; LOGICAL EXPRESSIONS; PAIR-WISE; T-WISE;

TESTING;

SOFTWARE TESTING;

EID: 84883323844     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICSTW.2013.83     Document Type: Conference Paper

References (39)

1. Y. Lei and K. C. Tai, "In-parameter-order: a test generation strategy for pairwise testing, " Proc. 3rd IEEE Int. High-Assurance Systems Engineering Symp. , IEEE Press, Nov. 1998, pp. 254-261, doi: 10. 1109/HASE. 1998. 731623.
2. D. R. Kuhn, Y. Lei, and R. Kacker, "Practical Combinatorial Testing-Beyond Pairwise, " IEEE IT Professional, Jun. 2008, pp. 19-23.
3. J. R. Maximoff, M. D. Trela, D. R. Kuhn, and R. Kacker, "A Method for Analyzing System State-space Coverage within a t-Wise Testing Framework, " IEEE International Systems Conference 2010, Apr. 4-11, 2010, San Diego.
4. M. Grindal, J. Offutt, and S. F. Andler, "Combination testing strategies: a survey, " Software Testing, Verification and Reliability, vol. 15, no. 3, Mar. 2005, pp. 167-199, doi: 10. 1002/stvr. 319.
5. S. A. Vilkomir, K. Kapoor, and J. P. Bowen, "Tolerance of controlflow testing criteria, " Proc. 27th Annu. Int. Computer Software and Applications Conf. (COMPSAC 2003), IEEE Press, Nov. 2003, pp. 182-187, doi: 10. 1109/CMPSAC. 2003. 1245339.
6. P. Thevenod-Fosse, H. Waeselynck, and Y. Crouzet, "An experimental study on software structural testing: deterministic versus random input generation, " Proc. 21st Int. Symp. Fault-Tolerant Computing (FTCS 91), IEEE Press, Jun. 1991, pp. 410-417, doi: 10. 1109/FTCS. 1991. 146694.
7. I. Ciupa, A. Pretschner, A. Leitner, M. Oriol, and B. Meyer, "On the predictability of random tests for object-oriented software, " Proc. 1st Int. Conf. Software Testing, Verification, and Validation (ICST 08), IEEE Press, Apr. 2008, pp. 72-81, doi: 10. 1109/ICST. 2008. 20.
8. Y. T. Yu and M. F. Lau, "Comparing several coverage criteria for detecting faults in logical decisions, " Proc. 4th Int. Conf. Quality Software (QSIC 04), IEEE Press, Sep. 2004, pp. 14-21, doi: 10. 1109/QSIC. 2004. 1357940.
9. J. J. Chilenski and S. P. Miller, "Applicability of modified condition/decision coverage to software testing, " Software Engineering Journal, vol. 9, no. 5, Sep. 1994, pp. 193-200.
10. S. A. Vilkomir and J. P. Bowen, "From MC/DC to RC/DC: formalization and analysis of control-flow testing criteria, " Formal Aspects of Computing, vol. 18, no. 1, Mar. 2006, pp. 42-62, doi: 10. 1007/s00165-005-0084- 7.
11. K. C. Tai and H. K. Su, "Test generation for boolean expressions, " Proc. COMPSAC '87, 1987, pp. 278-283.
12. Y. Yu, M. Lau, and T. Chen, "Automatic generation of test cases from Boolean specifications using the MUMCUT strategy, " Journal of Systems and Software, vol. 79, no. 6, Jun. 2006, pp. 820-840, doi:10. 1016/j. jss. 2005. 08. 016.
13. A. Dupuy and N. Leveson, "An empirical evaluation of the MC/DC coverage criterion on the HETE-2 satellite software, " Proc. 19th Digital Avionics Systems Conf. (DASC 00), IEEE Press, Oct. 2000, pp. 1B6/1-1B6/7, doi: 10. 1109/DASC. 2000. 886883.
14. M. P. E. Heimdahl, M. W. Whalen, A. Rajan, and M. Staats, "On MC/DC and implementation structure: An empirical study, " Proc. 27th IEEE/AIAA Digital Avionics Systems Conference (DASC 2008), 26-30 October 2008, St. Paul, Minnesota, USA, pp. 5. B. 3-1-5. B. 3-13, doi: 10. 1109/DASC. 2008. 4702848.
15. J. J. Chilenski, "An Investigation of Three Forms of the Modified Condition Decision Coverage (MCDC) Criterion, " DOT/FAA/AR-01/18 Technical Report, 2001.
16. E. Weyuker, T. Goradia, and A. Singh, "Automatically generating test data from a Boolean specification, " IEEE Trans. Software Engineering, vol. 20, no. 5, May 1994, pp. 353-363, doi: 10. 1109/32. 286420.
17. M. A. Vouk, K. C. Tai, and A. Paradkar, "Empirical studies of predicate-based software testing, ", Proc. 5th International Symposium on Software Reliability Engineering, Nov. 6-9, 1994, Monterey, California, USA, pp. 55-64, doi: 10. 1109/ISSRE. 1994. 341348
18. Y. Yu and M. Lau, "A comparison of MC/DC, MUMCUT and several other coverage criteria for logical decisions, " Journal of Systems and Software, vol. 79, no. 5, May 2006, pp. 577-590, doi: 10. 1016/j. jss. 2005. 05. 030.
19. D. R. Kuhn, "Fault classes and error detection capability of specification-based testing, " ACM Trans. Software Engineering and Methodology (TOSEM), vol. 8, no. 4, Oct. 1999, pp. 411-424, doi: 10. 1145/322993. 322996.
20. D. J. Richardson and M. C. Thompson, "The RELAY model of error detection and its application, " Proc. 2nd Workshop Software Testing, Verification, and Analysis, IEEE Press, Jul. 1988, pp. 223-230, doi: 10. 1109/WST. 1988. 5378.
21. D. J. Richardson and M. C. Thompson, "An analysis of test data selection criteria using the RELAY model of fault detection, " IEEE Trans. Software Engineering, vol. 19, no. 6, Jun 1993, pp. 533-553, doi: 10. 1109/32. 232020.
22. R. Mandl, "Orthogonal Latin Squares: an Application of Experiment Design to Compiler Testing, " Commun. ACM 28(10), 1985, pp. 1054-1058.
23. P. Schroeder, P. Bolaki, and V. Gopu, "Comparing the fault detection effectiveness of n-way and random test suites, " Proc. 2004 Int. Symp. Empirical Software Engineering (ISESE 04), IEEE Press, Aug. 2004, pp. 49-59, doi: 10. 1109/ISESE. 2004. 1334893.
24. D. R. Kuhn, R. Kacker, and Y. Lei, "Random vs. Combinatorial Methods for Discrete Event Simulation of a Grid Computer Network, " Proc. Mod Sim World 2009, NASA CP-2010-216205, Oct. 14-17 2009, Virginia Beach, USA, pp. 83-88.
25. D. M. Cohen, S. R. Dalal, M. L. Fredman, and G. C. Patton, "The AETG system: an approach to testing based on combinatorial design, " IEEE Trans. Software Engineering, vol. 23, no. 7, Jul. 1997, pp. 437-444, doi: 10. 1109/32. 605761.
26. D. M. Cohen, S. R. Dalal, J. Parelius, and G. C. Patton, "The combinatorial design approach to automatic test generation, " IEEE Software, vol. 13, no. 5, Sep. 1996, pp. 83-88, 10. 1109/52. 536462.
27. M. Ellims, D. Ince, and M. Petre, "The Effectiveness of T-Way Test Data Generation, " Proc. SAFECOMP 2008, 22-25 September 2008, Newcastle upon Tyne, UK, Springer LNCS 5219, pp. 16-29.
28. V. Hu, D. R. Kuhn, and T. Xie, "Property Verification for Generic Access Control Models, " Proc. IEEE/IFIP International Symposium on Trust, Security, and Privacy for Pervasive Applications, Shanghai, China, Dec. 17-20, 2008.
29. W. Ballance, W. Jenkins, and S. Vilkomir, "Probabilistic assessment of effectiveness of software testing for safety-critical systems, " Proc. 10th Int. Probabilistic Safety Assessment and Management Conf. (PSAM 10), 7-11 June 2010, Seattle, Washington, USA.
30. W. Ballance, S. Vilkomir, and W. Jenkins, "Effectiveness of Pairwise Testing for Software with Boolean Inputs, " Proc. Fifth International Conference on Software Testing, Verification and Validation (ICST 2012), April 17-21, 2012, Workshop on Combinatorial Testing (CT-2012), Montreal, Canada, pp. 580-585.
31. N. Kobayashi, T. Tsuchiya, and T. Kikuno, "Non-specification-based approaches to logic testing for software, " Information and Software Technology, vol. 44, no. 2, Feb. 2002, pp. 113-121, doi: 10. 1016/S0950-5849(01)00222-1.
32. N. G. Leveson, M. P. E. Heimdahl, H. Hildreth, and J. D. Reese, "Requirements specification for process-control systems, " IEEE Trans. Software Engineering, vol. 20, no. 9, Sep. 1994, pp. 684-707, doi: 10. 1109/32. 317428.
33. W. Jenkins, S. Vilkomir, and W. Ballance, "Fault Evaluator: a tool for experimental investigation of effectiveness in software testing, " Proc. 2010 IEEE Int. Conf. Progress in Informatics and Computing (PIC 2010), IEEE Press, Dec. 2010, pp. 1077-1083, doi: 10. 1109/PIC. 2010. 5688000.
34. J. Bach, "ALLPAIRS Test Case Generation Tool (Version 1. 2. 1), " Accessed on Jan. 2013, http://www. satisfice. com/tools/pairs. zip.
35. A. Williams, J. H. Lo, and A. Lareau, "TConfig, " Accessed on Jan. 2013, http://www. site. uottawa. ca/~awilliam/TConfig. jar.
36. NIST, "ACTS tool, " Accessed on Jan. 2013, http://csrc. nist. gov/groups/SNS/acts/.
37. Y. Lei, R. Kacker, D. R. Kuhn, V. Okun, and J. Lawrence, "IPOG: A general strategy for t-way software testing, " Proc. 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS '07), Tucson, AZ, USA, March 26-29, 2007, pp. 549-556.
38. R. Bryce, Y. Lei, D. R. Kuhn, and R. Kacker, "Combinatorial Testing, " Chap. 14, Handbook of Research on Software Engineering and Productivity Technologies: Implications of Globalization, Ramachandran, ed. , IGI Global, 2009, pp. 196-208.
39. R. C. Bryce, A. Rajan, and M. P. E. Heimdahl, "Interaction Testing in Model-Based Development: Effect on Model-Coverage, " Proc. 13th Asia-Pacific Software Engineering Conference (APSEC 2006), 6-8 Dec. 2006, Bangalore, India, pp. 259-268, doi: 10. 1109/APSEC. 2006. 42.