Integrating techniques and tools for testing automation

Software Testing Verification and Reliability

Volumn 17, Issue 1, 2007, Pages 3-39

  Polo, Macario ^a   Tendero, Sergio ^a   Piattini, Mario ^a  

^a UNIVERSITY OF CASTILLA LA MANCHA   (Spain)

Author keywords

JUnit; Mutation; Testing automation; Testing process

Indexed keywords

AUTOMATION; COMPUTER AIDED SOFTWARE ENGINEERING; COMPUTER PROGRAMMING LANGUAGES; SOFTWARE RELIABILITY;

TEST CASE EXECUTION FRAMEWORKS; TEST CASE QUALITY MEASURERS; TESTING AUTOMATION;

SOFTWARE TESTING;

EID: 33847372433     PISSN: 09600833     EISSN: 10991689     Source Type: Journal    
DOI: 10.1002/stvr.348     Document Type: Article

References (53)

1. Runeson P, Andersson C, Höst M. Test processes in software product evolution: A qualitative survey on the state of practice. Journal of Software Maintenance and Evolution: Research and Practice 2003; 15(1):41-59.
2. Rice RW. Surviving the top 10 challenges of software test automation. CrossTalk: The Journal of Defense Software Engineering 2002; 15(5):26-29.
3. Geras AM, Smith MR, Miller J. A survey of software testing practices in Alberta. Canadian Journal of Electrical and Computer Engineering 2004; 29(3):183-191.
4. Giraudo G, Tonella P. Designing and conducting an empirical study on test management automation. Empirical Software Engineering 2003; 8(1):59-81.
5. Meudec C. ATGen: Automatic test data generation using constraint logic programming and symbolic execution. Software Testing, Verification and Reliability 2001; 11(2):81-96.
6. Ng SP, Murnane T, Reed K, Grant D, Chen TY. A preliminary survey on software testing practices in Australia. Proceedings of the Australian Software Engineering Conference (ASWEC 2004), Melbourne, Australia, April 2004. IEEE Computer Society Press: Los Alamitos, CA, 2004; 116-125.
7. Ball T, Hoffman D, Ruskey F, Webber R, White L. State generation and automated class testing. Software Testing, Verification and Reliability 2000; 10(3):149-170.
8. Hoffman D, Strooper P, Wilkin S. Tool support for executable documentation of Java class hierarchies. Software Testing, Verification and Reliability 2005; 15(4):235-256.
9. Erdogmus H, Morisio M, Torchiano M. On the effectiveness of the test-first approach to programming. IEEE Transactions on Software Engineering 2005; 31(3):226-237.
10. Offutt J, Liu S, Abdurazik A, Ammann P. Generating test data from state-based specifications. Software Testing, Verification and Reliability 2003; 13(1):25-53.
11. Kirani SH, Tsai W-T. Method sequence specification and verification of classes. Journal of Object-Oriented Programming 1994; 7(6):28-38.
12. Grieskamp W, Gurevich Y, Schulte W, Veanes M. Testing with abstract state machines. Formal Methods and Tools for Computer Science: Proceedings of the Workshop on Abstract State Machines (ASM 2001), Las Palmas de Gran Canaria, Canary Islands, February 2001; 257-261.
13. Hong HS, Lee I, Sokolsky O, Cha SD. Automatic test generation from statecharts using model checking. Proceedings of the First International Workshop on Formal Approaches to Testing of Software (FATES 2001), Aalborg, Denmark, August 2001. BRICS: Basic Research in Computer Science, 2001; 15-30.
14. Burton S, Clark J, McDermid J. Automatic generation of tests from statechart specifications. Proceedings of the First International Workshop on Formal Approaches to Testing of Software (FATES 2001), Aalborg, Denmark, August 2001. BRICS: Basic Research in Computer Science, 2001; 31-46.
15. Beck K, Gamma E. Test infected: Programmers love writing tests. Java Report 1998; 3(7):51-56.
16. Beck K. Extreme Programming Explained: Embrace Change. Addison-Wesley: Boston, MA, 2000.
17. Parasoft. Automating and Improving Java Unit Testing: Using JTest with JUnit. http://www.parasoft.com [18 April 2006].
18. SilverMark Inc. SilverMark's Enhanced JUnit Version 3.7. SilverMark Inc., 2002.
19. Oriat C. Jartege: A tool for random generation of unit tests for Java classes. Proceedings of the 2nd International Workshop on Software Quality, Erfurt, Germany, September 2005 (Lecture Notes in Computer Science, vol. 3712). Springer: Berlin, 2005; 242-256.
20. Leavens GT. The Java Modeling Language (JML) home page. http://www.cs.iastate.edu/~leavens/JML [2 December 2005].
21. Cornett S. Code Coverage Analysis. http://www.bullseye.com/ webCoverage.html [15 June 2005].
22. DeMillo RA, Offutt AJ. Experimental results from an automatic test case generator. ACM Transactions on Software Engineering and Methodology 1993; 2(2):109-127.
23. Daniels FJ, Tai KC. Measuring the effectiveness of method test sequences derived from sequencing constraints. Proceedings of Technology of Object-Oriented Languages and Systems (TOOLS 1999), Santa Barbara, CA, August 1999. IEEE Computer Society Press: Los Alamitos, CA, 1999; 74-83.
24. Kim S-W, Clark JA, McDermid JA. Investigating the effectiveness of object-oriented testing strategies using the mutation method. Software Testing, Verification and Reliability 2001; 11(4):207-225.
25. Vincenzi AMR, Maldonado JC, Barbosa EF, Delamaro ME. Unit and integration testing strategies for C programs using mutation. Software Testing, Verification and Reliability 2001; 11(4):249-268.
26. Andrews JH, Briand LC, Labiche Y. Is mutation an appropriate tool for testing experiments? Proceedings of the 27th International Conference on Software Engineering (ICSE 2005), St. Louis, MO, May 2005. ACM Press: New York, 2005; 402-411.
27. Juristo N, Moreno AM, Vegas S. A survey on testing technique empirical studies: How limited is our knowledge? Proceedings of the International Symposium on Empirical Software Engineering (ISESE 2002), Nara, Japan, October 2002. IEEE Computer Society Press: Los Alamitos, CA, 2002; 161-172.
28. Choi B, Mathur AP, Pattison B. PMothra: Scheduling mutants for execution on a hypercube. Proceedings of the ACM SIGSOFT Third Symposium on Software Testing, Analysis, and Verification, Key West, FL, December 1989. ACM Press: New York, 1989; 58-65.
29. Duncan IMM. Strong mutation testing strategies. PhD Thesis, Department of Computer Science, University of Durham, U.K., 1993.
30. Mresa ES, Bottaci L. Efficiency of mutation operators and selective mutation strategies: An empirical study. Software Testing, Verification and Reliability 1999; 9(4):205-232.
31. Weiss SN, Fleyshgakker VN. Improved serial algorithms for mutation analysis. Proceedings of the International Symposium on Software Testing and Analysis (ISSTA 1993), Cambridge, MA, June 1993. ACM Press: New York, 1993; 149-158.
32. Baudry B, Fleurey F, Jézéquel J-M, Le Traon Y. From genetic to bacteriological algorithms for mutation-based testing. Software Testing, Verification and Reliability 2005; 15(2):73-96.
33. Offutt AJ, Rothermel G, Untch RH, Zapf C. An experimental determination of sufficient mutant operators. ACM Transactions on Software Engineering and Methodology 1996; 5(2):99-118.
34. Mathur AP. Performance, effectiveness, and reliability issues in software testing. Proceedings of the 15th Annual International Computer Software and Applications Conference (COMPSAC 1991), Tokyo, Japan, September 1991. IEEE Computer Society Press: Los Alamitos, CA, 1991; 604-605.
35. Wong WE, Mathur AP. Reducing the cost of mutation testing: An empirical study. Journal of Systems and Software 1995; 31(3):185-196.
36. Ma Y-S, Offatt J, Kwon YR. MuJava: An automated class mutation system. Software Testing, Verification and Reliability 2005; 15(2):97-133.
37. Krauser EW, Mathur AP, Rego VJ. High performance software testing on SIMD machines. IEEE Transactions on Software Engineering 1991; 17(5):403-423.
38. Howden WE. Weak mutation testing and completeness of test sets. IEEE Transactions on Software Engineering 1982; 8(4):371-379.
39. Offutt AJ, Lee SD. An empirical evaluation of weak mutation. IEEE Transactions on Software Engineering 1994; 20(5):337-344.
40. Hirayama M, Yamamoto T, Okayasu J, Mizuno O, Kikuno T. Elimination of crucial faults by a new selective testing method. Proceedings of the International Symposium on Empirical Software Engineering (ISESE 2002), Nara, Japan, October 2002. IEEE Computer Society Press: Los Alamitos, CA, 2002; 183-191.
41. Offutt AJ, Pan J. Automatically detecting equivalent mutants and infeasible paths. Software Testing, Verification and Reliability 1997; 7(3):165-192.
42. NUnit. NUnit, testing resources for Extreme Programming. http://www.junit.org [5 January 2003].
43. Myers GJ. The Art of Software Testing. Wiley: New York, 1979.
44. Offutt J. A practical system for mutation testing: Help for the common programmer. Proceedings of the 12th International Conference on Testing Computer Software, Washington, DC, June 1995. IEEE Computer Society Press: Los Alamitos, CA, 1995; 99-109.
45. Chow TS. Testing software design modeled by finite-state machines. IEEE Transactions on Software Engineering 1978; 4(3):178-187.
46. Boyapati C, Khurshid S, Marinov D. Korat: Automated testing based on Java predicates. Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2002), Rome, Italy, July 2002. ACM Press: New York, 2002; 123-133.
47. Tse T, Xu Z. Test case generation for class-level object-oriented testing. Proceedings of the 9th International Software Quality Week, San Francisco, CA, May 1996. Software Research Inc.: San Francisco, CA, 1996; 4T4.0-4T4.12.
48. Korel B, Al-Yami AM. Assertion-oriented automated test data generation. Proceedings of the 18th International Conference on Software Engineering (ICSE 1996), Berlin, Germany, March 1996. IEEE Computer Society Press: Los Alamitos, CA, 1996; 71-80.
49. Tracey N, Clark J, Mander K, McDermid J. Automated test data generation for exception conditions. Software - Practice and Experience 2000; 30(1):61-79.
50. Basili VR, Shull F, Lanubile F. Building knowledge through families of experiments. IEEE Transactions on Software Engineering 1999; 25(4):456-473.
51. Höst M, Regnell B, Wohlin C. Using students as subjects: A comparative study of students and professionals in lead-time impact assessment. Empirical Software Engineering 2000; 5(3):201-204.
52. Kitchenham B, Pfleeger SL, Pickard L, Jones P, Hoaglin D, El-Emam K, Rosenberg J. Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering 2002; 28(8):721-734.
53. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A. Experimentation in Software Engineering: An Introduction. Kluwer Academic: Norwell, MA, 2000.