Genetic algorithms for randomized unit testing

IEEE Transactions on Software Engineering

Volumn 37, Issue 1, 2011, Pages 80-94

  Andrews, James H ^a   Menzies, Tim ^b   Li, Felix C H ^a  

^a UNIVERSITY OF WESTERN ONTARIO   (Canada)

^b West Virginia University   (United States)

Author keywords

feature subset selection; genetic algorithms; randomized testing; search based optimization; Software testing; testing tools

Indexed keywords

FEATURE SUBSET SELECTION; META-HEURISTIC SEARCH; RANDOMIZED TESTING; RELATIVE FREQUENCIES; SEARCH-BASED OPTIMIZATION; TEST COVERAGE; TESTING SOFTWARE; TESTING TOOLS; UNIT TESTING;

COMPUTER SOFTWARE SELECTION AND EVALUATION; FEATURE EXTRACTION; OPTIMIZATION; SOFTWARE ENGINEERING; SOFTWARE TESTING;

GENETIC ALGORITHMS;

EID: 79551507207     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2010.46     Document Type: Article

References (49)

1. B.P. Miller, L. Fredriksen, and B. So, "An Empirical Study of the Reliability of UNIX Utilities," Comm. ACM, vol. 33, no. 12, pp. 32-44, Dec. 1990.
2. J.H. Andrews, S. Haldar, Y. Lei, and C.H.F. Li, "Tool Support for Randomized Unit Testing," Proc. First Int'l Workshop Randomized Testing, pp. 36-45, July 2006.
3. C. Pacheco, S.K. Lahiri, M.D. Ernst, and T. Ball, "Feedback-Directed Random Test Generation," Proc. 29th Int'l Conf. Software Eng., pp. 75-84, May 2007.
4. A. Groce, G.J. Holzmann, and R. Joshi, "Randomized Differential Testing as a Prelude to Formal Verification," Proc. 29th Int'l Conf. Software Eng., pp. 621-631, May 2007.
5. C.C. Michael, G. McGraw, and M.A. Schatz, "Generating Software Test Data by Evolution," IEEE Trans. Software Eng., vol. 27, no. 12, pp. 1085-1110, Dec. 2001.
6. W. Visser, C.S. Pasareanu, and R. Pelánek, "Test Input Generation for Java Containers Using State Matching," Proc. Int'l Symp. Software Testing and Analysis, pp. 37-48, July 2006.
7. R.-K. Doong and P.G. Frankl, "The ASTOOT Approach to Testing Object-Oriented Programs," ACM Trans. Software Eng. and Methodology, vol. 3, no. 2, pp. 101-130, Apr. 1994.
8. S. Antoy and R.G. Hamlet, "Automatically Checking an Implementation against its Formal Specification," IEEE Trans. Software Eng., vol. 26, no. 1, pp. 55-69, Jan. 2000.
9. K. Claessen and J. Hughes, "QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs," Proc. Fifth ACM SIGPLAN Int'l Conf. Functional Programming, pp. 268-279, Sept. 2000.
10. K. Sen, D. Marinov, and G. Agha, "CUTE: A Concolic Unit Testing Engine for C," Proc. 13th ACM SIGSOFT Int'l Symp. Foundations of Software Eng., pp. 263-272, Sept. 2005.
11. J. Andrews, F. Li, and T. Menzies, "Nighthawk: A Two-Level Genetic-Random Unit Test Data Generator," Proc. 22nd IEEE/ACM Int'l Conf. Automated Software Eng., http://menzies.us/pdf/07asenighthawk. pdf, 2007.
12. J. Andrews and T. Menzies, "On the Value of Combining Feature Subset Selection with Genetic Algorithms: Faster Learning of Coverage Models," Proc. Fifth Int'l Conf. Predictor Models in Software Eng., http://menzies.us/ pdf/09fssga.pdf, 2009.
13. Program Test Methods, W.C. Hetzel, ed., Prentice-Hall, 1973.
14. R. Hamlet, "Random Testing," Encyclopedia of Software Eng., Wiley, pp. 970-978, 1994.
15. E.J. Weyuker, "On Testing Non-Testable Programs," The Computer J., vol. 25, no. 4, pp. 465-470, Nov. 1982.
16. C. Csallner and Y. Smaragdakis, "JCrasher: An Automatic Robustness Tester for Java," Software Practice and Experience, vol. 34, no. 11, pp. 1025-1050, 2004.
17. I. Ciupa, A. Leitner, M. Oriol, and B. Meyer, "Artoo: Adaptive Random Testing for Object-Oriented Software," Proc. 30th ACM/ IEEE Int'l Conf. Software Eng., pp. 71-80, May 2008.
18. M.D. Ernst, J. Cockrell, W.G. Griswold, and D. Notkin, "Dynamically Discovering Likely Program Invariants to Support Program Evolution," IEEE Trans. Software Eng., vol. 27, no. 2, pp. 99-123, Feb. 2001.
19. T. Ball, "A Theory of Predicate-Complete Test Coverage and Generation," Proc. Third Int'l Symp. Formal Methods for Components and Objects, pp. 1-22, Nov. 2004.
20. J.H. Andrews and Y. Zhang, "General Test Result Checking with Log File Analysis," IEEE Trans. Software Eng., vol. 29, no. 7, pp. 634-648, July 2003.
21. L.A. Clarke, "A System to Generate Test Data and Symbolically Execute Programs," IEEE Trans. Software Eng., vol. 2, no. 3, pp. 215-222, Sept. 1976.
22. J.C. King, "Symbolic Execution and Program Testing," Comm. ACM, vol. 19, no. 7, pp. 385-394, 1976.
23. B. Korel, "Automated Software Test Generation," IEEE Trans. Software Eng., vol. 16, no. 8, pp. 870-879, Aug. 1990.
24. N. Gupta, A.P. Mathur, and M.L. Soffa, "Automated Test Data Generation Using an Iterative Relaxation Method," Proc. Sixth Int'l Symp. Foundations of Software Eng., pp. 224-232, Nov. 1998.
25. W.K. Leow, S.C. Khoo, and Y. Sun, "Automated Generation of Test Programs from Closed Specifications of Classes and Test Cases," Proc. 26th Int'l Conf. Software Eng., pp. 96-105, May 2004.
26. W. Visser, C.S. Pasareanu, and S. Khurshid, "Test Input Generation with Java PathFinder," Proc. ACM/SIGSOFT Int'l Symp. Software Testing and Analysis, pp. 97-107, July 2004.
27. P. Godefroid, N. Klarlund, and K. Sen, "DART: Directed Automated Random Testing," Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, pp. 213-223, June 2005.
28. D. Owen and T. Menzies, "Lurch: A Lightweight Alternative to Model Checking," Proc. 15th Int'l Conf. Software Eng. and Knowledge Eng., pp. 158-165, July 2003.
29. J.H. Holland, Adaptation in Natural and Artificial Systems. Univ. of Michigan Press, 1975.
30. D.E. Goldberg, Genetic Algorithm in Search, Optimization, and Machine Learning. Addison-Wesley, 1989.
31. L. Rela, "Evolutionary Computing in Search-Based Software Engineering," master's thesis, Lappeenranta Univ. of Technology, 2004.
32. R.P. Pargas, M.J. Harrold, and R.R. Peck, "Test-Data Generation Using Genetic Algorithms," J. Software Testing, Verification and Reliability, vol. 9, pp. 263-282, Dec. 1999.
33. Q. Guo, R.M. Hierons, M. Harman, and K. Derderian, "Computing Unique Input/Output Sequences Using Genetic Algorithms," Proc. Third Int'l Workshop Formal Approaches to Testing of Software, pp. 164-177, 2004.
34. P. Tonella, "Evolutionary Testing of Classes," Proc. ACM/ SIGSOFT Int'l Symp. Software Testing and Analysis, pp. 119-128, July 2004.
35. A. Watkins and E.M. Hufnagel, "Evolutionary Test Data Generation: A Comparison of Fitness Functions," Software Practice and Experience, vol. 36, pp. 95-116, Jan. 2006.
36. P.G. Frankl and S.N. Weiss, "An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing," IEEE Trans. Software Eng., vol. 19, no. 8, pp. 774-787, Aug. 1993.
37. A. Siami Namin and J. Andrews, "The Influence of Size and Coverage on Test Suite Effectiveness," Proc. Int'l Symp. Software Testing and Analysis, pp. 57-68, 2009.
38. G. Rothermel, M.J. Harrold, J. Ostrin, and C. Hong, "An Empirical Study of the Effects of Minimization on the Fault Detection Capabilities of Test Suites," Proc. Int'l Conf. Software Maintenance, pp. 34-43, Nov. 1998.
39. D.R. Cok, "Adapting JML to Generic Types and Java 1.6," Proc. Seventh Int'l Workshop Specification and Verification of Component-Based Systems, pp. 27-34, Nov. 2008.
40. K. Havelund and T. Pressburger, "Model Checking Java Programs Using Java PathFinder," Int'l J. Software Tools for Technology Transfer, vol. 2, no. 4, pp. 366-381, 2000.
41. K.A. DeJong and W.M. Spears, "An Analysis of the Interacting Roles of Population Size and Crossover in Genetic Algorithms," Proc. First Workshop Parallel Problem Solving from Nature, pp. 38-47, 1990.
42. Cobertura Development Team "Cobertura Web Site," cobertura. sourceforge.net, Feb. 2007.
43. L. Burdy, Y. Cheon, D.R. Cok, M.D. Ernst, J.R. Kiniry, and G.T. Leavens, "An Overview of JML Tools and Applications," Int'l J. Software Tools for Technology Transfer, vol. 7, no. 3, pp. 212-232, June 2005.
44. M. Hall and G. Holmes, "Benchmarking Attribute Selection Techniques for Discrete Class Data Mining," IEEE Trans. Knowledge and Data Eng., vol. 15, no. 6, pp. 1437-1447, Nov./Dec. 2003.
45. A. Miller, Subset Selection in Regression, second ed., Chapman & Hall, 2002.
46. K. Kira and L. Rendell, "A Practical Approach to Feature Selection," Proc. Ninth Int'l Conf. Machine Learning, pp. 249-256, 1992.
47. I. Kononenko, "Estimating Attributes: Analysis and Extensions of Relief," Proc. Seventh European Conf. Machine Learning. pp. 171-182, 1994.
48. S. Cornett, "Minimum Acceptable Code Coverage," http:// www.bullseye.com/minimum.html, 2006.
49. S. Berner, R. Weber, and R.K. Keller, "Enhancing Software Testing by Judicious use of Code Coverage Information," Proc. 29th Int'l Conf. Software Eng., pp. 612-620, May 2007.