Does automated white-box test generation really help software testers?

2013 International Symposium on Software Testing and Analysis, ISSTA 2013 - Proceedings

Volumn , Issue , 2013, Pages 291-301

  Fraser, Gordon ^a   Staats, Matt ^b   McMinn, Phil ^a   Arcuri, Andrea ^c   Padberg, Frank ^d  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

^b Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

^c SIMULA RESEARCH LABORATORY   (Norway)

^d KARLSRUHE INSTITUTE OF TECHNOLOGY   (Germany)

Author keywords

automated test generation; branch coverage; empirical software engineering; Unit testing

Indexed keywords

AUTOMATED TEST GENERATIONS; BRANCH COVERAGE; CONTROLLED EXPERIMENT; EMPIRICAL SOFTWARE ENGINEERING; RESEARCH COMMUNITIES; STRUCTURAL ASPECTS; TEST GENERATIONS; UNIT TESTING;

SOFTWARE ENGINEERING; TOOLS;

SOFTWARE TESTING;

EID: 84881283024     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2483760.2483774     Document Type: Conference Paper

References (33)

1. S. Afshan, P. McMinn, and M. Stevenson. Evolving readable string test inputs using a natural language model to reduce human oracle cost. In Int. Conference on Software Testing, Verification and Validation (ICST), 2013. (To appear).
2. A. Arcuri and L. Briand. A hitchhiker's guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability. (To appear).
3. L. Baresi, P. L. Lanzi, and M. Miraz. Testful: an evolutionary test approach for Java. In IEEE International Conference on Software Testing, Verification and Validation (ICST), pages 185-194, 2010.
4. C. Csallner and Y. Smaragdakis. JCrasher: an automatic robustness tester for Java. Software: Practice and Experience, 34(11):1025-1050, 2004.
5. J. T. de Souza, C. L. Maia, F. G. de Freitas, and D. P. Coutinho. The human competitiveness of search based software engineering. In International Symposium on Search Based Software Engineering (SSBSE), pages 143-152, 2010.
6. G. Fraser and A. Arcuri. Evosuite: Automatic test suite generation for object-oriented software. In ACM Symposium on the Foundations of Software Engineering (FSE), pages 416-419, 2011.
7. G. Fraser and A. Arcuri. Sound empirical evidence in software testing. In ACM/IEEE International Conference on Software Engineering (ICSE), pages 178-188, 2012.
8. G. Fraser and A. Arcuri. Whole test suite generation. IEEE Transactions on Software Engineering, 39(2):276-291, 2013.
9. G. Fraser and A. Zeller. Exploiting common object usage in test case generation. In IEEE International Conference on Software Testing, Verification and Validation (ICST), pages 80-89. IEEE Computer Society, 2011.
10. G. Fraser and A. Zeller. Mutation-driven generation of unit tests and oracles. IEEE Transactions on Software Engineering, 28(2):278-292, 2012.
11. M. Harman, S. Mansouri, and Y. Zhang. Search-based software engineering: Trends, techniques and applications. ACM Computing Surveys (CSUR), 45(1):11, 2012.
12. M. Harman and P. McMinn. A theoretical and empirical study of search based testing: Local, global and hybrid search. IEEE Transactions on Software Engineering, 36(2):226-247, 2010.
13. M. Islam and C. Csallner. Dsc+mock: A test case + mock class generator in support of coding against interfaces. In International Workshop on Dynamic Analysis (WODA), pages 26-31, 2010.
14. R. Just, F. Schweiggert, and G. Kapfhammer. MAJOR: An efficient and extensible tool for mutation analysis in a Java compiler. In International Conference on Automated Software Engineering (ASE), pages 612-615, 2011.
15. B. Kitchenham, S. Pfleeger, L. Pickard, P. Jones, D. Hoaglin, K. El Emam, and J. Rosenberg. Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering, 28(8):721-734, 2002.
16. J. Koza. Human-competitive results produced by genetic programming. Genetic Programming and Evolvable Machines, 11(3):251-284, 2010.
17. K. Lakhotia, P. McMinn, and M. Harman. An empirical investigation into branch coverage for C programs using CUTE and AUSTIN. Journal of Systems and Software, 83(12):2379-2391, 2010.
18. P. McMinn. Search-based software test data generation: A survey. Software Testing, Verification and Reliability, 14(2):105-156, 2004.
19. E. F. Miller, Jr. and R. A. Melton. Automated generation of testcase datasets. In International Conference on Reliable Software, pages 51-58. ACM, 1975.
20. A. Namin and J.H.Andrews. The influence of size and coverage on test suite effectiveness. In ACM International Symposium on Software Testing and Analysis (ISSTA). ACM, 2009.
21. C. Pacheco and M. D. Ernst. Randoop: feedback-directed random testing for Java. In Object-Oriented Programming Systems, Languages, and Applications (OOPSLA), pages 815-816. ACM, 2007.
22. C. Parnin and A. Orso. Are automated debugging techniques actually helping programmers? In ACM International Symposium on Software Testing and Analysis (ISSTA), pages 199-209, 2011.
23. C. Pasareanu and N. Rungta. Symbolic pathfinder: symbolic execution of java bytecode. In IEEE/ACM International Conference on Automated Software Engineering (ASE), volume 10, pages 179-180, 2010.
24. F. Pastore, L. Mariani, and G. Fraser. Crowdoracles: Can the crowd solve the oracle problem? In IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE, 2013. (To appear).
25. R. Ramler, D. Winkler, and M. Schmidt. Random test case generation and manual unit testing: Substitute or complement in retrofitting tests for legacy code? In EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA), pages 286-293. IEEE, 2012.
26. G. Sautter, K. Böhm, F. Padberg, and W. Tichy. Empirical evaluation of semi-automated XML annotation of text documents with the GoldenGATE editor. Research and Advanced Techn. for Digital Libraries, pages 357-367, 2007.
27. C. Seaman. Qualitative methods in empirical studies of software engineering. IEEE Transactions on Software Engineering, 25(4):557-572, 1999. (Pubitemid 129304118)
28. D. Sjoberg, J. Hannay, O. Hansen, V. By Kampenes, A. Karahasanovic, N. Liborg, and A. C Rekdal. A survey of controlled experiments in software engineering. IEEE Transactions on Software Engineering, 31(9):733-753, 2005. (Pubitemid 41555415)
29. M. Staats, G. Gay, and M. Heimdahl. Automated oracle creation support, or: how I learned to stop worrying about fault propagation and love mutation testing. In ACM/IEEE International Conference on Software Engineering (ICSE), pages 870-880, 2012.
30. M. Staats, S. Hong, M. Kim, and G. Rothermel. Understanding user understanding: determining correctness of generated program invariants. In ACM International Symposium on Software Testing and Analysis (ISSTA), pages 188-198. ACM, 2012.
31. N. Tillmann and N. J. de Halleux. Pex- white box test generation for. NET. In International Conference on Tests And Proofs (TAP), pages 134-253, 2008.
32. P. Tonella. Evolutionary testing of classes. In ACM International Symposium on Software Testing and Analysis (ISSTA), pages 119-128, 2004.
33. J. Wegener, A. Baresel, and H. Sthamer. Evolutionary test environment for automatic structural testing. Information and Software Technology, 43(14):841-854, 2001. (Pubitemid 33050964)