CrowdOracles: Can the crowd solve the oracle problem?

Proceedings - IEEE 6th International Conference on Software Testing, Verification and Validation, ICST 2013

Volumn , Issue , 2013, Pages 342-351

  Pastore, Fabrizio ^a   Mariani, Leonardo ^a   Fraser, Gordon ^b  

^a UNIVERSITY OF MILANO BICOCCA   (Italy)

^b UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

crowd sourcing; test case generation; test oracles

Indexed keywords

CROWD SOURCING; CURRENT BEHAVIORS; GENERIC PROPERTIES; ORACLE PROBLEM; TEST CASE GENERATION; TEST GENERATIONS; TEST ORACLES; VIABLE SOLUTIONS;

SOFTWARE TESTING;

PROBLEM SOLVING;

EID: 84883402341     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICST.2013.13     Document Type: Conference Paper

References (19)

1. N. Tillmann and J. D. Halleux, "Pex: white box test generation for .NET," in Proc. International Conference on Tests and Proofs. Springer-Verlag, 2008, pp. 134-253.
2. P. Godefroid, N. Klarlund, and K. Sen, "DART: directed automated random testing," in Proc. Conference on Programming Language Design and Implementation, 2005, pp. 213-223.
3. G. Fraser and A. Arcuri, "EvoSuite: automatic test suite generation for object-oriented software," in Proc. ACM SIGSOFT Symposium and the European Conference on Foundations of Software Engineering. ACM, 2011, pp. 416-419.
4. C. Pacheco and M. D. Ernst, "Randoop: feedback-directed random testing for Java," in Proc. Object-oriented Programming Systems and Applications, 2007, pp. 815-816.
5. E. Miller and W. E. Howden, Software Testing and Validation Techniques, 2nd ed. IEEE Computer Society, 1981.
6. K. T. Stolee and S. Elbaum, "Exploring the use of crowdsourcing to support empirical studies in software engineering," in Proc. International Symposium on Empirical Software Engineering and Measurement, 2010, pp. 35:1-35:4.
7. J. Howe, "The rise of crowdsourcing," Wired, vol. 14, no. 6, June 2006.
8. Y.-S. Ma, J. Offutt, and Y.-R. Kwon, "MuJava: a mutation system for Java," in Proc. International Conference on Software Engineering, tool demo, 2006, pp. 827-830.
9. A. J. Offutt, A. Lee, G. Rothermel, R. H. Untch, and C. Zapf, "An experimental determination of sufficient mutant operators," ACM Transactions on Software Engineering and Methodology, vol. 5, no. 2, pp. 99-118, 1996.
10. T. Xie, "Augmenting automatically generated unit-test suites with regression oracle checking," in Proc. European Conference on Object-Oriented Programming, 2006, pp. 380-403.
11. G. Fraser and A. Zeller, "Mutation-driven generation of unit tests and oracles," IEEE Transactions on Software Engineering, vol. 38, no. 2, pp. 278-292, 2011.
12. R. Evans and A. Savoia, "Differential testing: a new approach to change detection," in Proc. European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering, 2007, pp. 549-552.
13. K. Taneja and T. Xie, "DiffGen: Automated regression unittest generation," in Proc. IEEE/ACM International Conference on Automated Software Engineering, 2008, pp. 407-410.
14. C. Pacheco and M. D. Ernst, "Eclat: Automatic generation and classification of test inputs," in Proc. European Conference on Object-Oriented Programming, 2005, pp. 504-527.
15. B. Daniel, V. Jagannath, D. Dig, and D. Marinov, "ReAssert: Suggesting repairs for broken unit tests," in Proc. Int. Conference on Automated Software Engineering, 2009, pp. 433-444.
16. M. J. Franklin, D. Kossmann, T. Kraska, S. Ramesh, and R. Xin, "CrowdDB: answering queries with crowdsourcing," in Proc. Int. Conf. on Management of Data, 2011, pp. 61-72.
17. J. Heer and M. Bostock, "Crowdsourcing graphical perception: using mechanical turk to assess visualization design," in Proc. International Conference on Human Factors in Computing Systems, 2010, pp. 203-212.
18. J. J. Horton and L. B. Chilton, "The labor economics of paid crowdsourcing," in Proc. ACM conference on Electronic commerce. ACM, 2010, pp. 209-218.
19. R. Pandita, X. Xiao, H. Zhong, T. Xie, S. Oney, and A. Paradkar, "Inferring method specifications from natural language API descriptions," in Proc. International Conference on Software Engineering, June 2012, pp. 815-825.