Threats to the validity and value of empirical assessments of the accuracy of coverage-based fault locators

2013 International Symposium on Software Testing and Analysis, ISSTA 2013 - Proceedings

Volumn , Issue , 2013, Pages 314-324

  Steimann, Friedrich ^a   Frenkel, Marcus ^a   Abreu, Rui ^b  

^a UNIVERSITY OF HAGEN   (Germany)

^b UNIVERSITY OF PORTO   (Portugal)

Author keywords

Fault localization; Threats to validity

Indexed keywords

COVERAGE-BASED FAULT LOCALIZATIONS; EMPIRICAL ASSESSMENT; FAULT LOCALIZATION; FAULT LOCATOR; LOWER AND UPPER BOUNDS; LOWER BOUNDS; THREATS TO VALIDITY;

SOFTWARE TESTING;

EID: 84881283059     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2483760.2483767     Document Type: Conference Paper

References (32)

1. R Abreu, P Zoeteweij, AJC van Gemund "An evaluation of similarity coefficients for software fault localization" in: Proc. of PRDC (2006) 39-46.
2. R Abreu, P Zoeteweij, R Golsteijn, AJC van Gemund "A practical evaluation of spectrum-based fault localization" Journal of Systems and Software 82:11 (2009) 1780-1792.
3. R Abreu, P Zoeteweij, AJC van Gemund "Spectrum-based multiple fault localization" in: Proc. of ASE (2009) 88-99.
4. S Ali, JH Andrews, T Dhandapani, W Wang "Evaluating the accuracy of fault localization techniques" in: Proc. of ASE (2009) 76-87.
5. JH Andrews, LC Briand, Y Labiche "Is mutation an appropriate tool for testing experiments?" in: Proc. of ICSE (2005) 402-411.
6. GK Baah, A Podgurski, MJ Harrold "Mitigating the confounding effects of program dependences for effective fault localization" in: Proc. of SIGSOFT FSE (2011) 146-156.
7. J Bentley More Programming Pearls: Confessions of a Coder (Addison Wesley 1988).
8. MY Chen, E Kiciman, E Fratkin, A Fox, A., EA Brewer "Pinpoint: Problem determination in large, dynamic internet services" in: Proc. of DSN (2002) 595-604.
9. H Cleve, A Zeller "Locating causes of program failures" in: Proc. of ICSE (2005) 342-351.
10. V Dallmeier, C Lindig, A Zeller "Lightweight defect localization for Java" in: Proc. of ECOOP (2005) 528-550.
11. N DiGiuseppe, JA Jones "On the influence of multiple faults on coverage-based fault localization" in: Proc. of ISSTA (2011) 210-220.
12. G Fraser, A Arcuri "Sound empirical evidence in software testing" in: Proc. of ICSE (2012) 178-188.
13. A Gonzalez-Sanchez, R Abreu, HG Gross, AJC von Gemund "Prioritizing tests for fault localization through ambiguity group reduction" in: Proc. of ASE (2011) 83-92.
14. R Gore, PF Reynolds Jr. "Reducing confounding bias in predicate-level statistical debugging metrics" in: Proc. of ICSE (2012) 463-473.
15. JA Jones, MJ Harrold, JT Stasko "Visualization of test information to assist fault localization" in: Proc. of ICSE (2002) 467-477.
16. JA Jones, MJ Harrold "Empirical evaluation of the tarantula automatic fault-localization technique" in: Proc. of ASE (2005) 273-282.
17. JA Jones, MJ Harrold, JF Bowring "Debugging in parallel" in: Proc. of ISSTA (2007) 16-26.
18. JA Jones Semi-Automatic Fault Localization PhD Thesis (Georgia Institute of Technology, 2008).
19. L Naish, H Jie L, K Ramamohanarao "A model for spectra-based software diagnosis" ACM Trans. Softw. Eng. Methodol. 20(3):11 (2011).
20. AS Namin, S Kakarla "The use of mutation in testing experiments and its sensitivity to external threats" in: Proc. of ISSTA (2011) 342-352.
21. C Parnin, A Orso "Are automated debugging techniques actually helping programmers?" in: Proc of ISSTA (2011) 199-209.
22. M Renieris, SP Reiss "Fault localization with nearest neighbor queries" in: Proc. of ASE (2003) 30-39.
23. D Saff "Theory-infected: or how I learned to stop worrying and love universal quantification" in: OOPSLA Companion (2007) 846-847.
24. RA Santelices, JA Jones, Y Yu, MJ Harrold "Lightweight fault-localization using multiple coverage types" in: Proc. of ICSE (2009) 56-66.
25. F Steimann, T Eichstädt-Engelen, M Schaaf "Towards raising the failure of unit tests to the level of compiler-reported errors" in: Proc. of TOOLS Europe (2008) 60-79.
26. F Steimann, M Bertschler "A simple coverage-based locator for mul-tiple faults" in: Proc of ICST (2009) 366-375.
27. F Steimann, M Frenkel "Improving coverage-based localization of multiple faults using algorithms from Integer Linear Programming" in: Proc. of ISSRE (2012).
28. X Wang, SC Cheung, WK Chan, Z Zhang "Taming coincidental correctness: Coverage refinement with context patterns to improve fault localization" in: Proc. of ICSE (2009) 45-55.
29. X Xie, T Chen, FC Kuo, B Xu "A theoretical analysis of the risk evaluation formulas for spectrum-based fault Localization" ACM TOSEM (in print).
30. X Xu, V Debroy, WE Wong, D Guo "Ties within fault localization rankings: Exposing and addressing the problem" International Journal of Software Engineering and Knowledge Engineering 21:6 (2011) 803-827.
31. S Yoo, M Harman, D Clark "Fault localization prioritization: Comparing information theoretic and coverage based approaches" ACM TOSEM (in press).
32. Z Zhang, WK Chan, TH Tse, YT Yu, P Hu "Non-parametric statistical fault localization" Journal of Systems and Software 84:6 (2011) 885-905.