How well does test case prioritization integrate with statistical fault localization?

Information and Software Technology

Volumn 54, Issue 7, 2012, Pages 739-758

  Jiang, Bo ^a   Zhang, Zhenyu ^b   Chan, W K ^c   Tse, T H ^d   Chen, Tsong Yueh ^e  

^a BEIHANG UNIVERSITY   (China)

^b INSTITUTE OF SOFTWARE   (China)

^c CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^d UNIVERSITY OF HONG KONG   (Hong Kong)

^e SWINBURNE UNIVERSITY OF TECHNOLOGY   (Australia)

Author keywords

Adaptive random testing; Continuous integration; Coverage; Software process integration; Statistical fault localization; Test case prioritization

Indexed keywords

ADAPTIVE RANDOM TESTING; COVERAGE; FAULT LOCALIZATION; SOFTWARE PROCESS; TEST CASE PRIORITIZATION;

EXPERIMENTS; STATISTICS;

SOFTWARE TESTING;

EID: 84862821558     PISSN: 09505849     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.infsof.2012.01.006     Document Type: Article

References (51)

1. R. Abreu, P. Zoeteweij, R. Golsteijn, and A.J.C. van Gemund A practical evaluation of spectrum-based fault localization Journal of Systems and Software 82 11 2009 1780 1792
2. R. Abreu, P. Zoeteweij, and A.J.C. van Gemund Spectrum-based multiple fault localization Proceedings of the 24th IEEE/ACM International Conference on Automated Software Engineering (ASE 2009) 2009 IEEE Computer Society Press Los Alamitos, CA 88 99
3. B. Baudry, F. Fleurey, and Y. Le Traon Improving test suites for efficient fault localization Proceedings of the 28th International Conference on Software Engineering (ICSE 2006) 2006 ACM Press New York, NY 82 91 (Pubitemid 46600905)
4. th USENIX Conference on Operating Systems Design and Implementation (OSDI 2008) 2008 USENIX Association Berkeley, CA 209 224
5. T.Y. Chen, F.-C. Kuo, R.G. Merkel, and T.H. Tse Adaptive random testing: the ART of test case diversity Journal of Systems and Software 83 1 2010 60 66
6. T.Y. Chen, H. Leung, and I.K. Mak Adaptive random testing M.J. Maher, Advances in Computer Science: Proceedings of the 9th Asian Computing Science Conference (ASIAN 2004) Lecture Notes in Computer Science vol. 3321 2004 Springer Berlin, Germany 320 329 (Pubitemid 39739341)
7. T.Y. Chen, and R.G. Merkel Quasi-random testing IEEE Transactions on Reliability 56 3 2007 562 568 (Pubitemid 47399764)
8. T.Y. Chen, and R.G. Merkel An upper bound on software testing effectiveness ACM Transactions on Software Engineering and Methodology 17 3 2008 1 27
9. P.K. Chittimalli, and M.J. Harrold Recomputing coverage information to assist regression testing IEEE Transactions on Software Engineering 35 4 2009 452 469
10. I. Ciupa, A. Leitner, M. Oriol, and B. Meyer Object distance and its application to adaptive random testing of object-oriented programs Proceedings of the 1st International Workshop on Random Testing (in conjunction with the 2006 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2006)) 2006 ACM Press New York, NY 55 63 (Pubitemid 46645426)
11. I. Ciupa, A. Leitner, M. Oriol, and B. Meyer Experimental assessment of random testing for object-oriented software Proceedings of the 2007 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2007) 2007 ACM Press New York, NY 84 94 (Pubitemid 47317582)
12. I. Ciupa, A. Leitner, M. Oriol, and B. Meyer ARTOO: adaptive random testing for object-oriented software Proceedings of the 30th International Conference on Software Engineering (ICSE 2008) 2008 ACM Press New York, NY 71 80
13. H. Cleve, and A. Zeller Locating causes of program failures Proceedings of the 27th International Conference on Software Engineering (ICSE 2005) 2005 ACM Press New York, NY 342 351
14. H. Do, and G. Rothermel On the use of mutation faults in empirical assessments of test case prioritization techniques IEEE Transactions on Software Engineering 32 9 2006 733 752 (Pubitemid 44604891)
15. P.M. Duvall, S. Matyas, and A. Glover Continuous Integration: Improving Software Quality and Reducing Risk 2007 Addison-Wesley Upper Saddle River, NJ
16. S.G. Elbaum, A.G. Malishevsky, and G. Rothermel Test case prioritization: a family of empirical studies IEEE Transactions on Software Engineering 28 2 2002 159 182 (Pubitemid 34198321)
17. S.G. Elbaum, G. Rothermel, S. Kanduri, and A.G. Malishevsky Selecting a cost-effective test case prioritization technique Software Quality Control 12 3 2004 185 210 (Pubitemid 38930558)
18. D. Farley, The deployment pipeline: extending the range of continuous integration. < http://www.scribd.com/doc/196739/The-Deployment-Pipeline-by- Dave-Farley-2007 >, 2007.
19. M. Fowler, Continuous integration. < http://www.martinfowler.com/ articles/continuousIntegration.html >, 2006.
20. Gonzalez-Sanchez, R. Abreu, H.-G. Gross, A. van Gemund, A diagnostic approach to test prioritization, Technical Report TUD-SERG-2010-007, Software Engineering Research Group, Delft University of Technology, Delft, the Netherlands, 2010.
21. A. Gonzalez-Sanchez, E. Piel, H.-G. Gross, and A.J.C. van Gemund Prioritizing tests for software fault localization Proceedings of the 10th International Conference on Quality Software (QSIC 2010) 2010 IEEE Computer Society Press Los Alamitos, CA 42 51
22. D. Jeffrey, N. Gupta, and R. Gupta Fault localization using value replacement Proceedings of the 2008 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2008) 2008 ACM Press New York, NY 167 178
23. B. Jiang, W.K. Chan, On the integration of test adequacy: test case prioritization and statistical fault localization, The 1st International Workshop on Program Debugging in China (IWPDC 2010), in: Proceedings of the 10th International Conference on Quality Software (QSIC 2010), IEEE Computer Society Press, Los Alamitos, CA, 2010, pp. 377-384.
24. B. Jiang, T.H. Tse, W. Grieskamp, N. Kicillof, Y. Cao, and X. Li Regression testing process improvement for specification evolution of real-world protocol software Proceedings of the 10th International Conference on Quality Software (QSIC 2010) 2010 IEEE Computer Society Press Los Alamitos, CA 62 71
25. B. Jiang, Z. Zhang, W.K. Chan, and T.H. Tse Adaptive random test case prioritization Proceedings of the 24th IEEE/ACM International Conference on Automated Software Engineering (ASE 2009) 2009 IEEE Computer Society Press Los Alamitos, CA 233 244
26. B. Jiang, Z. Zhang, T.H. Tse, T.Y. Chen, How well do test case prioritization techniques support statistical fault localization, in: Proceedings of the 33rd Annual International Computer Software and Applications Conference (COMPSAC 2009), vol. 1, IEEE Computer Society Press, Los Alamitos, CA, 2009, pp. 99-106.
27. J.A. Jones, and M.J. Harrold Empirical evaluation of the Tarantula automatic fault-localization technique Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering (ASE 2005) 2005 ACM Press New York, NY 273 282
28. J.A. Jones, M.J. Harrold, and J. Stasko Visualization of test information to assist fault localization Proceedings of the 24th International Conference on Software Engineering (ICSE 2002) 2002 ACM Press New York, NY 467 477 (Pubitemid 35009133)
29. I. Leon, W. Masri, and A. Podgurski An empirical evaluation of test case filtering techniques based on exercising complex information flows Proceedings of the 27th International Conference on Software Engineering (ICSE 2005) 2005 ACM Press New York, NY 412 421
30. S. Li, N. Bian, Z. Chen, D. You, and Y. He A simulation study on some search algorithms for regression test case prioritization Proceedings of the 10th International Conference on Quality Software (QSIC 2010) 2010 IEEE Computer Society Press Los Alamitos, CA 72 81
31. Z. Li, M. Harman, and R.M. Hierons Search algorithms for regression test case prioritization IEEE Transactions on Software Engineering 33 4 2007 225 237 (Pubitemid 46523213)
32. B. Liblit, M. Naik, A.X. Zheng, A. Aiken, and M.I. Jordan Scalable statistical bug isolation Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2005) 2005 ACM Press New York, NY 15 26
33. A.H. Lipkus A proof of the triangle inequality for the Tanimoto distance Journal of Mathematical Chemistry 26 1 1999 263 265
34. th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC 2005/FSE-13) 2005 ACM Press New York, NY 286 295 (Pubitemid 43218839)
35. L. Naish, H.J. Lee, K. Ramamohanarao, A model for spectra-based software diagnosis, ACM Transactions on Software Engineering and Methodology 20 (3) (2010). Article no. 11.
36. M. Renieris, and S.P. Reiss Fault localization with nearest neighbor queries Proceedings of the 18th IEEE International Conference on Automated Software Engineering (ASE 2003) 2003 IEEE Computer Society Press Los Alamitos, CA 30 39
37. G. Rothermel, R.H. Untch, C. Chu, and M.J. Harrold Test case prioritization: an empirical study Proceedings of the 15th IEEE International Conference on Software Maintenance (ICSM 1999) 1999 IEEE Computer Society Press Los Alamitos, CA 179 188 (Pubitemid 30547719)
38. A. Srivastava, and J. Thiagarajan Effectively prioritizing tests in development environment Proceedings of the 2002 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2002) 2002 ACM Press New York, NY 97 106
39. K.R. Walcott, M.L. Soffa, G.M. Kapfhammer, and R.S. Roos TimeAware test suite prioritization Proceedings of the 2006 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2006) 2006 ACM Press New York, NY 1 12
40. W.E. Wong, V. Debroy, and B. Choi A family of code coverage-based heuristics for effective fault localization Journal of Systems and Software 83 2 2010 188 208
41. W.E. Wong, J.R. Horgan, S. London, and H. Agrawal A study of effective regression testing in practice Proceedings of the 8th International Symposium on Software Reliability Engineering (ISSRE 1997) 1997 IEEE Computer Society Press Los Alamitos, CA 264 274
42. S. Yan, Z. Chen, Z. Zhao, C. Zhang, and Y. Zhou A dynamic test cluster sampling strategy by leveraging execution spectra information Proceedings of the 3rd International Conference on Software Testing, Verification, and Validation (ICST 2010) 2010 IEEE Computer Society Press Los Alamitos, CA 147 154
43. I. You, Z. Chen, B. Xu, B. Luo, and C. Zhang An empirical study on the effectiveness of time-aware test case prioritization techniques Proceedings of the 2011 ACM Symposium on Applied Computing (SAC 2011) 2011 ACM Press New York, NY
44. Y. Yu, J.A. Jones, and M.J. Harrold An empirical study of the effects of test-suite reduction on fault localization Proceedings of the 30th International Conference on Software Engineering (ICSE 2008) 2008 ACM Press New York, NY 201 210
45. C. Zhang, Z. Chen, Z. Zhao, S. Yan, J. Zhang, and B. Xu An improved regression test selection technique by clustering execution profiles Proceedings of the 10th International Conference on Quality Software (QSIC 2010) 2010 IEEE Computer Society Press Los Alamitos, CA 171 179
46. L. Zhang, S.-S. Hou, C. Guo, T. Xie, and H. Mei Time-aware test-case prioritization using integer linear programming Proceedings of the 2009 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2009) 2009 ACM Press New York, NY 213 224
47. Z. Zhang, W.K. Chan, T.H. Tse, P. Hu, and X. Wang Is non-parametric hypothesis testing model robust for statistical fault localization? Information and Software Technology 51 11 2009 1573 1585
48. Z. Zhang, W.K. Chan, T.H. Tse, B. Jiang, and X. Wang Capturing propagation of infected program states Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC 2009/FSE-17) 2009 ACM Press New York, NY 43 52
49. Z. Zhang, W.K. Chan, T.H. Tse, Y.T. Yu, and P. Hu Non-parametric statistical fault localization Journal of Systems and Software 84 6 2011 885 905
50. Z. Zhang, B. Jiang, W.K. Chan, T.H. Tse, and X. Wang Fault localization through evaluation sequences Journal of Systems and Software 83 2 2010 174 187
51. Z.Q. Zhou, Using coverage information to guide test case selection in adaptive random testing, The 7th International Workshop on Software Cybernetics (IWSC 2010), in: Proceedings of the 34th Annual Computer Software and Applications Conference (COMPSAC 2010), IEEE Computer Society Press, Los Alamitos, CA, 2010, pp. 208-213.