An empirical study of test case filtering techniques based on exercising information flows

IEEE Transactions on Software Engineering

Volumn 33, Issue 7, 2007, Pages 454-477

  Masri, Wes ^a   Podgurski, Andy ^b   Leon, David ^c  

^a AMERICAN UNIVERSITY OF BEIRUT   (Lebanon)

^b Case Western Reserve University   (United States)

^c Torre Movistar piso 3 Oficina 2 ^*  (Venezuela)

Author keywords

Coverage based filtering; Distribution based filtering; Dynamic slicing; Empirical studies; Information flow; Software testing; Test case filtering; Test suite minimization

Indexed keywords

COMPUTER SOFTWARE SELECTION AND EVALUATION; OPTIMIZATION; SOFTWARE TESTING;

COVERAGE-BASED FILTERING; DISTRIBUTION-BASED FILTERING; DYNAMIC SLICING; INFORMATION FLOWS; TEST CASE FILTERING;

COMPUTER AIDED SOFTWARE ENGINEERING;

EID: 34250772325     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2007.1020     Document Type: Article

References (49)

1. H. Agrawal, J. Horgan, E. Krauser, and S. London, "Incremental Regression Testing," Proc. IEEE Conf. Software Maintenance, pp. 348-357, 1993.
2. S. Bates and S. Horwitz, "Incremental Program Testing Using Program Dependence Graphs," Proc. 20th ACM Symp. Principles of Programming Languages, pp. 384-396, Jan. 1993.
3. L.A. Clarke, A. Podgurski, D.J. Richardson, and S.J. Zeil, "A Formal Evaluation of Data Flow Path Selection Criteria," IEEE Trans. Software Eng., vol. 15, no. 11, pp. 1381-1332, Nov. 1989.
4. D.E. Denning and P.J. Denning, "Certification of Programs for Secure Information Flow," Comm. ACM, vol. 20, no. 7, pp. 504-513, 1977.
5. W. Dickinson, D. Leon, and A. Podgurski, "Finding Failures by Cluster Analysis of Execution Profiles," Proc. 23rd Int'l Conf. Software Eng., pp. 339-348, May 2001.
6. W. Dickinson, D. Leon, and A. Podgurski, "Pursuing Failure: The Distribution of Program Failures in a Profile Space," Proc. 10th European Software Eng. Conf./Ninth ACM SIGSOFT Symp. Foundations of Software Eng., pp. 246-255, Sept. 2001.
7. S. Elbaum, A.G. Malishevsky, and G. Rothermel, "Test Case Prioritization: A Family of Empirical Studies," IEEE Trans. Software Eng., vol. 28, no. 2, pp. 159-182, Feb. 2002.
8. J. Ferrante, K.J. Ottenstein, and J.D. Warren, "The Program Dependence Graph and Its Use in Optimization," ACM Trans. Programming Languages and Systems, vol. 9, no. 3, pp. 319-349, Oct. 1987.
9. P. Frankl and O. Iakounenko, "Further Empirical Studies of Test Effectiveness," Proc. Sixth ACM SIGSOFT Int'l Symp. Foundations of Software Eng., pp. 153-162, Nov. 1998.
10. P.G. Frankl and Y. Deng, "A Comparison of Delivered Reliability of Branch, Data Flow, and Operational Testing: A Case Study," Proc. Int'l Symp. Software Testing and Analysis, pp. 124-134, Aug. 2000.
11. T.L. Graves, M.J. Harrold, J.M. Kim, A. Porter, and G. Rothermel, "An Empirical Study of Regression Test Selection Techniques," ACM Trans. Software Eng. and Methodology, vol. 10, no. 2, pp. 184-208, Apr. 2001.
12. R. Gupta, M.J. Harrold, and M.L. Soffa, "Program Slicing-Based Regression Testing Techniques," J. Software Testing, Verification, and Reliability, vol. 6, no. 2, pp. 83-112, June 1996.
13. M.J. Harold, G. Rothermel, K. Sayre, R. Wu, and L. Yi, "An Empirical Investigation of the Relationship between Spectra Differences and Regression Faults," J. Software Testing, Verification, and Reliability, vol. 10, no. 3, pp. 171-194, Sept. 2000.
14. D.S. Hochbaum, Approximation Algorithms for NP-Hard Problems. PWS, 1997.
15. Jacks Project, IBM, www.ibm.com/developerworks/oss/cvs/jacks/, 2002.
16. Java Language Specification, Sun Microsystems, java.sun.com/docs/books/ jls/second_edition/html/j.title.doc.html, 2000.
17. javac, Java 2 Platform, Standard Edition, Sun Microsystems, java.sun.com/ j2se/1.3/, 1995-2002.
18. Jtidy, World Wide Web Consortium (Massachusetts Inst. of Technology, Institut Nat'l de Recherche en Informatique et en Automatique, Keio Univ.), jtidy.sourceforge.net, 1998-2000.
19. J.M. Kim and A. Porter, "A History-Based Test Prioritization Technique for Regression Testing in Resource Constrained Environments," Proc. Int'l Conf. Software Eng., pp. 119-129, May 2002.
20. B. Korel, "The Program Dependence Graph in Static Program Testing," Information Processing Letters, vol. 24, pp.103-108,Jan.1987.
21. B. Korel and J. Laski, "Dynamic Program Slicing," Information Processing Letters, vol. 29, pp. 155-163, Oct. 1988.
22. B. Korel and S. Yalamanchili, "Forward Computation of Dynamic Program Slices," Proc. Int'l Symp. Software Testing and Analysis, pp. 66-79, 1994.
23. J. Laski and B. Korel, "A Data Flow Oriented Program Testing Strategy," IEEE Trans. Software Eng., vol. 9, no. 3, pp. 347-354, May 1983.
24. D. Leon and A. Podgurski, "A Comparison of Coverage-Based and Distribution-Based Techniques for Filtering and Prioritizing Test Cases," Proc. Int'l Symp. Software Reliability Eng., pp. 442-454, Nov. 2003.
25. D. Leon, A. Podgurski, and L.J. White, "Multivariate Visualization in Observation-Based Testing," Proc. 22nd Int'l Conf. Software Eng., pp. 116-125, June 2000.
26. D. Leon, W. Masri, and A. Podgurski, "An Empirical Evaluation of Test Case Filtering Techniques Based on Exercising Complex Information Flows," Proc. 27th Int'l Conf. Software Eng., pp. 412-421, May 2005.
27. W. Masri, A. Podgurski, and D. Leon, "Detecting and Debugging Insecure Information Flows," Proc. 15th IEEE Int'l Symp. Software Reliability Eng. (ISSRE '04), pp. 198-209, 2004.
28. W. Masri and A. Podgurski, "An Empirical Study of the Relationship between Information Flow and Program Dependence," Proc. Fourth Int'l Workshop Dynamic Analysis (WODA '06), May 2006.
29. W. Masri, N. Nahas, and A. Podgurski, "Memoized Forward Computation of Program Slices," Proc. 17th IEEE Int'l Symp. Software Reliability Eng. (ISSRE '06), 2006.
30. W. Masri and A. Podgurski, "Using Dynamic Information Flow Analysis to Detect Attacks against Applications," Proc. Workshop Software Eng. for Secure Systems, pp. 1-7, May 2005.
31. L.J. Morell, "A Theory of Fault-Based Testing," IEEE Trans. Software Eng., vol. 16, no. 8, pp. 844-857, Aug. 1990.
32. S.C. Ntafos, "On Required Element Testing," IEEE Trans. Software Eng., vol. 10, no. 6, pp. 795-803, Nov. 1984.
33. A. Podgurski and L.A. Clarke, "A Formal Model of Program Dependencies and Its Implications for Software Testing, Debugging, and Maintenance," IEEE Trans. Software Eng., vol. 16, no. 9, pp. 965-979, 1990.
34. A. Podgurski, D. Leon, P. Francis, W. Masri, M. Minch, J. Sun, and B. Wang, "Automated Support for Classifying Software Failure Reports," Proc. Int'l Conf. Software Eng., pp. 465-475, May 2003.
35. S. Rapps and E.J. Weyuker, "Selecting Software Test Data Using Data Flow Information," IEEE Trans. Software Eng., vol. 11, no. 4, pp. 367-375, Apr. 1985.
36. G. Rothermel, M.J. Harrold, J. Ostrin, and C. Hong, "An Empirical Study of the Effects of Minimization on the Fault Detection Capabilities of Test Suites," Proc. Int'l Conf. Software Maintenance, pp. 34-43, Nov. 1998.
37. G. Rothermel and M.J. Harrold, "A Safe, Efficient Algorithm for Regression Test Selection," Proc. Int'l Conf. Software Maintenance pp. 358-367, Sept. 1993.
38. G. Rothermel, R. Untch, C. Chu, and M.J. Harrold, "Prioritizing Test Cases for Regression Testing," IEEE Trans. Software Eng., vol. 27, no. 10, pp. 929-948, Oct. 2001.
39. G. Rothermel and M.J. Harrold, "Selecting Tests and Identifying Test Coverage Requirements for Modified Software," Proc. Int'l Symp. Software Testing and Analysis, pp. 169-184, Aug. 1994.
40. S. Sinha, M.J. Harrold, and G. Rothermel, "Interprocedural Control Dependence," ACM Trans. Software Eng. Methodology, pp. 209-254, 2000.
41. Software-Artifact Infrastructure Repository: SIR, http://sir.unl.edu, 2006.
42. The Byte Code Engineering Library (BCEL), Apache Jakarta Project, http://jakarta.apache.org/bcel, 2005.
43. M. Thompson, D.J. Richardson, and L.A. Clarke, "An Information Flow Model of Fault Detection," Proc. Int'l Symp. Software Testing and Analysis, pp. 182-192, June 1993.
44. M. Weiser, "Program Slicing," IEEE Trans. Software Eng., vol. 10, no. 4, pp. 352-357, Oct. 1984.
45. W.E. Wong, J.R. Horgan, S. London, and A.P. Mathur, "Effect of Test Set Minimization on Fault Detection Effectiveness," Proc. 17th Int'l Conf. Software Eng., pp. 41-50, 1995.
46. W.E. Wong, J.R. Horgan, S. London, and A.P. Mathur, "Effect of Test Set Size Minimization on Fault Detection Effectiveness," Software Practice and Experience, vol. 28, no. 4, pp. 347-369, Apr. 1998.
47. W.E. Wong, J.R. Horgan, A.P. Mathur, and A. Pasquini, "Test Set Size Minimization and Fault Detection Effectiveness: A Case Study in a Space Application," Proc. 21st Ann. Int'l Computer Software and Applications Conf., pp. 522-528, Aug. 1997.
48. Xerces, Apache XML Project, xml.apache.org/xerces-j, 2005.
49. XML Conformance Test Suite, http://www.w3.org/XML/Test, 2005.