On establishing a benchmark for evaluating static analysis alert prioritization and classification techniques

ESEM'08: Proceedings of the 2008 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement

Volumn , Issue , 2008, Pages 41-50

  Heckman, Sarah ^a   Williams, Laurie ^a  

^a North Carolina State University   (United States)

Author keywords

Alert classification; Alert prioritization; Automated static analysis; Benchmark creation; False positive mitigation

Indexed keywords

SOFTWARE ENGINEERING; STATIC ANALYSIS;

ALERT CLASSIFICATION; ALERT PRIORITIZATION; AUTOMATED STATIC ANALYSIS; BENCHMARK CREATION; FALSE POSITIVE MITIGATION;

COMPUTER SOFTWARE SELECTION AND EVALUATION;

EID: 62949138527     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1414004.1414013     Document Type: Conference Paper

References (24)

1. N. Ayewah, W. Pugh, J. D. Morgenthaler, J. Penix, and Y. Zhou, "Evaluating Static Analysis Defect Warnings On Production Software," Proceedings of the 7th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, San Diego, CA, USA, June 13-14, 2007, pp. 1-8.
2. B. W. Boehm and R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed: Addison-Wesley, 2003.
3. G. Boetticher, T. Menzies, and T. Ostrand, "PROMISE Repository of Empirical Software Engineering Data," http://promisedata.org/ repository, West Virginia University, Department of Computer Science, 2007.
4. C. Boogerd and L. Moonen, "Prioritizing Software Inspection Results using Static Profiling," Proceedings of the 6th IEEE Workshop on Source Code Analysis and Manipulation, Philadelphia, PA, USA, September 27-29, 2006, pp. 149-160.
5. S. S. Heckman, "Adaptively Ranking Alerts Generated from Automated Static Analysis," in ACM Crossroads, vol. 14, no. 1, 2007, pp. 16-20.
6. D. Hovemeyer and W. Pugh, "Finding Bugs is Easy," Proceedings of the 19th ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications, Vancouver, British Columbia, Canada, October 24-28, 2004, pp. 132-136.
7. M. Hutchins, H. Foster, T. Goradia, and T. Ostrand, "Experiments on the Effectiveness of Dataflow- and Controlflow-Based Test Adequacy Criteria," Proceedings of the 19th International Conference on Software Engineering, Sorrento, Italy, May 16-21, 1994, pp. 191-200.
8. IEEE, "IEEE Standard 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology," 1990.
9. IEEE, "IEEE 1028-1997 (R2002) IEEE Standard for Software Reviews," 2002.
10. S. Kim and M. D. Ernst, "Prioritizing Warning Categories by Analyzing Software History," Proceedings of the International Workshop on Mining Software Repositories, Minneapolis, MN, USA, May 19-20, 2007, p. 27.
11. S. Kim and M. D. Ernst, "Which Warnings Should I Fix First?," Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, Dubrovnik, Croatia, September 3-7, 2007, pp. 45-54.
12. S. Kim, T. Zimmermann, J. E. James Whitehead, and A. Zeller, "Predicting Faults from Cached History," Proceedings of the 29th International Conference on Software Engineering, Minneapolis, MN, USA, May 23-25, 2007, pp. 489-498.
13. T. Kremenek, K. Ashcroft, J. Yang, and D. Engler, "Correlation Exploitation in Error Ranking," Proceedings of the 12th ACM SIGSOFTInternational Symposium on Foundations of Software Engineering, Newport Beach, CA, USA, 2004, pp. 83-93.
14. T. Kremenek and D. Engler, "Z-Ranking: Using Statistical Analysis to Counter the Impact of Static Analysis Approximations," Proceedings of the 10th International Static Analysis Symposium, San Diego, California, 2002.
15. S. Lu, Z. Li, F. Oin, L. Tan, P. Zhou, and Y. Zhou, "BugBench: Benchmarks for Evaluating Bug Detection Tools," Proceedings of the Workshop on the Evaluation of Software Defect Detection Tools, Chicago, Illinois, 2005.
16. M. Naik and A. Aiken, "Effective Static Race Detection for Java," Proceedings of the ACM SIGPLAN 2006 Conference on Programming Language Design and Implementation, Ottawa, Canada, June 10-16, 2006, pp. 308-319.
17. G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold, "Prioritizing Test Cases For Regression Testing," IEEE Transactions on Software Engineering, vol. 27, no. 10, pp. 929-948, October, 2001.
18. S. E. Sim, S. Easterbrook, and R. C. Holt, "Using Benchmarking to Advance Research: A Challenge to Software Engineering," Proceedings of the 25th International Conference on Software Engineering, Portland, Oregon, USA, May 3-10, 2003, pp. 74-83.
19. W. F. Tichy, "Should Computer Scientists Experiment More?," in Computer, vol. 31, no. 5, 1998, pp. 32-40.
20. S. Wagner and M. A. Florian Deissenboeck, Johann Wimmer, Markus Schwalb, "An Evaluation of Two Bug Pattern Tools for Java," Proceedings of the 1st IEEE International Conference on Software Testing, Verification, and Validation, Lillehammer, Norway, to appear, 2008.
21. C. C. Williams and J. K. Hollingsworth, "Automatic Mining of Souce Code Repositories to Improve Bug Finding Techniques," IEEE Transactions on Software Engineering, vol. 31, no. 6, pp. 466-480, 2005.
22. I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, 2nd ed. Amsterdam: Morgan Kaufmann, 2005.
23. J. Zheng, L. Williams, N. Nagappan, W. Snipes, J. Hudepohl, and M. Vouk, "On the Value of Static Analysis for Fault Detection in Software," IEEE Transactions on Software Engineering, vol. 32, no. 4, pp. 240-253, April, 2006.
24. T. Zimmermann, R. Premraj, and A. Zeller, "Predicting Defects in Eclipse," Proceedings of the 3rd International Workshop on Predictor Models in Software Engineering, Minneapolis, MN, USA, May 20, 2007, p. 9.