A systematic literature review of actionable alert identification techniques for automated static code analysis

Information and Software Technology

Volumn 53, Issue 4, 2011, Pages 363-387

  Heckman, Sarah ^a   Williams, Laurie ^a  

^a NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

Actionable alert identification; Actionable alert prediction; Automated static analysis; Systematic literature review; Unactionable alert mitigation; Warning prioritization

Indexed keywords

AUTOMATION; CLASSIFICATION (OF INFORMATION); DATA FUSION; GRAPH THEORY; LEARNING SYSTEMS; SOFTWARE DESIGN;

CONTEXTUAL INFORMATION; EVALUATION METRICS; IDENTIFICATION TECHNIQUES; PRIORITIZATION; SOFTWARE DEVELOPMENT LIFE CYCLE; STATIC CODE ANALYSIS; SYSTEMATIC LITERATURE REVIEW; UNACTIONABLE ALERT MITIGATION;

STATIC ANALYSIS;

EID: 79951811783     PISSN: 09505849     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.infsof.2010.12.007     Document Type: Review

References (61)

1. A. Aggarwal, P. Jalote, Integrating static and dynamic analysis for detecting vulnerabilities, in: Proceedings of the 30th Annual International Computer Software and Applications Conference, Chicago, Illinois, USA, September 17-21, 2006, pp. 343-350. (Pubitemid 46661666)
2. C. Artho, A. Biere, Applying static analysis to large-scale, multi-threaded java programs, in: Proceedings of the 13th Australian Conference on Software Engineering, Canberra, Australia, August 27-28, 2001, pp. 68-75.
3. R.M. Bell, T.J. Ostrand, E.J. Weyuker, Looking for bugs in all the right places, in: Proceedings of the International Symposium on Software Testing and Analysis, 2006, pp. 61-71. (Pubitemid 46645519)
4. C. Boogerd, L. Moonen, Prioritizing software inspection results using static profiling, in: Proceedings of the 6th IEEE Workshop on Source Code Analysis and Manipulation, Philadelphia, PA, USA, September 27-29, 2006, pp. 149-160.
5. C. Boogerd, L. Moonen, Ranking software inspection results using execution likelihood, in: Proceedings of the Philips Software Conference, November, 2006, p. 10.
6. C. Boogerd, L. Moonen, Assessing the value of coding standards: an empirical study, in: Proceedings of the IEEE International Conference on Software Maintenance, Beijing, China, September 28-October 4, 2008, pp. 277-286.
7. C. Boogerd, L. Moonen, On the use of data flow analysis in static profiling, in: Proceedings of the Eighth IEEE International Working Conference on Source Code Analysis and Manipulation, Beijing, China, September 28-29, 2008, pp. 79-88.
8. P. Chen, H. Han, Y. Wang, S. Shen, X. Yin, B. Mao, L. Xie, intfinder: automatically detecting integer bugs in x86 binary program, in: Proceedings of the International Conference on Information and Communications Security, Beijing, China, December, 2009, pp. 336-345.
9. B. Chess, and G. McGraw Static analysis for security IEEE Security & Privacy 2 6 2004 76 79 (Pubitemid 40010916)
10. B. Chess, and J. West Secure Programming with Static Analysis first ed. 2007 Addison-Wesley Upper Saddle River, NJ
11. J. Cohen A coefficient of agreement for nominal scales Educational and Psychological Measurement 20 1960 213 220
12. C. Csallner, and Y. Smaragdakis JCrasher: an automatic robustness tester for Java Software - Practice and Experience 34 11 2004 1025 1050
13. C. Csallner, Y. Smaragdakis, Check 'n' crash: combining static checking and testing, in: Proceedings of the 27th International Conference on Software Engineering, St. Louis, MO, USA, 2005, pp. 422-431.
14. C. Csallner, Y. Smaragdakis, and T. Xie DSD-crasher: a hybrid analysis tool for bug finding ACM Transactions on Software Engineering and Methodology 17 2 2008 1 36
15. D. Engler, B. Chelf, A. Chou, S. Hallem, Checking System Rules Using System-Specific, Programmer-Written Compiler Extensions, in Operating Systems Design and Implementation, San Diego, CA, 2000.
16. M.D. Ernst, J. Cockrell, W.G. Griswold, and D. Notkin Dynamically discovering likely program invariants to support program evolution IEEE Transactions on Software Engineering 27 2 2001 99 123 (Pubitemid 32254259)
17. C. Flanagan, K.R.M. Leino, M. Lillibridge, G. Nelson, J.B. Saxe, R. Stata, Extended static checking for java, in: Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation, Berlin, Germany, 2002, pp. 234-245. (Pubitemid 34991522)
18. J.L. Fleiss, J. Cohen, and B.S. Everitt Large sample standard errors of kappa and weighted kappa Psychological Bulletin 72 1969 323 327
19. E. Haugh, M. Bishop, Testing C programs for buffer overflow vulnerabilities, in: Proceedings of the Network and Distributed System Security Symposium, San Diego, CA, February, 2003, pp. 123-130.
20. S. Heckman, L. Williams, A measurement framework of alert characteristics for false positive mitigation models, in: North Carolina State University TR-2008-23, October 6, 2008.
21. S. Heckman, L. Williams, On establishing a benchmark for evaluating static analysis alert prioritization and classification techniques, in: Proceedings of the 2nd International Symposium on Empirical Software Engineering and Measurement, Kaiserslautern, Germany, October 9-10, 2008, pp. 41-50.
22. S. Heckman, L. Williams, A model building process for identifying actionable static analysis alerts, in: Proceedings of the 2nd IEEE International Conference on Software Testing, Verification and Validation, Denver, CO, USA, 2009, pp. 161-170.
23. S.S. Heckman, A Systematic Model Building Process for Predicting Actionable Static Analysis Alerts, Dissertation, Computer Science, North Carolina State University, 2009.
24. G.J. Holzmann The SPIN Model Checker: Primer and Reference Manual 2003 Addison-Wesley
25. D. Hovemeyer, W. Pugh, Finding bugs is easy, in: Proceedings of the 19th ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications, Vancouver, British Columbia, Canada, October 24-28, 2004, pp. 132-136.
26. IEEE, IEEE Standard 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology, in no., 1990.
27. IEEE, IEEE 1028-1997 (R2002) IEEE Standard for Software Reviews, no., 2002.
28. IEEE, ISO/IEC 24765:2008 Systems and Software Engineering Vocabulary, 2008.
29. S.C. Johnson Lint, a C Program Checker 1978 Bell Laboratories Murray Hill, NJ
30. Y. Jung, J. Kim, J. Shin, K. Yi, Taming false alarms from a domain-unaware C analyzer by a Bayesian statistical post analysis, in: Proceedings of the 12th International Static Analysis Symposium, Imperial College London, UK, 2005, pp. 203-217.
31. S. Kim, M.D. Ernst, Prioritizing warning categories by analyzing software history, in: Proceedings of the International Workshop on Mining Software Repositories, Minneapolis, MN, USA, May 19-20, 2007, p. 27.
32. S. Kim, M.D. Ernst, Which warnings should I fix first?, in: Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, Dubrovnik, Croatia, September 3-7, 2007, pp. 45-54.
33. B. Kitchenham, Procedures for Performing Systematic Reviews, Joint Technical Report, Keele University Technical Report (TR/SE-0401) and NICTA Technical Report (0400011T.1) July 2004, 2004.
34. D. Kong, Q. Zheng, C. Chen, J. Shuai, M. Zhu, "isa: a source code static vulnerability detection system based on data fusion, in: Proceedings of the 2nd International Conference on Scalable Information Systems, Suzhou, China, June 6-8, 2007, p. 55.
35. T. Kremenek, K. Ashcraft, J. Yang, D. Engler, Correlation exploitation in error ranking, in: Proceedings of the 12th ACM SIGSOFT International Symposium on Foundations of Software Engineering, Newport Beach, CA, USA, 2004, pp. 83-93. (Pubitemid 40787570)
36. T. Kremenek, D. Engler, Z-ranking: using statistical analysis to counter the impact of static analysis approximations, in: Proceedings of the 10th International Static Analysis Symposium, San Diego, California, 2003, pp. 295-315.
37. G. Liang, L. Wu, Q. Wu, Q. Wang, T. Xie, H. Mei, Automatic construction of an effective training set for prioritizing static analysis warnings, in: Proceedings of the 25th IEEE/ACM International Conference on Automated Software Engineering, Antwerp, Belgium, September, 2010, pp. 93-102.
38. N. Meng, Q. Wang, Q. Wu, H. Mei, An approach to merge results of multiple static analysis tools (short paper), in: Proceedings of the Eight International Conference on Quality Software, Oxford, UK, August 12-13, 2008, pp. 169-174.
39. M.G. Nanda, M. Gupta, S. Sinha, S. Chandra, D. Schmidt, P. Balachandran, Making defect-finding tools work for you, in: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering, Cape Town, South Africa, May 2-8, vol. 2, 2010, pp. 99-108.
40. H. Ogasawara, M. Aizawa, A. Yamada, Experiences with program static analysis, in: Proceedings of the 1998 Software Metrics Symposium, Bethesda, MD, USA, November 20-21, 1998, pp. 109-112.
41. T.J. Ostrand, E.J. Weyuker, R.M. Bell, Where the bugs are, in: Proceedings of the International Symposium on Software Testing and Analysis, 2004, pp. 86-96. (Pubitemid 41121380)
42. C. Pacheco, M.D. Ernst, Eclat: automatic generation and classification of test inputs, in: Proceedings of the 19th European Conference on Object-Oriented Programming, Glasgow, Scotland, July 27-29, 2005, pp. 504-527. (Pubitemid 41435995)
43. G. Rothermel, R.H. Untch, C. Chu, and M.J. Harrold Prioritizing test cases for regression testing IEEE Transactions on Software Engineering 27 10 2001 929 948 (Pubitemid 33048252)
44. N. Rungta, E.G. Mercer, A meta heuristic for effectively detecting concurrency errors, in: Proceedings of the 4th International Haifa Verification Conference on Hardware and Software: Verification and Testing, Haifa, Israel, October, 2008, pp. 23-37.
45. J.R. Ruthruff, J. Penix, J.D. Morgenthaler, S. Elbaum, G. Rothermel, Predicting accurate and actionable static analysis warnings: an experimental approach, in: Proceedings of the 30th International Conference on Software Engineering, Leipzig, Germany, May 10-18, 2008, pp. 341-350.
46. M. Shaw, Writing good software engineering research papers: minitutorial, in: Proceedings of the 25th International Conference on Software Engineering, Portland, Oregon, USA, May 3-10, 2003, pp. 726-736.
47. S.E. Sim, S. Easterbrook, R.C. Holt, Using benchmarking to advance research: a challenge to software engineering, in: Proceedings of the 25th International Conference on Software Engineering, Portland, Oregon, USA, May 3-10, 2003, pp. 74-83.
48. W.F. Tichy Should computer scientists experiment more? Computer 31 5 1998 32 40 (Pubitemid 128556565)
49. S.B. Vardeman, and J.M. Jobe Basic Engineering Data Collection and Analysis first ed. 2001 Duxbury Pacific Grove, CA
50. J. Viega, J.T. Floch, Y. Kohno, G. McGraw, "its4: a static vulnerability scanner for C and C++ code, in: Proceedings of the 16th Annual Conference on Computer Security Applications, New Orleans, LA, USA, December 11-15, 2000, pp. 257-267.
51. W. Visser, K. Havelund, G. Brat, S. Park, Model checking programs, in: Proceedings of the 15th IEEE International Conference on Automated Software Engineering, Grenoble, France, September, 2000, pp. 3-11.
52. D. Wagner, S.F. Jeffrey, E.A. Brewer, A. Aiken, A first step towards automated detection of buffer overrun vulnerabilities, in: Proceedings of the Network and Distributed Systems Security Conference, San Diego, CA, USA, February 2-4, 2000, 2000, pp. 3-17.
53. S. Wagner, F. Deissenboeck, M. Aichner, J. Wimmer, M. Schwalb, An evaluation of two bug pattern tools for java, in: Proceedings of the 1st International Conference on Software Testing, Verification, and Validation, Lillehammer, Norway, April 9-11, 2008, pp. 248-257.
54. D.W. Wall, Predicting program behavior using real or estimated profiles, in: Proceedings of the ACM SIGPLAN 1991 Conference on Programming Language Design and Implementation, Toronto, Ontario, Canada, June 26-28, 1991, pp. 59-70.
55. C.C. Williams, and J.K. Hollingsworth Automatic mining of source code repositories to improve bug finding techniques IEEE Transactions on Software Engineering 31 6 2005 466 480 (Pubitemid 41245069)
56. I.H. Witten, and E. Frank Data Mining: Practical Machine Learning Tools and Techniques second ed. 2005 Morgan Kaufman Amsterdam
57. S. Xiao, C. Pham, Performing high efficiency source code static analysis with intelligent extensions, in: Proceedings of the 11th Asia-Pacific Software Engineering Conference, Busan, Korea, November 30-December 3, 2004, pp. 346-355. (Pubitemid 40700525)
58. K. Yi, H. Choi, J. Kim, and Y. Kim An empirical study on classification methods for alarms from a bug-finding static C analyzer Information Processing Letters 102 2-3 2007 118 123 (Pubitemid 46357475)
59. L. Yu, J. Zhou, Y. Yi, J. Fan, Q. Wang, A hybrid approach to detecting security defects in programs, in: Proceedings of the 9th International Conference on Quality Software, Jeju, Korea, August 24-25, 2009, pp. 1-10.
60. J. Zheng, L. Williams, N. Nagappan, W. Snipes, J. Hudepohl, and M. Vouk On the value of static analysis for fault detection in software IEEE Transactions on Software Engineering 32 4 2006 240 253
61. T. Zimmermann, R. Premraj, A. Zeller, Predicting defects in eclipse, in: Proceedings of the 3rd International Workshop on Predictor Models in Software Engineering, Minneapolis, MN, USA, May 20, 2007, p. 9.