Security Events and Vulnerability Data for Cybersecurity Risk Estimation

Risk Analysis

Volumn 37, Issue 8, 2017, Pages 1606-1627

  Allodi, Luca ^a   Massacci, Fabio ^b  

^a EINDHOVEN UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b UNIVERSITY OF TRENTO   (Italy)

Author keywords

Attack likelihood; cybersecurity events; quantitative risk; vulnerabilities

Indexed keywords

ESTIMATION; NETWORK SECURITY; RISK ASSESSMENT; RISKS;

ATTACK LIKELIHOOD; CYBER SECURITY; FINANCIAL INSTITUTION; QUALITATIVE RISK ASSESSMENT; QUANTITATIVE APPROACH; QUANTITATIVE RISK; SECURITY OPERATION CENTER; VULNERABILITIES;

RISK PERCEPTION;

APPETITE; ARTICLE; COMPUTER SECURITY; FEMALE; INTERNET; MALE; ORGANIZATION; QUANTITATIVE ANALYSIS; RISK ASSESSMENT;

EID: 85027396040     PISSN: 02724332     EISSN: 15396924     Source Type: Journal    
DOI: 10.1111/risa.12864     Document Type: Article

References (106)

1. Van Goethem T, Chen P, Nikiforakis N, Desmet L, Joosen W. Large-scale security analysis of the web: Challenges and findings. Pp. 110–126 in Trust and Trustworthy Computing. Springer, 2014.
2. Wang L, Islam T, Long T, Singhal A, Jajodia S. An attack graph-based probabilistic security metric. Pp. 283–296 in Proceedings of the 22nd IFIP WG 11.3 Working Conference on Data and Applications Security. Vol. 5094 of Lecture Notes in Computer Science. Springer Berlin/Heidelberg, 2008.
3. Chen Py, Kataria G, Krishnan R. Correlated failures, diversification, and information security risk management. MIS Quarterly: Management Information Systems, 2011; 35(2): 397–422.
4. Apostolakis GE. How useful is quantitative risk assessment? Risk Analysis, 2004; 24(3):515–520.
5. Lomnitz C. Global Tectonics and Earthquake Risk, Vol. 5. Amsterdam, The Netherlands: Elsevier, 1974.
6. Cavusoglu H, Mishra B, Raghunathan S. The value of intrusion detection systems in information technology security architecture. Information Systems Research, 2005; 16(1):28–46.
7. Khorshidi HA, Gunawan I, Ibrahim MY. Data-driven system reliability and failure behavior modeling using FMECA. IEEE Transactions on Industrial Informatics, 2016; 12(3): 1253–1260.
8. Susto GA, Schirru A, Pampuri S, McLoone S. Supervised aggregative feature extraction for big data time series regression. IEEE Transactions on Industrial Informatics, 2016; 12(3):1243–1252.
9. Valeur F, Vigna G, Kruegel C, Kemmerer RA. Comprehensive approach to intrusion detection alert correlation. IEEE Transactions on Dependable and Secure Computing, 2004; 1(3):146–169.
10. Axelsson S. The base-rate fallacy and the difficulty of intrusion detection. ACM Transactions on Information and System Security (TISSEC), 2000; 3(3):186–205.
11. Pauli D. Hackers tear shreds off Verizon's data breach report top 10 bug list. Register, 2016. Available at: http://www.theregister.co.uk/2016/05/12/verizon_dbir_criticised/, Accessed November 2016.
12. Martin B. A note on the Verizon DBIR 2016 vulnerabilities claims. OSVDB, 2016. https://doi.org/blog.osvdb.org/2016/04/27/a-note-on-the-verizon-dbir-2016-vulnerabilities-claims/. Available at: https://doi.org/blog.osvdb.org/2016/04/27/a-note-on-the-verizon-dbir-2016-vulnerabilities-claims/, Accessed November 2016.
13. Bowen P, Hash J, Wilson M. Information Security Handbook: A Guide for Managers. Gaithersburg, MD: NIST, 2006.
14. Anthony Tony Cox L. What's wrong with risk matrices? Risk Analysis, 2008; 28(2):497–512.
15. Duijm NJ. Recommendations on the use and design of risk matrices. Safety Science, 2015; 76:21–31.
16. Team CS. Common Vulnerability Scoring System v3.0: Specification Document. First.org, 2015.
17. Bozorgi M, Saul LK, Savage S, Voelker GM. Beyond heuristics: Learning to classify vulnerabilities and predict exploits. Pp. 105–114 in Proceedings of the 16th ACM International Conference on Knowledge Discovery and Data Mining, 2010.
18. Houmb SH, Franqueira VN, Engum EA. Quantifying security risk level from CVSS estimates of frequency and impact. Journal of Systems and Software, 2010; 83(9):1622–1634.
19. Allodi L, Massacci F. Comparing vulnerability severity and exploits using case-control studies. ACM Transaction on Information and System Security (TISSEC), 2014; 17(1):1–20.
20. Council PCISS. PCI Data Security Standard (DSS): Requirements and Security Assessment Procedures. PCI-DSS, 2010. Available from: https://doi.org/www.pcisecuritystandards.org/documents/pci_dss_v2.pdf, Accessed April 2017.
21. org F. Common Vulnerability Scoring System v3.0: Specification Document. FIRST, 2015 Available at http://www.first.org/cvss, Accessed April 2017.
22. Giacalone M, Mammoliti R, Massacci F, Paci F, Perugino R, Selli C. Security triage: A report of a lean security requirements methodology for cost-effective security analysis. Pp. 25–27 in Proc. of ACM/IEE ESEM'14, 2014.
23. Cherdantseva Y, Burnap P, Blyth A, Eden P, Jones K, Soulsby H, et al. A review of cyber security risk assessment methods for SCADA systems. Computers & Security, 2016; 56:1–27.
24. Kunreuther H. Risk analysis and risk management in an uncertain world. Risk Analysis, 2002; 22(4):655–664. Available at: https://doi.org/10.1111/0272-4332.00057, Accessed April 2017.
25. Kelley D, Moritz R. Best practices for building a security operations center. Information Systems Security, 2006; 14(6): 27–32.
26. Jacobs P, Arnab A, Irwin B. Classification of security operation centers. Pp. 1–7 in Venter HS, Loock M, Coetzee M (eds). 2013 Information Security for South Africa. Johannesburg, South Africa: IEEE, 2013.
27. Aven T. On some recent definitions and analysis frameworks for risk, vulnerability, and resilience. Risk Analysis, 2011; 31(4):515–522.
28. Ransbotham S, Mitra S. Choice and chance: A conceptual model of paths to information security compromise. Information Systems Research, 2009; 20:121–139.
29. Rios Insua D, Rios J, Banks D. Adversarial risk analysis. Journal of the American Statistical Association, 2009; 104(486):841–854.
30. Rao NS, Poole SW, Ma CY, He F, Zhuang J, Yau DK. Defense of cyber infrastructures against cyber-physical attacks using game-theoretic models. Risk Analysis, 2016; 36(4):694–710.
31. Merrick J, Parnell GS. A comparative analysis of PRA and intelligent adversary methods for counterterrorism risk management. Risk Analysis, 2011; 31(9):1488–1510.
32. Brown GG, Cox Jr LAT. How probabilistic risk assessment can mislead terrorism risk analysts. Risk Analysis, 2011; 31(2):196–204.
33. Verizon. 2016 Data Breach Investigation Report. Verizon, 2016. Available at: http://www.verizonenterprise.com/verizon-insights-lab/dbir/2016/, Accessed November 2016.
34. Bilge L, Dumitras T. Before we knew it: An empirical study of zero-day attacks in the real world. Pp. 833–844 in Proceedings of the 19th ACM Conference on Computer and Communications Security (CCS'12). ACM, 2012.
35. Paté-Cornell E. On “black swans” and “perfect storms”: Risk analysis and management when statistics are not enough. Risk Analysis, 2012; 32(11):1823–1833.
36. Tankard C. Advanced persistent threats and how to monitor and deter them. Network Security, 2011; 2011(8):16–19.
37. Grier C, Ballard L, Caballero J, Chachra N, Dietrich CJ, Levchenko K, et al. Manufacturing compromise: The emergence of exploit-as-a-service. Pp. 821–832 in Proceedings of the 19th ACM Conference on Computer and Communications Security (CCS'12). ACM, 2012.
38. Provos N, Mavrommatis P, Rajab MA, Monrose F. All your iFRAMEs point to us. Pp. 1–15 in Proceedings of the 17th USENIX Security Symposium, 2008.
39. Nicholson A, Webber S, Dyer S, Patel T, Janicke H. SCADA security in the light of cyber-warfare. Computers & Security, 2012; 31(4):418–436.
40. Verizon. 2015 Data Breach Investigations Report. Verizon, 2015. Available at: http://www.verizon.com/about/news/2015-data-breach-report-info/, Accessed November 2016.
41. Symantec. Symantec Corporation Internet Security Threat Report 2013. 2012 Trends, 2013.
42. TrendMicro. Exploit Kit. TrendMicro, 2016. Available at: http://www.trendmicro.com/vinfo/us/security/definition/exploit-kit, Accessed November 2016.
43. Kotov V, Massacci F. Anatomy of exploit kits. Preliminary analysis of exploit kits as software artefacts. Pp. 181–196 in Proc. of ESSoS 2013, 2013.
44. Sophos. Sophos, editor. Location-based threats: How cybercriminals target you based on where you live. Sophos, 2016. Available at: https://doi.org/blogs.sophos.com/2016/05/03/location-based-ransomware-threat-research/, Accessed November 2016.
45. Erickson J. Hacking: The Art of Exploitation. San Francisco, CA: No Starch Press, 2008.
46. Allodi L. The Heavy Tails of Vulnerability Exploitation. Pp. 133–148 in Proceedings of the 2015 Engineering Secure Software and Systems Conference (ESSoS'15), 2015.
47. Nayak K, Marino D, Efstathopoulos P, Dumitraş T. Some vulnerabilities are different than others. Pp. 426–446 in Proceedings of the 17th International Symposium on Research in Attacks, Intrusions and Defenses. Springer, 2014.
48. Bhatt S, Manadhata PK, Zomlot L. The operational role of security information and event management systems. IEEE Security Privacy, 2014; 12(5):35–41.
49. ISO/IEC. 27005:2011–Information Technology–Security Techniques–Information Security Management Systems–Requirements. ISO/IEC, 2011.
50. ISO/IEC. 31000:2009: Risk Management. ISO/IEC, 2009.
51. Stoneburner G, Goguen A, Feringa A. Risk Management Guide for Information Technology Systems. Gaithersburg, MD: NIST, 2002.
52. ISACA. COBIT 5: A Business Framework for the Governance and Management of Enterprise IT. Rolling Meadows, IL: ISACA, 2012.
53. Sherwood J, Clark A, Lynas D. Enterprise Security Architecture: A Business-Driven Approach. Boca Raton, FL: CRC Press, 2005.
54. Curtis P, Carey M. Risk assessment in practice. Committee of Sponsoring Organizations of the Treadway Commission, 2012. Available at: http://www.coso.org.
55. Aven T, Cox LA. National and Global risk studies: How can the field of risk analysis contribute? Risk Analysis, 2016; 36(2):186–190. Available at: https://doi.org/10.1111/risa.12584, Accessed April 2017.
56. Quinn SD, Scarfone KA, Barrett M, Johnson CS. SP 800-117. Guide to Adopting and Using the Security Content Automation Protocol (SCAP) Version 1.0. NIST, 2010.
57. Naaliel M, Joao D, Henrique M. Security benchmarks for web serving systems. Pp. 1–12 in Proceedings of the 25th IEEE International Symposium on Software Reliability Engineering (ISSRE'14), 2014.
58. Patcha A, Park JM. An overview of anomaly detection techniques: Existing solutions and latest technological trends. Computer Networks, 2007; 51(12):3448–3470.
59. Aven T, Zio E. Foundational issues in risk assessment and risk management. Risk Analysis, 2014; 34(7):1164–1172.
60. Manadhata PK, Wing JM. An attack surface metric. IEEE Transactions on Software Engineering, 2011; 37:371–386.
61. Howard M, Pincus J, Wing JM. Measuring relative attack surfaces. Pp. 109–137 in Lee Dt, Shieh SP, Tygar JD (eds). Computer Security in the 21st Century. Boston, MA: Springer US, 2005.
62. Manadhata PK, Wing JM, Flynn MA, McQueen MA. Measuring the attack surfaces of two FTP daemons. Pp. 3–10 in Proceedings of the 2nd Workshop on Quality of Protection, 2006.
63. Baiardi F, Telmon C, Sgandurra D. Hierarchical, model-based risk management of critical infrastructures. Reliability Engineering & System Safety, 2009; 94(9):1403–1415.
64. Poolsappasit N, Dewri R, Ray I. Dynamic security risk management using Bayesian attack graphs. IEEE Transactions on Dependable and Secure Computing, 2012; 9(1):61–74.
65. Ten CW, Manimaran G, Liu CC. Cybersecurity for critical infrastructures: Attack and defense modeling. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 2010; 40(4):853–865.
66. Serra E, Jajodia S, Pugliese A, Rullo A, Subrahmanian VS. Pareto-optimal adversarial defense of enterprise systems. ACM Transactions on Information and System Security, 2015; 17(3):11:1–11:39. Available at: https://doi.acm.org/10.1145/2699907, Accessed April 2017.
67. Hewett R, Rudrapattana S, Kijsanayothin P. Cyber-security analysis of smart grid SCADA systems with game models. Pp. 109–112 in Proceedings of the 9th Annual Cyber and Information Security Research Conference. ACM, 2014.
68. Dewri R, Poolsappasit N, Ray I, Whitley D. Optimal security hardening using multi-objective optimization on attack tree models of networks. Pp. 204–213 in Proceedings of the 14th ACM Conference on Computer and Communications Security. CCS '07. ACM, 2007. Available from: https://doi.acm.org/10.1145/1315245.1315272, Accessed April 2017.
69. Holm H. A large-scale study of the time required to compromise a computer system. IEEE Transactions on Dependable and Secure Computing, 2014; 11(1):2–15.
70. Henry MH, Haimes YY. A comprehensive network security risk model for process control networks. Risk Analysis, 2009; 29(2):223–248.
71. McQueen MA, Boyer WF, Flynn MA, Beitel GA. Quantitative cyber risk reduction estimation methodology for a small SCADA control system. Proceedings of the 39nd Hawaii International Conference on System Sciences, 2006; 9:226.
72. Byres EJ, Franz M, Miller D. The use of attack trees in assessing vulnerabilities in SCADA systems. Pp. 3–10 in Proceedings of the International Infrastructure Survivability Workshop. Citeseer, 2004.
73. Ten CW, Liu CC, Govindarasu M. Vulnerability assessment of cybersecurity for SCADA systems using attack trees. Pp. 1–8 in Power Engineering Society General Meeting, 2007. IEEE, 2007.
74. Christey S, Martin B. Buying into the bias: Why vulnerability statistics suck. BlackHat, 2013. Available at: https://doi.org/www.blackhat.com/us-13/archives.html#Martin, Accessed April 2017.
75. Grossklags J, Christin N, Chuang J. Secure or insure?: A game-theoretic analysis of information security games. Pp. 209–218 in Proceedings of the 17th International Conference on World Wide Web. ACM, 2008.
76. Beresnevichiene Y, Tsifountidis F. Network security risk assessment. Google Patents, 2014. US Patent 8,650,637.
77. Kang PY, Sim WT, Kim WH. Security risk evaluation method for effective threat management. Google Patents, 2007. US Patent App. 11/941,193.
78. Tambe M, Paruchuri P, Ordóñez F, Kraus S, Pearce J, Marecki J. Agent security via approximate solvers. Google Patents, 2012. US Patent 8,195,490.
79. Wiemer D, Robert JM, McFarlane B, Gustave C, Chow S, Tang J. Application of cut-sets to network interdependency security risk assessment. Google Patents, 2005. US Patent App. 11/232,004.
80. Cohen G, Meiseles M, Reshef E. System and method for risk detection and analysis in a computer network. Google Patents, 2005. US Patent 6,952,779.
81. NTIA. NTIA vulnerability disclosure call for comments. NTIA, 2016. Available at: https://doi.org/www.ntia.doc.gov/files/ntia/publications/fr_meeting_vulnerability_disclosure_msp_04082016.pdf, Accessed December 2016.
82. Tversky A, Kahneman D. Judgment under uncertainty: Heuristics and biases. Science. 1974; 185(4157):1124–1131. Available at: http://science.sciencemag.org/content/185/4157/1124, Accessed April 2017.
83. Cox Jr LAT. Some limitations of “risk= threat× vulnerability× consequence” for risk analysis of terrorist attacks. Risk Analysis, 2008; 28(6):1749–1761.
84. Romanosky S. Examining the costs and causes of cyber incidents. Journal of Cybersecurity, 2016; 2(2):121–135.
85. Romanosky S, Hoffman D, Acquisti A. Empirical analysis of data breach litigation. Journal of Empirical Legal Studies, 2014; 11(1):74–104.
86. Yayla AA, Hu Q. The impact of information security events on the stock value of firms: The effect of contingency factors. Journal of Information Technology, 2011; 26(1): 60–77.
87. Davis G, Garcia A, Zhang W. Empirical analysis of the effects of cyber security incidents. Risk Analysis, 2009; 29(9):1304–1316. Available at: https://doi.org/10.1111/j.1539-6924.2009.01245.x, Accessed April 2017.
88. Ezell BC, Bennett SP, Von Winterfeldt D, Sokolowski J, Collins AJ. Probabilistic risk analysis and terrorism risk. Risk Analysis, 2010; 30(4):575–589. Available at: https://doi.org/10.1111/j.1539-6924.2010.01401.x, Accessed April 2017.
89. Brown GG, Cox, LAT Jr. How probabilistic risk assessment can mislead terrorism risk analysts. Risk Analysis, 2011; 31(2):196–204. Available at: https://doi.org/10.1111/j.1539-6924.2010.01492.x, Accessed April 2017.
90. Franzetti C. Operational Risk Modelling and Management. Boca Raton, FL: CRC Press, 2016.
91. Allodi L, Massacci F, Williams J. The work-averse cyber attacker model. Evidence from two million attack signatures. Pp. 1–35 in WEIS, 2017. Available at: https://doi.org/ssrn.com/abstract=2862299, Accessed April 2017.
92. Allodi L, Kotov V, Massacci F. MalwareLab: Experimentation with cybercrime attack tools. Pp. 1–8 in Proceedings of the 2013 6th Workshop on Cybersecurity Security and Test, 2013.
93. Dumitras T, Efstathopoulos P. Ask WINE: Are we safer today? Evaluating operating system security through big data analysis. P. 11 in Proceeding of the 2012 USENIX Workshop on Large-Scale Exploits and Emergent Threats. LEET'12, 2012.
94. Baker W, Howard M, Hutton A, Hylender CD. 2012 Data Breach Investigation Report. Verizon, 2012.
95. Panjwani S, Tan S, Jarrin KM, Cukier M. An experimental evaluation to determine if port scans are precursors to an attack. Pp. 602–611 in Dependable Systems and Networks, 2005. IEEE, 2005.
96. Harris B, Hunt R. TCP/IP security threats and attack methods. Computer Communications, 1999; 22(10):885–897.
97. Haimes YY. On the complex definition of risk: A systems-based approach. Risk Analysis, 2009; 29(12):1647–1654.
98. Brito J, Watkins T. Loving the cyber bomb: The dangers of threat inflation in cybersecurity policy. Harvard National Security Journal, 2011; 3:39–84.
99. Smith S, Winchester D, Bunker D, Jamieson R. Circuits of power: A study of mandated compliance to an information systems security “de jure” standard in a government organization. MIS Quarterly, 2010; 34(3):463–486.
100. Boehmer W. Appraisal of the effectiveness and efficiency of an information security management system based on ISO 27001. Pp. 224–231 in 2010 Fourth International Conference on Emerging Security Information, Systems and Technologies, 2008.
101. Mellado D, Fernández-Medina E, Piattini M. A comparison of software design security metrics. Pp. 236–242 in Proceedings of the Fourth European Conference on Software Architecture. ECSA '10. ACM, 2010. Available at: https://doi.acm.org/10.1145/1842752.1842797, Accessed April 2017.
102. Anderson R. Security economics—A personal perspective. Pp. 139–144 in Proceedings of the 28th Annual Computer Security Applications Conference, 2012.
103. Nappa A, Johnson R, Bilge L, Caballero J, Dumitras T. The attack of the clones: A study of the impact of shared code on vulnerability patching. Pp. 692–708 in Proceedings of the 36th IEEE Symposium on Security and Privacy, 2015.
104. Ning P, Xu D. Learning attack strategies from intrusion alerts. Pp. 200–209 in Proceedings of the 10th ACM Conference on Computer and Communications Security. ACM, 2003.
105. Kieyzun A, Guo PJ, Jayaraman K, Ernst MD. Automatic creation of SQL injection and cross-site scripting attacks. Pp. 199–209 in Software Engineering, 2009. ICSE 2009. IEEE, 2009.
106. Ornaghi A, Valleri M. Man in the middle attacks demos. Blackhat, 2003; 19.