Cloud log forensics: Foundations, state of the art, and future directions

ACM Computing Surveys

Volumn 49, Issue 1, 2016, Pages

  Khan, Suleman ^a   Gani, Abdullah ^a   Wahab, Ainuddin Wahid Abdul ^a   Bagiwa, Mustapha Aminu ^a   Shiraz, Muhammad ^b   Khan, Samee U ^c   Buyya, Rajkumar ^d   Zomaya, Albert Y ^e  

^a UNIVERSITY OF MALAYA   (Malaysia)

^b FEDERAL URDU UNIVERSITY OF ARTS   (Pakistan)

^c North Dakota State University   (United States)

^d UNIVERSITY OF MELBOURNE   (Australia)

^e UNIVERSITY OF SYDNEY   (Australia)

Author keywords

Authenticity; Big data; Cloud computing; Cloud log forensics; Confidentiality; Correlation of cloud logs; Integrity

Indexed keywords

BIG DATA; CLOUD COMPUTING; NETWORK SECURITY; TRANSPORTATION; WELL LOGGING;

AUTHENTICITY; CLOUD SERVICE PROVIDERS; CONFIDENTIALITY; INTEGRITY; INVESTIGATION PROCESS; MALICIOUS ACTIVITIES; MALICIOUS BEHAVIOR; SECURITY REQUIREMENTS;

COUPLINGS;

EID: 84971268778     PISSN: 03600300     EISSN: 15577341     Source Type: Journal    
DOI: 10.1145/2906149     Document Type: Article

References (107)

1. A. Burton. 2014. Real-time log management and analytics at any scale. (2014). Retrieved November 16, 2015, from https://logentries.com/.
2. A. Chuvakin, K. Schmidt, and Chris Phillips. 2013. Logging and Log Management: The Authoritative Guide to Understanding the Concepts Surrounding Logging and Log Management. Syngress, 460 pages.
3. A. Gani, G. M. Nayeem, M. Shiraz, M. Sookhak, M. Whaiduzzaman, and S. Khan. 2014. A review on interworking and mobility techniques for seamless connectivity in mobile cloud computing. J. Network Comput. Appl. 43 (2014), 84-102.
4. A. Holovaty. 2014. Django Makes It Easier to Build Better Web Apps More Quickly and with Less Code. (2014). Retrieved November 16, 2015, from https://www.djangoproject.com/.
5. A. Oliner, A. Ganapathi, and W. Xu. 2012. Advances and challenges in log analysis. Commun. ACM 55, 2 (2012), 55-61.
6. A. Patrascu and V. V. Patriciu. 2014. Logging framework for cloud computing forensic environments. In Proceeding of the IEEE 10th International Conference on Communications (COMM). 1-4.
7. A. Patrascu and V. V. Patriciu. 2015. Logging for cloud computing forensic systems. Int. J. Comput. Commun. Control 10, 2 (2015), 222-229.
8. A. Prasad and P. Chakrabarti. 2014. Extending access management to maintain audit logs in cloud computing. Int. J. Adv. Comput. Sci. Appl. 5, 3 (2014), 144-147.
9. A. Rafael. 2013. Secure log architecture to support remote auditing. Math. Comput. Model. 57, 7 (2013), 1578-1591.
10. A. Stanojevic. 2013. Banca Intesa counters threats with HP ArcSight. Case Study. Hewlett-Packard. 4 pages. Retrieved November 16, 2015, from http://www8.hp.com/h20195/V2/GetPDF.aspx/4AA4-6020ENUS.pdf.
11. A. Williams. 2013. Loggly, a Splunk Competitor, Raises $10.5m for Cloud-Centric Approach to Log Management. (2013). Retrieved November 16, 2015, from http://techcrunch.com/2013/09/03/loggly-a-splunkcompetitor-raises-10-5m-for-cloud-centric-approach-to-log-management/.
12. Amazon. 2015. Amazon Simple Notification Service. (2015). Retrieved November 16, 2015, from http://aws.amazon.com/sns/.
13. B. Mizerany. 2014. Put this in your pipe and smoke it. (2014). Retrieved November 16, 2015, from http://www.sinatrarb.com/.
14. B. Mollamustafaoglu. 2014. We make alerts work for you. (2014). Retrieved November 16, 2015, from https://www.opsgenie.com/.
15. B. R. Carrier. 2006. Risks of live digital forensic analysis. Commun. ACM 49, 2 (2006), 56-61.
16. C. C. Yun, J. Y. C. Chang, B. B. C. Chiu, D. Y. Shue, Y. Kaneyasu, and J.W.Warfield. 2014. Ensuring integrity of security event log upon download and delete. (2014). U.S. Patent No. 8, 856, 086.
17. C. Oppenheimer. 2009. Loggly reveals what matters. (2009). Retrieved November 16, 2015, from https://www.loggly.com/.
18. C. Rong, S. T. Nguyen, and M. G. Jaatun. 2013. Beyond lightning: A survey on security challenges in cloud computing. Comput. Electr. Eng. 39, 1 (2013), 47-54.
19. D. J. Scales, M. Xu, and M. D. Ginzton. 2013. Low overhead fault tolerance through hybrid checkpointing and replay. U.S. Patent No. 8, 499, 297 (2013).
20. D. Birk. 2011. Technical challenges of forensic investigations in cloud computing environments. In Workshop on Cryptography and Security in Clouds. Zurich, Switzerland, 1-6.
21. D. Birk and C. Wegener. 2011. Technical issues of forensic investigations in cloud computing environments. In Proceeding of the IEEE 6th International Workshop on Systematic Approaches to Digital Forensic Engineering (SADFE). Washington, DC, USA, 1-10.
22. E. Casey. 2009. Handbook of Digital Forensics and Investigation. Academic Press, San Diego, CA, 600 pages.
23. E. J. Janger and P.M. Schwartz. 2001. Gramm-Leach-Bliley act, information privacy, and the limits of default rules. The. Minn. L. Rev. 86 (2001), 1219.
24. E. Lindvall. 2014. How Papertrail makes life easier. (2014). Retrieved November 16, 2015, from https://papertrailapp.com/.
25. G. Rocher. 2005. A powerful Groovy-basedWeb application framework for the JVM. (2005). Retrieved November 16, 2015, from https://grails.org/.
26. G. Samudra. 2005. Extending Log4j to create custom logging components. In Logging in Java with the JDK 1.4 Logging API and Apache Log4j. Apress. 235-284.
27. H. A. Jahdali, A. Albatli, P. Garraghan, P. Townend, L. Lau, and Jie Xu. 2014. Multi-tenancy in cloud computing. In Proceeding of the IEEE 8th International Symposium on Service Oriented System Engineering. Oxford, United Kingdom, 344-351.
28. H. Chung, J. Park, S. Lee, and C. Kang. 2012. Digital forensic investigation of cloud storage services. Digital Invest. 9, 2 (2012), 81-95.
29. H. H. Mao, C. J. Wu, E. E. Papalexakis, C. Faloutsos, K. C. Lee, and T. C. Kao. 2014. MalSpot: Multi2 malicious network behavior patterns analysis. In Advances in Knowledge Discovery and Data Mining. Springer, Berlin, (2014), 1-14.
30. I. A. T. Hashem, I. Yaqoob, N. B. Anuar, S. Mokhtar, A. Gani, and S. U. Khan. The rise of "big data" on cloud computing: Review and open research issues. Inform. Syst. 47 (2015), 98-115.
31. I. M. Abbadi. 2014. Cloud Management and Security. John Wiley & Sons, New York, 238 pages.
32. I. Ray, K. Belyaev, M. Strizhov, D. Mulamba, and M. Rajaram. 2013. Secure logging as a service-delegating log management to the cloud. IEEE Syst. J. 7 (2013), 323-334.
33. J. Dykstra and A. T. Sherman. 2011. Understanding issues in cloud forensics: Two hypothetical case studies. J. Network Forens. 3, 1 (2011), 19-31.
34. J. Gerring. 2007. Case Study Research. Principles and Practices. Cambridge University Press, Cambridge, 278 pages.
35. J. Hash, P. Bowen, A. Johnson, C. D. Smith, and D. I. Steinberg. 2008. An Introductory Resource Guide for Implementing the Health Insurance Portability and Accountability Act (HIPAA) Security Rule. Doctoral Dissertation, National Institute of Standards and Technology, 117 pages.
36. J. H. Beaver. 2015. Lessons on Efficient Log Analysis from Monex Insight. Case Study Report. Loggly Research. 3 pages. https://www.loggly.com/blog/lessons-efficient-log-analysis-monex-insight/.
37. J. Sissel. 2014. Process any data, from any source. (2014). Retrieved November 16, 2015, from https://www.elastic.co/products/logstash.
38. J. South. 2013. Heartland Payment Systems Hardens Applications and Blocks Attacks with the Aid of HP Security Software. Technical Report. IDC Go-To-Market Services. http://www8.hp.com/h20195/V2/GetPDF.aspx/4AA5-1356ENW.pdf.
39. J. Spring. 2011. Monitoring cloud computing by layer, part 1. IEEE Security Privacy 9, 2 (2011), 66-68.
40. J. Stoppelman. 2004. AWS Case Study: Yelp. Case Study. Amazon. Retrieved November 16, 2015, from https://aws.amazon.com/solutions/case-studies/yelp/.
41. J. T. Force and T. Initiative. 2013. Security and privacy controls for federal information systems and organizations. NIST Spec. Publ. 800 (2013), 53.
42. J. Turnbull. 2005. Understanding logging and log monitoring. Hardening Linux. A-press, Berkeley, California, 584 pages.
43. J. W. Joo, J. H. Park, S. K. Suk, and D. G. Lee. 2014. LISS: Log data integrity support scheme for reliable log analysis of osp. J. Converg. 5, 4 (2014), 1-5.
44. J. Wei, Y. Zhao, K. Jiang, R. Xie, and Y. Jin. 2011. Analysis farm: A cloud-based scalable aggregation and query platform for network log analysis. In Proceedings of the IEEE International Conference on Cloud and Service Computing (CSC). Hong Kong, 354-359.
45. J. Yang. N. Plasson, G. Gillis, N. Talagala, and S. Sundararaman. 2014. Don't stack your log on my log. In USENIX Workshop on Interactions of NVM/Flash with Operating Systems and Workloads (INFLOW). Broomfield, USA.
46. J. Yin. 2014. Cloud based logging service. US Patent 20, 140, 366, 118 (2014).
47. K. Kent, S. Chevalier, T. Grance, and H. Dang. 2006. Guide to integrating forensic techniques into incident response. NIST Spec. Publ. (2006), 800-886.
48. K.Kent andM. Souppaya. 2014. Guide to computer security log management. National Institute of Standards and Technology (2014). 72 pages.
49. K. L. K. Ryan, P. Jagadpramana, and B. S. Lee. 2011a. Flogger: A file-centric logger for monitoring file access and transfers within cloud computing environments. In Proceedings of the International Joint Conference of IEEE TrustCom-11/11/IEEE ICESS-11/FCST-11. 765-771.
50. K. L. K. Ryan, M. Kirchberg, and B. S. Lee. 2011b. From system-centric to data-centric logging-accountability, trust & security in cloud computing. In Proceedings of the IEEE Defense Science Research Conference and Expo (DSR). Singapore, 1-4.
51. K. Popovic and Z. Hocenski. 2010. Cloud computing security issues and challenges. In Proceedings of the IEEE 33rd International Convention (MIPRO). Opatija, Croatia, 344-349.
52. K. Ruan, J. Carthy, T. Kechadi, and M. Crosbie. 2011. Cloud forensics. Advances in Digital Forensics VII. Springer, Berlin, 35-46.
53. K. Ruan, J. James, J. Carthy, and T. Kechadi. 2012. Key terms for service level agreements to support cloud forensics. Advances in Digital Forensics VIII. Springer, Berlin, 201-212.
54. K. Saurabh and C. Beedgen. 2014. Master your data continous intelligence. (2014). Retrieved November 16, 2015, from https://www.sumologic.com/.
55. M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia. 2010. A view of cloud computing. Commun. ACM 53, 4 (2010), 50-58.
56. M. Baum. 2014. Analyze & troubleshoot your cloud applications. Technical Report. SplunkStorm. https://www.splunk.com/web-assets/pdfs/secure/Storm-Product-Fact-Sheet.pdf.
57. M. Bradley and A. Dent. 2010. Payment Card Industry Data Security: What it is and its impact on retail merchants. Technical Report. Royal Holloway Series. http://cdn.ttgtmedia.com/searchsecurityuk/downloads/RHUL-Bradley-2010.pdf.
58. M. Damshenas, A. Dehghantanha, R. Mahmoud, and S. B. Shamsuddin. 2012. Forensics investigation challenges in cloud computing environments. In Proceedings of the IEEE International Conference on Cyber Security, Cyber Warfare and Digital Forensics (CyberSec). 190-194.
59. M. Ellis. 2013. IBM Operations Analytics-Log Analysis. (2013). Retrieved November 16, 2015, from http://www-03.ibm.com/software/products/en/ibm-operations-analytics-log-analysis.
60. M. Lemoudden, N. Bouazza, and B. E. Ouahidi. 2014. Towards achieving discernment and correlation in cloud logging. In Proceedings of the Applications of Information Systems in Engineering and Bioscience. Gdansk, Poland, 202-207.
61. M. Sato and T. Yamauchi. 2013. Secure log transfer by replacing a library in a virtual machine. In Advances in Information and Computer Security. Springer, Berlin, 1-18.
62. M. Shiraz, A. Gani, A. Shamim, S. Khan, and R. W. Ahmad. 2015. Energy efficient computational offloading framework for mobile cloud computing. J. Grid Comput. 13, 1 (2015), 1-18.
63. M. Taylor, J. Haggerty, D. Gresty, and D. Lamb. 2011. Forensic investigation of cloud computing systems. Network Security 2011, 3 (2011), 4-10.
64. M. Vrable, S. Savage, and G. M. Voelker. 2012. BlueSky: A cloud-backed file system for the enterprise. In Proceedings of the 10th USENIX Conference on File and Storage Technologies. San Jose, CA, USA, 19-19.
65. N. Prabha, C. Timotta, T. Rajan, and A. Jaleef PK. 2014. Encrypted query processing based log management in the cloud for improved potential for confidentiality. Int. J. Comput. Appl. Technol. Res. 3, 5. (2014), 309-311.
66. N. Santos, K. P. Gummadi, and R. Rodrigues. 2009. Towards trusted cloud computing. In Proceedings of the 2009 Conference on Hot Topics in Cloud Computing. 3-3.
67. P. Heath. 2014. Monitor your apps every single second. (2014). Retrieved November 16, 2015, from http://www.bmc.com/truesightpulse/customers/.
68. P. M. Trenwith and H. S. Venter. 2014. A digital forensic model for providing better data provenance in the cloud. In Proceedings of the IEEE Information Security for South Africa (ISSA). 1-6.
69. P. Mell and T. Grace. 2011. The NIST definition of cloud computing. NIST Special Publication 800-145 (2011).
70. Q. Han, M. Shiraz, A. Gani, M. Whaiduzzaman, and S. Khan. 2014. Sierpinski triangle based data center architecture in cloud computing. J. Supercomput. 69, 2 (2014), 887-907.
71. R. A. Popa, J. R. Lorch, D. Molnar, H. J.Wang, and L. Zhuang. 2011. Enabling security in cloud storage SLAs with cloudproof. In Usenix Annual Technical Conference. 242 (2011).
72. R. Buyya, C. S. Yeo, and S. Venugopalirk. 2008. Market-Oriented cloud computing: Vision, hype, and reality for delivering IT services as computing utilities. In Proceeding of the IEEE 10th International Conference on High Performance Computing and Communications. 5-13.
73. R. Buyya, C. S. Yeo, S. Venugopalirk, J. Broberg, and I. Brandic. 2009. Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Generation Comput. Syst. 25, 6 (2009), 599-616.
74. R. Dahl. 2014. Node.js on the Road. (2014). Retrieved November 16, 2015 from https://www.joyent.com/noderoad.
75. R. Marty. 2011. Cloud application logging for forensics. In Proceedings of the 2011 ACM Symposium on Applied Computing. ACM, New York, NY, 178-184.
76. R. Vaarandi andM. Pihelgas. 2014. Using security logs for collecting and reporting technical security metrics. In Proceedings of the IEEE Military Communications Conference (MILCOM). 294-299.
77. S. Ahmad, B. Ahmad, S. M. Saqib, and R. M. Khattak. 2012. Trust model: Cloud's provider and cloud's user. Int. J. Adv. Sci. Technol. 44, (2012), 69-80.
78. S. Butterfield, E. Costello, C. Henderson, and S. Mourachov. 2014. Slack so yeah, we tried slack. (2014). Retrieved November 16, 2015, from https://slack.com/.
79. S. Khan, A. Gani, A. W. A. Wahab, and M. A. Bagiwa. 2015. SIDNFF: Source identification network forensics framework for cloud computing. In Proceeding of the IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW). 418-419.
80. S. Khan, A. Gani, A.W. A.Wahab, M. Shiraz, and I. Ahmad. 2016. Network forensics: Review, taxonomy, and open challenges. (in press).
81. S. Khan, E. Ahmad, M. Shiraz, A. Gani, A.W. A.Wahab, and M. A. Bagiwa. 2014a. Forensic challenges in mobile cloud computing. In Proceeding of the IEEE International Conference on Computer, Communication, and Control Technology (I4CT 2014). 343-347.
82. S. Khan, K. Hayat, S. A. Madani, S. U. Khan, and J. Kolodziej. 2012. The median resource failure check pointing. In 26th European Conference on Modelling and Simulation (ECMS). 483-489.
83. S. Khan, M. Shiraz, A. W. A. Wahab, A. Gani, Q. Han, and Z. B. A. Rahman. 2014b. A comprehensive review on adaptability of network forensics frameworks for mobile cloud computing. Sci. World J. 2014, 547062 (2014), 27.
84. S. Ramgovind, M. M. Eloff, and E. Smith. 2010. The management of security in cloud computing. In Proceedings of the IEEE Information Security for South Africa (ISSA). 1-7.
85. S. Simou, C. Kalloniatis, E. Kavakli, and S. Gritzalis. 2014. Cloud forensics: Identifying the major issues and challenges. In Advanced Information Systems Engineering. Springer, Berlin, 271-284.
86. S. Sundareswaran, A. C. Squicciarini, and D. Lin. 2012. Ensuring distributed accountability for data sharing in the cloud. IEEE Trans. Depend. Secure Comput. 9, 4 (2012). 556-568.
87. S. T. On, J. Xu, B. Choi, H. Hu, and B. He. 2012. Flag commit: Supporting efficient transaction recovery in flash-based dbmss. IEEE Trans. Knowled. Data Eng. 24, 9 (2012), 1624-1639.
88. S. Thorpe, I. Ray, T. Grandison, and A. Barbir. 2011a. The virtual machine log auditor. In Proceeding of the IEEE 1st International Workshop on Security and Forensics in Communication Systems. 1-7.
89. S. Thorpe, I. Ray, and T. Grandison. 2011b. A synchronized log cloud forensic framework. The International Conference on Cybercrime, Security & Digital Forensics. 14 pages.
90. S. Thorpe, I. Ray, and T. Grandison. 2011c. Enforcing data quality rules for a synchronized VM log audit environment using transformation mapping techniques. In Computational Intelligence in Security for Information Systems. Springer, Berlin, 265-271.
91. S. Thorpe, I. Ray, T. Grandison, and A. Barbir. 2012a. Cloud log forensics metadata analysis. In Proceedings of the IEEE Computer Software and Applications Conference Workshops (COMPSACW). 194-199.
92. S. Thorpe, I. Ray, T. Grandison, A. Barbir, and R. France. 2013b. Hypervisor event logs as a source of consistent virtual machine evidence for forensic cloud investigations. In Data and Applications Security and Privacy XXVII. Springer, Berlin, 97-112.
93. S. Thorpe, I. Ray, I. Ray, and T. Grandison. 2011d. A formal temporal log data model for the global synchronized virtual machine environment. Int. J. Inform. Assur. Secur. 6, 2 (2011), 398-406.
94. S. Thorpe, I. Ray, I. Ray, T. Grandison, A. Barbir, and R. France. 2012b. Formal parameterization of log synchronization events within a distributed forensic compute cloud database environment. In Digital Forensics and Cyber Crime. Springer, Berlin, 156-171.
95. S. Thorpe, T. Grandison, A. Campbell, J. Williams, K. Burrell, and I. Ray. 2013a. Towards a forensic-based service oriented architecture framework for auditing of cloud logs. In Proceeding of the IEEE 9th World Congress on Services. 75-83.
96. T. Nielsen. 2014. Everything you need to build, run, and scale. (2014). Retrieved November 16, 2015, from https://www.heroku.com/.
97. T. R. Wyatt. 2009. Mission: Messaging: Circular Logs Vs Linear Logs. (2014). Retrieved November 16th, 2015 from http://www.ibm.com/developerworks/websphere/techjournal/0904-mismes.html.
98. T. Sang. 2013. A log-based approach to make digital forensics easier on cloud computing. In Proceeding of the IEEE 3rd International Conference on Intelligent System Design and Engineering Applications (ISDEA). 91-94.
99. T. Simon. 2014. KPI Dashboards that put your data to work. Retrieved November 16, 2015, from https://www.geckoboard.com/.
100. U. Flegel. 2002. Pseudonymizing unix log files. In Infrastructure Security. Springer, Berlin, 162-179.
101. V. Wesley, T. Harris, L. Long Jr., and R. Green. 2014. Hypervisor security in cloud computing systems. ACM Comput. Surv. (2014), 1-22.
102. X. Lin, P. Wang, and B. Wu. 2013. Log analysis in cloud computing environment with hadoop and spark. In Proceedings of the IEEE 5th International Conference on Broadband Network & Multimedia Technology (IC-BNMT2013). 273-276.
103. Z. Nik. 2011. Detection of network security breaches based on analysis of network record logs. U.S. Patent No. 7, 904, 479 (2011).
104. Z. Shams, A. K. Dutta, and R. Hasan. 2013. SecLaaS: Secure logging-as-a-service for cloud forensics. In Proceedings of the 8th ACM SIGSAC Symposium on Information, Computer and Communications Security. ACM, New York, NY, 219-230.
105. Z. Shams, M. Mernik, and R. Hasan. 2014. Towards building a forensics aware language for secure logging. Comput. Sci. Inform. Syst. 11, 4 (2014), 1291-1314.
106. Z. Shen, L. Li, F. Yan, and X. Wu. 2010. Cloud computing system based on trusted computing platform. In Proceeding of the IEEE Intelligent Computation Technology and Automation (ICICTA). 942-945.
107. Z. Zibin, J. Zhu, and M. R. Lyu. 2013. Service-generated big data and big data-as-a-service: An overview. In Proceedings of the IEEE International Congress on Big Data (BigData Congress). 403-410.