A big data architecture for large scale security monitoring

Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014

Volumn , Issue , 2014, Pages 56-63

  Marchal, Samuel ^a,b   Jiang, Xiuyan ^a   State, Radu ^a   Engel, Thomas ^a  

^a UNIVERSITY OF LUXEMBOURG   (Luxembourg)

^b UNIVERSITÉ DE LORRAINE   (France)

Author keywords

[No Author keywords available]

Indexed keywords

BIG DATA; DIGITAL STORAGE; DISTRIBUTED DATABASE SYSTEMS; INFORMATION MANAGEMENT; INTRUSION DETECTION; NETWORK ARCHITECTURE;

DATA ARCHITECTURES; DATA CORRELATIONS; DISTRIBUTED DATA STORAGES; DISTRIBUTED SYSTEMS; ENTERPRISE NETWORKS; HOLISTIC ANALYSIS; NETWORK INTRUSION DETECTION; SECURITY MONITORING;

NETWORK SECURITY;

EID: 84923911158     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/BigData.Congress.2014.18     Document Type: Conference Paper

References (36)

1. J. P. Anderson, "Computer security threat monitoring and surveillance, " Fort Washington, Pennsylvania, Tech. Rep., 1980.
2. D. E. Denning, "An intrusion-detection model, " IEEE Transactions in Software Engineering, vol. 13, no. 2, Feb. 1987.
3. M. Roesch, "Snort-lightweight intrusion detection for networks, " in Proceedings of the 13th USENIX conferen ce on System administration, ser. LISA '99, 1999, pp. 229-238.
4. V. Paxson, "Bro: a system for detecting network intruders in real-time, " in Proceedings of the 7th conference on USENIX Security Symposium-Volume 7, ser. SSYM'98, 1998.
5. "Suricata, open source ids/ips/nsm engine. " [Online]. Available: http://www.suricata-ids.org
6. M. Antonakakis,R. Perdisci,D. Dagon,W. Lee,N. Feamster Building a dynamic reputation system for DNS in Proceedings of the 19th Usenix Security Symposium 2010.
7. L. Bilge, E. Kirda, C. Kruegel, and M. Balduzzi, "Exposure: Finding malic ious domains using passive DNS analysis," in Proceedings of NDSS, 2011.
8. A. Sperotto, G. Schaffrath, R. Sadre, C. Morariu, A. Pras, and B. Stiller, " An overview of ip flow-based intrusion detection," Communications Surveys Tutorials, IEEE, vol. 12, no. 3, pp. 343-356, Third 2010.
9. J. Franç ois, S. Wang, R. State, and T. Engel, "Bottr ack: Tracking botnets using netflow and pagerank," in Proceedings of the 10th International IFIP TC 6 Conference on Networking-Volume Part I, ser. NETWORKING' 11. Berlin, Heidelberg: Springer-Verlag, 2011, pp. 1-14.
10. C. Kreibich and J. Crowcroft, "Honeycomb: creating intrusion detection signatures using honeypots, " SIGCOMM Comput. Commun. Rev., vol. 34, no. 1, pp. 51-56, Jan. 2004.
11. F. Weimer, "Passive dns replication, " in Proceedings of the 17th Annual FIRST Conference on Computer Security Incident Handling, 2005.
12. R. Edmonds, "ISC Passive DNS Architecture, " Internet Systems Consortium, Inc., Tech. Rep., 2012. [Online]. Available: https://security.isc.org/Passive DNS/passive-dns-Architecture.pdf
13. A. Lakshman and P. Malik, "Cassandra: structured storage system on a p2p network, " in Proceedings o f the 28th ACM symposium on Principles of distributed computing, ser. PODC '09. New York, NY, USA: ACM, 2009, pp. 5-5.
14. "Cisco netflow. " [Online]. Available: http://www.cisco.com/web/go/netflow
15. "Dionaea, catches bugs. " [Online]. Available: http://dionaea.carnivore.it/
16. S. Marchal, J. François, C. Wagner, R. State, A. Dulaunoy, T. Engel, and O. Festor, " DNSSM: A large scale passive DNS security monitoring framework," ser. NOMS'12, 2012.
17. Y. Lee and Y. Lee, "Toward scalable internet traffic measurement and analysis with hadoop, " SIGCOMM Comput. Commun. Rev., vol. 43, no. 1, pp. 5-13, 2012.
18. T. White, Hadoop: The Definitive Guide. O'Reilly Media, June 2009.
19. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica, " Spark: cluster computing with working sets," in Proceedings of the 2nd USENIX conference on Hot topics in cloud computing, ser. HotCloud'10, 2010.
20. J. Dean and S. Ghemawat, "MapReduce: simplified data processing on large clusters, " in Symposium on Opearting Systems Design & Implementation (OSDI). USENIX Association, 2004.
21. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy, " Hive: a warehousing solution over a mapreduce framework," Proc. VLDB Endow., vol. 2, no. 2, pp. 1626-1629, Aug. 2009.
22. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins, " Pig latin: a not-so-foreign language for data processing," in Proceedings of the 2008 ACM SIGMOD international conference on Management of data, ser. SIGMOD '08. New York, NY, USA: ACM, 2008, pp. 1099-1110.
23. C. Engle, A. Lupher, R. Xin, M. Zaharia, M. J. Franklin, S. Shenker, and I. Stoica, " Shark: fast data analysis using coarse-grained distributed memory," in Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, ser. SIGMOD '12, 201 2, pp. 689-692.
24. A. Valdes and K. Skinner, "Probabilistic alert correlation, " in Proceedings of the 4th International Symposium on Recent Advances in Intrusion Detection, 2001, pp. 54-68.
25. D. Xu and P. Ning, "Alert correlation through triggering events and common resources, " in Proceedings of the 20th Annual Computer Security Applications Conference, 2004, pp. 360-369.
26. T. Chyssler, S. Burschka, M. Semling, T. Lingvall, and K. Burbeck, " Alarm reduction and correlation in intrusion detection systems," in Proceedings of Detection of Intrusions and Malware Vulnerability Assessment workshop (DIMVA), 2004, pp. 9-24.
27. C. Endorf,E. Schultz,J. Mellander Intrusion detection & prevention. McGraw-Hill 2004.
28. L. Ricciulli, "A service model for network secur ity applications," in Proceedings of the Sixth Annual Workshop on Cyber Security and Information Intelligence Research, ser. CSIIRW '10, 2010.
29. G. Vasiliadis, M. Polychronakis, and S. Ioannidis, "Midea: A m ultiparallel intrusion detection architecture," in Proceedings of the 18th ACM Conference on Computer and Communications Security, ser. CCS '11. New York, NY, USA: ACM, 2011, pp. 297-308.
30. M. A. Jamshed, J. Lee, S. Moon, I. Yun, D. Kim, S. Lee, Y. Yi, and K. Park, " Kargus: A highly-scalable software-based intrusion detection system," in Proceedings of the 2012 ACM Conference on Computer and Communications Securi ty, ser. CCS '12. New York, NY, USA: ACM, 2012, pp. 317-328.
31. H. Jiang, G. Zhang, G. Xie, K. Salamatian, and L. Mathy, " Scalable high-performance parallel design for network intrusion detection systems on many-core processors," in Proceedings of the Ninth ACM/IEEE Symposium on Architectures for Networking and Communications Systems, ser. ANCS '13. Piscataway, NJ, USA: IEEE Press, 2013, pp. 137-146.
32. H. Gill, D. Lin, X. Han, C. Nguyen, T. Gill, and B. T. Loo, " Scalanytics: A declarative multi-core platform for scalable composable traffic analytics," in Proceedings of the 22Nd International Symposium on Highperformance Parallel and Distributed Computing, ser. HPDC '13. New York, NY, USA: ACM, 2013, pp. 61-72.
33. A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. DeWitt, S. Madden, and M. Stonebraker, " A comparison of approaches to large-scale data analysis," in Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data, ser. SIGMOD '09. New York, NY, USA: ACM, 2009, pp. 165-178.
34. S. Blanas, J. M. Patel, V. Ercegovac, J. Rao, E. J. Shekita, and Y. Tian, " A comparison of join algorithms for log processing in mapreduce," in Proceedings of the 2010 ACM SIGMOD International Con ference on Management of Data, ser. SIGMOD '10. New York, NY, USA: ACM, 2010, pp. 975-986.
35. M. Zaharia, T. Das, H. Li, S. Shenker, and I. Stoica, " Discretized streams: An efficient and fault-tolerant model for stream processing on large clusters," in Proceedings of the 4th USENIX Conference on Hot Topics in Cloud Ccomputing, ser. HotCloud'12. Berkeley, CA, USA: USENIX Association, 2012, pp. 10-10.
36. "Storm. " [Online]. Available: http://storm-project.net/