Reference Architecture and Classification of Technologies, Products and Services for Big Data Systems

Big Data Research

Volumn 2, Issue 4, 2015, Pages 166-186

  Pääkkönen, Pekka ^a   Pakkala, Daniel ^a  

^a VTT TECHNICAL RESEARCH CENTRE OF FINLAND   (Finland)

Author keywords

Big data; Classification; Literature survey; Reference architecture

Indexed keywords

CLASSIFICATION (OF INFORMATION); NETWORK ARCHITECTURE; SOCIAL NETWORKING (ONLINE); SURVEYS;

BUSINESS CASE; DATA SYSTEMS; FACEBOOK; IMPLEMENTATION ARCHITECTURE; LINKEDIN; LITERATURE SURVEY; PRODUCT AND SERVICES; REFERENCE ARCHITECTURE; TECHNOLOGY PRODUCTS; TECHNOLOGY SERVICE;

BIG DATA;

EID: 84946822182     PISSN: None     EISSN: 22145796     Source Type: Journal    
DOI: 10.1016/j.bdr.2015.01.001     Document Type: Review

References (92)

1. Sumbaly R., Kreps J., Shah S. The "Big Data" Ecosystem at LinkedIn. 2013 ACM SIGMOD International Conference on Management of Data 22-27 June, 2013.
2. Amatriain X. Big & Personal: data and models behind Netflix recommendations. The 2nd International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications 11 August, 2013.
3. Mishne G. Fast data in the era of big data: Twitter's real-time related query suggestion architecture. The 2013 ACM SIGMOD International Conference on Management of Data 22-27 June, 2013.
4. Simoncelli D., Dusi M., Gringoli F., Niccolini S. Stream-monitoring with BlockMon: convergence of network measurements and data analytics platforms. ACM SIGCOMM Commun. Rev. 2013, 43:29-35.
5. Zeng C., et al. FIU-miner: a fast, integrated, and user-friendly system for data mining in distributed environment. 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 11-14 August, 2013.
6. Thusoo A., et al. Data warehousing and analytics infrastructure at Facebook. 2010 ACM SIGMOD International Conference on Management of Data 6-11 June, 2010.
7. Meier M. Towards a big data reference architecture October 2013, Master's thesis, Eindhoven University of Technology.
8. Schmidt R., Möhring M. Strategic alignment of cloud-based architectures for big data. 17th IEEE International Enterprise Distributed Object Computing Conference Workshops 9-13 September, 2013.
9. Demchenko Y., Ngo C., Membrey P. Architecture framework and components for the Big Data Ecosystem September 12, 2013, SNE Technical Report, University of Amsterdam.
10. Cuesta C.E., Martinez-Prieto M.A., Fernandez J.D. Towards an architecture for managing big semantic data in real-time. 7th European Conference on Software Architecture 1-5 July, 2013.
11. Chen C.L.P., Zhang C. Data-intensive applications, challenges, techniques and technologies: a survey on Big Data. Inf. Sci. 2014, 275:314-347.
12. Angelov A., Grefen P., Greefhorst D. A framework for analysis and design of software reference architectures. Inf. Softw. Technol. 2012, 54:417-431.
13. Begoli E. A short survey on the state of the art in architectures and platforms for large scale data analysis and knowledge discovery from data. The 10th Working IEEE/IFIP Conference on Software Architecture & 6th European Conference on Software Architecture (WICSA/ECSA) 20-24 August, 2012.
14. Chen M., Mao S., Liu Y. Big data: a survey. Mob. Netw. Appl. 2014, 18.
15. Wu X., Wu G., Ding W. Data mining with big data. IEEE Trans. Knowl. Data Eng. 2014, 28:97-106.
16. Cuzzocrea A., Sacca D., Ullman J.D. Big data: a research agenda. The 17th International Database Engineering & Applications Symposium 09-11 October, 2013.
17. Galster M., Avgeriou P. Empirically-grounded reference architectures: a proposal. Joint ACM SIGSOFT Conference on Quality of Software Architectures and ACM SIGSOFT Conference on Quality of Software Architectures and ACM SIGSOFT Symposium on Architecting Critical Systems June 20-24, 2011.
18. Zimmermann A., et al. Towards and integrated service-oriented reference enterprise architecture. International Workshop on Ecosystem Architectures 19 August, 2013.
19. Doshi K.A., et al. Blending SQL and NewSQL approaches reference architectures for enterprise big data challenges. The International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery 10-12 October, 2013.
20. Zhong T., et al. On mixing high-speed updates and in-memory queries a big-data architecture for real-time analytics. IEEE International Conference on Big Data 6-9 October, 2013.
21. Lin J., Ryaboy D. Scaling big data mining infrastructure: the Twitter experience. ACM SIGKDD Explor. Newsl. 2013, 14:6-19.
22. Lee G.L., Lin J., Liu C., Lorek A., Ryaboy D. The unified logging infrastructure for data analytics at Twitter. The 38th International Conference on Very Large Databases 27-31 August, 2012.
23. Gupta P., et al. WTF: the who to follow service at Twitter. The International World Wide Web Conference 13-17 May, 2013.
24. Dusi M., et al. BlockMon: flexible and high-performance big data stream analytics platform and its use cases. NEC Tech. J. 2012, 7:102-106.
25. Lee Y., Lee Y. Toward scalable internet traffic measurement and analysis with hadoop. ACM SIGCOMM Commun. Rev. 2013, 43:5-13.
26. Lee Y., Kang W., Lee Y. A hadoop-based packet trace processing tool. International Workshop on Traffic Monitoring and Analysis April 27, 2011.
27. Kreps J., Narkhede N., Rao J. Kafka: a distributed messaging system for log processing. The 6th International Workshop on Networking Meets Databases 12 June, 2011.
28. Wu L., et al. Avatara: OLAP for web-scale analytics products. 38th International Conference on Very Large Databases 27-31 August, 2012.
29. Busch M., et al. EarlyBird: real-time search at Twitter. 2012 IEEE 28th International Conference on Data Engineering 1-5 April, 2012.
30. Amatriain X. System architectures for personalized recommendations Available via Netflix, accessed 25 Nov., 2013. http://techblog.netflix.com/2013/03/system-architectures-for.html.
31. Boulon J., et al. Chukwa: a large-scale monitoring system. Cloud Computing and its Applications 22-23 October, 2008.
32. Woznica A., Nguyen R., Kalousis A. Model mining for robust feature selection. The 18th ACM SIGKDD Conference on Knowledge Discovery and Data Mining 12-16 August, 2012.
33. Mavani M., Ragha L. MapReduce frame work: investigating suitability for faster data analytics. Adv. Comput. Commun. Control CCIS 2013, 361:119-130.
34. Condie T., et al. MapReduce online. The 7th USENIX Conference on Networked Systems Design and Implementation 28-30 April, 2010.
35. Lam W., et al. Muppet: MapReduce-style processing of fast data. The 38th International Conference on Very Large Databases 27-31 August, 2012.
36. Neumeyer L., Robbins B., Nair A., Kesari A. S4: distributed stream computing platform. The IEEE International Conference on Data Mining Workshops 13 December, 2010.
37. Chauhan J., Chowdhury S.A., Makaroff D. Performance evaluation of Yahoo! S4: a first look. The 7th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing 12-14 November, 2012.
38. SQLStream, processing and analysing streams of CDRs in real time accessed 13/04/2014. http://www.sqlstream.com/wp-content/uploads/2014/02/SQLstream_Whitepaper_CDR-Analytics-update.pdf.
39. Qian Z., et al. TimeStream: reliable stream computation in the cloud. Eurosys Conference 14-17 April, 2013.
40. Zaharia M., et al. Discretized streams: fault-tolerant streaming computation at scale. 24th ACM Symposium on Operating System Principles 3-8 November, 2013.
41. Murray D.G., et al. Naiad: a timely dataflow system. The 24th ACM Symposium on Operating System Principles 3-6 November, 2013.
42. Matsuura H., Ganse M., Suzumura T. A highly efficient consolidated platform for stream computing and hadoop. The 26th International Parallel and Distributed Processing Symposium 21-25 May, 2012.
43. Angles R., Gutierrez C. Survey of graph database models. ACM Comput. Surv. 2008, 40:1-39.
44. Stonebraker M., Madden S., Dubey P. Intel "Big Data" science and technology center vision and execution plan. SIGMOD Rec. 2013, 42:44-49.
45. Angles R. A comparison of current graph database models. The 28th IEEE International Conference on Data Engineering Workshops 1-5 April, 2012.
46. Shao B., Wang H., Xiao Y. Managing and mining large graphs: systems and implementations. SIGMOD 2012 20-24 May, 2012.
47. Dayarathna M., Suzumura T. Graph database benchmarking on cloud environments with XGDBench. Autom. Softw. Eng. 2013, 21.
48. Angles R., Prat-Perez A., Dominguez-Sal D., Larriba-Pey J. Benchmarking database systems for social network applications. The 1st International Workshop on Graph Data Management Experience and Systems June 23, 2013.
49. Abreu D.D., et al. Choosing between graph databases and RDF engines for consuming and mining linked data. 4th International Workshop on Consuming Linked Data 22 October, 2013.
50. Huang J., Abadi D.J., Ren K. Scalable SPARQL querying of large graphs. The 37th International Conference on Very Large Data Bases 29 August-3 September, 2011.
51. Zeng K., et al. A distributed graph engine for web scale RDF data. 39th International Conference on Very Large Databases 26-30 August, 2013.
52. Shao B., Wang H., Li Y. Trinity: a distributed graph engine on a memory cloud. 2013 ACM SIGMOD International Conference on Management of Data 22-27 June, 2013.
53. Malewicz G., et al. Pregel: a system for large-scale graph processing. SIGMOD 2010 6-11 June, 2010.
54. Low Y., et al. Distributed GraphLab: a framework for machine learning and data mining in the cloud. The 38th International Conference on Very Large Databases 27-31 August, 2012.
55. Kyrölä A., Blelloch G., Guestrin C. Proceedings of the 10th USENIX Symposium on Operating Systems Design and Implementation 8-10 October, 2012.
56. Chaudhuri S., Dayal U., Narasayya V. An overview of business intelligence technology. Commun. ACM 2011, 54:88-98.
57. Choo J., Park H. Customizing computational methods for visual analytics with big data. IEEE Comput. Graph. Appl. 2013, 33:22-28.
58. Beyer J., et al. Exploring the connectome petascale volume visualization of microscopy data streams. IEEE Comput. Graph. Appl. 2013, 33:50-61.
59. Chen Y., Alspaugh S., Katz R. Interactive analytical processing in big data systems: a cross-industry study of MapReduce workloads. The 38th International Conference on Very Large Databases (VLDB'12) 27-31 August, 2012.
60. Qin X., Zhou X. A survey on benchmarks for big data and some more considerations. 14th International Conference on Intelligent Data Engineering and Automated Learning 20-23 October, 2013.
61. Wang L., et al. BigDataBench: a big data benchmark suite from internet services. 20th IEEE International Symposium on High Performance Computer Architecture (HPCA-2014) 15-19 February, 2014.
62. Ghazal A., et al. BigBench: towards an industry standard benchmark for big data analytics. SIGMOD 2013 22-27 June, 2013.
63. Rabl T., Jacobsen H. Big data generation. Workshop on Big Data Benchmarking 8-9 May, 2012.
64. Baru C., et al. Benchmarking big data systems and the BigData Top100 list. Big Data J. 2013, 1.
65. Armstrong T.G., Ponnekanti V., Borthakur D. LinkBench: a database benchmark based on the Facebook social graph. SIGMOD 2013 22-27 June, 2013.
66. Quan J., Shi Y., Zhao M., Yang W. The implications from benchmarking three big data systems. IEEE International Conference on Big Data 6-9 October, 2013.
67. Walters J.P., et al. A comparison of virtualization technologies for HPC. 22nd International Conference on Advanced Information Networking and Applications 25-28 March, 2008.
68. Xavier M.G. Performance evaluation of container-based virtualization for high performance computing environments. 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing 27 February-1 March, 2013.
69. Che J., et al. A synthetical performance evaluation of OpenVZ, Xen and KVM. The IEEE Asia-Pacific Services Computing Conference 6-10 December, 2010.
70. Younge A.J., et al. Analysis of virtualization technologies for high performance computing environments. IEEE 4th International Conference on Cloud Computing 4-9 July, 2011.
71. Fayyad-Kazan H., Perneel L., Timmerman M. Full and para-virtualization with Xen: a performance comparison. J. Emerg. Trends Comput. Infor. Sci. 2013, 4:719-727.
72. Li J., et al. Performance overhead among three hypervisors: an experimental study using hadoop benchmarks. IEEE International Congress on Big Data 6-9 October, 2013.
73. Palit H.N., et al. Evaluating hardware-assisted virtualization for deploying HPC-as-a-Service. Virtualization Technologies in Distributed Computing 18 June, 2013.
74. Padala P., et al. Performance Evaluation of Virtualization Technologies for Server Consolidation April 11, 2007, HP Laboratories Palo Alto.
75. Buyya R., et al. Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst. 2009, 25:599-616.
76. Voras E., et al. Evaluating open-source cloud computing solutions. 34th International Convention MIPRO 2011 23-27 May, 2011.
77. Dukaric R., Juric M.B. Towards a unified taxonomy and architecture of cloud frameworks. Future Gener. Comput. Syst. 2013, 29:1196-1210.
78. Sempolinski P., Thain D. A comparison and critique of eucalyptus, OpenNebula and Nimbus. 2nd IEEE International Conference on Cloud Computing Technology and Science 30 November-3 December, 2013.
79. Wen X., et al. Comparison of open-source cloud management platforms: OpenStack and OpenNebula. 9th International Conference on Fully Systems and Knowledge Discovery 29-31 May, 2012.
80. von Laszewski G., et al. Comparison of multiple cloud networks. 5th International Conference on Cloud Computing 24-29 June, 2012.
81. Huang Q., et al. Evaluating open source cloud computing solutions for geosciences. Comput. Geosci. 2013, 59:41-52.
82. Huang Q., et al. An experimental study of open-source cloud platforms for dust storm forecasting. AMC SIGSPATIAL 6-9 November, 2012.
83. Borkar V., Carey M.J., Li C. Inside "Big Data Management": ogres, onions, or parfaits?. 15th International Conference on Extending Database Technology 27-30 March, 2012.
84. Ewen S. Iterative parallel data processing with stratosphere: an inside look. SIGMOD 2013 22-27 June, 2013.
85. Lohrmann B., Warneke D., Kao O. Nephele streaming: stream processing under QoS constraints at scale. J. Cluster Comput. 2013, 16.
86. Boden C., Markl V., Karnstedt M., Fernandez M. Large-scale social-media analytics on stratosphere. The International World Wide Web Conference 13-17 May, 2013.
87. Feinleib D. Big data landscape accessed 01/01/2014. http://blogs-images.forbes.com/davefeinleib/files/2012/07/Big-Data-Landscape-Jul-4-2012.00111.png.
88. Lindsay R.M., Ehrenberg A.S.C. The design of replicated studies. Am. Stat. 1993, 47:217-228.
89. Westerlund M., et al. A generalized scalable software architecture for analyzing temporally structured big data in the cloud. New Perspect. Inform. Syst. Technol. 2014, 1:559-569.
90. Alsubaiee S., et al. AsterixDB: a scalable, open source BDMS. The 40th International Conference on Very Large Databases (VLDB'14) 1-5 September, 2014.
91. Abiteboul S. Querying semi-structured data. International Conference on Database Theory 8-10 January, 1997.
92. Chen J., et al. Big data challenge: a data management perspective. Front. Comput. Sci. 2013, 2:157-164.