Big data processing in cloud computing environments

Proceedings of the 2012 International Symposium on Pervasive Systems, Algorithms, and Networks, I-SPAN 2012

Volumn , Issue , 2012, Pages 17-23

  Ji, Changqing ^a,b   Li, Yu ^c   Qiu, Wenming ^c   Awada, Uchechukwu ^c   Li, Keqiu ^c  

^a DALIAN MARITIME UNIVERSITY   (China)

^b DALIAN UNIVERSITY   (China)

^c DALIAN UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

Big Data; Cloud Computing; Data Management; Distributed Computing

Indexed keywords

BIG DATUM; CLOUD ARCHITECTURES; CLOUD COMPUTING ENVIRONMENTS; CLOUD COMPUTING PLATFORMS; CLOUD DATA MANAGEMENTS; CLOUD DATABASE; CRITICAL CHALLENGES; DATA STORAGE; EFFECTIVE MANAGEMENT; EMERGING APPLICATIONS; ISSUES AND CHALLENGES; KEY ISSUES; LARGE-SCALE DATUM; MAP-REDUCE; OPTIMIZATION STRATEGY; PARALLEL PROCESSING; RAPID GROWTH; RESEARCH DIRECTIONS; SOCIAL NETWORK ANALYSIS;

ALGORITHMS; BIOINFORMATICS; CLOUD COMPUTING; DATA PROCESSING; DISTRIBUTED COMPUTER SYSTEMS; INFORMATION MANAGEMENT; SOCIAL NETWORKING (ONLINE); UBIQUITOUS COMPUTING;

COMPUTER SYSTEMS;

EID: 84874615486     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/I-SPAN.2012.9     Document Type: Conference Paper

References (41)

1. "Big data: science in the peta byte era," Nature 455 (7209): 1, 2008.
2. Douglas and Laney, "The importance of 'big data': A definition," 2008.
3. D. Kossmann, T. Kraska, and S. Loesing, "An evaluation of alternative architectures for transaction processing in the cloud," in Proceedings of the 2010 international conference on Management of data. ACM, 2010, pp. 579-590.
4. S. Ghemawat, H. Gobioff, and S. Leung, "The google file system," in ACM SIGOPS Operating Systems Review, vol. 37, no. 5. ACM, 2003, pp. 29-43.
5. J. Dean and S. Ghemawat, "Mapreduce: simplified data processing on large clusters," Communications of the ACM, vol. 51, no. 1, pp. 107-113, 2008.
6. D. Borthakur, "The hadoop distributed file system: Architecture and design," Hadoop Project Website, vol. 11, 2007.
7. A. Rabkin and R. Katz, "Chukwa: A system for reliable large-scale log collection," in USENIX Conference on Large Installation System Administration, 2010, pp. 1-15.
8. S. Sakr, A. Liu, D. Batista, and M. Alomari, "A survey of large scale data management approaches in cloud environments," Communications Surveys & Tutorials, IEEE, vol. 13, no. 3, pp. 311-336, 2011.
9. Y. Cao, C. Chen, F. Guo, D. Jiang, Y. Lin, B. Ooi, H. Vo, S. Wu, and Q. Xu, "Es2: A cloud data storage system for supporting both oltp and olap," in Data Engineering (ICDE), 2011 IEEE 27th International Conference on. IEEE, 2011, pp. 291-302.
10. F. Chang, J. Dean, S. Ghemawat, W. Hsieh, D. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. Gruber, "Bigtable: A distributed structured data storage system," in 7th OSDI, 2006, pp. 305-314.
11. B. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H. Jacobsen, N. Puz, D. Weaver, and R. Yerneni, "Pnuts: Yahoo!'s hosted data serving platform," Proceedings of the VLDB Endowment, vol. 1, no. 2, pp. 1277-1288, 2008.
12. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels, "Dynamo: amazon's highly available keyvalue store," in ACM SIGOPS Operating Systems Review, vol. 41, no. 6. ACM, 2007, pp. 205-220.
13. Y. Lin, D. Agrawal, C. Chen, B. Ooi, and S. Wu, "Llama: leveraging columnar storage for scalable join processing in the mapreduce framework," in Proceedings of the 2011 international conference on Management of data. ACM, 2011, pp. 961-972.
14. D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman, L. Youseff, and D. Zagorodnov, "The eucalyptus open-source cloud-computing system," in Cluster computing and the Grid, 2009. CCGRID'09. 9th IEEE/ACM International Symposium on. IEEE, 2009, pp. 124-131.
15. P. Sempolinski and D. Thain, "A comparison and critique of eucalyptus, opennebula and nimbus," in Cloud computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on. Ieee, 2010, pp. 417-426.
16. D. DeWitt and M. Stonebraker, "Mapreduce: A major step backwards," The Database Column, vol. 1, 2008.
17. M. Stonebraker, D. Abadi, D. DeWitt, S. Madden, E. Paulson, A. Pavlo, and A. Rasin, "Mapreduce and parallel dbmss: friends or foes," Communications of the ACM, vol. 53, no. 1, pp. 64-71, 2010.
18. A. Abouzeid, K. Bajda-Pawlikowski, D. Abadi, A. Silberschatz, and A. Rasin, "Hadoopdb: an architectural hybrid of mapreduce and dbms technologies for analytical workloads," Proceedings of the VLDB Endowment, vol. 2, no. 1, pp. 922-933, 2009.
19. Y. Xu, P. Kostamaa, and L. Gao, "Integrating hadoop and parallel dbms," in Proceedings of the 2010 international conference on Management of data. ACM, 2010, pp. 969-974.
20. J. Dittrich, J. Quiane-Ruiz, A. Jindal, Y. Kargin, V. Setty, and J. Schad, "Hadoop++: Making a yellow elephant run like a cheetah (without it even noticing)," Proceedings of the VLDB Endowment, vol. 3, no. 1-2, pp. 515-529, 2010.
21. D. Logothetis and K. Yocum, "Ad-hoc data processing in the cloud," Proceedings of the VLDB Endowment, vol. 1, no. 2, pp. 1472-1475, 2008.
22. C. Ji, T. Dong, Y. Li, Y. Shen, K. Li, W. Qiu, W. Qu, and M. Guo, "Inverted grid-based knn query processing with mapreduce," in ChinaGrid, 2012 Seventh ChinaGrid Annual Conference on. IEEE, 2012, pp. 25-33.
23. S. Das, Y. Sismanis, K. Beyer, R. Gemulla, P. Haas, and J. McPherson, "Ricardo: integrating r and hadoop," in Proceedings of the 2010 international conference on Management of data. ACM, 2010, pp. 987-998.
24. A. Stupar, S. Michel, and R. Schenkel, "Rankreduce-processing k-nearest neighbor queries on top of mapreduce," in Proceedings of the 8th Workshop on Large-Scale Distributed Systems for Information Retrieval, 2010, pp. 13-18.
25. R. Ferreira Cordeiro, C. Traina Junior, A. Machado Traina, J. Lopez, U. Kang, and C. Faloutsos, "Clustering very large multi-dimensional datasets with mapreduce," in Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2011, pp. 690-698.
26. C. Wang, J. Wang, X. Lin, W. Wang, H. Wang, H. Li, W. Tian, J. Xu, and R. Li, "Mapdupreducer: detecting near duplicates over massive datasets," in Proceedings of the 2010 international conference on Management of data. ACM, 2010, pp. 1119-1122.
27. C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis, "Evaluating mapreduce for multi-core and multiprocessor systems," in High Performance Computer Architecture, 2007. HPCA 2007. IEEE 13th International Symposium on. IEEE, 2007, pp. 13-24.
28. B. He, W. Fang, Q. Luo, N. Govindaraju, and T. Wang, "Mars: a mapreduce framework on graphics processors," in Proceedings of the 17th international conference on Parallel architectures and compilation techniques. ACM, 2008, pp. 260-269.
29. H. Yang, A. Dasdan, R. Hsiao, and D. Parker, "Map-reducemerge: simplified relational data processing on large clusters," in Proceedings of the 2007 ACM SIGMOD international conference on Management of data. ACM, 2007, pp. 1029-1040.
30. D. Jiang, A. Tung, and G. Chen, "Map-Join-Reduce: Toward scalable and efficient data analysis on large clusters," Knowledge and Data Engineering, IEEE Transactions on, vol. 23, no. 9, pp. 1299-1311, 2011.
31. T. Nykiel, M. Potamias, C. Mishra, G. Kollios, and N. Koudas, "Mrshare: Sharing across multiple queries in mapreduce," Proceedings of the VLDB Endowment, vol. 3, no. 1-2, pp. 494-505, 2010.
32. Y. Xu, P. Zou, W. Qu, Z. Li, K. Li, and X. Cui, "Samplingbased partitioning in mapreduce for skewed data," in China-Grid, 2012 Seventh ChinaGrid Annual Conference on. IEEE, 2012, pp. 1-8.
33. J. Ekanayake, H. Li, B. Zhang, T. Gunarathne, S. Bae, J. Qiu, and G. Fox, "Twister: a runtime for iterative mapreduce," in Proceedings of the 19th ACM International Symposium on High Performance Distributed computing. ACM, 2010, pp. 810-818.
34. Y. Bu, B. Howe, M. Balazinska, and M. Ernst, "Haloop: Efficient iterative data processing on large clusters," Proceedings of the VLDB Endowment, vol. 3, no. 1-2, pp. 285-296, 2010.
35. G. Malewicz, M. Austern, A. Bik, J. Dehnert, I. Horn, N. Leiser, and G. Czajkowski, "Pregel: a system for largescale graph processing," in Proceedings of the 2010 international conference on Management of data. ACM, 2010, pp. 135-146.
36. T. Condie, N. Conway, P. Alvaro, J. Hellerstein, K. Elmeleegy, and R. Sears, "Mapreduce online," in Proceedings of the 7th USENIX conference on Networked systems design and implementation, 2010, pp. 21-21.
37. T. Condie, N. Conway, P. Alvaro, J. Hellerstein, J. Gerth, J. Talbot, K. Elmeleegy, and R. Sears, "Online aggregation and continuous query support in mapreduce," in ACM SIGMOD, 2010, pp. 1115-1118.
38. D. Jiang, B. Ooi, L. Shi, and S. Wu, "The performance of mapreduce: An in-depth study," Proceedings of the VLDB Endowment, vol. 3, no. 1-2, pp. 472-483, 2010.
39. R. Vernica, M. Carey, and C. Li, "Efficient parallel setsimilarity joins using mapreduce," in SIGMOD conference. Citeseer, 2010, pp. 495-506.
40. C. Zhang, F. Li, and J. Jestes, "Efficient parallel knn joins for large data in mapreduce," in Proceedings of the 15th International Conference on Extending Database Technology. ACM, 2012, pp. 38-49.
41. X. Zhou, J. Lu, C. Li, and X. Du, "Big data challenge in the management perspective," Communications of the CCF, vol. 8, pp. 16-20, 2012.