From the cloud to the atmosphere: Running MapReduce across data centers

IEEE Transactions on Computers

Volumn 63, Issue 1, 2014, Pages 74-87

  Jayalath, Chamikara ^a   Stephen, Julian ^a   Eugster, Patrick ^a  

^a Purdue University   (United States)

Author keywords

Big data; Data center; Geodistributed; MapReduce

Indexed keywords

BIG DATUM; DATA CENTERS; DATA TRANSFORMATION; EXECUTING SEQUENCE; GEODISTRIBUTED; MAP-REDUCE; MAPREDUCE FRAMEWORKS; OPTIMIZATION FRAMEWORK;

OPTIMIZATION;

DATA HANDLING;

EID: 84890513039     PISSN: 00189340     EISSN: None     Source Type: Journal    
DOI: 10.1109/TC.2013.121     Document Type: Article

References (29)

1. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," Proc. Sixth Conf. Symp. Operating Systems Design and Implementation (OSDI), 2004.
2. "Hadoop," Apache Software Foundation, http://hadoop.apache.org, 2013.
3. S. Agarwal, J. Dunagan, N. Jain, S. Saroiu, A. Wolman, and H. Bhogan, "Volley: Automated Data Placement for Geo-Distributed Cloud Services," Proc. Seventh USENIX Conf. Networked Systems Design and Implementation (NSDI), 2010.
4. "Data from Year 2000 US Census," http://aws.amazon.com/data sets/Economics/2290, 2013.
5. "Department of Defense Information Enterprise Strategic Plan 2011-2012," http://dodcio.defense.gov/docs/DodIESP-r16.pdf, 2013.
6. W. Lloyd, M.J. Freedman, M. Kaminsky, and D.G. Andersen, "Don't Settle for Eventual: Scalable Causal Consistency for Wide- Area Storage with COPS," Proc. ACM 23rd Symp. Operating Systems Principles (SOSP), 2011.
7. V. Ramasubramanian, T. Rodeheffer, D. Terry, M. Walraed- Sullivan, T. Wobber, C. Marshall, and A. Vahdat, "Cimbiosys: A Platform for Content-Based Partial Replication," Proc. Sixth USENIX Symp. Networked Systems Design and Implementation (NSDI), 2009.
8. "Hadoop Across Data-Centers," http://mail-archives.apache. org/mod-mbox/hadoop-general/201001.mbox// %3C4B4B6AC7.6020801@lifeless.net%3E, 2013.
9. "Hadoop: The Definitive Guide,"http://oreilly.com/catalog/ 9780596521981, 2013.
10. "PageRank Algorithm," http://en.wikipedia.org/wiki/PageRank, 2013.
11. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins, "Pig Latin: A Not-so-Foreign Language for Data Processing," Proc. ACM SIGMOD Int'l Conf. Management of Data, 2008.
12. "Pig," Apache Software Foundation, http://pig.apache.org, 2013.
13. "Vicci Cloud Computing Testbed," http://www.vicci.org, 2013.
14. Y. Sovran, R. Power, M.K. Aguilera, and J. Li, "Transactional Storage for Geo-Replicated Systems," Proc. ACM 23rd Symp. Operating Systems Principles (SOSP), 2011.
15. J. Ousterhout, P. Agrawal, D. Erickson, C. Kozyrakis, J. Leverich, D. Mazières, S. Mitra, A. Narayanan, G. Parulkar, M. Rosenblum, S.M. Rumble, E. Stratmann, and R. Stutsman, "The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM," SIGOPS Operating Systems Rev., vol. 43, no. 4, pp. 92- 105, Jan. 2010.
16. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, "Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks," Proc. ACM Second SIGOPS/EuroSys European Conf. Computer Systems (Eurosys), 2007.
17. "Teradata," http://www.teradata.com, 2013.
18. C. He, D. Weitzel, D. Swanson, and Y. Lu, "HOG: Distributed Hadoop MapReduce on the Grid," Proc. IEEE Workshop Many-Task Computing Grids and Supercomputers (MTAGS), 2012.
19. J. Cao, S.A. Jarvis, S. Saini, and G.R. Nudd, "GridFlow: Workflow Management for Grid Computing," Proc. 3st Int'l Symp. Cluster Computing and the Grid (CCGrid), 2003.
20. "WS-BPEL," https://www.oasis-open.org/committees/wsbpel/, 2013.
21. M. Zaharia, A. Konwinski, A.D. Joseph, R.H. Katz, and I. Stoica, "Improving MapReduce Performance in Heterogeneous Environments," Proc. Eighth USENIX Conf. Operating Systems Design and Implementation (OSDI), 2008.
22. T. Condie, N. Conway, P. Alvaro, J. Hellerstein, K. Elmeleegy, and R. Sears, "MapReduce Online," Proc. Seventh USENIX Conf. Networked Systems Design and Implementation (NSDI), 2010.
23. H.-C. Yang, A. Dasdan, R.-L. Hsiao, and D.S. Parker, "Map- Reduce-Merge: Simplified Relational Data Processing on Large Clusters," Proc. ACM SIGMOD Int'l Conf. Management of Data, 2007.
24. H. Chang, M. Kodialam, R. Kompella, T. Lakshman, M. Lee, and S. Mukherjee, "Scheduling in MapReduce-like Systems for Fast Completion Time," Proc. IEEE INFOCOM, 2011.
25. E.W. Dijkstra, "A Note on Two Problems in Connection with Graphs," Numerische Math., vol. 11, pp. 1-269, 1959.
26. "EC2 Instance Types," http://aws.amazon.com/ec2/instancetypes/, 2013.
27. "Weather Data," http://aws.amazon.com/data sets/Climate/ 2759, 2013.
28. "Properties of the Iris," http://archive.ics.uci.edu/ml/data sets/ Iris, 2013.
29. J. Dejun, G. Pierre, and C. Chi, "EC2 Performance Analysis for Resource Provisioning of Service-Oriented Applications," Proc. Int'l Conf. Service-Oriented Computing (ICSOC/ServiceWave), 2009.