Cura: A cost-optimized model for MapReduce in a cloud

Proceedings - IEEE 27th International Parallel and Distributed Processing Symposium, IPDPS 2013

Volumn , Issue , 2013, Pages 1275-1286

  Palanisamy, Balaji ^a   Singh, Aameek ^b   Liu, Ling ^a   Langston, Bryan ^b  

^a Georgia Institute of Technology   (United States)

^b IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLOUD DATA CENTERS; CLOUD SERVICE MODELS; CLUSTER CONFIGURATIONS; COMPLEX CLUSTERS; INFRASTRUCTURE COSTS; PERFORMANCE ENHANCEMENTS; PRODUCTION WORKLOADS; RESOURCE UTILIZATIONS;

DISTRIBUTED DATABASE SYSTEMS; DISTRIBUTED PARAMETER NETWORKS; OPTIMIZATION; RESOURCE ALLOCATION;

COST REDUCTION;

EID: 84884898758     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2013.20     Document Type: Conference Paper

References (46)

1. B. Igou "User Survey Analysis: Cloud-Computing Budgets Are Growing and Shifting; Traditional IT Services Providers Must Prepare or Perish". Gartner Report, 2010
2. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In OSDI, 2004.
3. G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha and E. Harris. Reining in the Outliers inMap-Reduce Clusters using Mantri. In OSDI, 2010.
4. K. Kambatla, A. Pathak and H. Pucha. Towards Optimizing Hadoop Provisioning in the Cloud. In HotCloud, 2009.
5. K. Morton, A. Friesen, M. Balazinska, D. Grossman. Estimating the Progress of MapReduce Pipelines. In ICDE, 2010.
6. Pig User Guide. http://pig.apache.org/.
7. A. Thusoo et. al. Hive - A Warehousing Solution Over a MapReduce Framework In VLDB, 2009.
8. D. Borthakur et al. Apache Hadoop goes realtime at Facebook In SIGMOD, 2011.
9. S. Melnik et al. Dremel: interactive analysis of web-scale datasets In VLDB, 2010.
10. C. Curino, E. P. C. Jones, R. Popa, N. Malviya, E. Wu, S. Madden, H. Balakrishnan, N. Zeldovich Relational Cloud: A Database-as-a-Service for the Cloud In CIDR, 2011.
11. S. Aulbach, T. Grust, D. Jacobs, A. Kemper, J. Rittinger Multi-Tenant Databases for Software as a Service: Schema-Mapping Techniques In SIGMOD, 2008.
12. P. Xiong, Y. Chi, S. Zhu, J. Tatemura, C. Pu, H. Hacigumus ActiveSLA: A Profit-Oriented Admission Control Framework for Database-as-a-Service Providers In SOCC, 2011.
13. T. Wood, P. Shenoy, A. Venkataramani and M. Yousif Black-box and Gray-box Strategies for Virtual Machine Migration. In NSDI, 2007.
14. Amazon Elastic MapReduce. http://aws.amazon.com/elasticmapreduce/
15. Amazon Elastic Compute Cloud. http://aws.amazon.com/ec2/
16. Hadoop. http://hadoop.apache.org.
17. M. Zaharia, A. Konwinski, A. D. Joseph, R. Katz, I. Stoica. Improving MapReduce Performance in Heterogeneous Environments. In OSDI, 2008.
18. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg. Quincy: fair scheduling for distributed computing clusters. In SOSP, 2009.
19. I. Roy, Srinath. Setty, A. Kilzer, V. Shmatikov, E. Witchel Airavat: Security and Privacy for MapReduce NSDI, 2010.
20. SELinux user guide. http://selinuxproject.org/page/Main-Page.
21. M. R. Garey, D. S. Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman. ISBN 0-7167-1045-5.
22. S. L. Garfinkel An Evaluation of Amazon's Grid Computing Services: EC2, S3 and SQS . Technical Report, TR-08-07, Harvard University, available at: ftp://ftp.deas.harvard.edu/techreports/tr-08-07.pdf
23. F. Tian and K. Chen Towards Optimal Resource Provisioning for Running MapReduce Programs in Public Clouds In CLOUD, 2011
24. H. Herodotou and S. Babu On Optimizing MapReduce Programs / Hadoop Jobs. In VLDB, 2011.
25. H. Herodotou et al Starfish: A Selftuning System for Big Data Analytics. In CIDR, 2011.
26. H. Herodotou, F. Dong and S. Babu No One (Cluster) Size Fits All: Automatic Cluster Sizing for Data-intensive Analytics. In SOCC, 2011.
27. V. Jalaparti, H. Ballani, P. Costa, T. Karagiannis, A. Rowstron Bazaar: Enabling Predictable Performance in Datacenters MSR Cambridge, UK, Technical Report MSR-TR-2012-38.
28. A. Popescu, V. Ercegovac, A. Balmin, M. Branco, A. Ailamaki Same Queries, Different Data: Can we Predict Runtime Performance? SMDB, 2012.
29. A. Verma, L. Cherkasova, and R. H. Campbell Resource Provisioning Framework for MapReduce Jobs with Performance Goals In Middleware, 2011.
30. Y. Chen, A. Ganapathi, R. Griffith, R. Katz The Case for Evaluating MapReduce Performance Using Workload Suites In MASCOTS, 2011.
31. S. Kavulya, J. Tan, R. Gandhi, P. Narasimhan An Analysis of Traces from a Production MapReduce Cluster In CCGrid, 2010.
32. T. Sandholm and K. Lai. Mapreduce optimization using dynamic regulated prioritization. In ACM SIGMETRICS/Performance, 2009.
33. B. Palanisamy, A. Singh, L. Liu and B. Jain Purlieus: locality-aware resource allocation for MapReduce in a cloud. In SC, 2011.
34. Google BigQuery . https://developers.google.com/bigquery/.
35. Y. Chen, S. Alspaugh, D. Borthakur and R. Katz Energy Effciency for Large-Scale MapReduce Workloads with Significant Interactive Analysis In EUROSYS, 2012.
36. A. Singh, M. Korupolu, and D. Mohapatra. Server-storage virtualization: Integration and load balancing in data centers. In SC, 2008.
37. A. Mu'alem, D. Feitelson, Utilization, Predictability, Workloads, and User Runtime Estimates in Scheduling the IBM SP2 with Backfilling. In TPDS, 2001
38. J. Skovira, W. Chan, H. Zhou, D. Lifka The EASY - LoadLeveler API Project In IPPS, 1996.
39. S. Anastasiadis, K. Sevcik Parallel Application Scheduling on Networks of Workstations. In JPDC, 1997
40. E. Rosti, E. Smirni, L. Dowdy, G. Serazzi, B. Carlson Robust Partitioning Policies of Multiprocessor Systems. In Performance Evaluation, 1993.
41. S. Srinivasan, V. Subramani, R. Kettimuthu, P. Holenarsipur, P. Sadayappan Effective Selection of Partition Sizes for Moldable Scheduling of Parallel Jobs. In HIPC, 2002.
42. W. Cirne Using Moldability to Improve the Performance of Supercomputer Jobs. Ph.D. Thesis. UCSD, 2001
43. M. Mao, M. Humphrey Auto-Scaling to Minimize Cost and Meet Application Deadlines in Cloud Workflows. In SC, 2011.
44. B. Sotomayor, K. Keahey, I. Foster Combining Batch Execution and Leasing Using Virtual Machines
45. K. Chard, K. Bubendorfer, P. Komisarczuk High Occupancy Resource Allocation for Grid and Cloud systems, a Study with DRIVE. In HPDC, 2010.
46. T. Hacker, K. Mahadik Flexible Resource Allocation for Reliable Virtual Cluster Computing Systems. In SC, 2011