Purlieus: Locality-aware resource allocation for mapreduce in a cloud

Proceedings of 2011 SC - International Conference for High Performance Computing, Networking, Storage and Analysis

Volumn , Issue , 2011, Pages

  Palanisamy, Balaji ^a   Singh, Aameek ^b   Liu, Ling ^a   Jain, Bhushan ^b  

^a Georgia Institute of Technology   (United States)

^b IBM RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLOUD DATA; INPUT DATAS; JOB EXECUTION; LOCALITY-AWARENESS; MAP-REDUCE; NETWORK TRAFFIC; RESOURCE ALLOCATION SYSTEMS; RUNTIME PERFORMANCE; VIRTUAL MACHINES;

COMPUTER NETWORKS; COMPUTER SOFTWARE SELECTION AND EVALUATION; COUPLINGS; RESOURCE ALLOCATION;

CLOUD COMPUTING;

EID: 83155193241     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2063384.2063462     Document Type: Conference Paper

References (38)

1. B. Igou "User Survey Analysis: Cloud-Computing Budgets Are Growing and Shifting; Traditional IT Services Providers Must Prepare or Perish; Gartner Report, 2010.
2. http://en.wikipedia.org/wiki/Loop-device.
3. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In OSDI, 2004.
4. G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha and E. Harris. Reining in the Outliers inMap-Reduce Clusters using Mantri. In OSDI, 2010.
5. http://en.wikipedia.org/wiki/Big-data.
6. S. Babu. Towards Automatic Optimization of MapReduce Programs. In SOCC, 2010.
7. http://en.wikipedia.org/wiki/Clickstream.
8. K. Kambatla, A. Pathak and H. Pucha. Towards Optimizing Hadoop Provisioning in the Cloud. In HotCloud, 2009.
9. Cloudera. http://www.cloudera.com/blog/2010/08/hadoopfor-fraud-detection- and-prevention/.
10. K. Morton, A. Friesen, M. Balazinska, D. Grossman. Estimating the Progress of MapReduce Pipelines. In ICDE, 2010.
11. Hadoop DFS User Guide. http://hadoop.apache.org/.
12. T. Wood, P. Shenoy, A. Venkataramani and M. Yousif. Black-box and Gray-box Strategies for Virtual Machine Migration. In NSDI, 2007.
13. Y. Chen, R. Griffith, J. Liu, R. H. Katz and A. D. Joseph. Understanding TCP Incast Throughput Collapse in Datacenter Networks. In WREN, 2009.
14. Amazon Elastic MapReduce. http://aws.amazon.com/elasticmapreduce/.
15. Amazon Elastic Compute Cloud. http://aws.amazon.com/ec2/.
16. Amazon Simple Storage Service.http://aws.amazon.com/s3/.
17. T. Gunarathne, T. Wu, J. Qiu, G. Fox MapReduce in the Clouds for Science. In CloudCom, 2010.
18. M. Cardosa, P. Narang, A. Chandra, H. Pucha and A. Singh. STEAMEngine: Optimizing MapReduce provisioning in the cloud. Dept. of CSE, Univ. of Minnesota, 2010.
19. M. Al-Fares, A. Loukissas and A. Vahdat. A scalable, commodity data center network architecture. In SIGCOMM, 2008.
20. C. Guo, H. Wu, K. Tan, L. Shiy, Y. Zhang, S. Luz. DCell: A Scalable and Fault-Tolerant Network Structure for Data Centers. In SIGCOMM, 2008.
21. A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, S. Sengupta. VL2: A Scalable and Flexible Data Center Network . In SIGCOMM, 2009.
22. M. Al-Fares, S. Radhakrishnan, B. Raghavan, N. Huang, A. Vahdat. Hedera: Dynamic Flow Scheduling for Data Center Networks. In NSDI, 2010.
23. Hadoop. http://hadoop.apache.org.
24. M. Zaharia, A. Konwinski, A. D. Joseph, R. Katz, I. Stoica. Improving MapReduce Performance in Heterogeneous Environments. In OSDI, 2008.
25. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg. Quincy: fair scheduling for distributed computing clusters. In SOSP, 2009.
26. G. Wang, A. Butt, P. Pandey, K. Gupta. A Simulation Approach to Evaluating Design Decisions in MapReduce Setups. MASCOTS, 2009.
27. R. J. Mokken. Cliques, clubs and clans. In Quality and Quantity, 1973.
28. M. R. Garey, D. S. Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman. ISBN 0-7167-1045-5.
29. M. A. Kozuch, M. P. Ryan, R. Gass et al. Tashi: Location-aware Cluster Management. In ACDC, 2009.
30. K. Kambatla, A. Pathak, and H. Pucha. Towards optimizing hadoop provisioning in the cloud. In HotCloud, 2009.
31. H. Herodotou, H. Lim, G. Luo, N. Borisov, L. Dong, F. B. Cetin, S. Babu Starfish: A Selftuning System for Big Data Analytics. In CIDR, 2011.
32. G. Khanna, K. Beaty, G. Kar, and A. Kochut. Application performance management in virtualized server environments. In NOMS, 2006.
33. T. Sandholm and K. Lai. Mapreduce optimization using dynamic regulated prioritization. In ACM SIGMETRICS/Performance, 2009.
34. Scheduling in hadoop. http://www.cloudera.com/blog/tag/scheduling/.
35. A. Singh, M. Korupolu, and D. Mohapatra. Server-storage virtualization: Integration and load balancing in data centers. In IEEE/ACM Supercomputing, 2008.
36. A. Verma, P. Ahuja, and A. Neogi. pMapper: Power and Migration Cost Aware Placement of Applications in Virtualized Systems. In ACM Middleware, 2008.
37. A. Phanishayee, H. Shah, E. Krevat, D. Andersen, G. Ganger, G. Gibson, B. Mueller, V. Vasudevan. Safe and Effective Fine-grained TCP Retransmissions for Datacenter Communication. In SIGCOMM 2009.
38. G. Lee, N. Tolia, P. Ranganathan, R. Katz. Topology-Aware Resource Allocation for Data-intensive workloads. In APSys, 2010.