Topology-aware resource allocation for data-intensive workloads

Computer Communication Review

Volumn 41, Issue 1, 2011, Pages 120-124

  Lee, Gunho ^a   Tolia, Niraj ^b   Ranganathan, Parthasarathy ^b   Katz, Randy H ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b HEWLETT PACKARD LABORATORIES   (United States)

Author keywords

Hadoop; IaaS; Infrastructure as a service; MapReduce; Modeling; Performance; Topology awareness; Virtualization

Indexed keywords

HADOOP; IAAS; INFRASTRUCTURE-AS-A-SERVICE; MAP-REDUCE; MODELING; PERFORMANCE; TOPOLOGY AWARENESS; VIRTUALIZATIONS;

TOPOLOGY;

RESOURCE ALLOCATION;

EID: 79551553478     PISSN: 01464833     EISSN: 01464833     Source Type: Conference Proceeding    
DOI: 10.1145/1925861.1925881     Document Type: Article

References (20)

1. http://aws.amazon.com/ec2/.
2. http://www.opencirrus.org/.
3. M. Al-Fares, A. Loukissas, and A. Vahdat. A scalable, commodity data center network architecture. In Proceedings of the ACM SIGCOMM 2008 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM), pages 63-74, Seattle, WA, Aug. 2008.
4. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In Proceedings of the 19th ACM Symposium on Operating Systems Principles, pages 164-177, Bolton Landing, NY, 2003.
5. J. S. Bucy, J. Schindler, S. W. Schlosser, G. R. Ganger, and et al. The disksim simulation environment version 4.0 reference manual. Technical Report CMU-PDL-08-101, Parallel Data Laboratory, Carnegie Mellon University, Pit
6. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In Proceedings of the 6th Symposium on Operating System Design and Implementation (OSDI 04), pages 137-150, San Francisco, CA, Dec. 2004.
7. D. E. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, 1989.
8. A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta. VL2: A scalable and flexible data center network. In Proceedings of the ACM SIGCOMM 2009 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM), pages 51-62, Barcelona, Spain, Aug. 2009.
9. C. Guo, G. Lu, D. Li, H. Wu, X. Zhang, C. T. Yunfeng Shi, Y. Zhang, and S. Lu. BCube: A high performance, server-centric network architecture for modular data centers. In SIGCOMM, pages 63-74, Barcelona, Spain, Aug. 2009.
10. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg. Quincy: Fair scheduling for distributed computing clusters. In Proceedings of 22nd ACM Symposium on Operating Systems Principles (SOSP 09), Big Sky, MT, Oct. 2009.
11. K. Kambatla, A. Pathak, and H. Pucha. Towards optimizing hadoop provisioning in the cloud. In Proceedings of the 1st USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 09), San Diego, CA, June 2009.
12. S.-C. Lin, W. Punch, and E.D.Goodman. Coarse-grain parallel genetic algorithms: Categorization and new approach. In Proceedings of the Sixth IEEE Symposium on Parallel and Distributed Processing, pages 28-37, 1994.
13. C. Liu, L. Yang, I. T. Foster, and D. Angulo. Design and evaluation of a resource selection framework for grid applications. In Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing (HPDC-11), pages 63-72, Edinburgh, Scotland, July 2002.
14. D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman, L. Youseff, and D. Zagorodnov. The eucalyptus open-source cloud-computing system. In Proceedings of the 9th IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID 09), pages 124131, Shanghai, China, May 2009.
15. T. Ristenpart, E. Tromer, H. Shacham, and S. Savage. Hey, you, get off of my cloud: Exploring information leakage in third-party compute clouds. In Proceedings of the ACM Conference on Computer and Communications Security, Chicago, IL, Nov. 2009.
16. T. Sandholm and K. Lai. Mapreduce optimization using regulated dynamic prioritization. In Proceedings of the 11th International Joint Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), pages 299-310, 2009.
17. C. A. Santos, A. Sahai, X. Zhu, D. Beyer, V. Machiraju, and S. Singhal. Policy-based resource assignment in utility computing environments. In Proceedings of the 15th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management (DSOM 04), pages 100-111, Davis, CA, Nov. 2004.
18. O. Sonmez, H. Mohamed, and D. Epema. Communication-aware job placement policies for the KOALA grid scheduler. In Proceedings of the Second IEEE International Conference on e-Science and Grid Computing, page 79, Amsterdam, Netherlands, Dec. 2006.
19. G. Wang, A. R. Butt, P. Pandey, and K. Gupta. A simulation approach to evaluating design decisions in mapreduce setups. In Proceedings of the 17th International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2009), London, UK, Sept. 2009.
20. M. Zaharia, D. Borthakur, J. S. Sarma, K. Elmeleegy, S. Shenker, and I. Stoica. Job scheduling for multi-user mapreduce clusters. Technical Report UCB/EECS-2009-55, University of California, Berkeley, Apr. 2009.