Topology-aware resource allocation for data-intensive workloads

Proceedings of the 1st ACM Asia-Pacific Workshop on Systems, APSys '10, Co-located with SIGCOMM 2010

Volumn , Issue , 2010, Pages 1-5

  Lee, Gunho ^a   Tolia, Niraj ^b   Ranganathan, Parthasarathy ^b   Katz, Randy H ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b HEWLETT PACKARD LABORATORIES   (United States)

Author keywords

hadoop; IAAS; infrastructure as a service; mapreduce; modeling; performance; topology awareness; virtualization

Indexed keywords

EID: 78149326439     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1851276.1851278     Document Type: Conference Paper

References (20)

1. http://aws.amazon.com/ec2/.
2. http://www.opencirrus.org/.
3. M. Al-Fares, A. Loukissas, and A. Vahdat. A scalable, commodity data center network architecture. In Proceedings of the ACM SIGCOMM 2008 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM), pages 63-74, Seattle, WA, Aug. 2008.
4. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In Proceedings of the 19th ACM Symposium on Operating Systems Principles, pages 164-177, Bolton Landing, NY, 2003.
5. J. S. Bucy, J. Schindler, S. W. Schlosser, G. R. Ganger, and et al. The disksim simulation environment version 4.0 reference manual. Technical Report CMU-PDL-08-101, Parallel Data Laboratory, Carnegie Mellon University, Pittsburgh, PA, May 2008.
6. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In Proceedings of the 6th Symposium on Operating System Design and Implementation (OSDI '04), pages 137-150, San Francisco, CA, Dec. 2004.
7. D. E. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, 1989.
8. A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta. VL2: A scalable and flexible data center network. In Proceedings of the ACM SIGCOMM 2009 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM), pages 51-62, Barcelona, Spain, Aug. 2009.
9. C. Guo, G. Lu, D. Li, H. Wu, X. Zhang, C. T. Yunfeng Shi, Y. Zhang, and S. Lu. BCube: A high performance, server-centric network architecture for modular data centers. In SIGCOMM, pages 63-74, Barcelona, Spain, Aug. 2009.
10. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg. Quincy: Fair scheduling for distributed computing clusters. In Proceedings of 22nd ACM Symposium on Operating Systems Principles (SOSP '09), Big Sky, MT, Oct. 2009.
11. K. Kambatla, A. Pathak, and H. Pucha. Towards optimizing hadoop provisioning in the cloud. In Proceedings of the 1st USENIX Workshop on Hot Topics in Cloud Computing (HotCloud '09), San Diego, CA, June 2009.
12. S.-C. Lin, W. Punch, and E.D.Goodman. Coarse-grain parallel genetic algorithms: Categorization and new approach. In Proceedings of the Sixth IEEE Symposium on Parallel and Distributed Processing, pages 28-37, 1994.
13. C. Liu, L. Yang, I. T. Foster, and D. Angulo. Design and evaluation of a resource selection framework for grid applications. In Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing (HPDC-11), pages 63-72, Edinburgh, Scotland, July 2002.
14. D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman, L. Youseff, and D. Zagorodnov. The eucalyptus open-source cloud-computing system. In Proceedings of the 9th IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID '09), pages 124-131, Shanghai, China, May 2009.
15. T. Ristenpart, E. Tromer, H. Shacham, and S. Savage. Hey, you, get off of my cloud: Exploring information leakage in third-party compute clouds. In Proceedings of the ACM Conference on Computer and Communications Security, Chicago, IL, Nov. 2009.
16. T. Sandholm and K. Lai. Mapreduce optimization using regulated dynamic prioritization. In Proceedings of the 11th International Joint Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), pages 299-310, 2009.
17. C. A. Santos, A. Sahai, X. Zhu, D. Beyer, V. Machiraju, and S. Singhal. Policy-based resource assignment in utility computing environments. In Proceedings of the 15th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management (DSOM '04), pages 100-111, Davis, CA, Nov. 2004.
18. O. Sonmez, H. Mohamed, and D. Epema. Communication-aware job placement policies for the KOALA grid scheduler. In Proceedings of the Second IEEE International Conference on e-Science and Grid Computing, page 79, Amsterdam, Netherlands, Dec. 2006.
19. G. Wang, A. R. Butt, P. Pandey, and K. Gupta. A simulation approach to evaluating design decisions in mapreduce setups. In Proceedings of the 17th International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2009), London, UK, Sept. 2009.
20. M. Zaharia, D. Borthakur, J. S. Sarma, K. Elmeleegy, S. Shenker, and I. Stoica. Job scheduling for multi-user mapreduce clusters. Technical Report UCB/EECS-2009-55, University of California, Berkeley, Apr. 2009.