Scheduling jobs across geo-distributed datacenters

ACM SoCC 2015 - Proceedings of the 6th ACM Symposium on Cloud Computing

Volumn , Issue , 2015, Pages 111-124

  Hung, Chien Chun ^a   Golubchik, Leana ^a   Yu, Minlan ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLOUD COMPUTING; SCHEDULING;

DISTRIBUTED SITES; EXTENSIVE SIMULATIONS; IMPROVING PERFORMANCE; JOB SCHEDULING; JOB SCHEDULING ALGORITHMS; NEAR-OPTIMAL PERFORMANCE; SCHEDULING JOBS; SHORTEST REMAINING PROCESSING TIME;

SCHEDULING ALGORITHMS;

EID: 84958976586     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2806777.2806780     Document Type: Conference Paper

References (39)

1. http://aws.amazon.com/about-aws/global-infrastructure/Amazonn Global Infrastructure.
2. https://code.google.com/p/googleclusterdata/Google Cluster Workload Traces.
3. http://hadoop.apache.org/Hadoop Cluster Computing System.
4. http://www.microsoft.com/en-us/server-cloud/cloudos/global-datacenters.aspx Microsoft Cloud Platform.
5. G. Ananthanarayanan, S. Kandula, A. G. Greenberg, I. Stoica, Y. Lu, B. Saha, and E. Harris. Reining in the outliers in mapreduce clusters using mantri. In USENIX OSDI, 2010.
6. G. Ananthanarayanan, A. Ghodsi, A. Wang, D. Borthakur, S. Kandula, S. Shenker, and I. Stoica. Pacman: coordinated memory caching for parallel jobs. In USENIX NSDI, 2012.
7. G. Ananthanarayanan, A. Ghodsi, S. Shenker, and I. Stoica. Effective straggler mitigation: Attack of the clones. In USENIX NSDI, 2013.
8. G. Ananthanarayanan, C.-C. Hung, X. Ren, I. Stoica, A.Wierman, and M. Yu. Grass: Trimming stragglers in approximation analytics. In USENIX NSDI, 2014.
9. N. Bansal and M. Harchol-Balter. Analysis of SRPT scheduling: Investigating unfairness. ACM, 2001.
10. M. Chowdhury, Y. Zhong, and I. Stoica. Efficient coflow scheduling with varys. In ACM SIGCOMM, 2014.
11. J. C. Corbett, J. Dean, M. Epstein, A. Fikes, C. Frost, J. Furman, S. Ghemawat, A. Gubarev, C. Heiser, P. Hochschild, et al. Spanner: Googles globally distributed database. ACM Transactions on Computer Systems (TOCS), 2013.
12. N. Garg, A. Kumar, and V. Pandit. Order scheduling models: hardness and algorithms. In FSTTCS 2007: Foundations of Software Technology and Theoretical Computer Science. Springer, 2007.
13. A. Gupta, F. Yang, J. Govig, A. Kirsch, K. Chan, K. Lai, S. Wu, S. G. Dhoot, A. R. Kumar, A. Agiwal, et al. Mesa: Geo-replicated, near real-time, scalable data warehousing. In Proceedings of the VLDB Endowment, 2014.
14. M. Hajjat, D. Maltz, S. Rao, K. Sripanidkulchai, et al. Dealer: Application-aware request splitting for interactive cloud applications. In ACM CoNEXT, 2012.
15. M. Harchol-Balter, B. Schroeder, N. Bansal, and M. Agrawal. Size-based scheduling to improve web performance. ACM Transactions on Computer Systems (TOCS), 2003.
16. J.-H. Hwang, U. Cetintemel, and S. Zdonik. Fast and reliable stream processing over wide area networks. In IEEE ICDE Workshop, 2007.
17. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. In ACM SIGOPS Operating Systems Review, 2007.
18. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg. Quincy: Fair scheduling for distributed computing clusters. In ACM SOSP, 2009.
19. S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh, S. Venkata, J. Wanderer, J. Zhou, M. Zhu, et al. B4: Experience with a globally-deployed software defined wan. In ACM SIGCOMM, 2013.
20. T. Kraska, G. Pang, M. J. Franklin, S. Madden, and A. Fekete. Mdcc: Multi-data center consistency. In ACM EuroSys, 2013.
21. M. Lin, A. Wierman, and B. Zwart. The average response time in a heavy-traffic SRPT queue. ACM SIGMETRICS Performance Evaluation Review, 2010.
22. M. Lin, A. Wierman, and B. Zwart. Heavy-traffic analysis of mean response time under shortest remaining processing time. Performance Evaluation, 2011.
23. M. Mastrolilli, M. Queyranne, A. S. Schulz, O. Svensson, and N. A. Uhan. Minimizing the sum of weighted completion times in a concurrent open shop. Operation Research Letter, 2010.
24. S. Muralidhar, W. Lloyd, S. Roy, C. Hill, E. Lin, W. Liu, S. Pan, S. Shankar, V. Sivakumar, L. Tang, et al. f4: Facebook warm blob storage system. In USENIX OSDI, 2014.
25. P. Pietzuch, J. Ledlie, J. Shneidman, M. Roussopoulos, M.Welsh, and M. Seltzer. Network-aware operator placement for stream-processing systems. In IEEE ICDE, 2006.
26. Q. Pu, G. Ananthanarayanan, P. Bodik, S. Kandula, A. Akella, P. Bahl, and I. Stoica. Low latency geo-distributed data analytics. In ACM SIGCOMM, 2015.
27. A. Rabkin, M. Arye, S. Sen, V. S. Pai, and M. J. Freedman. Aggregation and degradation in jetstream: Streaming analytics in the wide area. In USENIX NSDI, 2014.
28. C. Reiss, A. Tumanov, G. R. Ganger, R. H. Katz, and M. A. Kozuch. Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In Proceedings of the Third ACM Symposium on Cloud Computing, 2012.
29. T. A. Roemer. A note on the complexity of the concurrent open shop problem. Springer, 2006.
30. L. Schrage. A proof of the optimality of the shortest remaining processing time discipline. Operations Research, 1968.
31. L. E. Schrage and L. W. Miller. The queue m/g/1 with the shortest remaining processing time discipline. Operations Research, 1966.
32. B. Schroeder and M. Harchol-Balter. Web servers under overload: How scheduling can help. ACM Transactions on Internet Technology (TOIT), 2006.
33. B. Sharma, V. Chudnovsky, J. L. Hellerstein, R. Rifaat, and C. R. Das. Modeling and synthesizing task placement constraints in google compute clusters. In Proceedings of the 2nd ACM Symposium on Cloud Computing, 2011.
34. J. Tan, X. Meng, and L. Zhang. Delay tails in mapreduce scheduling. ACM SIGMETRICS Performance Evaluation Review, 2012.
35. A. Vulimiri, C. Curino, B. Godfrey, T. Jungblut, J. Padhye, and G. Varghese. Global analytics in the face of bandwidth and regulatory constraints. In To Appear in NSDI, 2015.
36. A. Vulimiri, C. Curino, B. Godfrey, K. Karanasos, and G. Varghese. Wanalytics: Analytics for a geo-distributed dataintensive world. In To Appear in CIDR, 2015.
37. A. Wierman and M. Harchol-Balter. Classifying scheduling policies with respect to unfairness in an m/gi/1. In ACM SIGMETRICS Performance Evaluation Review, 2003.
38. M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, and I. Stoica. Delay scheduling: A simple technique for achieving locality and fairness in cluster scheduling. In ACM EuroSys, 2010.
39. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. Mc-Cauley, M. J. Franklin, S. Shenker, and I. Stoica. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In USENIX NSDI, 2012.