Sparrow: Distributed, low latency scheduling

SOSP 2013 - Proceedings of the 24th ACM Symposium on Operating Systems Principles

Volumn , Issue , 2013, Pages 69-84

  Ousterhout, Kay ^a   Wendell, Patrick ^a   Zaharia, Matei ^a   Stoica, Ion ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HIGH AVAILABILITY; HIGHLY PARALLELS; LARGE-SCALE DATUM; LOW LATENCY; NEAR-OPTIMAL PERFORMANCE; RANDOMIZED SAMPLING;

SCHEDULING;

EID: 84889640333     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2517349.2522716     Document Type: Conference Paper

References (27)

1. Apache Thrift. http://thrift.apache.org.
2. G. Ananthanarayanan, A. Ghodsi, S. Shenker, and I. Stoica. Why Let Resources Idle? Aggressive Cloning of Jobs with Dolly. In HotCloud, 2012.
3. G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, and E. Harris. Reining in the Outliers in Map-Reduce Clusters using Mantri. In Proc. OSDI, 2010.
4. D. Bradley, T. S. Clair, M. Farrellee, Z. Guo, M. Livny, I. Sfiligoi, and T. Tannenbaum. An Update on the Scalability Limits of the Condor Batch System. Journal of Physics: Conference Series, 331(6), 2011.
5. J. Dean and L. A. Barroso. The Tail at Scale. Communications of the ACM, 56(2), February 2013.
6. A. Demers, S. Keshav, and S. Shenker. Analysis and Simulation of a Fair Queueing Algorithm. In Proc. SIGCOMM, 1989.
7. D. L. Eager, E. D. Lazowska, and J. Zahorjan. Adaptive Load Sharing in Homogeneous Distributed Systems. IEEE Transactions on Software Engineering, 1986.
8. B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A. D. Joseph, R. Katz, S. Shenker, and I. Stoica. Mesos: A Platform For Fine-Grained Resource Sharing in the Data Center. In Proc. NSDI, 2011.
9. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg. Quincy: Fair Scheduling for Distributed Computing Clusters. In Proc. SOSP, 2009.
10. M. A. Jette, A. B. Yoo, and M. Grondona. SLURM: Simple Linux Utility for Resource Management. In Proc. Job Scheduling Strategies for Parallel Processing, Lecture Notes in Computer Science, pages 44-60. Springer, 2003.
11. M. Kornacker and J. Erickson. Cloudera Impala: Real Time Queries in Apache Hadoop, For Real. http://blog.cloudera.com/blog/2012/10/cloudera-impala- real-time-queries-in-apache-hadoop-for-real/, October 2012.
12. S. Melnik, A. Gubarev, J. J. Long, G. Romer, S. Shivakumar, M. Tolton, and T. Vassilakis. Dremel: Interactive Analysis of Web-Scale Datasets. Proc. VLDB Endow., 2010.
13. M. Mitzenmacher. How Useful is Old Information? volume 11, pages 6-20, 2000.
14. M. Mitzenmacher. The Power of Two Choices in Randomized Load Balancing. IEEE Transactions on Parallel and Distributed Computing, 12(10):1094-1104, 2001.
15. M. Mitzenmacher. The Power of Two Random Choices: A Survey of Techniques and Results. In S. Rajasekaran, P. Pardalos, J. Reif, and J. Rolim, editors, Handbook of Randomized Computing, volume 1, pages 255-312. Springer, 2001.
16. A. C. Murthy. The Next Generation of Apache MapReduce. http://developer.yahoo.com/blogs/hadoop/next-generation-apache-hadoop-mapreduce- 3061.html, February 2012.
17. K. Ousterhout, A. Panda, J. Rosen, S. Venkataraman, R. Xin, S. Ratnasamy, S. Shenker, and I. Stoica. The Case for Tiny Tasks in Compute Clusters. In Proc. HotOS, 2013.
18. G. Park. A Generalization of Multiple Choice Balls-into-Bins. In Proc. PODC, pages 297-298, 2011.
19. L. Rudolph, M. Slivkin-Allalouf, and E. Upfal. A Simple Load Balancing Scheme for Task Allocation in Parallel Machines. In Proc. SPAA, 1991.
20. M. Schwarzkopf, A. Konwinski, M. Abd-El- Malek, and J. Wilkes. Omega: flexible, scalable schedulers for large compute clusters. In Proc. EuroSys, 2013.
21. B. Sharma, V. Chudnovsky, J. L. Hellerstein, R. Rifaat, and C. R. Das. Modeling and Synthesizing Task Placement Constraints in Google Compute Clusters. In Proc. SOCC, 2011.
22. D. Shue, M. J. Freedman, and A. Shaikh. Performance Isolation and Fairness for Multi-Tenant Cloud Storage. In Proc. OSDI, 2012.
23. T. White. Hadoop: The Definitive Guide. O'Reilly Media, 2009.
24. R. S. Xin, J. Rosen, M. Zaharia, M. J. Franklin, S. Shenker, and I. Stoica. Shark: SQL and Rich Analytics at Scale. In Proc. SIGMOD, 2013.
25. M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, and I. Stoica. Delay Scheduling: A Simple Technique For Achieving Locality and Fairness in Cluster Scheduling. In Proc. EuroSys, 2010.
26. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M. J. Franklin, S. Shenker, and I. Stoica. Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. In Proc. NSDI, 2012.
27. M. Zaharia, A. Konwinski, A. D. Joseph, R. Katz, and I. Stoica. Improving MapReduce Performance in Heterogeneous Environments. In Proc. OSDI, 2008.