Joint scheduling of processing and shuffle phases in MapReduce systems

Proceedings - IEEE INFOCOM

Volumn , Issue , 2012, Pages 1143-1151

  Chen, Fangfei ^a   Kodialam, Murali ^b   Lakshman, T V ^b  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

^b LUCENT TECHNOLOGIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

JOINT SCHEDULING; LARGE DATA; MAP-REDUCE; PERFORMANCE IMPROVEMENTS; PRACTICAL SCHEMES; THEORETICAL COMPLEXITY; THREE PHASE; THREE PHASIS;

APPROXIMATION ALGORITHMS; DATA HANDLING;

SCHEDULING;

EID: 84861632325     PISSN: 0743166X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/INFCOM.2012.6195473     Document Type: Conference Paper

References (17)

1. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," in USENIX OSDI, 2004.
2. H. Chang, M. S. Kodialam, R. R. Kompella, T. V. Lakshman, M. Lee, and S. Mukherjee, "Scheduling in mapreduce-like systems for fast completion time," in IEEE INFOCOM, 2011.
3. M. Chowdhury, M. Zaharia, J. Ma, M. I. Jordan, and I. Stoica, "Managing data transfers in computer clusters with orchestra," in ACM SIGCOMM, 2011.
4. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg, "Quincy: Fair Scheduling for Distributed Computing Clusters," in SOSP, 2009.
5. M. Zaharia, D. Borthakur, J. S. Sarma, K. Elmeleegy, S. Shenker, and I. Stoica, "Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling," in EuroSys'10: Proceedings of the 5th European conference on Computer systems, 2010, pp. 265-278.
6. G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, and E. Harris, "Reining in the outliers in map-reduce clusters using mantri," in USENIX OSDI, 2010.
7. J. Lenstra, A. Rinnooy Kan, and P. Brucker, "Complexity of machine scheduling problems," Annals of Discrete Mathematics, vol. 1, pp. 343-362, 1977.
8. Y. Kim, "Data migration to minimize the total completion time," Journal of Algorithms, vol. 55, no. 1, pp. 42-57, 2005.
9. P. Brucker, Scheduling algorithms. Springer, 2004.
10. R. Graham, "Bounds on multiprocessing timing anomalies," SIAM Journal on Applied Mathematics, vol. 17, no. 2, pp. 416-429, 1969.
11. J. Lenstra and A. Kan, "Complexity of scheduling under precedence constraints," Operations Research, pp. 22-35, 1978.
12. S. Chakrabarti, C. Phillips, A. Schulz, D. Shmoys, C. Stein, and J. Wein, "Improved scheduling algorithms for minsum criteria," Automata, Languages and Programming, pp. 646-657, 1996.
13. M. Queyranne and A. Schulz, "Approximation bounds for a general class of precedence constrained parallel machine scheduling problems," SIAM Journal on Computing, vol. 35, no. 5, pp. 1241-1253, 2006.
14. R. Gandhi and J. Mestre, "Combinatorial algorithms for data migration to minimize average completion time," Algorithmica, vol. 54, no. 1, pp. 54-71, 2009.
15. M. Hajiaghayi, R. Khandekar, G. Kortsarz, and V. Liaghat, "On a local protocol for concurrent file transfers," in Proceedings of the 23rd ACM symposium on Parallelism in algorithms and architectures. ACM, 2011, pp. 269-278.
16. A. Bar-Noy, M. Halldórsson, G. Kortsarz, R. Salman, and H. Shachnai, "Sum multicoloring of graphs," Journal of Algorithms, vol. 37, no. 2, pp. 422-450, 2000.
17. R. Gandhi, M. Halldórsson, G. Kortsarz, and H. Shachnai, "Improved bounds for scheduling conflicting jobs with minsum criteria," ACM Transactions on Algorithms (TALG), vol. 4, no. 1, p. 11, 2008.