S3 : AAn efficient shared scan scheduler on MapReduce framework

Proceedings of the International Conference on Parallel Processing

Volumn , Issue , 2011, Pages 325-334

  Shi, Lei ^a   Li, Xiaohui ^a   Tan, Kian Lee ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

DATA FILES; DATA-INTENSIVE COMPUTING; MAP-REDUCE; OPEN SOURCE IMPLEMENTATION; WAITING-TIME;

SCHEDULING;

BATCH DATA PROCESSING;

EID: 80155183353     PISSN: 01903918     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICPP.2011.42     Document Type: Conference Paper

References (16)

1. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," in OSDI '04, pp. 137-150.
2. http://hadoop.apache.org
3. http://www.greenplum.com/technology/mapreduce/
4. B. He, W. Fang, Q. Luo, N. K. Govindaraju, and T. Wang, "Mars: A MapReduce Framework on Graphics Processors," in PACT '08, pp. 260-269.
5. C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis, "Evaluating MapReduce for Multi-core and Multiprocessor Systems," in HPCA '07, pp. 13-24.
6. H.-c. Yang, A. Dasdan, R.-L. Hsiao, and D. S. Parker, "Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters," in SIGMOD '07, pp. 1029-1040.
7. F. N. Afrati and J. D. Ullman, "Optimizing Joins in a Map-Reduce Environment," in EDBT '10, pp. 99-110.
8. http://hadoopblog.blogspot.com/2010/05/facebook-hasworlds-largest-hadoop. html
9. P. Agrawal, D. Kifer, and C. Olston, "Scheduling Shared Scans of Large Data Files," Proc. VLDB Endow., vol. 1, pp. 958-969, August 2008.
10. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg, "Quincy: Fair Scheduling for Distributed Computing Clusters," in SOSP '09, pp. 261-276.
11. T. Nykiel, M. Potamias, C. Mishra, G. Kollios, and N. Koudas, "MRShare: Sharing across Multiple Queries in MapReduce," PVLDB, vol. 3, no. 1, pp. 494-505, 2010.
12. http://hadoop.apache.org/common/docs/r0.21.0/hod-scheduler.html
13. http://www.clusterresources.com/products/torque-resourcemanager.php
14. http://hadoop.apache.org/hdfs/
15. S. Ghemawat, H. Gobioff, and S.-T. Leung, "The Google File System," SIGOPS Oper. Syst. Rev., vol. 37, no. 5, pp. 29-43, 2003.
16. http://www.comp.nus.edu.sg/~shilei/document/s3.pdf