An empirical analysis of scheduling techniques for real-time cloud-based data processing

Proceedings - 2011 IEEE International Conference on Service-Oriented Computing and Applications, SOCA 2011

Volumn , Issue , 2011, Pages

  Phan, Linh T X ^a   Zhang, Zhuoyao ^a   Zheng, Qi ^a   Loo, Boon Thau ^a   Lee, Insup ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AMAZON EC2; DATA-INTENSIVE APPLICATION; EMPIRICAL ANALYSIS; HUMAN-IN-THE-LOOP; MAP-REDUCE; MISSION CRITICAL TASKS; MULTI PROCESSOR SCHEDULING; ONLINE SCHEDULING; PERFORMANCE IMPROVEMENTS; REAL TIME SCHEDULING; RESEARCH AGENDA; SCHEDULING TECHNIQUES; TIME-DEPENDENT; VIRTUALIZATIONS;

DATA PROCESSING; REAL TIME CONTROL; RESEARCH;

SCHEDULING;

EID: 84859942965     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SOCA.2011.6166240     Document Type: Conference Paper

References (48)

1. P. Agrawal, D. Kifer, and C. Olston. Scheduling shared scans of large data files. In VLDB, 2008.
2. Amazon Relational Database Service (Amazon RDS). http://aws. amazon.com/rds/.
3. J. H. Anderson and A. Srinivasan. Pfair scheduling: beyond periodic task systems. In RTCSA, 2000.
4. S. K. Baruah, N. K. Cohen, C. G. Plaxton, and D. A. Varvel. Proportionate progress: A notion of fairness in resource allocation. Algorithmica, 15:600-625, 1996. 10.1007/BF01940883.
5. Spyros Blanas, Jignesh M. Patel, Vuk Ercegovac, Jun Rao, Eugene J. Shekita, and Yuanyuan Tian. A comparison of join algorithms for log processing in mapreduce. In SIGMOD, 2010.
6. Yingyi Bu, Bill Howe, Magdalena Balazinska, and Michael Ernst. Haloop: Efficient iterative data processing on large clusters. PVLDB, 3(1):285-296, 2010.
7. S. Bygde, A. Ermedahl, and B. Lisper. An efficient algorithm for parametric wcet calculation. In RTCSA, 2009.
8. F. Checconi, T. Cucinotta, D. Faggioli, and G. Lipari. Hierarchical multiprocessor CPU reservations for the linux kernel. In OSPERT, 2009.
9. Shimin Chen, Phillip B. Gibbons, and Suman Nath. Pr-join: a non-blocking join achieving higher early result rate with statistical guarantees. In SIGMOD, 2010.
10. Songting Chen. Cheetah: A high performance, custom data warehouse on top of mapreduce. PVLDB, 3(2):1459-1468, 2010.
11. L. Cherkasova, D. Gupta, and A. Vahdat. Comparison of the three cpu schedulers in xen. SIGMETRICS Perform. Eval. Rev., 35:42-51, September 2007.
12. Tyson Condie, Neil Conway, Peter Alvaro, Joseph M. Hellerstein, Khaled Elmeleegy, and Russell Sears. Mapreduce online. In NSDI, pages 313-328, 2010.
13. Silviu S. Craciunas, Andreas Haas, Christoph M. Kirsch, Hannes Payer, Harald Rak, Andreas Rottmann, Ana Sokolova, Rainer Trummer, Joshua Love, and Raja Sengupta. Information-acquisition-as-a-service for cyber-physical cloud computing. In Hotcloud, 2011.
14. Dan Craytor. Transforming warfare with cloud services. ITEA Journal, 2011.
15. T. Cucinotta, G. Anastasi, and L. Abeni. Respecting temporal constraints in virtualised services. In COMPSAC, 2009.
16. Carlo Curino, Evan Jones, Raluca Ada Popa, Nirmesh Malviya, Eugene Wu, Samuel Madden, Hari Balakrishnan, and Nickolai Zeldovich. Relational Cloud: A Database Service for the Cloud. In CIDR, 2011.
17. Sudipto Das, Yannis Sismanis, Kevin S. Beyer, Rainer Gemulla, Peter J. Haas, and John McPherson. Ricardo: integrating r and hadoop. In SIGMOD, 2010.
18. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In OSDI, 2004.
19. Tim Dornemann, Ernst Juhnke, and Bernd Freisleben. On-demand resource provisioning for bpel workflows using amazon's elastic compute cloud. In CCGrid, 2009.
20. Francesco Fusco, Marc Ph. Stoecklin, and Michail Vlachos. Net-fli: On-the-fly compression, archiving and indexing of streaming network traffic. PVLDB, 2010.
21. D. Griffin and A. Burns. Realism in Statistical Analysis of Worst Case Execution Times. In WCET, 2010.
22. Hadoop. http://hadoop.apache.org/.
23. Thomas A. Henzinger, Anmol V. Singh, Vasu Singh, Thomas Wies, and Damien Zufferey. FlexPRICE: Flexible provisioning of resources in a cloud environment. In Proc. of IEEE International Conference on Cloud Computing (CLOUD), 2010.
24. David Irwin, Prashant Shenoy, Emmanuel Cecchet, and Michael Zink. Resource management in data-intensive clouds:opportunities and challenges. In LANMAN, 2010.
25. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg. Quincy: fair scheduling for distributed computing clusters. In SOSP, 2009.
26. Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, and Dennis Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In EuroSys, 2007.
27. Dawei Jiang, Beng Chin Ooi, Lei Shi, and Sai Wu. The performance of mapreduce: An in-depth study. PVLDB, 3(1):472-483, 2010.
28. B. Kao and H. Garcia-Molina. An overview of real-time database systems. In Real Time Computing, 1994.
29. J. Lee, A. Easwaran, and I. Shin. Llf schedulability analysis on multiprocessor platforms. In RTSS, 2010.
30. Youngseok Lee, Wonchul Kang, and Hyeongu Son. An internet traffic analysis method with mapreduce. In NOMS, 2010.
31. C. L. Liu and James W. Layland. Scheduling algorithms for multi-programming in a hard-real-time environment. J. ACM, 20:46-61, Jan 1973.
32. Ke Liu, Hai Jin, Jinjun Chen, Xiao Liu, Dong Yuan, and Yun Yang. A Compromised-Time-Cost Scheduling Algorithm in SwinDeW-C for Instance-Intensive Cost-Constrained Workflows on Cloud Computing Platform. J. of High Performance Computing Applications, 24:445-456, Nov 2010.
33. A. F. Mills and J. H. Anderson. A Stochastic Framework for Multiprocessor Soft Real-Time Scheduling. In RTAS, 2010.
34. Ioannis Moschakis and Helen Karatza. Evaluation of gang scheduling performance and cost in a cloud computing system. The Journal of Supercomputing, 2010.
35. Christopher Olston, Benjamin Reed, Utkarsh Srivastava, Ravi Kumar, and Andrew Tomkins. Pig latin: a not-so-foreign language for data processing. In SIGMOD, 2008.
36. Linh T.X. Phan, Zhuoyao Zhang, Boon Thau Loo, and Insup Lee. Realtime mapreduce scheduling. Technical report, U. of Pennsylvania, 2010. MS-CIS-10-32.
37. L.T.X. Phan, I. Lee, and O. Sokolsky. Compositional analysis of multi-mode systems. In ECRTS, 2010.
38. Yahoo Capacity Scheduler. http://hadoop.apache.org/core/docs/current/ capacity-scheduler.html.
39. I. Shin, A. Easwaran, and I. Lee. Hierarchical scheduling framework for virtual clustering of multiprocessors. In ECRTS, 2008.
40. Chenyang Lu Sisu Xi, Justin Wilson and Christopher Gill. Rt-xen:realtime virtualization based on hierarchical scheduling. Technical report, Washington U., 2010. WUCSE-2010-38.
41. A. Verma, L. Cherkasova, and R. Campbell. Aria: Automatic resource inference a nd allocation for mapreduce environments. In ICAC, 2011.
42. Guozhang Wang, Marcos Antonio Vaz Salles, Benjamin Sowell, Xun Wang, Tuan Cao, Alan J. Demers, Johannes Gehrke, and Walker M. White. Behavioral simulations in mapreduce. PVLDB, 3(1):952-963, 2010.
43. Joel Wolf, Deepak Rajan, Kirsten Hildrum, Rohit Khandekar, Vibhore Kumar, Sujay Parekh, Kun-Lung Wu, and Andrey Balmin. Flex: A slot allocation scheduling optimizer for mapreduce workloads. In Middleware 2010, 2010.
44. Jungwoo Yang, Hyungseok Kim, Sangwon Park, Changki Hong, and Insik Shin. Implementation of compositional scheduling framework on virtualization. In CRTS, 2010.
45. Yuan Yu, Michael Isard, Dennis Fetterly, Mihai Budiu, Ulfar Erlingsson, Pradeep Kumar Gunda, and Jon Currey. Dryadlinq: A system for general-purpose distributed data-parallel computing. In OSDI, 2008.
46. Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy, Scott Shenker, and Ion Stoica. Job scheduling for multi-user mapreduce clusters. Technical Report UCB/EECS-2009-55, U. of California, Berkeley, 2009.
47. Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy, Scott Shenker, and Ion Stoica. Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. In EuroSys, 2010.
48. Matei Zaharia, Andy Konwinski, Anthony D. Joseph, Randy Katz, and Ion Stoica. Improving mapreduce performance in heterogeneous environments. In OSDI, 2008.