Data-aware resource scheduling for multicloud workflows: A fine-grained simulation approach

Proceedings of the International Conference on Cloud Computing Technology and Science, CloudCom

Volumn 2015-February, Issue February, 2015, Pages 887-892

  Tang, Wei ^a   Jenkins, Jonathan ^a   Meyer, Folker ^a   Ross, Robert ^a   Kettimuthu, Rajkumar ^a   Winkler, Linda ^a   Yang, Xi ^b   Lehman, Thomas ^b   Desai, Narayan ^a,c  

^a ARGONNE NATIONAL LABORATORY   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

^c ERICSSON RESEARCH   (United States)

Author keywords

Cloud computing; Data aware scheduling; Resource management; Scientific workflow; Workflow simulation

Indexed keywords

CLOUD COMPUTING; ENVIRONMENTAL MANAGEMENT; NATURAL RESOURCES MANAGEMENT; RESOURCE ALLOCATION; SCHEDULING;

CLOUD INFRASTRUCTURES; DATA-AWARE SCHEDULING; RESOURCE MANAGEMENT; RESOURCE MANAGEMENT AND SCHEDULING; RESOURCE-SCHEDULING; SCIENTIFIC AND ENGINEERING APPLICATIONS; SCIENTIFIC WORKFLOWS; WORKFLOW SIMULATION;

INFORMATION MANAGEMENT;

EID: 84937940704     PISSN: 23302194     EISSN: 23302186     Source Type: Conference Proceeding    
DOI: 10.1109/CloudCom.2014.19     Document Type: Conference Paper

References (26)

1. AWE project. https://github. com/MG-RAST/AWE
2. CODES project. http://press3. mcs. Anl. gov/codes/
3. MG-RAST website. http://metagenomics. Anl. gov
4. Shock project. https://github. com/MG-RAST/Shock
5. R. Calheiros, R. Ranjan, A. Beloglazov, C. Rose, and R. Buyya,"CloudSim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms," Software: Practice and Experience, 41(1):23-50, 2011
6. C. Carothers, D. Bauer, and S. Pearce, "ROSS: A highperformance, low memory, modular time warp system," in Proc. of Workshop on Paral. And Distri. Simulation, 2000
7. J. Cope, N. Liu, S. Lang, P. Carns, C. Carothers, R. Ross, "CODES: Enabling co-design of multi-layer exascale storage srchitectures," in Proc. of Workshop on Emerging Supercomputing Technologies, 2011
8. W. Chen and E. Deelman, "WorkflowSim: A toolkit for simulating scientific workflows in distributed environments," in IEEE International Conference on e-Science, 2012
9. D. Duplyakin, P. Marshall, K. Keahey, H. Tufo, and A. Alzabarah, "Rebalancing in a multi-cloud environment," in Proc. of ACM Workshop on Scientific Cloud Computting, 2013
10. W. J. Kent, "BLAT-the BLAST-like alignment tool," Genome Research, 12(4):656-664, 2002
11. K. Keahey, P. Armstrong, J. Bresnahan, D. LaBissoniere, and P. Riteau, "Infrastructure outsourcing in multi-cloud environment," in Proc. of Workshop on Cloud Serv. Fed. 8th Open Cirrus Summit, 2012
12. T. Kosar and M. Balman, "A new paradigm: Data-aware scheduling in grid computing," Future Generation Computer Systems, 25(2009):406-413, 2009
13. D. Lezzi, F. Lordan, R. Rafanell, and R. Badia, "Execution of scientific workflows on federated multi-cloud Infrastructures," in Euro-Par 2013: Parallel Processing Workshops, volume 8374 of Lecture Notes in Computer Science, pages 136-145, Springer, 2014
14. K. Maheshwari, E.-S. Jung, J. Meng, V. Vishwanath, and R. Kettimuthu, "Improving multisite workflow performance using model-based scheduling," in Proc. of Int'l Conf. on Parallel Programming, 2014
15. F. Meyer, D. Paarmann, M. D'Souza, R. Olson, E. M. Glass, M. Kubal, T. Paczian, A. Rodriguez, R. Stevens, A. Wilke, J. Wilkening, and R. Edwards, "The metagenomics RAST server-a public resource for the automatic phylogenetic and functional analysis of metagenomes," BMC Bioinformatics, 9(386):1-8, 2008
16. N. Mohamed, H. Lin, and W.-C. Feng, "Accelerating dataintensive genome analysis in the cloud," in Proc. of Int'l Conf. on Bioinformatics and Computational Biology, 2013
17. L. Ramakrishnan, P. Zbiegel, S. Campbell, R. Bradshaw, R. S. Canon, S. Coghlan, I. Sakrejda, N. Desai, T. Declerck, and A. Liu, "Magellan: Experiences from a science cloud," in Proc. of International Workshop on Scientific Cloud Computing, 2011
18. W. Tang, J. Bischof, N. Desai, K. Mahadik, W. Gerlach, T. Harrison, A. Wilke, and F. Meyer, "Workload characterization for MG-RAST metagenomic data analytics service in the cloud," in Proc. of IEEE Int'l Conf. on Big Data, 2014
19. W. Tang, Z. Lan, N. Desai, and D. Buettner, "Fault-Aware, Utility-Based Job Scheduling on Blue Gene/P Systems," in Proc. of IEEE Int'l Conf. on Cluster Computing, 2009
20. W. Tang, J. Wilkening, N. Desai, W. Gerlach, A. Wilke, and F. Meyer, "A scalable data analysis platform for metagenomics," in Proc. of IEEE International Conference on Big Data, 2013
21. Dan Tsafrir, "Modeling, evaluating, and improving the performance of supercomputer scheduling," Technical Report 2006-78, School of Computer Science and Engineering, the Hebrew University (PhD Thesis), 2006
22. A. Wilke, E. Glass, D. Bartels, J. Bischof, D Braithwaite, M. D'Souza, W. Gerlach, T. Harrison, K. Keegan, H. Matthews, R. Kottmann, T. Paczian, W. Tang, W. Trimble, P. Yilmaz, J. Wilkening, N. Desai, and F. Meyer, "A metagenomics portal for a democratized sequencing world," Methods in Enzymology, 531:487-523, 2013
23. A. Yoo, M. Jette, and M. Grondona, "SLURM: Simple Linux utility for resource management," Job Scheduling Strategies for Parallel Processing, volume 2862 of Lecture Notes in Computer Science, pages 44-60, Springer-Verlag, 2003
24. A. C. Zhou and B. He, "Transformation-based monetary cost optimizations for workflows in the cloud," IEEE Transactions on Cloud Computing 2, no. 1, 85-98, 2014
25. H. Zou, Y. Yu, W. Tang, and H. Chen, "Improving I/O performance with adaptive data compression for big data applications," in Proc. of International Parallel and Distributed Processing Symposium Workshops and Phd Forum, 2014
26. H. Zou, Y. Yu, W. Tang, and H. Chen, "FlexAnalytics: A Flexible Data Analytics Framework for Big Data Applications with I/O Performance Improvement," Big Data Research, 1:4-13, 2014.