An adaptive parallel execution strategy for cloud-based scientific workflows

Concurrency and Computation: Practice and Experience

Volumn 24, Issue 13, 2012, Pages 1531-1550

  De Oliveira, Daniel ^a   Ogasawara, Eduardo ^a,b   Ocaña, Kary ^a   Baião, Fernanda ^c   Mattoso, Marta ^a  

^a FEDERAL UNIVERSITY OF RIO DE JANEIRO   (Brazil)

^b CENTRO FEDERAL DE EDUCAÇÃO TECNOLÓGICA CELSO SUCKOW DA FONSECA   (Brazil)

^c FEDERAL UNIVERSITY OF THE STATE OF RIO DE JANEIRO   (Brazil)

Author keywords

cloud computing; parallelism; scientific workflows

Indexed keywords

CLOUD COMPUTING; MIDDLEWARE; WORK SIMPLIFICATION;

EXPERIMENTAL EVALUATION; MULTIPROCESSOR MACHINES; PARALLEL EXECUTIONS; PARALLEL TECHNIQUES; PARALLELISM; SCIENTIFIC EXPERIMENTS; SCIENTIFIC WORKFLOW MANAGEMENTS; SCIENTIFIC WORKFLOWS;

INFORMATION MANAGEMENT;

EID: 84865333940     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.1880     Document Type: Conference Paper

References (33)

1. Hey T, Tansley S, Tolle K,. The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research: Seattle/WA, United States, 2009.
2. Cooperstock FI,. General relativistic dynamics: extending Einstein's legacy throughout the universe. World Scientific: New Jersey, 2009.
3. Lin C, Lu S, Fei X, Pai D, Hua J,. A Task Abstraction and Mapping Approach to the Shimming Problem in Scientific Workflows. Proceedings of the International Conference on Services Computing 2009, Bangalore, India, 2009; 284-291.
4. Taylor IJ, Deelman E, Gannon DB, Shields M,. Workflows for e-Science: Scientific Workflows for Grids, 1st ed. Springer: London, 2007.
5. Mattoso M, Werner C, Travassos GH, Braganholo V, Murta L, Ogasawara E, Oliveira D, Cruz SMSD, Martinho W,. Towards Supporting the Life Cycle of Large-scale Scientific Experiments. International Journal of Business Process Integration and Management 2010; 5 (1): 79-92.
6. Walker E, Guiang C,. Challenges in executing large parameter sweep studies across widely distributed computing environments. Workshop on Challenges of large applications in distributed environments, Monterey, California, USA, 2007; 11-18. (Pubitemid 47315531)
7. Raicu I, Foster I, Zhao Y,. Many-task computing for grids and supercomputers. Workshop on Many-Task Computing on Grids and Supercomputers, Austin, Texas, 2008; 1-11.
8. Vaquero LM, Rodero-Merino L, Caceres J, Lindner M,. A break in the clouds: towards a cloud definition. SIGCOMM Computer Communication Review 2009; 39 (1): 50-55.
9. Geelan J,. Twenty-One Experts Define Cloud Computing. Cloud Computing Journal 2009. http://cloudcomputing.sys-con.com/node/612375?page=0,2
10. El-Khamra Y, Kim H, Jha S, Parashar M,. Exploring the Performance Fluctuations of HPC Workloads on Clouds, Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science, Washington, DC, USA, 2010; 383-387.
11. Jackson KR, Ramakrishnan L, Muriki K, Canon S, Cholia S, Shalf J, Wasserman HJ, Wright NJ,. Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud. Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science, Washington, DC, USA, 2010; 159-168.
12. He Q, Zhou S, Kobler B, Duffy D, McGlynn T,. Case study for running HPC applications in public clouds, Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, New York, NY, USA, 2010; 395-401.
13. Amazon EC2. Amazon Elastic Compute Cloud (Amazon EC2), http://aws.amazon.com/ec2/, 2010.
14. Zhao Y, Raicu I, Foster I,. Scientific Workflow Systems for 21st Century, New Bottle or New Wine?, IEEE Congress on Services, 2008; 467-471.
15. Oliveira D, Ogasawara E, Baião F, Mattoso M,. SciCumulus: A Lightweigth Cloud Middleware to Explore Many Task Computing Paradigm in Scientific Workflows, International Conference on Cloud Computing, Miami, FL, 2010; 378-385.
16. Oliveira D, Ogasawara E, Baiao F, Mattoso M,. An Adaptive Approach for Workflow Activity Execution in Clouds, International Workshop on Challenges in e-Science-SBAC, Petrõpolis, RJ-Brazil, 2010; 9-16.
17. Oliveira D, Ocana K, Ogasawara E, Dias J, Baião F, Mattoso M,. A Performance Evaluation of X-ray Crystallography Scientific Workflow using SciCumulus, International Conference on Cloud ComputingInternational Conference on Cloud Computing, Washington D.C, 2011.
18. Buyya R, Ranjan R, Calheiros R,. Modeling and Simulation of Scalable Cloud Computing Environments and the CloudSim Toolkit: Challenges and Opportunities. Proceedings of High Performance Computing & Simulation 2009, Leipzig, Germany, 2009.
19. Meyer LA, Rössle SC, Bisch PM, Mattoso M,. Parallelism in Bioinformatics Workflows, High Performance Computing for Computational Science-VECPAR 2004, 2005; 583-597.
20. Callahan SP, Freire J, Santos E, Scheidegger CE, Silva CT, Vo HT,. VisTrails: visualization meets data management. SIGMOD, Chicago, Illinois, USA, 2006; 745-747.
21. Freire J, Koop D, Santos E, Silva CT,. Provenance for Computational Tasks: A Survey. Computing in Science and Engineering 2008; 10 (3): 11-21. (Pubitemid 351559435)
22. Carpenter B, Getov V, Judd G, Skjellum A, Fox G,. MPJ: MPI-like message passing for Java. Concurrency: Practice and Experience 2000; 12 (11): 1019-1038. (Pubitemid 32032847)
23. SubCloud. Shared Enterprise File System for Amazon S3 Cloud Storage | SubCloud, http://www.subcloud.com/, 2011.
24. Jain D, Lamour V,. Computational tools in protein crystallography. Methods in Molecular Biology (Clifton, N.J.) 2010; 673: 129-156.
25. Azevedo W,. Prof. Dr. Walter Filgueira de Azevedo Jr. URL: http://wfdaj.sites.uol.com.br/. Access: 30 Jul 2011, 2011.
26. Lima A, Mattoso M, Valduriez P,. Adaptive Virtual Partitioning for OLAP Query Processing in a Database Cluster. SBBDBrazilian Symposium on Databases, 2004; 92-105.
27. Abramson D, Enticott C, Altinas I,. Nimrod/K: towards massively parallel dynamic grid workflows. Proceedings of International Conference for High Performance Computing, Networking, Storage and Analysis, Austin, Texas, 2008; 1-11.
28. Altintas I, Berkley C, Jaeger E, Jones M, Ludascher B, Mock S,. Kepler: an extensible system for design and execution of scientific workflows, Scientific and Statistical Database Management, Greece, 2004; 423-424.
29. Hoffa C, Mehta G, Freeman T, Deelman E, Keahey K, Berriman B, Good J,. On the use of cloud computing for scientific workflows. IEEE Fourth International Conference on eScience (eScience 2008), Indianapolis, USA, 2008; 7-12.
30. Deelman E, Mehta G, Singh G, Su M, Vahi K,. Pegasus: Mapping Large-Scale Workflows to Distributed Resources, Workflows for e-Science. Springer, 2007; 376-394.
31. Warneke D, Kao O,. Nephele: efficient parallel data processing in the cloud. Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers, New York, NY, USA, 2009; 1-10.
32. Lee C, Suzuki J, Vasilakos A, Yamamoto Y, Oba K,. An evolutionary game theoretic approach to adaptive and stable application deployment in clouds. Proceeding of the 2nd workshop on Bio-inspired algorithms for distributed systems, Washington, DC, USA, 2010; 29-38.
33. Ogasawara E, Oliveira D, Chirigati F, Barbosa CE, Elias R, Braganholo V, Coutinho A, Mattoso M,. Exploring many task computing in scientific workflows. Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers, Portland, Oregon, 2009; 1-10.