Scientific-workflow-management-as-a-service in the cloud

Proceedings - 2nd International Conference on Cloud and Green Computing and 2nd International Conference on Social Computing and Its Applications, CGC/SCA 2012

Volumn , Issue , 2012, Pages 97-104

  Zhao, Yong ^a   Li, Youfu ^a   Tian, Wenhong ^a   Xue, Ruini ^a  

^a UNIVERSITY OF ELECTRONIC SCIENCE AND TECHNOLOGY OF CHINA   (China)

Author keywords

Cloud workflow; Scientific Workflow; Swift; Workflow as a Service

Indexed keywords

BIG DATUM; CLOUD PLATFORMS; CLOUD WORKFLOW; ESSENTIAL SUPPORT; HIGH-THROUGHPUT; JOB SCHEDULING; ON DEMANDS; RESOURCE MANAGEMENT; SCIENTIFIC WORKFLOW MANAGEMENTS; SCIENTIFIC WORKFLOWS; SWIFT; TASK DEPENDENCIES; TASK-SCHEDULING; VIRTUAL CLUSTERS; WORKFLOW SPECIFICATION; WORKFLOW-AS-A-SERVICE;

DISTRIBUTED COMPUTER SYSTEMS; IMAGE PROCESSING; MANAGEMENT; NASA; WORK SIMPLIFICATION;

SCHEDULING;

EID: 84874635218     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CGC.2012.70     Document Type: Conference Paper

References (35)

1. OpenNebula, http://www.OpenNebula.org, 2012
2. Openstack, http://www.openstack.org, 2012
3. http://www.psc.edu/general/software/packages/genbank/,2012
4. Large Hadron Collider, http://lhc.web.cern.ch, 2012
5. Shawn Rogers, Big Data is Scaling BI and Analytics, Information Management, Sept 1, 2011.
6. I. Foster, Y. Zhao, I. Raicu, S. Lu. "Cloud Computing and Grid Computing 360-Degree Compared ", IEEE Grid Computing Environments (GCE08) 2008, co-located with IEEE/ACM Supercomputing 2008.
7. G. Bell, T. Hey, A. Szalay, Beyond the Data Deluge, Science, Vol. 323, no. 5919, pp. 1297-1298, 2009.
8. E. Deelman et al. Pegasus: A framework for mapping complex scientific workflows onto distributed systems, Scientific Programming, Volume 13 Issue 3, July 2005.
9. B. Ludäscher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger, M. Jones, E. A. Lee, J. Tao, Y. Zhao, Scientific workflow management and the Kepler system, Concurrency and Computation: Practice and Experience,Special Issue: Workflow in Grid Systems, Volume 18, Issue 10, pages 1039-1065, 25 August 2006.
10. J. Freire, C. T. Silva, S. P. Callahan, E. Santos, C. E. Scheidegger and H. T. Vo, Managing Rapidly-Evolving Scientific Workflows, Provenance and Annotation of Data, Lecture Notes in Computer Science, 2006, Volume 4145/2006, 10-18, DOI: 10.1007/11890850-2
11. D. Hull, K. Wolstencroft, R. Stevens, C. Goble, M. Pocock, P. Li, and T. Oinn, "Taverna: a tool for building and running workflows of services.," Nucleic Acids Research, vol. 34, iss. Web Server issue, pp. 729-732, 2006.
12. Y. Zhao, X. Fei, I. Raicu, S. Lu, Opportunities and Challenges in Running Scientific Workflows on the Cloud, IEEE International Conference on Cyber-enabled distributed computing and knowledge discovery (CyberC), 2011.
13. C. Hoffa, G. Mehta, T. Freeman, E. Deelman, K. Keahey, B. Berriman, J. Good, "On the Use of Cloud Computing for Scientific Workflows", 3rd International Workshop on Scientific Workflows and Business Workflow Standards in e-Science (SWBES), Indianapolis, Indiana, USA, December 2008.
14. K. Keahey, T. Freeman. Science Clouds: Early Experiences in Cloud Computing for Scientific Applications, Cloud Computing and Its Applications 2008 (CCA-08), Chicago, IL. October 2008.
15. J.-S. Vöckler, G. Juve, E. Deelman, M. Rynge, G. B. Berriman. "Experiences Using Cloud Computing for A Scientific Workflow Application", Invited Paper, ACM Workshop on Scientific Cloud Computing (ScienceCloud) 2011.
16. L. Ramakrishnan, C. Koelbel, Y.-S. Kee, R. Wolski, D. Nurmi, D. Gannon, G. Obertelli, A. YarKhan, A. Mandal, T.M. Huang, K. Thyagaraja, and D. Zagorodnov, "VGrADS: Enabling e-Science Workflows on Grids and Clouds with Fault Tolerance," Proc. Conf. High Performance Computing Networking, Storage and Analysis (SC'09), 2009.
17. M. Palankar, A. Iamnitchi, M. Ripeanu, S. Garfinkel. Amazon S3 for science grids: a viable solution? In Proceedings of the 2008 international workshop on Data-aware distributed computing (DADC '08). 2008.
18. A. Iosup, S. Ostermann, M. N. Yigitbasi, R. Prodan, T. Fahringer, D. H. Epema, "Performance Analysis of Cloud Computing Services for Many- Tasks Scientific Computing," IEEE Transactions on Parallel and Distributed Systems, pp. 931-945, June, 2011.
19. E. Deelman, G. Singh, M. Livny, B. Berriman, and J. Good. The cost of doing science on the Cloud: the Montage example. In Proceedings of the 2008 ACM/IEEE conference on Supercomputing, SC '08, pages 50:1-50:12, Piscataway, NJ, USA, 2008.
20. C. Vecchiola, S. Pandey, and R. Buyya. High-Performance Cloud Computing: A View of Scientific Applications. In International Symposium onParallel Architectures, Algorithms, and Networks,2009.
21. K. R. Jackson, L. Ramakrishnan, K. Muriki, S. Canon et al. Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud. In CloudCom, pages 159-168. IEEE, 2010.
22. Keahey, K., and T. Freeman. Contextualization: Providing One-click Virtual Clusters. in eScience. 2008. Indianapolis, IN, 2008.
23. G. Juve and E. Deelman. Wrangler: Virtual Cluster Provisioning for the Cloud. In HPDC, 2011.
24. Bresnahan, J., Freeman, T., LaBissoniere, D., Keahey, K., Managing Appliance Launches in Infrastructure Clouds, Teragrid 2011. Salt Lake City, UT. July 2011.
25. C. Lin, S. Lu, X. Fei, A. Chebotko, D. Pai, Z. Lai, F. Fotouhi, and J. Hua, "A Reference Architecture for Scientific Workflow Management Systems and the VIEW SOA Solution", IEEE Transactions on Services Computing (TSC), 2(1), pp.79-92, 2009.
26. C. Lin, S. Lu, Z. Lai, A. Chebotko, X. Fei, J. Hua, F. Fotouhi, "Service-Oriented Architecture for VIEW: a Visual Scientific Workflow Management System", In Proc. of the IEEE 2008 International Conference on Services Computing (SCC), Honolulu, Hawaii, USA, July 2008, pp.335-342.
27. I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, M. Wilde. " Falkon: a Fast and Light-weight tasK executiON framework, IEEE/ACM SuperComputing 2007.
28. I. Raicu, Y. Zhao, I. Foster, A. Szalay. "Accelerating Large-scale Data Exploration through Data Diffusion ", International Workshop on Data-Aware Distributed Computing 2008, co-locate with ACM/IEEE International Symposium High Performance Distributed Computing (HPDC) 2008.
29. M. Wilde, I. Foster, K. Iskra, P. Beckman, Z. Zhang, Allan Espinosa, Mihael Hategan, Ben Clifford, Ioan Raicu. "Parallel Scripting for Applications at the Petascale and Beyond ", IEEE Computer Nov. 2009 Special Issue on Extreme Scale Computing, 2009.
30. NASA MODIS dataset, http://modis.gsfc.nasa.gov/, 2012.
31. Y. Zhao, J. Dobson, I. Foster, L. Moreau, M. Wilde, A Notation and System for Expressing and Executing Cleanly Typed Workflows on Messy Scientific Data, SIGMOD Record, Volume 34, Number 3, September 2005.
32. Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. v. Laszewski, I. Raicu, T. Stef-Praun, M. Wilde. "Swift: Fast, Reliable, Loosely Coupled Parallel Computation ", IEEE Workshop on Scientific Workflows 2007.
33. Hadoop, http://hadoop.apache.org/, 2012
34. D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman, L. Youseff, D. Zagorodnov, The Eucalyptus Open-Source Cloud- Computing Platform, In Proceedings of Cloud Computing and Its Applications 2008.
35. Oliveira, D. Ocaña, K., Ogasawara, E., Dias, J., Baião, F., Mattoso, M.: A Performance Evaluation of X-Ray Crystallography Scientific Workflow Using SciCumulus. IEEE CLOUD 2011, p. 708-715