Scientific workflow design with data assembly lines

Proceedings of the 4th Workshop on Workflows in Support of Large-Scale Science, WORKS '09, in Conjunction with SC 2009

Volumn , Issue , 2009, Pages

  Zinn, Daniel ^a   Bowers, Shawn ^b   McPhillips, Timothy ^c   Ludäscher, Bertram ^a,c  

^a Davis   (United States)

^b Department of Computer Science   (Denmark)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ASSEMBLY LINE; CONDITIONAL EXECUTION; CONFIGURATION MECHANISMS; CONVENTIONAL APPROACH; DATA CHANGE; DATA-DRIVEN; DESIGN CHALLENGES; DESIGN PARADIGM; ERROR-PRONE PROCESS; NESTED DATA; PRACTICAL EXPERIENCE; REAL-WORLD; SCIENTIFIC WORKFLOWS; SEMI-STRUCTURED; VIRTUAL ASSEMBLY; VIRTUAL DATA; WORK-FLOWS;

ASSEMBLY; ASSEMBLY MACHINES; MACHINE DESIGN;

MANAGEMENT;

EID: 74049117535     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1645164.1645178     Document Type: Conference Paper

References (38)

1. D. Abramson, B. Bethwaite, C. Enticott, S. Garic, and T. Peachey. Parameter space exploration using scientific workflows. In Intl. Conf. on Computational Science, LNCS 5544, pages 104-113, 2009.
2. R. Armstrong, D. Gannon, A. Geist, K. Keahey, S. Kohn, L. McInnes, S. Parker, and B. Smolinski. Toward a common component architecture for high-performance scientific computing. In In Intl. Sympos. on High Performance Distr. Computation, 1999.
3. S. Bowers and B. Ludäscher. An ontology driven framework for data transformation in scientific workflows. In Intl. Workshop on Data Integration in the Life Sciences (DILS), LNCS 2994, 2004.
4. S. Bowers and B. Ludäscher. Actor-oriented design of scientific workflows. In Intl. Conf. on Conceptual Modeling (ER), LNCS 3716, 2005.
5. S. Bowers, B. Ludäscher, A. H. Ngu, and T. Critchlow. Enabling scientific workflow reuse through structured composition of dataflow and control-flow. In Post-ICDE Workshop on Workflow and Data Flow for Scientific Applications (SciFlow), Atlanta, GA, April 2006.
6. S. Bowers, T. McPhillips, S. Riddle, M. Anand, and B. Ludäscher. Kepler/pPOD: Scientific workflow and provenance support for assembling the tree of life. In Intl. Provenance and Annotation Workshop (IPAW), 2008.
7. C. Brooks, E. A. Lee, X. Liu, S. Neuendorffer, Y. Zhao, and H. Zheng. Heterogeneous concurrent modeling and design in java (volume 1: Introduction to ptolemy ii). Technical Report No. UCB/EECS-2008-28, April 2008.
8. P. Couvares, T. Kosar, A. Roy, J. Weber, and K. Wenger. Workflow Management in Condor, pages 357-375. In Taylor et al. [33], 2007.
9. S. B. Davidson, S. C. Boulakia, A. Eyal, B. Ludäscher, T. M. McPhillips, S. Bowers, M. K. Anand, and J. Freire. Provenance in scientific workflow systems. IEEE Data Engineering Bulletin, 30(4):44-50, 2007.
10. E. Deelman. Pegasus: A framework for mapping complex scientific workflows onto distributed systems. Scientific Programming, 13(3):219-237, 2005.
11. E. Deelman, D. Gannon, M. Shields, and I. Taylor. Workflows and e-Science: An overview of workflow system features and capabilities. Future Gen. Computer Systems, 25(5):528-540, 2009.
12. J. Felsenstein. PHYLIP (phylogeny inference package) version 3.6. Distributed by the author. Department of Genome Sciences, University of Washington, Seattle, 2004.
13. J. Freire, C. Silva, S. Callahan, E. Santos, C. Scheidegger, and H. Vo. Managing rapidly-evolving scientific workflows. In Intl. Provenance and Annotation Workshop (IPAW), LNCS 4145, pages 10-18, 2006.
14. Y. Gil, V. Ratnakar, E. Deelman, G. Mehta, and J. Kim. Wings for Pegasus: Creating large-scale scientific applications using semantic representations of computational workflows. In National Conference on Artificial Intelligence, pages 1767-1774, 2007.
15. J. Hidders, N. Kwasnikowska, J. Sroka, J. Tyszkiewicz, and J. V. den Bussche. Petri net + nested relational calculus = dataflow. In OTM Conferences, pages 220-237, 2005.
16. D. Hull, R. Stevens, P. Lord, C. Wroe, and C. Goble. Treating "shimantic web" syndrome with ontologies. In Advanced Knowledge Technologies Workshop on Semantic Web Services, 2004.
17. G. Kahn. The semantics of a simple language for parallel programming. In J. L. Rosenfeld, editor, Proc. of the IFIP Congress 74, pages 471-475. North-Holland, 1974.
18. D. Koop, C. Scheidegger, S. Callahan, J. Freire, and C. Silva. Viscomplete: Automating suggestions for visualization pipelines. IEEE Transactions on Visualization and Computer Graphics, 14(6):1691-1698, 2008.
19. C. Lin, S. Lu, X. Fei, D. Pai, and J. Hua. A task abstraction and mapping approach to the shimming problem in scientific workflows. In IEEE Intl. Conf. on Services Computing, Bangalore, India, 2009.
20. B. Ludäscher and I. Altintas. On providing declarative design and programming constructs for scientific workflows based on process networks. Technical Report SciDAC-SPA-TN-2003-01, SDSC, 2003.
21. B. Ludäscher, I. Altintas, S. Bowers, J. Cummings, T. Critchlow, E. Deelman, D. D. Roure, J. Freire, C. Goble, M. Jones, S. Klasky, T. McPhillips, N. Podhorszki, C. Silva, I. Taylor, and M. Vouk. Scientific process automation and workflow management. In A. Shoshani and D. Rotem, editors, Scientific Data Management: Challenges, Existing Technology, and Deployment, Computational Science Series, chapter 13. Chapman & Hall/CRC, 2009.
22. B. Ludäscher, S. Bowers, and T. McPhillips. Scientific workflows. In Encyclopedia of Database Systems. Springer, 2009.
23. T. McPhillips and S. Bowers. An Approach for Pipelining Nested Collections in Scientific Workflows. SIGMOD Record, 34(3):12-17, 2005.
24. T. McPhillips, S. Bowers, and B. Ludäscher. Collection-oriented scientific workflows for integrating and analyzing biological data. In 3rd Intl. Workshop on Data Integration in the Life Sciences, 2006.
25. T. McPhillips, S. Bowers, D. Zinn, and B. Ludäscher. Scientific workflow design for mere mortals. Future Generation Computer Systems, 25(5):541-551, 2009.
26. J. P. Morrison. Flow-Based Programming - A New Approach to Application Development. Van Nostrand Reinhold, 1994.
27. T. Oinn, M. Greenwood, M. Addis, M. N. Alpdemir, J. Ferris, K. Glover, C. Goble, A. Goderis, D. Hull, D. Marvin, P. Li, P. Lord, M. R. Pocock, M. Senger, R. Stevens, A. Wipat, and C. Wroe. Taverna: Lessons in creating a workflow environment for the life sciences. Concurrency and Computation: Practice & Experience, 18(10):1067-1100, August 2006.
28. N. Podhorszki, B. Ludäscher, and S. Klasky. Workflow Automation for Processing Plasma Fusion Simulation Data. In 2nd Workshop on Workflows in Support of Large-Scale Science (WORKS), June 2007.
29. D. D. Roure, C. Goble, and R. Stevens. The design and realisation of the myexperiment virtual research environment for social sharing of workflows. Future Generation Computer Systems, 25:561-567, 2009.
30. A. Stamatakis, M. Ott, and T. Ludwig. RAxML-OMP: An Efficient Program for Phylogenetic Inference on SMPs. Lecture Notes in Computer Science, 3606:288-302, 2005.
31. K. Stevens, D. Cutler, M. Zwick, P. de Jong, K. Huang, M. Koriabine, B. Ludäscher, C. Marston, S. Lee, D. Okou, K. Osoegawa, J. Warrington, D. Begun, and C. Langley. DPGP cyberinfrastructure and open source toolkit for chip based resequencing. In Advances in Genome Biology and Technology (AGBT), 2007.
32. Taverna. http://taverna.sourceforge.net.
33. I. J. Taylor, E. Deelman, D. B. Gannon, and M. Shields, editors. Workflows for e-Science: Scientific Workflows for Grids. Springer, 2007.
34. Triana project. http://www.trianacode.org.
35. K. van Hee, J. Hidders, G.-J. Houben, J. Paredaens, and P. Thiran. On the relationship between workflow models and document types. Information Systems, 34(1):178-208, March 2009.
36. D. Zinn, S. Bowers, S. Köhler, and B. Ludäscher. Parallelizing XML processing pipelines via MapReduce. Journal of Computer and System Sciences, 2009. Special issue on Scientific Workflow. Accepted for publication.
37. D. Zinn, S. Bowers, and B. Ludäscher. XML-Based Computation for Scientific Workflows. In Intl. Conf. on Data Engineering (ICDE), 2010. to appear.
38. D. Zinn, S. Bowers, T. M. McPhillips, and B. Ludäscher. X-CSR: Dataflow Optimization for Distributed XML Process Pipelines. In Intl. Conf. on Data Engineering (ICDE), pages 577-580, 2009. Also see Technical Report CSE-2008-15, UC Davis.