Karma2: Provenance management for data-driven workflows

International Journal of Web Services Research

Volumn 5, Issue 2, 2008, Pages 1-22

  Simmhan, Yogesh L ^a   Plale, Beth ^b   Gannon, Dennis ^b  

^a MICROSOFT RESEARCH   (United States)

^b INDIANA UNIVERSITY   (United States)

Author keywords

Data management; Metadata; Process mining; Provenance; Workflows

Indexed keywords

INFORMATION MANAGEMENT;

DATA DRIVEN E SCIENCE; PROCESS MINING; PROVENANCE; PUBLISH-SUBSCRIBE; SERVICE-WORKFLOW; WORK-FLOWS; WORKFLOW ENGINES; WORKFLOW EXECUTION;

METADATA;

EID: 42449141856     PISSN: 15457362     EISSN: 15465004     Source Type: Journal    
DOI: 10.4018/jwsr.2008040101     Document Type: Article

References (44)

1. Armstrong, R., et al. (1999). Toward a common component architecture for high-performance scientific computing. HPDC.
2. BEA Systems, IBM Corporation, Microsoft Corporation, SAP AG, & Siebel Systems (2003). Business process execution language for Web services (BPEL 1.1). Technical Report.
3. Bose, R., & Frew, J. (2004). Composing lineage metadata with XML for custom satellite-derived data products. SSDBM.
4. Bose, R., & Frew, J. (2005). Lineage retrieval for scientific data processing: A survey. ACM Computing Surveys, 37, 1-28.
5. Bose, R., Foster, I., & Moreau. L. (2006). Report on the international provenance and annotation workshop. SIGMOD Record, 35(3), 51-53.
6. Box, D., et al. (2004). Web services eventing (WS-eventing). Technical Report.
7. Brandes, U., Eiglsperger, M., Herman, I., Himsolt, M., & Marshall, M.S. (2002). GraphML progress report: Structural layer proposal. LNCS, 2265, 501-512.
8. Catlett, C. (2002). The TeraGrid: A primer. Retrieved from www.teragrid.org
9. Churches, D., et al. (2006). Programming scientific and distributed workflow with Triana Services. Concurrency and Computation: Practice and Experience, 18(10), 1021-1037.
10. Cohen-Boulakia, S., Cohen, S., & Davidson, S. (2007). Addressing the provenance challenge using zoom. Concurrency and Control: Practice and Experience.
11. de Rose, L.A., & Reed, D.A. (1999). SvPablo: A multi-language architecture-independent performance analysis system. ICPP.
12. Droegemeier, K.K., et al. (2005). Service-oriented environments for dynamically interacting with mesoscale weather. Computing in Science and Engineering, 7(6), 12-29.
13. Eugster, P.T., Felber, P.A., Guerraoui, R., & Kermarrec, A.M. (2003). The many faces of publish/subscribe. ACM Computing Surveys, 35, 114-131.
14. Foster, I., Kesselman, C., Nick, J., & Tuecke S. (2002). The physiology of the grid: An open grid services architecture for distributed systems integration. Global Grid Forum.
15. Foster, I.T., Vöckler, J., Wilde, M., & Zhao, Y. (2003). The virtual data grid: A new model and architecture for data-intensive collaboration. CIDR.
16. Gannon, D., et al. (2005). Service oriented architectures for science gateways on grid systems. ICSOC.
17. Goble, C. (2002). Position statement: Musings on provenance, workflow and (semantic Web) annotations for bioinformatics. Workshop on Data Derivation and Provenance.
18. Greenwood, M., et al. (2003). Provenance of e-science experiments - experience from Bioinformatics. UK OST e-Science 2nd AHM.
19. Groth, P., Luck, M., & Moreau, L. (2004). A protocol for recording provenance in service-oriented grids. OPODIS.
20. Groth, P., Miles, S., Fang, W., Wong, S.C., Zauner, K.-P., & Moreau, L. (2005). Recording and using provenance in a protein compressibility experiment. HPDC.
21. Gunter, D., Tierney, B., Jackson, K., Lee, J., & Stoufer, M. (2002). Dynamic monitoring of high-performance distributed applications. HPDC.
22. Huang, Y., Slominski, A., Herath, C., & Gannon, D. (2006). WS-Messenger: A Web services based messaging system for service-oriented grid computing. CCGrid.
23. IBM (2005). Web services data collector. Retrieved from www.alphaworks.ibm.com/tech/wsdatacollector
24. Kandaswamy, G., Fang, L., Huang, Y., Shirasuna, S., Marru, S., & Gannon, D. (2006). Building Web services for scientific grid applications. IBM Journal of Research and Development, 50(2/3), 249-260.
25. Liu, D.T., Franklin, M.J., Garlick, J., & Abdulla, G.M. (2005). Scaling up data-centric middleware on a cluster computer. Technical Report, Lawrence Livermore National Laboratory.
26. Ludäscher, B., et al. (2006). Scientific workflow management and the Kepler system. Concurrency and Computation: Practice and Experience, 18(10), 1039-1065.
27. Miles, S., Groth, P., Branco, M., & Moreau, L. (2005). The requirements of recording and using provenance in e-science experiments. Technical Report, Electronics and Computer Science, University of Southampton.
28. Moreau, L., & Foster, I. (2006). Provenance and annotation of data. Proceedings of the International Provenance and Annotation Workshop, LNCS 4145.
29. Moreau, L., & Ludascher, B. (2007). The first provenance challenge. Concurrency and Control: Practice and Experience.
30. OMG (2006). CORBA component model v4.0. Technical Report, OMG.
31. Pallickara, S., Fox, G., Yildiz, B., Pallickara, S.L., Patel, S., & Yemme, D. (2005). On the costs for reliable messaging in Web/grid service environments. e-Science.
32. Plale, B. (2005). Resource requirements study for LEAD storage repository. Technical Report 001, Linked Environments for Atmospheric Discovery.
33. Ribler, R.L., Vetter, J.S., Simitci, H., & Reed, D.A. (1998). Autopilot: Adaptive control of distributed applications. HPDC.
34. Shirasuna, S., & Gannon, D. (2006). XBaya: A graphical workflow composer for the Web services architecture. Technical Report 004, Linked Environments for Atmospheric Discovery.
35. Simmhan, Y., Plale, B., & Gannon, D. (2005). A survey of data provenance in e-science. SIGMOD Record, 34(3), 31-36.
36. Simmhan, Y.L., Pallickara, S.L., Vijayakumar, N.N., & Plale, B. (2006a). Data management in dynamic environment-driven computational science. Proceedings of the IFIP Working Conference on Grid-Based Problem Solving Environments (WoCo9).
37. Simmhan, Y.L., Plale, B., & Gannon, D. (2006b). Towards a quality model for effective data selection in collaboratories. Proceedings of the IEEE Workflow and Data Flow for Scientific Applications (SciFlow) Workshop.
38. Simmhan, Y.L., Plale, B., & Gannon, D. (2007). Query capabilities of the Karma provenance framework. Concurrency and Control: Practice and Experience.
39. Simmhan, Y.L., Plale, B., Gannon, D., & Marru, S. (2006c). Performance evaluation of the Karma provenance framework for scientific workflows. IPAW and LNCS 4145.
40. Slominski, A. (2006). Adapting BPEL to scientific workflows. Workflows for e-Science.
41. Szomszor, M., & Moreau, L. (2003). Recording and reasoning over data provenance in Web and grid services. ODBASE.
42. West, K. (2005). Scoping out the planet. Scientific American.
43. Yu, J., & Buyya, R. (2005). A taxonomy of scientific workflow systems for grid computing. SIGMOD Record, 34(3), 44-49.
44. Zhao, J., Wroe, C., Goble, C.A., Stevens, R., Quan, D., & Greenwood, R.M. (2004). Using semantic Web technologies for representing e-science provenance. ISWC.