Storing, reasoning, and querying OPM-compliant scientific workflow provenance using relational databases

Future Generation Computer Systems

Volumn 27, Issue 6, 2011, Pages 781-789

  Lim, Chunhyeok ^a   Lu, Shiyong ^a   Chebotko, Artem ^b   Fotouhi, Farshad ^a  

^a Wayne State University   (United States)

^b UNIVERSITY OF TEXAS PAN AMERICAN   (United States)

Author keywords

Metadata management; OPM; OPM compliant provenance storage; Provenance; RDBMS; Scientific workflow

Indexed keywords

METADATA MANAGEMENT; OPM; OPM-COMPLIANT PROVENANCE STORAGE; PROVENANCE; RDBMS; SCIENTIFIC WORKFLOWS;

INTEROPERABILITY; MANAGEMENT; QUERY LANGUAGES; QUERY PROCESSING;

METADATA;

EID: 79953196833     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2010.10.013     Document Type: Conference Paper

References (43)

1. E. Deelman, A.L. Chervenak, Data management challenges of data-intensive scientific workflows, in: Proc. of the International Symposium on Cluster Computing and the Grid, CCGRID, 2008, pp. 687692.
2. S.B. Davidson, J. Freire, Provenance and scientific workflows: challenges and opportunities, in: Proc. of the ACM SIGMOD International Conference on Management of Data, June 2008, pp. 13451350.
3. Z. Zhao, A. Belloum, and M. Bubak Special section on workflow systems and applications in e-Science Future Generation Computer Systems (FGCS) 25 5 2009 525 527
4. E. Deelman, D. Gannon, M.S. Shields, and I. Taylor Workflows and e-Science: an overview of workflow system features and capabilities Future Generation Computer Systems (FGCS) 25 5 2009 528 540
5. T.M. McPhillips, S. Bowers, D. Zinn, and B. Ludäscher Scientific workflow design for mere mortals Future Generation Computer Systems (FGCS) 25 5 2009 541 551
6. E. Elmroth, F. Hernndez, and J. Tordsson Three fundamental dimensions of scientific workflow interoperability: model of computation, language, and execution environment Future Generation Computer Systems (FGCS) 26 2 2010 245 256
7. C. Lin, S. Lu, X. Fei, A. Chebotko, D. Pai, Z. Lai, F. Fotouhi, and J. Hua A reference architecture for scientific workflow management systems and the VIEW SOA solution IEEE Transactions on Services Computing (TSC) 2 1 2009 79 92
8. E. Deelman, Y. Gil, M. Zemankova, NSF workshop on the challenges of scientific workflows, May 2006.
9. G. Bell, T. Hey, and A. Szalay Beyond the data deluge Science 323 6 2009 1297 1298
10. P. Groth, and L. Moreau Recording process documentation for provenance IEEE Transactions on Parallel and Distributed Systems (TPDS) 20 9 2009 1246 1259
11. Y. Simmhan, B. Plale, and D. Gannon A survey of data provenance in e-Science ACM SIGMOD Record 34 3 2005
12. S. Bowers, T. McPhillips, S. Riddle, M. Anand, B. Ludäscher, Kepler/pPOD: scientific workflow and provenance support for assembling the tree of life, in: Proc. of the International Provenance and Annotation Workshop, IPAW, June 2008.
13. P. Missier, K. Belhajjame, J. Zhao, C. Goble, Data lineage model for Taverna workflows with lightweight annotation requirements, in: Proc. of the International Provenance and Annotation Workshop, IPAW, 2008.
14. Y. Simmhan, B. Plale, and D. Gannon Query capabilities of the Karma provenance framework Concurrency and Computation: Practice and Experience 20 5 2008 441 451 (Pubitemid 351428358)
15. J. Freire, C.T. Silva, S.P. Callahan, E. Santos, C.E. Scheidegger, H.T. Vo, Managing rapidly-evolving scientific workflows, in: Proc. of the International Provenance and Annotation Workshop, IPAW, 2006.
16. Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. vonLaszewski, I. Raicu, T. Stef-Praun, M. Wilde, Swift: fast, reliable, loosely coupled parallel computation, in: Proc. of the IEEE International Workshop on Scientific Workflows, SWF, 2007, pp. 199206.
17. S. Miles, P. Groth, M. Branco, and L. Moreau The requirements of recording and using provenance in e-Science experiments International Journal of Grid Computing 5 1 2007 1 25 (Pubitemid 46909010)
18. J. Kim, E. Deelman, Y. Gil, G. Mehta, and V. Ratnakar Provenance trails in the Wings/Pegasus workflow system Concurrency and Computation: Practice and Experience 20 5 2008 587 597 (Pubitemid 351428371)
19. B. Plale, S. Miles, C. Goble, P. Missier, R. Barga, Y. Simmhan, J. Futrelle, R.E. McGrath, J. Myers, P. Paulson, S. Bowers, B. Ludäscher, N. Kwasnikowska, J.V. Bussche, T. Ellkvist, J. Freire, P. Groth, L. Moreau (Ed.) The open provenance model (v1.01), Technical Report, University of Southampton, July 2008.
20. The third international provenance and annotation workshop (IPAW 2010), Lecture Notes in Computer Science, LNCS 6378, (2010) (in press).
21. Provenance challenge wiki, 2006. http://twiki.ipaw.info/bin/view/ Challenge/WebHome.
22. S. Miles, P. Groth, S. Munroe, S. Jiang, T. Assandri, and L. Moreau Extracting causal graphs from an open provenance data model Concurrency and Computation: Practice and Experience 20 5 2008 577 586 (Pubitemid 351428370)
23. N. Kwasnikowska, J.V. Bussche, Mapping the NRC dataflow model to the open provenance model, in: Proc. of the International Provenance and Annotation Workshop (IPAW), 2008.
24. J. Alhiyafi, A scientific workflow system for genomic data analysis, Ph.D. Thesis, Wayne State University, 2010.
25. The third provenance challenge, 2009. http://twiki.ipaw.info/bin/view/ Challenge/ThirdProvenanceChallenge.
26. L. Moreau, B. Clifford, J. Freire, J. Futrelle, Y. Gil, P. Groth, N. Kwasnikowska, S. Miles, P. Missier, J. Myers, B. Plale, Y. Simmhan, E. Stephan, and J.V. Bussche The open provenance model core specification (v1.1) Future Generation Computer Systems (FGCS) 2010
27. The XML schema of the open provenance model (v1.01), June 2008. http://openprovenance.org/model/v1.01.a.
28. M. Atay, A. Chebotko, D. Liu, S. Lu, and F. Fotouhi Efficient schema-based XML-to-relational data mapping Information Systems (IS) 32 3 2007 458 476 (Pubitemid 44821544)
29. C. Lim, S. Lu, A. Chebotko, F. Fotouhi, Storing, reasoning, and querying OPM-compliant scientific workflow provenance using relational databases, Technical Report TR-SWR-082010-LLCF, August 2010. Available from: http://paris.cs.wayne.edu/~dv6585/TR/OPMProv.pdf.
30. Pan-STARRS project home page, 2009. http://ps1sc.org/.
31. Y. Simmhan, B. Plale, and D. Gannon Karma2: provenance management for data driven workflows International Journal of Web Services Research 5 2 2008 1 22
32. M. Anand, S. Bowers, T. McPhillips, B. Ludäscher, Exploring scientific workflow provenance using hybrid queries over nested data and lineage graphs, in: Proc. of the International Conference on Scientific and Statistical Database Management, SSDBM, 2009.
33. M. Anand, S. Bowers, T. McPhillips, B. Ludäscher, Efficient provenance storage over nested data collections, in: Proc. of the International Conference on Extending Database Technology, EDBT, 2009.
34. P. Missier, N.W. Paton, K. Belhajjame, Fine-grained and efficient lineage querying of collection-based workflow provenance, in: Proc. of the International Conference on Extending Database Technology, EDBT, 2010, pp. 299310.
35. J. Zhao, C. Goble, R. Stevens, and D. Turi Mining Taverna's semantic web of provenance Concurrency and Computation: Practice and Experience 20 5 2008 463 472 (Pubitemid 351428360)
36. B. Cao, B. Plale, G. Subramanian, E. Robertson, Y. Simmhan, Provenance information model of Karma version 3, in: Proc. of the IEEE International Workshop on Scientific Workflows, SWF, July 2009.
37. C. Silva, J. Freire, and S. Callahan Provenance for visualizations: reproducibility and beyond IEEE Computing in Science and Engineering 9 5 2007 82 89 (Pubitemid 47340566)
38. P. Groth, S. Miles, and L. Moreau A model of process documentation to determine provenance in mash-ups ACM Transactions on Internet Technology (TOIT) 9 1 2009 1 31
39. C. Lim, S. Lu, A. Chebotko, F. Fotouhi, Prospective and retrospective provenance collection in scientific workflow environments, in: Proc. of the IEEE International Conference on Services Computing, SCC, 2010, pp. 449456.
40. A. Chebotko, X. Fei, C. Lin, S. Lu, F. Fotouhi, Storing and querying scientific workflow provenance metadata using an RDBMS, in: Proc. of the IEEE International Conference on e-Science and Grid Computing, e-Science, 2007, pp. 611618.
41. A. Chebotko, S. Lu, X. Fei, and F. Fotouhi RDFProv: a relational RDF store for querying and managing scientific workflow provenance Data & Knowledge Engineering (DKE) 69 8 2010 836 865
42. B. He, Q. Luo, and B. Choi Adaptive index utilization in memory-resident structural joins IEEE Transactions on Knowledge and Data Engineering (TKDE) 19 6 2007 772 788 (Pubitemid 46685655)
43. Y. Chen, Y. Chen, An efficient algorithm for answering graph reachability queries, in: Proc. of the International Conference on Data Engineering, ICDE, 2008, pp. 893902.