Characterizing workflow-based activity on a production e-infrastructure using provenance data

Future Generation Computer Systems

Volumn 29, Issue 8, 2013, Pages 1931-1942

  Madougou, Souley ^a   Shahand, Shayan ^a   Santcroos, Mark ^a   Van Schaik, Barbera ^a   Benabdelkader, Ammar ^a   Van Kampen, Antoine ^a,b   Olabarriaga, Sílvia ^a  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

Grid computing; Provenance; Scientific workflow management systems; Workflow failure analysis

Indexed keywords

DATA-ACQUISITION DEVICES; E-INFRASTRUCTURES; IMPORTANT FEATURES; PROVENANCE; SCIENTIFIC WORKFLOW MANAGEMENTS; STATISTICAL APPROACH; WORKFLOW EXECUTION; WORKFLOW MANAGEMENT SYSTEMS;

DIGITAL STORAGE; FAILURE ANALYSIS; GRID COMPUTING; WORK SIMPLIFICATION;

INFORMATION MANAGEMENT;

EID: 84878846709     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2013.04.019     Document Type: Article

References (35)

1. T. Hey, S. Tansley, and K. Tolle The Fourth Paradigm: Data-Intensive Scientific Discovery 2009 Microsoft Research 0982544200
2. C. Kesselman, and I. Foster The Grid: Blueprint for a New Computing Infrastructure 1998 Morgan Kaufmann Publishers
3. J. Yu, and R. Buyya A taxonomy of scientific workflow systems for grid computing SIGMOD Record 0163-5808 34 3 2005 44 49 (Pubitemid 43152531)
4. E. Deelman, D. Gannon, M. Shields, and I. Taylor Workflows and e-Science: an overview of workflow system features and capabilities Future Generation Computer Systems 25 5 2009 528 540
5. S.B. Davidson, and J. Freire Provenance and scientific workflows: challenges and opportunities Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data SIGMOD'08 2008 ACM 978-1-60558-102-6 1345 1350
6. L. Bavoil, S. Callahan, P. Crossno, J. Freire, C. Scheidegger, C. Silva, H. Vo, VisTrails: enabling interactive multiple-view visualizations, in: Proceedings of IEEE Visualization, 2005, pp. 135-142.
7. S. Bowers, T.M. McPhillips, S. Riddle, M.K. Anand, B. Ludäscher, Kepler/pPOD: scientific workflow and provenance support for assembling the tree of life, in: Procs of IPAW, 2008, pp. 70-77.
8. J. Kim, E. Deelman, Y. Gil, G. Mehta, and V. Ratnakar Provenance trails in the Wings-Pegasus system Concurrency and Computation: Practice and Experience 1532-0626 20 2008 587 597
9. E. Deelman, G. Singh, M. Hui Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G.B. Berriman, J. Good, A. Laity, J.C. Jacob, and D.S. Katz Pegasus: a framework for mapping complex scientific workflows onto distributed systems Scientific Programming Journal 13 2005 219 237 (Pubitemid 43022937)
10. D. Hull, K. Wolstencroft, R. Stevens, C. Goble, M.R. Pocock, P. Li, and T. Oinn Taverna: a tool for building and running workflows of services Nucleic Acids Research 2006 729 732
11. A. Benabdelkader, M. Santcroos, S. Madougou, A.H.C. van Kampen, and S.D. Olabarriaga A provenance approach to trace scientific experiments on a grid infrastructure Proceedings of the 2011 IEEE Seventh International Conference on eScience e-SCIENCE'11 2011 IEEE Computer Society 978-0-7695-4597-4 134 141
12. M. Gerhards, V. Sander, T. Matzerath, A. Belloum, D. Vasunin, and A. Benabdelkader Provenance opportunities for WS-VLAM: an exploration of an e-science and an e-business approach Proceedings of the 6th Workshop on Workflows in Support of Large-Scale Science WORKS'11 2011 ACM 978-1-4503-1100-7 57 66
13. P. Missier, S. Sahoo, J. Zhao, C. Goble, and A. Sheth Janus: from workflows to semantic provenance and linked open data Lecture Notes in Computer Science vol. 6378/2010 2010 129 141
14. D.A. Cieslak, N.V. Chawla, and D.L. Thain Troubleshooting thousands of jobs on production grids using data mining techniques Proceedings of the 2008 9th IEEE/ACM International Conference on Grid Computing GRID'08 2008 IEEE Computer Society 978-1-4244-2578-5 217 224
15. P. Townend, P. Groth, J. Xu, A provenance-aware weighted fault tolerance scheme for service-based applications, in: Eighth IEEE International Symposium on Object-Oriented Real-Time Distributed Computing, 2005, ISORC 2005, 2005, pp. 258-266. (Pubitemid 43803481)
16. P. Groth, M. Luck, L. Moreau, A protocol for recording provenance in service-oriented grids, in: Proceedings of the 8th International Conference on Principles of Distributed Systems, OPODIS04, 2004, pp. 124-139. (Pubitemid 41422929)
17. T. Samak, D. Gunter, M. Goode, E. Deelman, G. Mehta, F. Silva, and K. Vahi Failure prediction and localization in large scientific workflows Proceedings of the 6th Workshop on Workflows in Support of Large-Scale Science WORKS'11 2011 ACM 978-1-4503-1100-7 107 116
18. R. Ferreira~da Silva, T. Glatard, and F. Desprez Self-healing of operational workflow incidents on distributed computing infrastructures Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing CCGRID'12 2012 IEEE Computer Society 978-0-7695-4691-9 318 325
19. T. Glatard, J. Montagnat, D. Lingrand, and X. Pennec Flexible and efficient workflow deployment of data-intensive applications on grids with MOTEUR Journal of High Performance Computing Applications 22 3 2008 347 360
20. J. Hofer, T. Fahringer, A multi-perspective taxonomy for systematic classification of grid faults, in: 16th Euromicro Conf. on Parallel, Distributed and Network-Based Processing, 2008, pp. 126-130.
21. S. Madougou, M. Santcroos, A. Benabdelkader, B.D. van Schaik, S. Shahand, V. Korkhov, A.H. van Kampen, and S.D. Olabarriaga Provenance for distributed biomedical workflow execution HealthGrid Applications and Technologies Meet Science Gateways for Life Sciences, Vol. 175 2012 91 100
22. S.D. Olabarriaga, T. Glatard, and P.T. de Boer A virtual laboratory for medical image analysis IEEE Transactions on Information Technology in Biomedicine 14 4 2010 979 985
23. S. Shahand, M. Santcroos, A. van Kampen, and S. Olabarriaga A grid-enabled gateway for biomedical data analysis Journal of Grid Computing 1570-7873 2012 1 18
24. J. Montagnat, B. Isnard, T. Glatard, K. Maheshwari, and M.B. Fornarino A data-driven workflow language for grids based on array programming principles Proceedings of the 4th Workshop on Workflows in Support of Large-Scale Science, Vol 10 WORKS'09 2009 ACM 978-1-60558-717-2
25. T. Glatard, J. Montagnat, D. Emsellem, and D. Lingrand A service-oriented architecture enabling dynamic service grouping for optimizing distributed workflow execution Future Generation Computer Systems 0167-739X 24 7 2008 720 730
26. J.T. Moscicki, M. Lamanna, M. Bubak, and P.M.A. Sloot Processing moldable tasks on the grid: late job binding with lightweight user-level overlay Future Generation Computer Systems 27 6 2011 725 736
27. L. Moreau The Open Provenance Model core specification (v1.1) Future Generation Computer Systems 27 6 2011 743 756
28. P.G. Yogesh Simmhan, and L. Moreau Special section: the third provenance challenge on using the Open Provenance Model for interoperability Future Generation Computer Systems 0167-739X 27 6 2011 737 742
29. H. Li, and R. Durbin Fast and accurate short read alignment with Burrows-Wheeler transform Bioinformatics 1367-4803 25 14 2009 1754 1760
30. S.F. Altschul, W. Gish, W. Miller, E.W. Myers, and D.J. Lipman Basic local alignment search tool Journal of Molecular Biology 0022-2836 215 3 1990 403 410
31. B.D.C. van Schaik, M. Santcroos, V. Korkhov, A. Jongejan, M. Willemsen, A.H.C. van Kampen, S.D. Olabarriaga, Challenges in DNA sequence analysis on a production grid, in: PoS(EGICF12-EMITC2), 2012, pp. 1-19 (submitted for publication).
32. P.-N. Tan, M. Steinbach, and V. Kumar Introduction to Data Mining first ed. 2005 Addison-Wesley Longman Publishing Co., Inc. 0321321367
33. T. Hothorn, K. Hornik, and A. Zeileis Unbiased recursive partitioning: a conditional inference framework Journal of Computational and Graphical Statistics 15 3 2006 651 674 (Pubitemid 44554201)
34. T. Hothorn, K. Hornik, A. Zeileis, Party: a laboratory for recursive partytioning, 2006.
35. M.K. Anand, S. Bowers, and B. Ludäscher Database support for exploring scientific workflow provenance graphs Proceedings of the 24th International Conference on Scientific and Statistical Database Management SSDBM'12 2012 Springer-Verlag 978-3-642-31234-2 343 360