User-steering of HPC workflows: State-of-the-art and future directions

Proceedings of the 2nd ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies, SWEET 2013

Volumn , Issue , 2013, Pages

  Mattoso, Marta ^a   Ocãa, Kary ^a   Horta, Felipe ^a   Dias, Jonas ^a   Ogasawara, Eduardo ^c   Silva, Vitor ^a   De Oliveira, Daniel ^b   Costa, Flavio ^a   Araújo, Igor ^a  

^a FEDERAL UNIVERSITY OF RIO DE JANEIRO   (Brazil)

^b FLUMINENSE FEDERAL UNIVERSITY   (Brazil)

^c Federal Center of Technological Education   (Brazil)

Author keywords

provenance; scientific workflows; steering

Indexed keywords

COMPUTER PROGRAMMING;

LONG LASTING; MONITORING ANALYSIS; PROVENANCE; REAL TIME MONITORING; SCIENTIFIC WORKFLOWS; STATE OF THE ART; SUPPORTING TECHNOLOGY; USER DEMANDS; WORK-FLOWS; WORKFLOW EXECUTION;

BIG DATA;

EID: 85140868998     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2499896.2499900     Document Type: Conference Paper

References (39)

1. E. Deelman, D. Gannon, M. Shields, and I. Taylor, 2009, Workflows and e-Science: An overview of workflow system features and capabilities, Future Generation Computer Systems, v. 25, n. 5, p. 528-540.
2. I. Foster and C. Kesselman, 2004, The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann.
3. L.M. Vaquero, L. Rodero-Merino, J. Caceres, and M. Lindner, 2009, A break in the clouds: towards a cloud definition, SIGCOMM Comput. Commun. Rev., v. 39, n. 1, p. 50-55.
4. A.M.R. Dávila, P.N. Mendes, G. Wagner, D.A. Tschoeke, R.R.C. Cuadrat, F. Liberman, L. Matos, T. Satake, K.A.C.S. Ocaña, et al., 2008, ProtozoaDB: dynamic visualization and exploration of protozoan genomes, Nucleic Acids Research, v. 36, n. Database issue, p. D547-D552.
5. E. Ogasawara, D. Oliveira, F. Chirigati, C.E. Barbosa, R. Elias, V. Braganholo, A. Coutinho, and M. Mattoso, 2009, Exploring many task computing in scientific workflows, In: Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers, p. 1-10
6. Y. Gil, E. Deelman, M. Ellisman, T. Fahringer, G. Fox, D. Gannon, C. Goble, M. Livny, L. Moreau, et al., 2007, Examining the Challenges of Scientific Workflows, Computer, v. 40, n. 12, p. 24-32.
7. M. Wilde, M. Hategan, J.M. Wozniak, B. Clifford, D.S. Katz, and I. Foster, 2011, Swift: A language for distributed parallel scripting, Parallel Computing, n. 37(9), p. 633-652.
8. D. Oliveira, K. Ocaña, F. Baião, and M. Mattoso, 2012, A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds, Journal of Grid Computing, v. 10, n. 3, p. 521-552.
9. E. Deelman, G. Mehta, G. Singh, M.-H. Su, and K. Vahi, 2007, "Pegasus: Mapping Large-Scale Workflows to Distributed Resources", Workflows for e-Science, Springer, p. 376-394.
10. A. Ailamaki, V. Kantere, and D. Dash, 2010, Managing scientific data, Communications of the ACM, v. 53, n. 6 (Jun.), p. 68-78.
11. A. Ailamaki, 2011, Managing scientific data: lessons, challenges, and opportunities, In: Proceedings of the 2011 ACM SIGMOD International Conference on Management of data, p. 1045-1046
12. J. Freire, D. Koop, E. Santos, and C.T. Silva, 2008, Provenance for Computational Tasks: A Survey, Computing in Science and Engineering, v.10, n. 3, p. 11-21.
13. K.A.C.S. Ocaña, D. Oliveira, E. Ogasawara, A.M.R. Dávila, A.A.B. Lima, and M. Mattoso, 2011, "SciPhy: A Cloud-Based Workflow for Phylogenetic Analysis of Drug Targets in Protozoan Genomes", In: O. Norberto de Souza, G.P. Telles, and M. Palakal, eds., Advances in Bioinformatics and Computational Biology, chapter 6832, Berlin, Heidelberg: Springer, p. 66-70.
14. K.A.C.S. Ocaña, D. Oliveira, J. Dias, E. Ogasawara, and M. Mattoso, 2011, Optimizing Phylogenetic Analysis Using SciHmm Cloud-based Scientific Workflow, In: 2011 IEEE Seventh International Conference on e-Science (e-Science), p. 190-197
15. K.A.C.S. Ocaña, D. Oliveira, J. Dias, E. Ogasawara, and M. Mattoso, 2012, Discovering Drug Targets for Neglected Diseases Using a Pharmacophylogenomic Cloud Workflow, In: Proceedings of the IEEE 8th International Conference on e-Science
16. G. Guerra, F. Rochinha, R. Elias, D. Oliveira, E. Ogasawara, J. Dias, M. Mattoso, and A.L.G.A. Coutinho, 2012, Uncertainty Quantification in Computational Predictive Models for Fluid Dynamics Using Workflow Management Engine, International Journal for Uncertainty Quantification, v. 2, n. 1, p. 53-71.
17. E. Ogasawara, J. Dias, D. Oliveira, F. Porto, P. Valduriez, and M. Mattoso, 2011, An Algebraic Approach for Data-Centric Scientific Workflows, Proc. of VLDB Endowment, v. 4, n. 12, p. 1328-1339.
18. J. Dias, E. Ogasawara, D. Oliveira, F. Porto, A. Coutinho, and M. Mattoso, 2011, Supporting Dynamic Parameter Sweep in Adaptive and User-Steered Workflow, In: 6th Workshop on Workflows in Support of Large-Scale Science, p. 31-36
19. D. Oliveira, K.A.C.S. Ocaña, E. Ogasawara, J. Dias, J. Goncalves, and M. Mattoso, 2012, Cloud-based Phylogenomic Inference of Evolutionary Relationships: A Performance Study, In: Proceedings of the 2nd International Workshop on Cloud Computing and Scientific Applications (CCSA)
20. K.A.C.S. Ocaña, D. de Oliveira, F. Horta, J. Dias, E. Ogasawara, and M. Mattoso, 2012, "Exploring Molecular Evolution Reconstruction Using a Parallel Cloud-based Scientific Workflow", Advances in Bioinformatics and Computational Biology, chapter 7409, Berlin, Heidelberg: Springer, p. 179-191.
21. Amazon EC2, 2010, Amazon Elastic Compute Cloud (Amazon EC2), http://aws.amazon.com/ec2/.
22. D. Oliveira, E. Ogasawara, F. Baião, and M. Mattoso, 2010, Sci-Cumulus: A Lightweight Cloud Middleware to Explore Many Task Computing Paradigm in Scientific Workflows, In: 3rd International Conference on Cloud Computing, p. 378-385
23. V.C. Emeakaroha, P. \Labaj, M. Maurer, I. Brandic, and D.P. Kreil, 2011, Optimizing bioinformatics workflows for data analysis using cloud management techniques, In: Proceedings of the 6th workshop on Workflows in support of large-scale science, p. 37-46
24. K. Lee, R. Sakellariou, N.W. Paton, and A.A.A. Fernandes, 2007, Workflow adaptation as an autonomic computing problem, In: Proceedings of the 2nd workshop on Workflows in support of large-scale science, p. 29-34
25. H. Truong, P. Brunner, T. Fahringer, F. Nerieri, R. Samborski, B. Balis, M. Bubak, and K. Rozkwitalski, Dec., K-WfGrid Distributed Monitoring and Performance Analysis Services for Workflows in the Grid, In: Second IEEE International Conference on e-Science and Grid Computing, 2006. e-Science '06, p. 15-15
26. K. Vahi, I. Harvey, T. Samak, D. Gunter, K. Evans, D. Rogers, I. Taylor, M. Goode, F. Silva, et al., 2012, A General Approach to Real-time Workflow Monitoring
27. P. Missier, S. Soiland-Reyes, S. Owen, W. Tan, A. Nenadic, I. Dunlop, A. Williams, T. Oinn, and C. Goble, 2010, Taverna, reloaded, In: Proceedings of the 22nd international conference on Scientific and statistical database management, p. 471-481
28. S.M.S. da Cruz, P.M. Barros, P.M. Bisch, M.L.M. Campos, and M. Mattoso, 2008, Provenance Services for Distributed Workflows, In: Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid, p. 526-533
29. F. Costa, V. Silva, D. Oliveira, K. Ocaña, J. Dias, E. Ogasawara, and M. Mattoso, 2013, Capturing and Querying Workflow Runtime Provenance with PROV: a Practical Approach, In: Proc. of the International Workshop on Managing and Querying Provenance Data at Scale
30. S.P. Callahan, J. Freire, E. Santos, C.E. Scheidegger, C.T. Silva, and H.T. Vo, 2006, VisTrails: visualization meets data management, In: SIGMOD International Conference on Management of Data, p. 745-747
31. N. Fabian, K. Moreland, D. Thompson, A.C. Bauer, P. Marion, B. Geveci, M. Rasquin, and K.E. Jansen, Oct., The ParaView Coprocessing Library: A scalable, general purpose in situ visualization library, In: 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV), p. 89-96
32. H. Yu, C. Wang, R.W. Grout, J.H. Chen, and K.-L. Ma, May-June, In Situ Visualization for Large-Scale Combustion Simulations, IEEE Computer Graphics and Applications, v. 30, n. 3, p. 45-57.
33. B. Whitlock, J. Favre, and J. Meredith, 2011, Parallel In Situ Coupling of Simulation with a Fully Featured Visualization System, In: Proceedings of the Eurographics Symposium on Parallel Graphics and Visualization
34. F. Horta, J. Dias, K. Ocaña, D. Oliveira, E. Ogasaw, and M. Mattoso, 2012, Poster: Using Provenance to Visualize Data from Large-Scale Experiments, In: Poster Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
35. S. Bowers, B. Ludascher, A.H.H. Ngu, and T. Critchlow, 2006, Enabling ScientificWorkflow Reuse through Structured Composition of Dataflow and Control-Flow, In: Proceedings of the 22nd International Conference on Data Engineering Workshops, p. 70
36. T. Samak, D. Gunter, M. Goode, E. Deelman, G. Mehta, F. Silva, and K. Vahi, 2011, Failure prediction and localization in large scientific workflows, In: Proceedings of the 6th workshop on Workflows in support of large-scale science, p. 107-116
37. F. Costa, D. de Oliveira, K.A.C.S. Ocaña, E. Ogasawara, and M. Mattoso, 2012, "Enabling Re-executions of Parallel Scientific Workflows Using Runtime Provenance Data", In: P. Groth and J. Frew, eds., Provenance and Annotation of Data and Processes, Springer Berlin Heidelberg, p. 229-232.
38. F. Costa, D. Oliveira, K. Ocaña, E. Ogasawara, J. Dias, and Mattoso M., 2012, Handling Failures in Parallel Scientific Workflows Using Clouds
39. N. Kourtellis, L. Prieto, A. Iamnitchi, G. Zarrate, and D. Fraser, 2008, Data transfers in the grid: workload analysis of globus GridFTP, In: Proceedings of the 2008 international workshop on Data-aware distributed computing, p. 29-38