Dispel4py: An agile framework for data-intensive eScience

Proceedings - 11th IEEE International Conference on eScience, eScience 2015

Volumn , Issue , 2015, Pages 454-464

  Filgueira, Rosa ^a   Krause, Amrey ^a   Atkinson, Malcolm ^a   Klampanos, Iraklis ^a   Spinuso, Alessandro ^b   Sanchez Exposito, Susana ^c  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b ROYAL NETHERLANDS METEOROLOGICAL INSTITUTE KNMI   (Netherlands)

^c INSTITUTO DE ASTROFÍSICA DE ANDALUCÍA CSIC   (Spain)

Author keywords

Data intensive application; Distributed systems; ESciences workflows; Python frameworks; Run time analysis

Indexed keywords

DISTRIBUTED COMPUTER SYSTEMS; HIGH LEVEL LANGUAGES; MAPPING;

DATA-INTENSIVE APPLICATION; DISTRIBUTED SYSTEMS; PYTHON FRAMEWORKS; RUN-TIME ANALYSIS; WORK-FLOWS;

DATA HANDLING;

EID: 84959055167     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/eScience.2015.40     Document Type: Conference Paper

References (30)

1. I. Altintas, C. Berkley, E. Jaeger, M. Jones, B. Ludascher, and S. Mock, "Kepler: an extensible system for design and execution of scientific workflows," in Proceedings of 16th International Conference on Scientific and Statistical Database Management, June 2004, pp. 423-424
2. E. Deelman, G. Singh, M.-H. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G. B. Berriman, J. Good, A. C. Laity, J. C. Jacob, and D. S. Katz, "Pegasus: A framework for mapping complex scientific workflows onto distributed systems," Scientific Programming, vol. 13, no. 3, pp. 219-237, 2005
3. D. Churches and etal., "Programming scientific and distributed workflow with Triana services," Conc. &Comp.: P&E, vol. 18, no. 10, pp. 1021-1037, 2006
4. M. Wilde and etal., "Swift: A language for distributed parallel scripting," Parallel Comp., vol. 37, no. 9, pp. 633-652, 2011
5. Y. Simmhan and etal., "Building the Trident Scientific Workflow Workbench for Data Management in the Cloud," in ADVCOMP. IEEE, October 2009
6. X. Llorá, B. Acs, L. S. Auvil, B. Capitanu, M. E. Welge, and D. E. Goldberg, "Meandre: Semantic-Driven Data-Intensive Flows in the Clouds," in IEEE Fourth International Conference on eScience. IEEE Press, 2008, pp. 238-245
7. Z. Falt and etal., "Bobolang: A Language for Parallel Streaming Applications," in Proc. HPDC. ACM, 2014, pp. 311-314
8. D. De Roure and etal., "The Design and Realisation of the myExperiment Virtual Research Environment for Social Sharing of Workflows," FGCS, vol. 25, pp. 561-567, 2009
9. P. Fraternali and S. Paraboschi, "Chimera: A language for designing rule applications. " in Active Rules in Database Systems, 1999, pp. 309-322. [Online]. Available: http://dblp. uni-trier. de/db/books/collections/patton99. html#FraternaliP99
10. K. Belhajjame and etal., "A Suite of Ontologies for Preserving Workflow-Centric Research Objects," J. Web Semantics, in press 2015
11. L. Lefort and etal., "W3C Incubator Group Report-review of Sensor and Observation ontologies," W3C, Tech. Rep., 2010
12. D. Rogers and etal., "Bundle and Pool Architecture for Multi-Language, Robust, Scalable Workflow Executions," J. Grid Comput., vol. 11, no. 3, pp. 457-480, 2013
13. M. P. Atkinson, C. S. Liew, M. Galea, P. Martin, A. Krause, A. Mouat, O. Corcho, and D. Snelling, "Data-intensive architecture for scientific knowledge discovery," Distributed and Parallel Databases, vol. 30, no. 5-6, pp. 307-324, 2012. [Online]. Available: http://dx. doi. org/10. 1007/ s10619-012-7105-3
14. MPI Forum, "MPI: A message-passing interface standard," IJ of Supercomputer Applications, vol. 8, pp. 165-414, 1994
15. A. Misra, A. Misra, M. Blount, M. Blount, A. Kementsietsidis, A. Kementsietsidis, D. Sow, D. Sow, M. Wang, and M. Wang, "Advances and Challenges for Scalable Provenance in Stream Processing Systems," Provenance and Annotation of Data and Processes, vol. 01, pp. 253-265, 2008. [Online]. Available: http://www. springerlink. com/index/w5865j066612l840. pdf
16. W. D. Pauw, M. Letja, B. Gedik, and H. Andrade, "Visual debugging for stream processing applications," Runtime Verification, pp. 18-35, 2010. [Online]. Available: http: //link. springer. com/chapter/10. 1007/978-3-642-16612-9\-3
17. A. Spinuso and etal., "Provenance for seismological processing pipelines in a distributed streaming workflow," in Proc. EDBT '13. ACM, 2013, pp. 307-312
18. M. Atkinson, M. C. S. Claus, R. Filgueira, and et al., "Verce delivers a productive e-science environment for seismology research," in IEEE International eScience Conference, 2015
19. R. Filguiera, I. Klampanos, A. Krause, M. David, A. Moreno, and M. Atkinson, "Dispel4py: A python framework for data-intensive scientific computing," in Proceedings of the 2014 International Workshop on Data Intensive Scalable Computing Systems, ser. DISCS '14. Piscataway, NJ, USA: IEEE Press, 2014, pp. 9-16. [Online]. Available: http://dx. doi. org/10. 1109/DISCS. 2014. 12
20. J. Ruiz, J. Garrido, J. Santander-Vela, S. Sánchez-Expósito, and L. Verdes-Montenegro, "Astrotaverna-building workflows with virtual observatory services," Astronomy and Computing, vol. 7-8, no. 0, pp. 3-11, 2014, special Issue on The Virtual Observatory: I
21. A. Pak and P. Paroubek, "Twitter as a corpus for sentiment analysis and opinion mining," in Proceedings of the International Conference on Language Resources and Evaluation, LREC 2010, 17-23 May 2010, Valletta, Malta, N. Calzolari, K. Choukri, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis, M. Rosner, and D. Tapias, Eds. European Language Resources Association, 2010. [Online]. Available: http://www. lrec-conf. org/proceedings/ lrec2010/summaries/385. html
22. H. Cho, J. Lee, and S. Kim, "Enhancing lexicon-based review classification by merging and revising sentiment dictionaries," in Sixth International Joint Conference on Natural Language Processing, IJCNLP 2013, Nagoya, Japan, October 14-18, 2013. Asian Federation of Natural Language Processing / ACL, 2013, pp. 463-470. [Online]. Available: http://aclweb. org/anthology/I/I13/I13-1053. pdf
23. F. A. Nielsen, "Afinn," Richard Petersens Plads, Building 321, DK-2800 Kgs. Lyngby, mar 2011
24. S. Baccianella, A. Esuli, and F. Sebastiani, "Sentiwordnet 3. 0: An enhanced lexical resource for sentiment analysis and opinion mining," in Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10), N. C. C. Chair), K. Choukri, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis, M. Rosner, and D. Tapias, Eds. Valletta, Malta: European Language Resources Association (ELRA), may 2010
25. "Real time data compress," 2012, "http://fastcompression. blogspot. co. uk/p/lz4. html"
26. R. Filgueira, M. P. Atkinson, Y. Tanimura, and I. Kojima, "Applying selectively parallel I/O compression to parallel storage systems," in Euro-Par 2014 Parallel Processing-20th International Conference, Porto, Portugal, August 25-29, 2014. Proceedings, ser. Lecture Notes in Computer Science, F. M. A. Silva, I. de Castro Dutra, and V. S. Costa, Eds., vol. 8632. Springer, 2014, pp. 282-293. [Online]. Available: http://dx. doi. org/10. 1007/978-3-319-09873-9-24
27. J. Russell and R. Cohn, Snappy. Book on Demand, 2012. [Online]. Available: http://books. google. co. uk/books? id=PXajMQEACAAJ
28. Markus Franz Xaver Johannes Oberhumer, "LZO," 2002, http://www. oberhumer. com/opensource/lzo/lzodoc. php
29. R. Zigon, "Run length encoding," Dr. Dobb's Journal of Software Tools, vol. 14, no. 2, Feb. 1989
30. D. E. Knuth, "Dynamic huffman coding," J. Algorithms, vol. 6, no. 2, pp. 163-180, 1985.