Dispel4py: A python framework for data-intensive scientific computing

Proceedings of DISCS 2014: The 2014 International Workshop on Data-Intensive Scalable Computing Systems - Held in Conjuction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis

Volumn , Issue , 2014, Pages 9-16

  Filguiera, Rosa ^a   Klampanos, Iraklis ^a   Krause, Amrey ^a   David, Mario ^b   Moreno, Alexander ^c   Atkinson, Malcolm ^a  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b INSTITUT DE PHYSIQUE DU GLOBE DE PARIS   (France)

^c Georgia Institute of Technology   (United States)

Author keywords

data streaming; Data intensive computing; e Infrastructures; programming frameworks; Python; scientific workflows

Indexed keywords

EMBEDDED SYSTEMS; HIGH LEVEL LANGUAGES;

DATA STREAMING; DATA-INTENSIVE COMPUTING; E-INFRASTRUCTURES; PROGRAMMING FRAMEWORK; PYTHON; SCIENTIFIC WORKFLOWS;

DIGITAL STORAGE;

EID: 84946531115     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/DISCS.2014.12     Document Type: Conference Paper

References (18)

1. A. J. G. Hey, S. Tansley, and K. Tolle, The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research, 2009.
2. T. Segaran and J. Hammerbacher, Beautiful Data: The Stories Behind Elegant Data Solutions. O'Reilly, 2009.
3. A. Shoshani and D. Rotem, Scientific Data Management: Challenges, Technology and Deployment, ser. Computational Science Series. Chapman and Hall/CRC, 2010.
4. W. H. Dutton and P. W. Jeffreys, World Wide Research: Reshaping the Sciences and Humanities. MIT Press, 2010.
5. T. Oinn, M. Addis, J. Ferris, D. Marvin, M. Senger, M. Greenwood, T. Carver, K. Glover, M. R. Pocock, A. Wipat, and P. Li, "Taverna: A tool for the composition and enactment of bioinformatics workflows," Bioinformatics, vol. 20, no. 17, pp. 3045-3054, Nov. 2004. [Online]. Available: http://dx.doi.org/10.1093/bioinformatics/bth361
6. D. De Roure and C. Goble, "Software design for empowering scientists," IEEE Software, vol. 26, no. 1, pp. 88-95, 2009.
7. M. P. Atkinson, C. S. Liew, M. Galea, P. Martin, A. Krause, A. Mouat, Corcho, and D. Snelling, "Data-intensive architecture for scientific knowledge discovery." Distributed and Parallel Databases, vol. 30, no. 5-6, pp. 307-324, 2012. [Online]. Available: http://dblp.uni-trier.de/db/journals/dpd/dpd30.html#AtkinsonLGMKMCS12
8. Message Passing Interface Forum, "MPI: A message-passing interface standard," International Journal of Supercomputer Applications, vol. 8, pp. 165-414, 1994.
9. I. Altintas, C. Berkley, E. Jaeger, M. Jones, B. Ludascher, and S. Mock, "Kepler: an extensible system for design and execution of scientific workflows," in Proceedings of 16th International Conference on Scientific and Statistical Database Management, June 2004, pp. 423-424.
10. E. Deelman, G. Singh, M.-H. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G. B. Berriman, J. Good, A. C. Laity, J. C. Jacob, and D. S. Katz, "Pegasus: A framework for mapping complex scientific workflows onto distributed systems," Scientific Programming, vol. 13, no. 3, pp. 219-237, 2005.
11. D. Churches, G. Gombas, A. Harrison, J. Maassen, C. Robinson, M. Shields, I. Taylor, and I. Wang, "Programming scientific and distributed workflow with Triana services: Research Articles," Concurrency and Computation: Practice and Experience, vol. 18, no. 10, pp. 1021-1037, 2006.
12. M. Wilde, M. Hategan, J. M. Wozniak, B. Clifford, D. S. Katz, and I. Foster, "Swift: A language for distributed parallel scripting," Parallel Computing, vol. 37, no. 9, pp. 633 - 652, 2011. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0167819111000524
13. Y. Simmhan, R. Barga, C. van Ingen, E. Lazowska, and A. Szalay, "Building the Trident Scientific Workflow Workbench for Data Management in the Cloud," in International Conference on Advanced Engineering Computing and Applications in Sciences (ADVCOMP). IEEE, October 2009.
14. X. Llorá, B. Ács, L. S. Auvil, B. Capitanu, M. E. Welge, and D. E. Goldberg, "Meandre: Semantic-Driven Data-Intensive Flows in the Clouds," in IEEE Fourth International Conference on eScience. IEEE Press, 2008, pp. 238-245.
15. Z. Falt, D. Bednárek, M. Krulis, J. Yaghob, and F. Zavoral, "Bobolang: A language for parallel streaming applications," in Proceedings of the 23rd International Symposium on High-performance Parallel and Distributed Computing, ser. HPDC '14. New York, NY, USA: ACM, 2014, pp. 311-314. [Online]. Available: http://doi.acm.org/10.1145/2600212.2600711
16. P. Fraternali and S. Paraboschi, "Chimera: A language for designing rule applications." in Active Rules in Database Systems, 1999, pp. 309-322. [Online]. Available: http://dblp.uni-trier.de/db/books/collections/patton99.html#FraternaliP99
17. G. D. Bensen, M. H. Ritzwoller, M. P. Barmin, A. L. Levshin, F. Lin, M. P. Moschetti, N. M. Shapiro, and Y. Yang, "Processing seismic ambient noise data to obtain reliable broad-band surface wave dispersion measurements," Geophysical Journal International, vol. 169, no. 3, pp. 1239-1260, 2007.
18. "SEED Reference Manual v2.4, Appendix G: Data Only SEED Volumes (Mini-SEED)," http://www.iris.edu/manuals/SEED appG.htm, accessed August 2011.