FRIEDA: Flexible robust intelligent elastic data management in cloud environments

Proceedings - 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, SCC 2012

Volumn , Issue , 2012, Pages 1096-1105

  Ghoshal, Devarshi ^a   Ramakrishnan, Lavanya ^b  

^a INDIANA UNIVERSITY   (United States)

^b LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

Author keywords

Cloud; Data Management; Scheduling

Indexed keywords

CLOUD ENVIRONMENTS; COMPLEX APPLICATIONS; DATA MANAGEMENT STRATEGIES; DESIGN AND IMPLEMENTATIONS; DISTRIBUTED DATA MANAGEMENTS; MANAGEMENT FRAMEWORKS; MANAGEMENT SCHEME; SCIENTIFIC APPLICATIONS;

BIOINFORMATICS; CLOUDS; DIGITAL STORAGE; ELASTICITY; SCHEDULING; STORAGE MANAGEMENT;

INFORMATION MANAGEMENT;

EID: 84876575712     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SC.Companion.2012.132     Document Type: Conference Paper

References (30)

1. S. Canon, S. Cholia, J. Shalf, K. Jackson, L. Ramakrishnan, and V. Markowitz, "A performance comparison of massively parallel sequence matching computations on cloud computing platforms and hpc clusters using hadoop," in Using Clouds for Parallel Computations in Systems Biology Workshop, Held at SC09, 2009.
2. E. Deelman, G. Singh, M. Livny, B. Berriman, and J. Good, "The cost of doing science on the cloud: the montage example," in Proceedings of the 2008 ACM/IEEE conference on Supercomputing, ser. SC '08. Piscataway, NJ, USA: IEEE Press, 2008, pp. 50:1-50:12. [Online]. Available: http://dl.acm.org/ citation.cfm?id=1413370.1413421
3. K. Keahey, T. Freeman, J. Lauret, and D. Olson, "Virtual workspaces for scientific applications," J. Phys.: Conf. Ser., vol. 78, no. 1, pp. 012 038+, 2007. [Online]. Available: http://dx.doi.org/10.1088/1742-6596/78/1/ 012038
4. L. Ramakrishnan, P. T. Zbiegel, S. Campbell, R. Bradshaw, R. S. Canon, S. Coghlan, I. Sakrejda, N. Desai, T. Declerck, and A. Liu, "Magellan: Experiences from a science cloud," ser. ScienceCloud, 2011, pp. 49-58. [Online]. Available: http://doi.acm.org/10.1145/1996109.1996119
5. "Magellan final report," http://science.energy.gov/-/media/ ascr/pdf/program-documents/docs/Magellan Final Report.pdf.
6. I. Baldine, Y. Xin, A. Mandal, C. Renci, U. Chase, V. Marupadi, A. Yumerefendi, and D. Irwin, "Networked cloud orchestration: A geni perspective," in GLOBECOM Workshops (GC Wkshps), 2010 IEEE, dec. 2010, pp. 573-578.
7. J. Dean and S. Ghemawat, "Mapreduce: simplified data processing on large clusters," Commun. ACM, vol. 51, no. 1, pp. 107-113, Jan. 2008. [Online]. Available: http://doi.acm.org/10.1145/1327452.1327492
8. "Apache Hadoop. http://hadoop.apache.org."
9. Y. Gu and R. L. Grossman, "Lessons learned from a year's worth of benchmarks of large data clouds," in Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers, ser. MTAGS '09. New York, NY, USA: ACM, 2009, pp. 3:1-3:6. [Online]. Available: http://doi.acm.org/10.1145/ 1646468.1646471
10. E. Dede, M. Govindaraju, D. Gunter, and L. Ramakrishnan, "Riding the elephant: Managing ensembles with hadoop," in 4th Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS), 2011.
11. W. Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, and I. Foster, "The globus striped gridftp framework and server," in Proceedings of the 2005 ACM/IEEE conference on Supercomputing, ser. SC '05. Washington, DC, USA: IEEE Computer Society, 2005, pp. 54-. [Online]. Available: http://dx.doi.org/10.1109/SC.2005.72
12. (2012) The twisted website. [Online]. Available: http://twistedmatrix. com/trac/
13. D. Ghoshal, R. S. Canon, and L. Ramakrishnan, "I/o performance of virtualized cloud environments," in Proceedings of the second international workshop on Data intensive computing in the clouds, ser. DataCloud-SC '11. New York, NY, USA: ACM, 2011, pp. 71-80. [Online]. Available: http://doi.acm.org/10. 1145/2087522.2087535
14. K. R. Jackson, L. Ramakrishnan, K. J. Runge, and R. C. Thomas, "Seeking supernovae in the clouds: a performance study," in Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, ser. HPDC '10. New York, NY, USA: ACM, 2010, pp. 421-429. [Online]. Available: http://doi.acm.org/10.1145/1851476.1851538
15. "ORCA. https://geni-orca.renci.org/trac/."
16. J. Chase, L. Grit, D. Irwin, V. Marupadi, P. Shivam, and A. Yumerefendi, "Beyond virtual data centers: Toward an open resource control architecture," in International Conference on the Virtual Computing Initiative, ser. ICVCI 2007, 2007.
17. "ORCA Flukes. https://geni-orca.renci.org/trac/wiki/flukes."
18. BLAST. [Online]. Available: http://blast.ncbi.nlm.nih.gov/
19. D. Borthakur, The Hadoop Distributed File System: Architecture and Design. The Apache Software Foundation, 2007.
20. Open mpi. [Online]. Available: http://www.open-mpi.org/
21. J. Xie, S. Yin, X. Ruan, Z. Ding, Y. Tian, J. Majors, A. Manzanares, and X. Qin, "Improving mapreduce performance through data placement in heterogeneous hadoop clusters," in Parallel and Distributed Processing, Workshops and Phd Forum (IPDPSW), Atlanta, Georgia, April 2010, pp. 1-9.
22. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, "Dryad: distributed data-parallel programs from sequential building blocks," in Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, ser. EuroSys '07. New York, NY, USA: ACM, 2007, pp. 59-72. [Online]. Available: http://doi.acm.org/10.1145/1272996.1273005
23. A. Shoshani, A. Sim, and J. Gu, "Storage resource managers: Middleware components for grid storage," 2002.
24. A. Chervenak, R. Schuler, M. Ripeanu, M. Ali Amer, S. Bharathi, I. Foster, A. Iamnitchi, and C. Kesselman, "The globus replica location service: Design and experience," Parallel and Distributed Systems, IEEE Transactions on, vol. 20, no. 9, pp. 1260-1272, sept. 2009.
25. A. Rajasekar, R. Moore, C.-Y. Hou, C. A. Lee, R. Marciano, A. de Torcy, M. Wan, W. Schroeder, S.-Y. Chen, L. Gilbert, P. Tooby, and B. Zhu, "irods primer: Integrated rule-oriented data system," Synthesis Lectures on Information Concepts, Retrieval, and Services, vol. 2, no. 1, pp. 1-143, 2010. [Online]. Available: http://www. morganclaypool.com/doi/abs/10.2200/ S00233ED1V01Y200912ICR012
26. G. Singh, S. Bharathi, A. Chervenak, E. Deelman, C. Kesselman, M. Manohar, S. Patil, and L. Pearlman, "A metadata catalog service for data intensive applications," in Proceedings of the 2003 ACM/IEEE conference on Supercomputing, ser. SC '03. New York, NY, USA: ACM, 2003, pp. 33-. [Online]. Available: http://doi.acm.org/10.1145/1048935.1050184
27. A. Krishnan, "Gridblast: a globus-based high-throughput implementation of blast in a grid computing framework," Concurrency Computat.: Pract. Exper., vol. 43, no. 2, p. 16071623, Apr. 2005.
28. R. D. Bjornson, A. H. Sherman, S. B. Weston, N. Willard, and J. Wing, "Turboblast(r): A parallel implementation of blast built on the turbohub," in Proceedings of the 16th International Parallel and Distributed Processing Symposium, ser. IPDPS '02. Washington, DC, USA: IEEE Computer Society, 2002, pp. 325-. [Online]. Available: http://dl.acm.org/ citation.cfm?id=645610.661393
29. I. J. Taylor, E. Deelman, and D. B. Gannon, Workflows for e-Science: Scientific Workflows for Grids. Springer, Dec. 2006. [Online]. Available: http://www.amazon.com/exec/obidos/redirect?tag= citeulike07-20n&path=ASIN/ 1846285194
30. E. Deelman and A. Chervenak, "Data management challenges of dataintensive scientific workflows."