Large science databases - Are cloud services ready for them?

Scientific Programming

Volumn 19, Issue 2-3, 2011, Pages 147-159

  Thakar, Ani ^a   Szalay, Alex ^a   Church, Ken ^b   Terzis, Andreas ^b  

^a JOHNS HOPKINS UNIVERSITY   (United States)

^b Johns Hopkins University   (United States)

Author keywords

cloud computing; Databases

Indexed keywords

COMPUTER SYSTEMS; DISTRIBUTED DATABASE SYSTEMS; WEB SERVICES; WINDOWS OPERATING SYSTEM;

ASTRONOMICAL DATABASE; CLOUD SERVICES; DATA SETS; EFFICIENT ANALYSIS; LARGE DATABASE; MICROSOFT; PERFORMANCE COMPARISON; SLOAN DIGITAL SKY SURVEY; SQL SERVERS;

CLOUD COMPUTING;

EID: 79960778456     PISSN: 10589244     EISSN: None     Source Type: Journal    
DOI: 10.3233/SPR-2011-0325     Document Type: Article

References (18)

1. M. Armbrust, A. Fox, R. Griffith, A. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica and M. Zaharia. Above the clouds: A Berkeley view of cloud computing, Technical Report No. UCB/EECS-2009-28, University of California at Berkley, USA, February 10, 2009.
2. K. Church and J.A. Hamilton, Sneakernet: Clouds with mobility, 2009, available at: Http://idies.jhu.edu/meetings/idies09/ KChurch.pdf.
3. J. Dean and S. Ghemawat, MapReduce: Simplified data processing on large clusters, in: 6th Symposium on Operating System Design and Implementation, San Francisco, CA, 2004.
4. J. Gray, A.S. Szalay, A. Thakar, P. Kunszt, C. Stoughton, D. Slutz and J. van den Berg, Data mining the SDSS SkyServer database, in: Distributed Data and Structures 4: Records of the 4th International Meeting, W. Litwin and G. Levy, eds, Paris, France, March 2002, Carleton Scientific, 2003, pp. 189-210, ISBN 1-894145-13-5, also MSR-TR-2002-01, 2002, available at: Http://research. microsoft.com/pubs/64558/tr-2002-01.pdf.
5. N. Li and A.S. Szalay, Simple Provenance in scientific databases, in: Proc. Microsoft e-Science Workshop, Pittsburgh, T. Hey and S. Tansley, eds, Microsoft Research, Redmond, WA, USA, 2009, pp. 48-50.
6. E.H. Neilsen Jr., The Sloan Digital Sky Survey data archive server, Comput. Sci. Eng. 10(1) (2008), 13-17.
7. W. O'Mullane, N. Li, M.A. Nieto-Santisteban, A. Thakar, A.S. Szalay and J. Gray, Batch is back: CasJobs, serving multi- TB data on the Web, Microsoft Technical Report MSR-TR- 2005-19, Microsoft Research, 2005.
8. Open Cloud Consortium, http://opencloudconsortium.org/.
9. P. Schmid and A. Ross, Sequential transfer tests of 3.5 hard drives, 2009, available at: Http://www.tomshardware.com/ reviews/wd-4k-sector,2554-6. html.
10. Y. Simmhan, R. Barga, C. van Ingen, M. Nieto-Santisteban, L. Dobos, N. Li, M. Shipway, A.S. Szalay, S. Werner and J. Heasley, GrayWulf: Scalable software architecture for data intensive computing, in: Proceedings of the HICSS-42 Conference, Mini-Track on Data Intensive Computing, I. Gorton, ed., IEEE Computer Society, Washington, DC, USA, 2009, pp. 1-10.
11. Spider: Http://www.nccs.gov/computing-resources/jaguar/filesystems/.
12. A. Szalay, G. Bell, H. Huang, A. Terzis and A. White, Lowpower amdahl-balanced blades for data intensive computing, in: Proceedings of the 2nd Workshop on Power Aware Computing and Systems (HotPower'09), Big Sky, MT, USA, October 10, 2009.
13. A.S. Szalay, G. Bell, J. van den Berg, A. Wonders, R. Burns, D. Fay, J. Heasley, T. Hey, M. Nieto-SantiSteban, A. Thakar, C. Van Ingen and R. Wilton, GrayWulf: Scalable clustered architecture for data intensive computing, in: Proceedings of the HICSS-42 Conference, Mini-Track on Data Intensive Computing, I. Gorton, ed., IEEE Computer Society, Washington, DC, USA, 2009, pp. 1-10.
14. A. Szalay, J. Gray, G. Fekete, P. Kunszt, P. Kukol and A. Thakar, Indexing the sphere with the hierarchical triangular mesh, Microsoft Technical Report, 2005, available at: Http:// research.microsoft.com/apps/pubs/default. aspx?id=64531.
15. A.S. Szalay, A.R. Thakar and J. Gray, The SQLLoader data loading pipeline, Comput. Sci. Eng. 10(1) (2008), 38-48.
16. A.S. Szalay et al., Entry in supercomputing-09 data challenge, in: SuperComputing 2009 (SC09), Portland, OR, USA, November 14-20, 2009.
17. A.R. Thakar, The sloan digital sky survey: Drinking from the fire hose, Comput. Sci. Eng. 10(1) (2008), 9-12.
18. A.R. Thakar, A.S. Szalay, G. Fekete and J. Gray, The catalog archive server database management system, Comput. Sci. Eng. 10(1) (2008), 30-37.