GrayWulf: Scalable software architecture for data intensive computing

Proceedings of the 42nd Annual Hawaii International Conference on System Sciences, HICSS

Volumn , Issue , 2009, Pages

  Simmhan, Yogesh ^a   Barga, Roger ^a   Van Ingen, Catharine ^a   Nieto Santisteban, Maria ^b   Dobos, Laszlo ^b   Li, Nolan ^b   Shipway, Michael ^b   Szalay, Alexander S ^b   Werner, Sue ^b   Heasley, Jim ^c  

^a MICROSOFT RESEARCH   (United States)

^b JOHNS HOPKINS UNIVERSITY   (United States)

^c UNIVERSITY OF HAWAII   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BEOWULF CLUSTERS; DATA SETS; DATA-INTENSIVE COMPUTING; DESIGN PRINCIPLES; INFRASTRUCTURE SOFTWARES; PAN-STARRS; PARALLEL APPLICATIONS; SOFTWARE SERVICES;

ARCHITECTURAL DESIGN;

SOFTWARE ARCHITECTURE;

EID: 63349088202     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/HICSS.2009.235     Document Type: Conference Paper

References (23)

1. P. Watson, "Databases in Grid Applications: Locality and Distribution", Database: Enterprise, Skills and Innovation. British National Conference on Databases, UK, July 5-7, 2005. LNCS Vol. 3567 pp. 1-16.
2. P. M. Caulk, "The Design of a Petabyte Archive and Distribution System for the NASA ECS Project", Fourth NASA Goddard Conference on Mass Storage Systems and Technologies, College Park, Maryland, March 1995.
3. J. Becla and D. Wang, "Lessons Learned from Managing a Petabyte", CIDR 2005 Conference, Asilomar, 2005.
4. Panoramic Survey Telescope and Rapid Response System, http://pan-starrs.ifa.hawaii.edu/
5. A. S. Szalay and J. Gray, "Science in an Exponential World", Nature, 440, pp 23-24, 2006.
6. J. Gray, D. Liu, M. Nieto-Santisteban, A. S. Szalay, D. DeWitt and G. Heber, "Scientific data management in the coming decade", ACM SIGMOD, 34(4), pp 34-41, 2005.
7. T. Sterling, J. Salmon, D. J. Becker and D. F. Savarese, "How to Build a Beowulf: A Guide to the Implementation and Application of PC Clusters", MIT Press, Cambridge, MA, 1999, also http://beowulf.org/.
8. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters", 6th Symposium on Operating System Design and Implementation, San Francisco, 2004.
9. J. Gray and A. S. Szalay, "Where the Rubber Meets the Sky: Bridging the Gap between Databases and Science", MSR-TR-2004-110, October 2004, IEEE Data Engineering Bulletin, December 2004, Vol. 27.4, pp. 3-11.
10. M. Nieto-Santisteban, T. School, A. Szalay and A. Kemper, "20 Spatial Queries for an Astronomer's Bench(mark)", Proceedings of Astronomical Data Analysis Software and Systems XVII, London, UK, 23rd - 26th September 2007.
11. M. A. Nieto-Santisteban, J. Gray, A. S. Szalay, J. Annis, A. R. Thakar and W. J. O'Mullane, "When Database Systems Meet the Grid", Proceedings of ACM CIDR 2005, Asilomar, CA, January 2005). 11.
12. D. Jewitt, "Project Pan-STARRS and the Outer Solar System", Earth, Moon and Planets 92: 465-476, 2003.
13. A. R. Thakar, A. S. Szalay, P. Z. Kunszt and J. Gray, "The Sloan Digital Sky Survey Science Archive: Mgrating a Multi-Terabyte Astronomical Archive from Object to Relational DBMS", Computing in Science and Engineering, V5.5, Sept 2003, IEEE Press. pp. 16-29, 2003.
14. J. Adelman-McCarthy et al., "The Sixth Data Release of the Sloan Digital Sky Survey", The Astrophysical Journal Supplement Series, Volume 175, Issue 2, pp. 297-313, 2007.
15. W. O'Mullane, N. Li, M. Nieto-Santisteban, A. S. Szalay and A. R. Thakar, "Batch is Back: CASJobs, Serving Multi-TB Data on the Web", International conference on Web Services 2005: 33-40.
16. V. Singh, J. Gray, A. R. Thakar, A. S. Szalay, J. Raddick, W. Boroski, S. Lebedeva and B. Yanny, "SkyServer Traffic Report - The First Five Years", Microsoft Technical Report, MSR-TR-2006-190, 2006.
17. M. A. Nieto-Santisteban, A. Thakar, A. Szalay and J. Gray, "Large-Scale Query and XMatch, Entering the Parallel Zone," Astronomical Data Analysis Software and Systems XV ASP Conference Series, Vol 351, Proceedings of the Conference Held 2-5 October 2005 in San Lorenzo de El Escorial, Spain. Edited by Carlos Gabriel, Christophe Arviset, Daniel Ponz, and Enrique Solano. San Francisco: Astronomical Society of the Pacific, 2006. p.493.
18. Barga, R.S., Jackson, J., Araujo, N, Guo, D., Gautam, N, Grochow, K. and Lazowska, E., "Trident: Scientific Workflow Workbench for Oceanography", International Workshop on Scientific Workflows (SWF 2008).
19. P. Andrews, "Presenting Windows Workflow Foundation", Sams Publishing, 2005.
20. Y. Simmhan, B. Plale and D. Gannon, "A Survey of Data Provenance for eScience". SIGMOD Record, Vol. 34, No. 3, 2005.
21. http://hadoop.apache.org/
22. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, "Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks", EuroSys, 2007.
23. S. Lohr, "Google and I.B.M. Join in 'Cloud Computing' Research", New York Times article, October 8, 2007.