Designing and mining multi-terabyte astronomy archives: The sloan digital sky survey

SIGMOD Record (ACM Special Interest Group on Management of Data)

Volumn 29, Issue 2, 2000, Pages 451-462

  Szalay, Alexander S ^a   Kunszt, Peter Z ^a   Thakar, Ani ^a   Gray, Jim ^b   Slutz, Don ^b   Brunner, Robert J ^c  

^a JOHNS HOPKINS UNIVERSITY   (United States)

^b MICROSOFT RESEARCH   (United States)

^c CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Archive; Astronomy; Data analysis; Data mining; Database; Internet; Scaleable

Indexed keywords

ASTRONOMY; DATA HANDLING; DATA REDUCTION; DATABASE SYSTEMS; INFORMATION MANAGEMENT; INTERNET; SURVEYS;

ARCHIVE; DATA MANAGEMENT CHALLENGES; GEOGRAPHICAL LOCATIONS; HASHING TECHNIQUES; NUMERICAL ATTRIBUTES; PAIR-WISE COMPARISON; SCALEABLE; SLOAN DIGITAL SKY SURVEY;

DATA MINING;

EID: 0003873676     PISSN: 01635808     EISSN: None     Source Type: Journal    
DOI: 10.1145/335191.335439     Document Type: Review

References (18)

1. Acharya, S. R. Alonso, M. J. Franklin, S. B. Zdonik, "Broadcast Disks: Data Management for Asymmetric Communications Environments." SIGMOD Conference 1995: 199-210.
2. Arpaci-Dusseau, R. Arpaci-Dusseau, A., Culler, D. E., Hellerstein, J. M., Patterson, D. A. , "The Architectural Costs of Streaming I/O: A Comparison of Workstations, Clusters, and SMPs", Proc. Fourth International Symposium On High-Performance Computer Architecture (HPCA), Feb 1998.
3. Arpaci-Dusseau, R. H., Anderson, E.,. Treuhaft, N., Culler, D.A., Hellerstein, J.M., Patterson, D.A., Yelick, K., "Cluster I/O with River: Making the Fast Case Common." IOPADS '99 pp 10-22, http://now.cs.berkeley.edu/River/
4. Barclay, T. Barnes, R., Gray, J., Sundaresan, P., "Loading Databases Using Dataflow Parallelism." SIGMOD Record 23(4): 72-83 (1994)
5. DeWitt, D.J. Gray, J., "Parallel Database Systems: The Future of High Performance Database Systems." CACM 35(6): 85-98 (1992)
6. Csabai, I., Szalay, A.S. and Brunner, R., "Multidimensional Index For Highly Clustered Data With Large Density Contrasts," in Statistical Challenges in Astronomy II, eds. E. Feigelson and A. Babu, (Wiley), 447 (1997).
7. Greene, G. et al, "The GSC-I and GCS-II Databases: An Object Oriented Approach", in New Horizons from Multi-Wavelength Sky Surveys, eds. B. McLean et al, Kluwer, p.474 (1997)
8. Gropp, W., Huss-Lederman, H., MPI the Complete Reference: The MPI-2 Extensions, Vol. 2, MIT Press, 1998, ISBN: 0262571234
9. Graefe, G., "Query Evaluation Techniques for Large Databases". ACM Computing Surveys 25(2): 73-170 (1993)
10. Hartman, A. of Dell Computer, private communication. Intel Corporation has generously provided the SDSS effort at Johns Hopkins with Dell Computers.
11. Samet, H., Applications of Spatial Data Structures: Computer Graphics, Image Processing, and GIS, Addison-Wesley, Reading, MA, 1990. ISBN 0-201-50300-0.
12. Samet, H., The Design and Analysis of Spatial Data Structures, Addison-Wesley, Reading, MA, 1990. ISBN 0-201-50255-0.
13. http://research.microsoft.com/barc/SortBenchmark/
14. Stonebraker, M., Frew, J., Gardels, K., Meredith, J., "The Sequoia 2000 Benchmark." Proc. ACM SIGMOD, pp. 2-11, 1993.
15. Szalay, A.S. and Brunner, R.J.: "Exploring Terabyte Archives in Astronomy", in New Horizons from Multi-Wavelength Sky Surveys, IAU Symposium 179, eds. B. McLean and D. Golombek, p.455. (1997).
16. Szalay, A.S., Kunszt, P. and Brunner, R.J.: "Hierarchical Sky Partitioning," Astronomical Journal, to be submitted, 2000.
17. Wells, D. C., Greisen, E. W., and Harten, R. H., "FITS: A Flexible Image Transport System," Astronomy and Astrophysics Supplement Series, 44, 363-370, 1981.
18. Zadonic, S., Maier, D., Readings in Object Oriented Database Systems, Morgan Kaufmann, San Francisco, CA, 1990. ISBN 1-55860-000-0