Database support for data-driven scientific applications in the grid

Parallel Processing Letters

Volumn 13, Issue 2, 2003, Pages 245-271

  Narayanan, Sivaramakrishnan ^a   Kurc, Tahsin ^a   Catalyurek, Umit ^a   Saltz, Joel ^a  

^a OHIO STATE UNIVERSITY   (United States)

Author keywords

Data intensive applications; Distributed datasets; Distributed query execution; Grid computing

Indexed keywords

COMPUTER ARCHITECTURE; COMPUTER SOFTWARE; DATA STORAGE EQUIPMENT; DATA TRANSFER;

DISTRIBUTED DATASETS; DISTRIBUTED QUERY EXECUTION; GRID COMPUTING;

DISTRIBUTED DATABASE SYSTEMS;

EID: 0141682128     PISSN: 01296264     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0129626403001264     Document Type: Article

References (88)

1. B. Allcock, J. Bester, J. Bresnahan, A. L. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnel, and S. Tuecke. Secure, efficient data transport and replica management for high-performance data-intensive computing. In Proceedings of IEEE Mass Storage Conference, April 2001.
2. G. Allen, T. Dramlitsch, I. Foster, N. Karonis, M. Ripeanu, E. Seidel, and B. Toonen. Supporting efficient execution in heterogeneous distributed computing environments with Cactus and Globus. In Proceedings of the 2001 ACM/IEEE SC01 Conference. ACM Press, Nov. 2001.
3. Asia Pacific BioGrid. http://www.apgrid.org.
4. N. Beckmann, H.-P. Kriegel, R. Schneider, and B. Seeger. The R*-tree: An efficient and robust access method for points and rectangles. In Proceedings of the 1990 ACM-SIGMOD Conference, pages 332-331, May 1990.
5. W. H. Bell, D. Bosio, W. Hoschek, P. Kunszt, G. McCance, and N. Silander. Project spitfite - towards grid web service databases. http://www.cs.man.ac.uk/grid-db/documents.html.
6. R. Bennett, K. Bryant, A. Sussman, R. Das, and J. Saltz. Jovian: A framework for optimizing parallel I/O. In Proceedings of the 1994 Scalable Parallel Libraries Conference, pages 10-20. IEEE Computer Society Press, Oct. 1994.
7. M. Beynon, T. Kurc, A. Sussman, and J. Saltz. Design of a framework for data-intensive wide-area applications. In Proceedings of the 9th Heterogeneous Computing Workshop (HCW2000), pages 116-130. IEEE Computer Society Press, May 2000.
8. M. D. Beynon, R. Ferreira, T. Kurc, A. Sussman, and J. Saltz. DataCutter: Middleware for filtering very large scientific datasets on archival storage systems. In Proceedings of the Eighth Goddard Conference on Mass Storage Systems and Technologies/17th IEEE Symposium on Mass Storage Systems, pages 119-133. National Aeronautics and Space Administration, Mar. 2000. NASA/CP 2000-209888.
9. M. D. Beynon, T. Kurc, U. Catalyurek, C. Chang, A. Sussman, and J. Saltz. Distributed processing of very large datasets with DataCutter. Parallel Computing, 2001. To appear.
10. M. D. Beynon, T. Kurc, U. Catalyurek, A. Sussman, and J. Saltz. Efficient manipulation of large datasets on heterogeneous storage systems. In Proceedings of the 11th Heterogeneous Computing Workshop (HCW2002). IEEE Computer Society Press, Apr. 2002.
11. M. D. Beynon, T. Kurc, A. Sussman, and J. Saltz. Optimizing execution of component-based applications using group instances. In IEEE International Symposium on Cluster Computing and the Grid (CCGrid2001), Brisbane, Australia, May 2001.
12. Biomedical Informatics Research Network (BIRN). http://www.nbirn.net.
13. P. Brezany, A. Choudhary, and M. Dang. Parallelization of irregular codes including out-of-core data and index arrays. In Proceedings of PARCO'97, 1997.
14. T. Brinkhoff, H.-P. Kriegel, and B. Seeger. Parallel processing of spatial joins using R-trees. In Proceedings of the 1996 International Conference on Data Engineering, pages 258-265, Feb. 1996.
15. P. H. Carns, W. B. Ligon, R. B. Ross, and R. Thakur. PVFS: A parallel file system for linux clusters. In Proceedings of the 4th Annual Linux Showcase and Conference, pages 317-327, Atlanta, GA, Oct 2000.
16. H. Casanova and J. Dongarra. Applying Netsolve's network-enabled server. IEEE Computational Science & Engineering, 5(3):57-67, July-September 1998.
17. Common Component Architecture Forum. http://www.cca-forum.org.
18. E. Cerami. Web Services Essentials. O'Reilly & Associates Inc., 2002.
19. C. Chang, R. Ferreira, A. Sussman, and J. Saltz. Infrastructure for building parallel database systems for multi-dimensional data. In Proceedings of the Second Merged IPPS/SPDP Symposiums. IEEE Computer Society Press, Apr. 1999.
20. P. Corbett, D. Feitelson, S. Fineberg, Y. Hsu, B. Nitzberg, J.-P. Prost, M. Snir, B. Traversat, and P. Wong. Overview of the MPI-IO parallel I/O interface. In Proceedings of the IPPS'95 Third Annual Workshop on Input/Output in Parallel and Distributed Systems (IOPADS), pages 1-15, Apr. 1995.
21. P. F. Corbett and D. G. Feitelson. The Vesta parallel file system. ACM Transactions on Computer Systems, 14(3):225-264, Aug. 1996.
22. K. Czajkowski, S. Fitzgerald, I. Foster, and C. Kesselman. Grid information services for distributed resource sharing, 2001.
23. Data Access and Integration Services. http://www.cs.man.ac.uk/grid-bd/documents.html.
24. R. Das, Y.-S. Hwang, M. Uysal, J. Saltz, and A. Sussman. Applying the CHAOS/PARTI library to irregular problems in computational chemistry and computational aerodynamics. In Proceedings of the 1993 Scalable Parallel Libraries Conference, pages 45-56. IEEE Computer Society Press, Oct. 1993.
25. R. Das, M. Uysal, J. Saltz, and Y.-S. Hwang. Communication optimization for irregular scientific computations on distributed memory architectures. Journal of Parallel and Distributed Computing, 22(3):462-479, Sept. 1994.
26. The DataCurrent project. http://www.cs.umd.edu/projects/hpsl/ResearchAreas/DataCutter.htm
27. D. DeWitt and J. Gray. Parallel database systems: the future of high performance database systems. Communications of the ACM, 35(6):85-98, June 1992.
28. Distributed Oceanographic Data System. http://www.unidata.ucar.edu/packages/dods/.
29. G. Edjlali, A. Sussman, and J. Saltz. Interoperability of data parallel runtime libraries with Meta-Chaos. Technical Report CS-TR-3633 and UMIACS-TR-96-30, University of Maryland, Department of Computer Science and UMIACS, May 1996.
30. G. Edjlali, A. Sussman, and J. Saltz. Interoperability of data parallel runtime libraries. In Proceedings of the 11th International Parallel Processing Symposium, Geneva, Switzerland, Apr. 1997. IEEE Computer Society Press.
31. Earth Systems Grid (ESG). http://www.earthsystemgrid.org.
32. EUROGRID. http://www.eurogrid.org/.
33. C. Faloutsos and P. Bhagwat. Declustering using fractals. In the 2nd International Conference on Parallel and Distributed Information Systems, pages 18-25, San Diego, CA, Jan. 1993.
34. C. Faloutsos and S. Roseman. Fractals for secondary key retrieval. In the 8th ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, Philadelphia, PA, Mar. 1989.
35. R. Ferreira, T. Kurc, M. Beynon, C. Chang, A. Sussman, and J. Saltz. Object-relational queries into multi-dimensional databases with the active data repository. Parallel Processing Letters, 9(2):173-195, 1999.
36. S. Fink, S. Kohn, and S. Baden. Efficient run-time support for irregular block-structured applications. Journal of Parallel and Distributed Computing, 50(1):61-82, Apr. 1998.
37. I. Foster, C. Kesselman, J. Nick, and S. Tuecke. Grid services for distributed system integration. IEEE Computer, 36(6):37-46, June 2002. Open Grid Services Architecture (OGSA).
38. I. Foster, C. Kesselman, J. M. Nick, and S. Tuecke. The physiology of the Grid: An open grid services architecture for distributed systems integration. http://www.globus.org/research/papers/ogsa.pdf, 2002.
39. J. Frey, T. Tannenbaum I. Foster, M. Livny, and S. Tuecke. Condor-G: A computation management agent for multi-institutional grids. In Proceedings of the Tenth IEEE Symposium on High Performance Distributed Computing (HPDC10). IEEE Press, Aug 2001.
40. Global Grid Forum. http://www.gridforum.org.
41. The Globus Project. http://www.globus.org.
42. S. Graham, S. Simeonov, T. Boubez, D. Davis, G. Daniels, Y. Nakamura, and R. Neyama. Building Web Services with Java: Making Sense of XML, SOAP, WSDL, and UDDI. SAMS Publishing, 2002.
43. A. S. Grimshaw, W. A. Wulf, and the Legion Team. The Legion vision of a worldwide virtual computer. Communications of the ACM, 40(1):39-45, Jan. 1997.
44. Grid Physics Network (GriPhyN). http://www.griphyn.org.
45. E. Gutiérrez, O. Plata, and E. Zapata. A compiler method for the parallel execution of irregular reductions in scalable shared memory multiprocessors. In Proceedings of the 14th ACM International Conference on Supercomputing, pages 78-87, Santa Fe, New Mexico, May 2000.
46. M. Hall, S. Amarasinghe, B. Murphy, S. Liao, and M. Lam. Detecting coarse-grain parallelism using an interprocedural parallelizing compiler. In Proceedings of Supercomputing '95, San Diego, CA, Dec. 1995.
47. H. Han and C.-W. Tseng. Improving compiler and run-time support for adaptive irregular codes. In Proceedings of the International Conference on Parallel Architecture and Compilation Techniques, Paris, France, Oct. 1998.
48. High Performance Fortran Forum. High Performance Fortran language specification. Scientific Programming, 2(1-2):1-170, 1993.
49. S. Hiranandani, K. Kennedy, and C.-W. Tseng. Compiling Fortran D for MIMD distributed-memory machines. Communications of the ACM, 35(8):66-80, Aug. 1992.
50. E. G. Hoel and H. Samet. Data-parallel spatial join algorithms. In J. Chandra, editor, Proceedings of the 23rd International Conference on Parallel Processing. Volume 3: Algorithms and Applications, pages 227-234, Boca Raton, FL, USA, Aug. 1994. CRC Press.
51. J. Huber, C. L. Elford, D. A. Reed, A. A. Chien, and D. S. Blumethal. PPFS: A high performance portable parallel file system. In Proceedings of the 9th ACM International Conference on Supercomputing, pages 385-394, Barcelona, Spain, July 1995.
52. D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994 Symposium on Operating Systems Design and Implementation, pages 61-74. ACM Press, Nov. 1994.
53. T. Kurc, M. Beynon, A. Sussman, and J. Saltz. Datacutter and a client interface for the Storage Resource Broker with DataCutter services. Technical Report CS-TR-4133 and UMIACS-TR-2000-26, University of Maryland, Department of Computer Science and UMIACS, May 2000.
54. D.-R. Liu and S. Shekhar. A similarity graph-based approach to declustering problems and its applications towards parallelizing grid files. In Proceedings of the International Conference on Data Engineering, pages 373-381, Taipei, Taiwan, Mar. 1995. IEEE Computer Society Press.
55. Logistical Computing and Internetworking Lab. http://loci.cs.utk.edu/.
56. MEDIGRID. http://creatis-www.insa-lyon.fr/MEDIGRID/home.html.
57. B. Moon, H. Jagadish, C. Faloutsos, and J. Saltz. Analysis of the clustering properties of the Hilbert space-filling curve. IEEE Transactions on Knowledge and Data Engineering, 13(1):124-141, January/February 2001.
58. B. Moon and J. H. Saltz. Scalability analysis of declustering methods for multidimensional range queries. IEEE Transactions on Knowledge and Data Engineering, 10(2):310-327, March/April 1998.
59. S. Mukherjee, S. Sharma, M. Hill, J. Larus, A. Rogers, and J. Saltz. Efficient support for irregular applications on distributed-memory machines. In Proceedings of the Fifth ACM SIGPLAN Symposium on Principles & Practice of Parallel Programming (PPOPP), pages 68-79. ACM Press, July 1995. ACM SIGPLAN Notices, Vol. 30, No. 8.
60. M. C. Murphay and D. Rotem. Multiprocessor join scheduling. IEEE Transactions on Knowledge and Data Engineering, 5(2):322-338, Apr. 1993.
61. N. Nieuwejaar and D. Kotz. The Galley parallel file system. In Proceedings of the 1996 International Conference on Supercomputing, pages 374-381. ACM Press, May 1996.
62. R. Oldfield and D. Kotz. Armada: A parallel file system for computational. In Proceedings of CCGrid2001: IEEE International Symposium on Cluster Computing and the Grid, Brisbane, Australia, May 2001. IEEE Computer Society Press.
63. R. Ponnusamy, J. Saltz, A. Choudhary, Y.-S. Hwang, and G. Fox. Runtime support and compilation methods for user-specified irregular data distributions. IEEE Transactions on Parallel and Distributed Systems, 6(8):815-831, Aug. 1995.
64. E. Port, M. Knopp, and G. Brix. Dynamic contrast-enhanced mri using gd-dtpa: Interindividual variability of the arterial input function and consequences for the assessment of kinetics in tumors. Magn. Reson. Med., 45(6):1030-1038, 2001.
65. J.-P. Prost, R. Treumann, R. Hedges, B. Jia, and A. Koniges. MPI-IO/GPFS, an optimized implementation of MPI-IO on top of GPFS. In Proceedings of the 2001 ACM/IEEE SC01 Conference. ACM Press, Nov. 2001.
66. V. Raman, I. Narang, C. Crone, L. Haas, S. Malaika, T. Mukai, D. Wolfson, and C. Baru. Data access and management services on grid. http://wwwcs.man.ac.uk/grid-db/documents.html.
67. M. Ranganathan, A. Acharya, G. Edjlali, A. Sussman, and J. Saltz. Flexible and efficient coupling of data parallel programs. In Proceedings of Parallel Object-Oriented Methods and Applications (POOMA96), Feb. 1996.
68. J. M. d. Rosario and A. N. Choudhary. High-performance I/O for massively parallel computers: Problems and prospects. IEEE Computer, 27(3):59-68, Mar. 1994.
69. J. Saltz, G. Agrawal, C. Chang, R. Das, G. Edjlali, P. Havlak, Y.-S. Hwang, B. Moon, R. Ponnusamy, S. Sharma, A. Sussman, and M. Uysal. Advances in Computers, volume 45, chapter Programming Irregular Applications: Runtime Support, Compilation and Tools, pages 105-153. Academic Press, 1997.
70. J. Saltz, R. Ponnusamy, S. D. Sharma, B. Moon, Y.-S. Hwang, M. Uysal, and R. Das. A manual for the CHAOS runtime library. Technical Report CS-TR-3437 and UMIACS-TR-95-34, University of Maryland, Department of Computer Science and UMIACS, Mar. 1995.
71. M. Sato, H. Nakada, S. Sekiguchi, S. Matsuoka, U. Nagashima, and H. Takagi. Ninf: a network based information library for a global world-wide computing infrastructure. In Proceedings of HPCN'97 (LNCS-1225), pages 491-502, 1997.
72. D. Schneider and D. DeWitt. Tradeoffs in processing complex join queries via hashing in multiprocessor database machines. In Proceedings of the 16th VLDB Conference, pages 469-480, Melbourne, Australia, Aug. 1990.
73. K. E. Seamons, Y. Chen, P. Jones, J. Jozwiak, and M. Winslett. Server-directed collective I/O in Panda. In Proceedings of Supercomputing '95, San Diego, CA, Dec. 1995. IEEE Computer Society Press.
74. X. Shen and A. Choudhary. DPFS: A distributed parallel file system. In IEEE 30th International Conference on Parallel Processing (ICPP), Sept. 2001.
75. X. Shen and A. Choudhary, A distributed multi-storage i/o system for high performance data intensive computing. In International Symposium on Cluster Computing and the Grid (CCGrid 2002), May 2002.
76. J. Smith, A. Gounaris, P. Watson, N. W. Paton, A. A. Fernandes, and R. Sakellariou. Distributed query processing on the grid. http://www.cs.man.ac.uk/griddb/documents.html.
77. M. Snir, S. W. Otto, S. Huss-Lederman, D. W. Walker, and J. Dongarra. MPI: The Complete Reference. Scientific and Engineering Computation Series. MIT Press, 1996.
78. Shared Pathology Informatics Network (SPIN). http://www.sharedpath.org.
79. SRB: The Storage Resource Broker. http://www.npaci.edu/DICE/SRB/index.html.
80. M. Stonebraker and P. Brown. Object-Relational DBMSs, Tracking the Next Great Wave. Morgan Kaufman Publishers, Inc., 1998.
81. TeraGrid. http://www.teragrid.org.
82. D. Thain, J. Basney, S. Son, and M. Livny. Kangaroo approach to data movement on the grid. In Proceedings of the Tenth IEEE Symposium on High Performance Distributed Computing (HPDC10), 2001.
83. D. Thain, J. Bent, A. Arpaci-Dusseau, R. Arpaci-Dusseau, and M. Livny. Gathering at the well: Creating communities for grid i/o. In Proceedings of Supercomputing 2001, Denver, CO, November 2001.
84. R. Thakur, A. Choudhary, R. Bordawekar, S. More, and S. Kuditipudi. Passion: Optimized I/O for parallel applications. IEEE Computer, 29(6):70-78, June 1996.
85. M. Uysal, T. M. Kurc, A. Sussman, and J. Saltz. A performance prediction framework for data intensive applications on large scale parallel machines. In Proceedings of the Fourth Workshop on Languages, Compilers and Run-time Systems for Scalable Computers, pages 243-258. Springer-Verlag, May 1998.
86. S. Vazhkudai, S. Tuecke, and I. Foster. Replica selection in the globus data grid, 2001.
87. R. Wolski, N. Spring, and J. Hayes. The network weather service: A distributed resource performance forecasting service for metacomputing. Journal of Future Generation Computing Systems, 15(5-6):757-768, 1999.
88. H. Yu and L. Rauchwerger. Adaptive reduction parallelization techniques. In Proceedings of the 14th ACM International Conference on Supercomputing, pages 66-77, Santa Fe, New Mexico, May 2000.