High-performance remote access to climate simulation data: A challenge problem for data grid technologies

Parallel Computing

Volumn 29, Issue 10, 2003, Pages 1335-1356

  Chervenak, Ann ^a   Deelman, Ewa ^a   Kesselman, Carl ^a   Allcock, Bill ^b   Foster, Ian ^b   Nefedova, Veronika ^b   Lee, Jason ^c   Sim, Alex ^c   Shoshani, Arie ^c   Drach, Bob ^d   Williams, Dean ^d   Middleton, Don ^e  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b ARGONNE NATIONAL LABORATORY   (United States)

^c LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

^d LAWRENCE LIVERMORE NATIONAL LABORATORY   (United States)

^e NATIONAL CENTER FOR ATMOSPHERIC RESEARCH   (United States)

Author keywords

Earth System Grid; Globus Toolkit ; Grid computing; GridFTP

Indexed keywords

CLIMATOLOGY; DATA ACQUISITION; DATA TRANSFER; EARTH SCIENCES;

GRID COMPUTING;

PARALLEL PROCESSING SYSTEMS;

EID: 0141526079     PISSN: 01678191     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.parco.2003.06.001     Document Type: Conference Paper

References (31)

1. Climate data analysis tools (CDAT), http://esg.llnl.gov/cdat.
2. Earth System Grid (ESG), http://www.earthsystemgrid.org.
3. OpenGIS Web site, http://www.opengis.org.
4. Allcock W., Bester J., Bresnahan J., Chervenak A., Foster I., Kesselman C., Meder S., Nefedova V., Quesnel D., Tuecke S. Data management and transfer in high-performance computational grid environments. Parallel Computing. 28(5):2002;749-771.
5. Allcock W., Foster I., Nefedova V., Chervenak A., Deelman E., Kesselman C., Lee J., Sim A., Shoshani A., Drach B., Williams D. High-performance remote access to climate simulation data: a challenge problem for data grid technologies. SC'2001. 2001;ACM Press.
6. C. Baru, R. Moore, A. Rajasekar, M. Wan, The SDSC storage resource broker, in: 8th Annual IBM Centers for Advanced Studies Conference, Toronto, Canada, 1998.
7. Beynon M., Kurc T., Catalyurek U., Chang C., Sussman A., Saltz J. Distributed processing of very large datasets with DataCutter. Parallel Computing. 27(11):2001;1457-1478.
8. A. Chervenak, E. Deelman, I. Foster, L. Guy, W. Hoschek, A. Iamnitchi, C. Kesselman, P. Kunst, M. Ripenu, B. Schwartzkopf, H. Stockinger, K. Stockinger, B. Tierney, Giggle: a framework for constructing scalable replica location services, in: SC'02: High Performance Networking and Computing, 2002.
9. A. Chervenak, E. Deelman, C. Kesselman, L. Pearlman, G. Singh, A metadata catalog service for data intensive applications, Technical Report, Information Sciences Institute, 2003.
10. Chervenak A., Foster I., Kesselman C., Salisbury C., Tuecke S. The data grid: towards an architecture for the distributed management and analysis of large scientific data sets. Journal of Network and Computer Applications. (23):2001;187-200.
11. P. Cornillon, NVODS and OPenDAP (a.k.a. DODS), an overview, 2002, http://www.unidata.ucar.edu/packages/dods/papers/pcornillon/dods-teds/.
12. Czajkowski K., Fitzgerald S., Foster I., Kesselman C. Grid information services for distributed resource sharing. 10th IEEE International Symposium on High Performance Distributed Computing. San Francisco: 2001;181-184 IEEE Computer Society Press.
13. B. Eaton, J. Gregory, B. Drach, K. Taylor, S. Hankin, NetCDF climate and forecast (CF) metadata convention, 2003, http://www.cgd.ucar.edu/cms/eaton/cf-metadata/index.html.
14. I. Foster, C.A. Kesselman, Data grid reference architecture, Technical Report, GriPhyN-2001-12, 2001, www.griphyn.org.
15. Foster I., Kesselman C. Globus: a metacomputing infrastructure toolkit. International Journal of Supercomputer Applications. 11(2):1998;115-129.
16. I. Foster, C. Kesselman, G. Tsudik, S. Tuecke, A security architecture for computational grids, in: 5th ACM Conference on Computer and Communications Security, 1998, pp. 83-91.
17. D. Gunter, B. Tierney, B. Crowley, M. Holding, J. Lee, NetLogger: a toolkit for distributed system performance analysis, in: IEEE Mascots 2000: Eighth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, 2000.
18. Kurc T., Catalyurek U., Chang C., Sussman A., Saltz J. Exploration and visualization of very large datasets with the active data repository. IEEE Computer Graphics & Applications. 21(4):2001;24-33.
19. L. Pearlman, V. Welch, I. Foster, C. Kesselman, S. Tuecke, A community authorization service for group collaboration, in: IEEE 3rd International Workshop on Policies for Distributed Systems and Networks, 2002.
20. Shoshani A., Sim A., Gu J. Storage resource managers: essential components for the grid. Nabrzyski J., Schopf J., Weglarz J. Resource Management for Grid Computing. 2003.
21. A. Shoshani, A. Sim, J. Gu, Storage resource managers: middleware components for grid storage, in: Nineteenth IEEE Symposium on Mass Storage Systems, 2002.
22. Stockinger H., Samar A., Allcock B., Foster I., Holtman K., Tierney B. File and object replication in data grids. Journal of Cluster Computing. 5(3):2002;305-314.
23. Thain D., Livny M. Building Reliable Clients and Services. The Grid: Blueprint for a New Computing Infrastructure. second ed. 2004;Morgan Kaufmann.
24. Tierney B. TCP tuning guide for distributed applications on wide area networks. Usenix; login: Journal. 2001;33.
25. Unidata netCDF: an access interface for self-describing portable data, 2002 http://www.unidata.ucar.edu/packages/netcdf/.
26. Vazhkudai S., Tuecke S., Foster I. Replica selection in the Globus data grid. International Workshop on Data Models and Databases on Clusters and the Grid (DataGrid 2001). 2001;IEEE Press.
27. M. Wahl, T. Howes, S. Kille, Lightweight Directory Access Protocol (v3). Internet Engineering Task Force, RFC 2251, 1997.
28. Wolski R. Forecasting network performance to support dynamic scheduling using the Network Weather Service. 6th IEEE International Symposium on High Performance Distributed Computing. Portland, Oregon: 1997;IEEE Computer Society Press.
29. Wolski R., Spring N., Hayes J. The Network Weather Service: a distributed resource performance forecasting service for metacomputing. Future Generation Computer Systems. 15(5-6):1999;757-768.
30. M. Atkinson, Introduction to OGSA-DAI, 2003. www.nesc.ac.uk/talks/mpa/GGFGridSummerSchool21July03.pdf.
31. OGSA-DAI Web site, http://www.ogsadai.org.uk/index.php.