Efficient data IO for a Parallel Global Cloud Resolving Model

Environmental Modelling and Software

Volumn 26, Issue 12, 2011, Pages 1725-1735

  Palmer, Bruce ^a   Koontz, Annette ^a   Schuchardt, Karen ^a   Heikes, Ross ^b   Randall, David ^b  

^a PACIFIC NORTHWEST NATIONAL LABORATORY   (United States)

^b Department of Mechanical Engineering   (United States)

Author keywords

Data formatting; Geodesic grid; Global Cloud Resolving Model; Grid Specifications; High performance IO; Parallel IO libraries

Indexed keywords

AGGREGATION STRATEGY; APPLICATION PROGRAMMER INTERFACES; COMMUNITY STANDARDS; DATA FIELDS; DATA FORMATTING; EFFICIENT ANALYSIS; GEODESIC GRID; GLOBAL CLOUD RESOLVING MODEL; HIGH BANDWIDTH; HIGH PERFORMANCE IO; INPUT/OUTPUT; NEW PHYSICS; OUTPUT DATA; OUTPUT FREQUENCY;

CODES (SYMBOLS); PARALLEL PROCESSING SYSTEMS;

APPLICATION PROGRAMMING INTERFACES (API);

ALGORITHM; ATMOSPHERIC MODELING; CLOUD MICROPHYSICS; COMPUTER SYSTEM; DATA INTERPRETATION; DATA PROCESSING; GEODESY; INPUT-OUTPUT ANALYSIS; PARALLEL COMPUTING; PERFORMANCE ASSESSMENT;

EID: 84855499356     PISSN: 13648152     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.envsoft.2011.08.007     Document Type: Article

References (28)

1. Adcroft A., Campin J.M., Hill C., Marshall J. Implementation of an atmosphere-ocean general circulation model on the expanded spherical cube. Monthly Weather Review December 2004, 132(12):2845-2863.
2. Antypas K., Calder A.C., Dubey A., Fisher R., Ganapathy M.K., Gallagher J.B., Reid L.B., Riley K., Sheeler D., Taylor N. Scientific applications on the massively parallel BG/L machine. PDPTA 2006 2006, 292-298. June 26-29, Las Vegas, Nevada.
3. Balaji, V., Adcroft, A., Lian, Z., 2007. Gridspec: A Standard for the Description of Grids Used in Earth Systems Models. Workshop on Community Standards for Unstructured Grids, Oct 16,17, 2006.
4. Borrill J., Oliker L., Shalf J., Shan H. Investigation of leading HPC I/O performance using a scientific-application derived benchmark. Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC'07), November 10-16, 2003 Reno, Nevada 2003, IEEE Computer Society, Washington, DC.
5. Ching A., Choudhary A., Liao W., Ross R., Gropp W. Efficient structured data access in parallel file systems. Proceedings of the IEEE International Conference on Cluster Computing (CLUSTER '03). December 1-4, 2003, Hong Kong, China 2003, IEEE Computer Society, Washington, DC.
6. Dabdub D., Seinfeld J.H. Parallel computation in atmospheric chemical modeling. Parallel Computing 1996, 22(1):111-130.
7. Drake J., Jones P., Carr G. Overview of the software design of the community climate system model. International Journal of High Performance Computing Applications 2005, 19(3):177-186.
8. Dennis, J., Edwards, J., Loy, R., Jacob, R., Mirin, A., Craig, A., Vertenstein, M. An application level parallel I/O library for earth system models. International Journal of High Performance Computing Applications, in press.
9. Hammond G., Lichtner P. Field-scale model for the natural attenuation of uranium at the Hanford 300 area using high performance computing. Water Resources Research 2010, 46:W09527.
10. Jablonowski C., Williamson D.L. A baroclinic instability test case for atmospheric model dynamical cores. Quarterly Journal of the Royal Meteorological Society 2006, 132:2943-2975.
11. Lang S., Carns P., Latham L., Ross R., Harms K., Allcock W. I/O performance challenges at leadership scale. Proceedings SC 09 2009, November 14-20, 2009, Portland, OR.
12. Li J., Liao W., Choudhary A., Ross R., Thakur R., Gropp W., Latham R., Siegel A., Gallagher B., Zingale M. Parallel netCDF: a high-performance scientific I/O interface. Proceedings of the 2003 ACM/IEEE Conference on Supercomputing (SC'03), November 15-21, 2003, Phoenix, Arizona 2003, IEEE Computer Society, Washington, DC.
13. Liou K.N. Influence of cirrus clouds on weather and climate processes: a global perspective. Monthly Weather Review 1986, 114(6):1167-1199.
14. Lofstead J., Klasky S., Schwan K., Podhorszki N., Jin C. Flexible IO and integration for scientific codes through the adaptable IO System (ADIOS). Proceedings CLADE 2008 at HPDC, June 2008 2008, ACM, Boston, MA.
15. Lofstead J., Zheng F., Klasky K., Schwan K. Adaptable, metadata rich IO methods for portable high performance IO. Proceedings IPDPS'09 2009, May 2009, Rome, Italy.
16. Lofstead J., Zheng F., Liu Q., Klasky S., Oldfield R., Kordenbrock T., Schwan K., Wolf M. Managing variability in the IO performance of petascale storage systems. Proceedings of SC 10 2010, November 2010, New Orleans, LA.
17. Mache J., Lo V., Livingstone M., Garg S. The impact of spatial layout of jobs on parallel I/O performance. Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems (IOPADS '99), Atlanta, GA 1999, 45-56. ACM, New York, NY.
18. Maltrud M., McClean J. An eddy resolving global 1/10° ocean simulation. Ocean Modeling 2005, 8:31-54.
19. Morton G.M. AComputer Oriented Geodetic Data Base and a New Technique in File Sequencing 1966, IBM Ltd., Ottawa, Ontario.
20. Neal J., Fewtrell T., Bates P., Wright N. Acomparison of three parallelization methods for 2D flood inundation models. Environmental Modeling and Software 2010, 25:398-411.
21. Randall D.A., Ringler T.D., Heikes R.P., Jones P., Baumgardner J. Climate modeling with spherical geodesic grids. Computing in Science and Engineering 2002, 4(5):32-41.
22. Randall D., Khairoutdinov M., Arakawa A., Grabowski W. Breaking the cloud parameterization deadlock. Bulletin of the American Meteorological Society 2003, 84:1547-1564.
23. Saini S., Talcott D., Thakur R., Adamadis P., Rabenseifner R., Ciotti R. Parallel I/O performance characterization of Columbia and NEC SX-8 superclusters. International Parallel and Distributed Processing Symposium (IPDPS) 2006, March 26-30, Long Beach, CA.
24. Thakur R., Gropp W., Lusk E. Data sieving and collective I/O in ROMIO. Proceedings of the 7th Symposium on Frontiers of Massively Parallel Computation, February 1999 1999, 82-189. IEEE Computer Society, Washington, DC.
25. Von Bloh W., Rost S., Gerten D., Lucht W. Efficient parallelization of a dynamic global vegetation model with river routing. Environmental Modeling and Software 2010, 25:685-690.
26. Yang M., Koziol Q. Using collective IO inside a high performance IO software package - HDF5. Proceedings of Teragrid 2006: Advancing Scientific Discovery 2006, June 12-15, 2006, Indianapolis, IN.
27. Yu D. Parallelization of a two-dimensional flood inundation model based on domain decomposition. Environmental Modeling and Software 2010, 25:935-945.
28. Yu H., Sahoo K., Howson C., AlmásiCastaños J.G., Gupta M., Moreira J.E., Parker J.J., Engelsiepen T.E., Ross R.B., Thakur R., Latham R., Gropp W.D. High performance I/O for the BlueGene/L supercomputer. Proceedings of the 12th International Symposium on High-Performance Computer Architecture February 2006.