Towards improved and more routine Earth system model evaluation in CMIP

Earth System Dynamics

Volumn 7, Issue 4, 2016, Pages 813-830

  Eyring, Veronika ^a   Gleckler, Peter J ^b   Heinze, Christoph ^c,d   Stouffer, Ronald J ^e   Taylor, Karl E ^b   Balaji, V ^e,f   Guilyardi, Eric ^g,h   Joussaume, Sylvie ⁱ   Kindermann, Stephan ^j   Lawrence, Bryan N ^h,k   Meehl, Gerald A ^l   Righi, Mattia ^a   Williams, Dean N ^b  

^a GERMAN AEROSPACE CENTER DLR   (Germany)

^b LAWRENCE LIVERMORE NATIONAL LABORATORY   (United States)

^c UNIVERSITY OF BERGEN   (Norway)

^d Uni Research Climate   (Norway)

^e NOAA EARTH SYSTEM RESEARCH LABORATORY   (United States)

^f PRINCETON UNIVERSITY   (United States)

^g UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

^h UNIVERSITY OF READING   (United Kingdom)

ⁱ CEA SACLAY   (France)

^j Deutsches Klimarechenzentrum   (Germany)

^k RUTHERFORD APPLETON LABORATORY   (United Kingdom)

^l NATIONAL CENTER FOR ATMOSPHERIC RESEARCH   (United States)

more..

Author keywords

[No Author keywords available]

Indexed keywords

CLIMATE CHANGE; EARTH (PLANET); UNCERTAINTY ANALYSIS;

COUPLED MODEL INTERCOMPARISON PROJECT; MODEL INTER COMPARISONS; OPERATIONAL EVALUATION; PERFORMANCE ASSESSMENT; PROCESSING CAPABILITY; RESEARCH COMMUNITIES; SYSTEMATIC EVALUATION; UNCERTAINTY ESTIMATES;

CLIMATE MODELS;

CLIMATE MODELING; CLIMATE PREDICTION; CMIP; GENERAL CIRCULATION MODEL;

EID: 84994631081     PISSN: 21904979     EISSN: 21904987     Source Type: Journal    
DOI: 10.5194/esd-7-813-2016     Document Type: Article

References (64)

1. Aghedo, A. M., Bowman, K. W., Shindell, D. T., and Faluvegi, G. The impact of orbital sampling. monthly averaging and vertical resolution on climate chemistry model evaluation with satellite observations, Atmos. Chem. Phys., 11, 6493-6514, doi: 10.5194/acp-11-6493-2011, 2011
2. Anav, A., Friedlingstein, P., Kidston, M., Bopp, L., Ciais, P., Cox, P., Jones, C., Jung, M., Myneni, R., and Zhu, Z. Evaluating the Land and Ocean Components of the Global Carbon Cycle in the CMIP5 Earth System Models. J. Climate, 26, 6801-6843, 2013.
3. Bellenger, H., Guilyardi, E., Leloup, J., Lengaigne, M., and Vialard, J. ENSO representation in climate models: from CMIP3 to CMIP5. Clim. Dynam., 42, 1999-2018, 2014
4. Bodas-Salcedo, A., Webb, M. J., Bony, S., Chepfer, H., Dufresne, J. L., Klein, S. A., Zhang, Y., Marchand, R., Haynes, J. M., Pincus, R.and John, V. O. COSP Satellite simulation software for model assessment. B. Am. Meteor. Soc., 92, 1023-1043, 2011.
5. Bony, S., Colman, R., Kattsov, V. M., Allan, R. P., Bretherton, C. S., Dufresne, J.-L., Hall, A., Hallegatte, S. Holland, M. M., Ingram, W., Randall, D. A., Soden, B. J., Tselioudis, G., and Webb, M. J. How Well Do We Understand and Evaluate Climate Change Feedback Processes?. J. Climate, 19, 3445-3482, 2006
6. Bony, S.Stevens, B., Frierson, D. M.W., Jakob, C., Kageyama, M., Pincus, R., Shepherd, T. G., Sherwood, S. C., Siebesma, A. P., Sobel, A. H., Watanabe, M., andWebb, M. J. Clouds. circulation and climate sensitivity, Nat. Geosci., 8, 261-268, 2015
7. Braconnot, P., Harrison, S. P., Kageyama, M., Bartlein, P. J., Masson-Delmotte, V., Abe-Ouchi, A., Otto-Bliesner, B., and Zhao, Y. Evaluation of climate models using palaeoclimatic data. Nat. Clim. Change, 2, 417-424, 2012
8. Caldwell, P. M., Bretherton, C. S., Zelinka, M. D., Klein, S. A., Santer, B. D., and Sanderson, B. M. Statistical significance of climate sensitivity predictors obtained by data mining. Geophys. Res. Lett., 41, 1803-1808, 2014
9. Ciais, P., Reichstein, M., Viovy, N., Granier, A., Ogee, J., Allard, V.Aubinet, M., Buchmann, N., Bernhofer, C., Carrara, A., Chevallier, F., De Noblet, N., Friend, A. D., Friedlingstein, P., Grunwald, T., Heinesch, B., Keronen, P., Knohl, A., Krinner, G., Loustau, D.Manca, G., Matteucci, G., Miglietta, F., Ourcival, J. M., Papale, D., Pilegaard, K., Rambal, S., Seufert, G., Soussana, J. F., Sanz, M. J., Schulze, E. D., Vesala, T., and Valentini, R. Europewide reduction in primary productivity caused by the heat and drought in 2003. Nature, 437, 529-533, 2005
10. Collins, M., Achuta Rao, K., Ashok, K., Bhandari, S., Mitra, A. K., Prakash, S., Srivastava, R., and Turner, A. CORRESPONDENCE: Observational challenges in evaluating climate models. Nat. Clim. Change, 3, 940-941, 2013a
11. Collins, M., Knutti, R., Arblaster, J., Dufresne, J.-L., Fichefet, T., Friedlingstein, P. Gao, X., Gutowski, W. J., Johns, T., Krinner, G., Shongwe, M., Tebaldi, C., Weaver, A. J., and Wehner, M.: Long-Term Climate Change: Projections, Commitments and Irreversibility Climate Change 2013: The Physical Science Basis, Contribution ofWorking Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, edited by: Stocker, T. F., Qin, D., Plattner, G.-K., Tignor, M., Allen, S. K., Boschung, J., Nauels, A., Xia, Y., Bex, V., and Midgley, P. M., Cambridge University Press, Cambridge, UK and New York, NY, USA, 1029-1136, 2013b
12. Covey, C., Gleckler, P. J., Doutriaux, C., Williams, D. N., Dai, A., Fasullo, J., Trenberth, K., and Berg, A.: Metrics for the Diurnal Cycle of Precipitation: Toward Routine Benchmarks for Climate Models, J. Climate, 29, 4461-4471, 2016
13. Cox, P. M., Pearson, D., Booth, B. B., Friedlingstein, P., Huntingford, C., Jones, C. D., and Luke, C. M. Sensitivity of tropical carbon to climate change constrained by carbon dioxide variability. Nature, 494, 341-344, 2013
14. Eyring, V., Harris, N. R. P., Rex, M., Shepherd, T. G., Fahey, D. W., Amanatidis, G. T., Austin, J., Chipperfield, M. P., Dameris, M., Forster, P. M. D., Gettelman, A., Graf, H. F., Nagashima, T., Newman, P. A., Pawson, S., Prather, M. J., Pyle, J. A., Salawitch, R. J., Santer, B. D., and Waugh, D. W. A Strategy for Process-Oriented Validation of Coupled Chemistry-Climate Models. B. Am. Meteor. Soc., 86, 1117-1133, 2005
15. Eyring, V., Bony, S., Meehl, G. A., Senior, C. A., Stevens, B., Stouffer, R. J., and Taylor, K. E. Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci. Model Dev., 9, 1937-1958, doi: 10.5194/gmd-9-1937-2016, 2016a.
16. Eyring, V., Righi, M., Lauer, A., Evaldsson, M.Wenzel, S., Jones, C., Anav, A., Andrews, O., Cionni, I., Davin, E. L., Deser, C., Ehbrecht, C., Friedlingstein, P., Gleckler, P., Gottschaldt, K.-D., Hagemann, S., Juckes, M., Kindermann, S., Krasting, J., Kunert, D., Levine, R., Loew, A., Mäkelä, J., Martin, G., Mason, E., Phillips, A. S., Read, S., Rio, C., Roehrig, R., Senftleben, D., Sterl, A., Van Ulft, L. H., Walton, J., Wang, S., and Williams, K. D.: ESMValTool (v1.0) - A community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP, Geosci. Model Dev., 9, 1747-1802, doi: 10.5194/gmd-9-1747-2016, 2016b
17. Fasullo, J. T., and Trenberth, K. E.: A Less Cloudy Future: The Role of Subtropical Subsidence in Climate Sensitivity, Science, 338, 792-794, 2012
18. Fasullo, J. T., Sanderson, B. M., and Trenberth, K. E. Recent Progress in Constraining Climate Sensitivity With Model Ensembles. Current Climate Change Reports, 1, 268-275, 2015
19. Ferraro, R., Waliser, D. E., Gleckler, P., Taylor, K. E., and Eyring, V. Evolving obs4MIPs to Support the Sixth Coupled Model Intercomparison Project (CMIP6). B. Am. Meteor. Soc., ES131-ES133, doi: 10.1175/BAMS-D-14-00216.1, 2015
20. Flato, G., Marotzke, J., Abiodun, B., Braconnot, P., Chou, S. C., Collins, W., Cox, P., Driouech, F., Emori, S., Eyring, V., Forest, C., Gleckler, P., Guilyardi, E. Jakob, C., Kattsov, V., Reason, C., and Rummukainen, M. Evaluation of Climate Models. Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. edited by: Stocker, T. F., Qin, D., Plattner, G.-K., Tignor, M., Allen, S. K., Boschung, J., Nauels, A., Xia, Y., Bex, V., and Midgley, P. M., Cambridge University Press, Cambridge, UK and New York, NY, USA, 741-866, 2013
21. GCOS: Implementation Plan for the Global Observing System for Climate in Support of the UNFCCC, August 2010, 1-29, 2010
22. Gettelman, A., Eyring, V., Fischer, C., Shiona, H., Cionni, I., Neish, M., Morgenstern, O., Wood, S. W., and Li, Z. A community diagnostic tool for chemistry climate model validation. Geosci. Model Dev., 5, 1061-1073, doi: 10.5194/gmd-5-1061-2012, 2012
23. Gleckler, P. J., Taylor, K. E., and Doutriaux, C. Performance metrics for climate models. J. Geophys. Res., 113, D06104, 2008
24. Gleckler, P. J., Doutriaux, C., Durack P. J., Taylor K. E., Zhang, Y., Williams, D. N., Mason, E., and Servonnat, J. A more powerful reality test for climate models. Eos Trans. AGU, 97, 2016
25. Guilyardi, E., Wittenberg, A., Fedorov, A., Collins, M., Wang, C. Z., Capotondi, A., Van Oldenborgh, G. J., and Stockdale, T. Understanding El Nino in Ocean-Atmosphere General Circulation Models Progress and Challenges. B. Am. Meteor. Soc., 90, 325, 2009
26. Guilyardi, E., Balaji, V., Lawrence, B., Callaghan, S., Deluca, C., Denvil, S., Lautenschlager, M., Morgan, M., Murphy, S., and Taylor, K. E. Documenting Climate Models and Their Simulations. B. Am. Meteor. Soc., 94, 2013
27. Hall, A., and Qu, X. Using the current seasonal cycle to constrain snow albedo feedback in future climate change. Geophys. Res. Lett., 33, doi: 10.1029/2005gl025127, 2006
28. Hourdin, F., Mauritsen, T., Gettelman, A., Golaz, J.-C., Balaji, V., Duan, Q., Folini, D., Ji, D., Klocke, D., Qian, Y., Rauser, F., Rio, C., Tomassini, L., Watanabe, M.and Williamson, D. The art and science of climate model tuning. B. Am. Meteor. Soc., doi: 10.1175/bams-d-15-00135.1, 2016
29. IPCC: Climate Change 2013: The Physical Science Basis. Contribution ofWorking Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, Cambridge University Press, Cambridge, UK and New York, NY, USA, 1535, 2013
30. Ivanova, D. P., Gleckler, P. J., Taylor, K. E., Durack, P. J., and Marvel, K. D. Moving beyond the Total Sea Ice Extent in Gauging Model Biases. J. Climate, doi: 10.1175/jcli-d-16-0026.1, 2016
31. Jöckel, P., Kerkweg, A., Pozzer, A., Sander, R., Tost, H., Riede, H., Baumgaertner, A., Gromov, S., and Kern, B. Development cycle 2 of the Modular Earth Submodel System (MESSy2). Geosci. Model Dev., 3, 717-752, doi: 10.5194/gmd-3-717-2010, 2010
32. Karpechko, A. Y., Maraun, D., and Eyring, V. Improving Antarctic Total Ozone Projections by a Process-Oriented Multiple Diagnostic Ensemble Regression. J. Atmos. Sci., 70, 3959-3976, 2013
33. Klein, S. A., and Hall, A. Emergent Constraints for Cloud Feedbacks. Current Climate Change Reports, 1, 276-287, 2015
34. Klocke, D., Pincus, R., and Quaas, J. On Constraining Estimates of Climate Sensitivity with Present-Day Observations through Model Weighting. J. Climate, 24, 6092-6099, 2011
35. Knutti, R., Abramowitz, G., Collins, Eyring, V., Gleckler, P. J., Hewit B., and Mearns., L. Good Practice Guidance Paper on Assessing and Combining Multi Model Climate Projections. IPCC Working Group I Technical Support Unit, University of Bern, Bern, Switzerland, 0165-0009, 2010
36. Laney, D. The Importance of Big Data: A Definition, 2012
37. Lawrence. B. N., Balaji, V., Bentley, P., Callaghan, S., DeLuca, C., Denvil, S., Devine, G., Elkington, M., Ford, R. W.Guilyardi, E., Lautenschlager, M., Morgan, M., Moine, M.-P., Murphy, S., Pascoe, C., Ramthun, H., Slavin, P., Steenman-Clark, L., Toussaint, F., Treshansky, A., and Valcke, S.: Describing Earth system simulations with the Metafor CIM, Geosci. Model Dev., 5, 1493-1500, doi: 10.5194/gmd-5-1493-2012, 2012
38. Luo, Y. Q., Randerson, J. T., Abramowitz, G., Bacour, C., Blyth, E., Carvalhais, N., CiaisP., Dalmonech, D., Fisher, J. B., Fisher, R., Friedlingstein, P., Hibbard, K., Hoffman, F., Huntzinger, D., Jones, C. D., Koven, C., Lawrence, D., Li, D. J., Mahecha, M., Niu, S. L., Norby, R., Piao, S. L., Qi, X., Peylin, P., Prentice, I. C., Riley, W., Reichstein, M., Schwalm, C., Wang, Y. P., Xia, J. Y., Zaehle, S., and Zhou, X. H. A framework for benchmarking land models. Biogeosciences, 9, 3857-3874, doi: 10.5194/bg-9-3857-2012, 2012
39. Mahlstein, I., and Knutti, R. September Arctic sea ice predicted to disappear near 2 degrees C global warming above present. J. Geophys. Res.-Atmos., 117, doi: 10.1029/2011jd016709, 2012
40. Massonnet, F., Fichefet, T., Goosse, H., Bitz, C. M., Philippon-Berthier, G., Holland, M. M., and Barriat, P. Y. Constraining projections of summer Arctic sea ice. The Cryosphere, 6, 1383-1394, doi: 10.5194/tc-6-1383-2012, 2012
41. Mauritsen, T., Stevens, B., Roeckner, E., Crueger, T., Esch, M., Giorgetta, M., Haak, H., Jungclaus, J., Klocke, D., Matei, D., Mikolajewicz, U., Notz, D., Pincus, R., Schmidt, H., and Tomassini, L. Tuning the climate of a global model. J. Adv. Modl. Earth Syst., 4, doi: 10.1029/2012ms000154, 2012
42. Otto-Bliesner, B., Schneider, R., Brady, E., Kucera, M., Abe-Ouchi, A., Bard, E., Braconnot, P., Crucifix, M., Hewitt, C., Kageyama, M., Marti, O., Paul, A., Rosell-Melé, A., Waelbroeck, C., Weber, S., Weinelt, M., and Yu, Y. A comparison of PMIP2 model simulations and the MARGO proxy reconstruction for tropical sea surface temperatures at last glacial maximum. Clim. Dynam., 32, 799-815, 2009
43. Phillips, A. S., Deser, C., and Fasullo, J. Evaluating Modes of Variability in Climate Models. Eos Trans. AGU, 95, 453-455, 2014
44. Prabhat, Rubel, O., Byna, S., Wu, K. S., Li, F. Y., Wehner, M., and Bethel, W. TECA: A Parallel Toolkit for Extreme Climate Analysis. Procedia Comput. Sci., 9, 866-876, 2012
45. Righi, M., Eyring, V., Gottschaldt, K.-D., Klinger, C., Frank, F., Jöckel, P., and Cionni, I. Quantitative evaluation of ozone and selected climate parameters in a set of EMAC simulations. Geosci. Model Dev., 8, 733-768, doi: 10.5194/gmd-8-733-2015, 2015
46. Sanderson, B. M., Knutti, R., and Caldwell, P. Addressing Interdependency in a Multimodel Ensemble by Interpolation of Model Properties. J. Climate, 28, 5150-5170, 2015
47. Santer, B. D., Thorne, P. W., Haimberger L., Taylor, K. E., Wigley, T. M. L., Lanzante, J. R., Solomon, S., Free, M., Gleckler, P. J., Jones, P. D., Karl, T. R., Klein, S. A., Mears, C., Nychka, D., Schmidt, G. A., Sherwood, S. C.and Wentz, F. J. Consistency of modelled and observed temperature trends in the tropical troposphere. Int. J. Climatol., 28, 1703-1722, 2008
48. Schaeffer, R., Szklo, A. S., De Lucena, A. F. P., Borba, B. S. M. C., Nogueira, L. P. P., Fleming, F. P., Troccoli, A., Harrison, M., and Boulahya, M. S. Energy sector vulnerability to climate change: A review. Energy, 38, 1-12, 2012
49. Schutgens, N. A. J., Gryspeerdt, E., Weigum, N., Tsyro, S., Goto, D., Schulz, M., and Stier, P. Will a perfect model agree with perfect observations?. The impact of spatial sampling. Atmos. Chem. Phys., 16, 6335-6353, doi: 10.5194/acp-16-6335-2016, 2016
50. Sherwood, S. C., Bony, S., and Dufresne, J. L. Spread in model climate sensitivity traced to atmospheric convective mixing. Nature, 505, 37-42, 2014
51. SPARC-CCMVal: SPARC Report on the Evaluation of Chemistry-Climate Models edited by: Eyring V., Shepherd, T. G., and Waugh, D. W., SPARC Report No. 5, WCRP-132, WMO/TDNo 1526 2010.
52. Sperber, K., Annamalai, H., Kang, I. S., Kitoh, A., Moise, A., Turner, A., Wang, B., and Zhou, T. The Asian summer monsoon: an intercomparison of CMIP5 vs. CMIP3 simulations of the late 20th century. Clim. Dynam., 41, 2711-2744, 2013
53. Stouffer, R. J., Eyring, V., Meehl, G. A., Bony, S., Senior, C., Stevens, B., and Taylor, K. E. CMIP5 Scientific Gaps and Recommendations for CMIP6. BAMS, accepted, doi: 10.1175/bamsd-15-00013.1, 2016
54. Sun, Y., Solomon, S., Dai, A., and Portmann, R.W. How often will it rain?. J. Climate, 20, 4801-4818, 2007
55. Taylor, K. E., Stouffer, R. J., and Meehl, G. A. An Overview of Cmip5 and the Experiment Design. B. Am. Meteor. Soc., 93, 485-498, 2012
56. Tebaldi, C., and Knutti, R. The use of the multi-model ensemble probabilistic climate projections. Philos. T. R. Soc. A, 365, 2053-2075, 2007
57. Teixeira, J., Waliser, D., Ferraro, R., Gleckler, P., Lee, T., and Potter, G. Satellite Observations for CMIP5: The Genesis of Obs4MIPs. B. Am. Meteor. Soc., 95, 1329-1334, 2014
58. Waugh, D. W., and Eyring, V. Quantitative performance metrics for stratospheric-resolving chemistry-climate models. Atmos. Chem. Phys., 8, 5699-5713, doi: 10.5194/acp-8-5699-2008, 2008
59. Wenzel, S., Cox, P. M., Eyring, V., and Friedlingstein, P. Emergent constraints on climate-carbon cycle feedbacks in the CMIP5 Earth system models. J. Geophys. Res.-Biogeosci., 119, 2014
60. Wenzel, S., Eyring V., Gerber, E. P., and Karpechko, A. Y.: Constraining Future Summer Austral Jet Stream Positions in the CMIP5 Ensemble by Process-Oriented Multiple Diagnostic Regression, J. Climate 673-687, doi: 10.1175/JCLI-D-15-0412.1, 2016b
61. Wenzel, S., Eyring, V., Gerber, E. P., and Karpechko, A. Y.: Constraining Future Summer Austral Jet Stream Positions in the CMIP5 Ensemble by Process-Oriented Multiple Diagnostic Regression, J. Climate, 673-687, doi: 10.1175/JCLI-D-15-0412.1, 2016b
62. Williams, D. N.: Visualization and Analysis Tools for Ultrascale Climate Data, Eos, Trans. Am. Geophys. Union, 95, 377-378, 2014
63. Williams, D. N., Balaji, V., Cinquini, L., Denvil, S., Duffy, D., Evans, B., Ferraro, R., Hansen, R., Lautenschlager, M., and Trenham, C. A Global Repository for Planet-Sized Experiments and Observations. B. Am. Meteor. Soc., 803-816, doi: 10.1175/bamsd-15-00132.1, 2015
64. Williams, K. andWebb, M. A quantitative performance assessment of cloud regimes in climate models. Clim. Dynam., 33, 141-157, 2009