Skill assessment for coupled biological/physical models of marine systems

Journal of Marine Systems

Volumn 76, Issue 1-2, 2009, Pages 4-15

  Stow, Craig A ^a   Jolliff, Jason ^b   McGillicuddy Jr , Dennis J ^c   Doney, Scott C ^c   Allen, J Icarus ^d   Friedrichs, Marjorie A M ^e   Rose, Kenneth A ^f   Wallhead, Philip ^g  

^a NOAA EARTH SYSTEM RESEARCH LABORATORY   (United States)

^b NAVAL RESEARCH LABORATORY   (United States)

^c WOODS HOLE OCEANOGRAPHIC INSTITUTION   (United States)

^d PLYMOUTH MARINE LABORATORY   (United Kingdom)

^e VIRGINIA INSTITUTE OF MARINE SCIENCE   (United States)

^f LOUISIANA STATE UNIVERSITY   (United States)

^g NATIONAL OCEANOGRAPHY CENTRE   (United Kingdom)

Author keywords

Goodness of fit; Model uncertainty; Skill assessment; Skill metric

Indexed keywords

MODEL STRUCTURES; UNCERTAINTY ANALYSIS;

FORECASTING ABILITIES; GOODNESS-OF-FIT; HYPOTHESIS GENERATIONS; MARINE SYSTEMS; MODEL SKILLS; MODEL UNCERTAINTY; NUMERICAL EXPERIMENTATIONS; RIGOROUS MODELS; SKILL ASSESSMENT; SKILL LEVELS; SKILL METRIC;

APPLICATIONS;

BIOLOGICAL METHOD; ECOSYSTEM MODELING; MARINE ECOSYSTEM; NUMERICAL MODEL; PERFORMANCE ASSESSMENT; PHYSICAL METHOD;

EID: 59049097569     PISSN: 09247963     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jmarsys.2008.03.011     Document Type: Article

References (59)

1. Allen J.I., and Clarke K.R. Effects of demersal trawling on ecosystem functioning in the North Sea: a modelling study. Marine Ecology Progress Series 336 (2007) 63-75
2. Allen J.I., and Somerfield P.J. A multivariate approach to model skill assessment. J. Mar. Syst. 76 (2009) 83-94 (this volume)
3. Allen, J.I., Smyth, T.J., Siddorn, J.R., Holt, M. in press. How well can we forecast high biomass algal bloom events in a eutrophic coastal sea? Harmful algae.
4. Allen J.I., Holt J.T., Blackford J., and Proctor R. Error quantification of a high-resolution coupled hydrodynamic-ecosystem coastal-ocean model: part 2. Chlorophyll-a, nutrients and SPM. J. Mar. Syst. 68 3-4 (2007) 381-404
5. Allen J.I., Somerfield P.J., and Gilbert F.J. Quantifying uncertainty in high-resolution coupled hydrodynamic-ecosystem models. Journal of Marine Systems 64 (2007) 3-14
6. Allen J.I., Somerfield P.J., and Siddorn J. Primary and bacterial production in the Mediterranean Sea: a modelling study. Journal of Marine Systems 33-34 (2002) 473-495
7. Arhonditsis G.B., and Brett M.T. Evaluation of the current state of mechanistic aquatic biogeochemical modeling. Marine Ecology Progress Series 271 (2004) 13-26
8. Arhonditsis G.B., Adams-VanHarn B.A., Nielsen L., Stow C.A., and Reckhow K.H. Evaluation of the current state of mechanistic aquatic biogeochemical models: citation analysis and future perspectives. Environmental Science & Technology 40 (2006) 6547-6554
9. Beck M.B. Water-quality modeling - a review of the analysis of uncertainty. Water Resources Research 23 (1987) 1393-1442
10. Bevington P., and Robinson K.D. Data Reduction and Error Analysis for the Physical Sciences (2002), McGraw-Hill Science, New York 336 pp.
11. Blackford J.C., Allen J.I., and Gilbert F. Ecosystem dynamics at six contrasting sites: a generic modelling study. Journal of Marine Systems 52 (2004) 191-215
12. Brier G.W. Verification of forecasts expressed in terms of probability. Monthly Weather Review 78 (1950) 1-3
13. Brown C.D., and Davis H.T. Receiver operating characteristics curves and related decision measures: a tutorial. Chemometrics and Intelligent Laboratory Systems 80 (2006) 24-38
14. Chatfield C., and Collins A.J. Introduction to Multivariate Analysis (1980), Chapman and Hall, London
15. Clarke K.R. Nonparameteric multivariate analyses of changes in community structure. Australian Journal of Ecology 18 (1993) 117-143
16. Clarke K.R., and Gorley R.N. PRIMER v6: User Manual/Tutorial (2006), PRIMER-E Ltd, Plymouth 192 pp.
17. Doney S.C. A synoptic atmospheric surface forcing data set and physical upper ocean model for the U.S. JGOFS Bermuda Atlantic Time-Series Study (BATS) site. Journal of Geophysical Research, Oceans 101 (1996) 25,615-25,634
18. Doney S.C., Yeager S., Danabasoglu G., Large W.G., and McWilliams J.C. Mechanisms governing interannual variability of upper ocean temperature in a global hindcast simulation. J. Phys. Oceanogr. 37 (2007) 1918-1938
19. Duda R.O., Hart P.E., and Stork D.G. Pattern Classification (2001), John Wiley and Sons, New York 654 pp.
20. Emery, W.J., Thomson, R.E. 1998: Data analysis methods in physical oceanography, Pergamon Elsevier, Oxford UK, 634p.
21. Evans G.T. Defining misfit between biogeochemical models and data sets. Journal of Marine Systems 40-41 (2003) 49-54
22. Fewster R.M., and Buckland S.T. Similarity indices for spatial ecological data. Biometrics 57 (2001) 495-501
23. Foody G.M. Status of land cover classification accuracy assessment. Remote Sensing of Environment 80 (2002) 185-201
24. Friedrichs M.A.M., Carr M.E., Barber R.T., Scardi M., Antoine D., Armstrong R.A., Asanuma I., Behrenfeld M.J., Buitenhuis E.T., Chai F., Christian J.R., Ciotti A.M., Doney S.C., Dowell M., Dunne J., Gentili B., Gregg W., Hoepffner N., Ishizaka J., Kameda T., Lima I., Marra J., Mélin F., Moore J.K., Morel A., O'Malley R.T., O'Reilly J., Saba V.S., Schmeltz M., Smyth T.J., Tjiputra J., Waters K., Westberry T.K., and Winguth A. Assessing the uncertainties of model estimates of primary productivity in the Tropical Pacific Ocean. J. Mar. Syst. 76 (2009) 113-133 10.1016/j.jmarsys.2008.05.010 (this issue)
25. Friedrichs, M.A.M., Dusenberry, J.A., Anderson, L.A., Armstrong, R., Chai, F., Christian, J.R., Doney, S.C., Dunne, J., Fujii, M., Hood, R., McGillicuddy, D., Moore, J.K., Schartau, M., Spitz, Y.H., Wiggert, J.D. 2007. Assessment of skill and portability in regional marine biogeochemical models: the role of multiple planktonic groups. J. Geophys. Res. Oceans 112, C08001. doi:10.1029/2006JC003852
26. Friedrichs M.A.M., Hood R., and Wiggert J. Ecosystem model complexity versus physical forcing: quantification of their relative impact with assimilated Arabian Sea data. Deep-Sea Research II 53 (2006) 576-600
27. Gregg W.W., Friedrichs M.A.M., Robinson A.R., Rose K.A., Schlitzer R., Thompson K.R., and Doney S.C. Skill assessment in ocean biological data assimilation. J. Mar. Syst. 76 (2009) 16-33 10.1016/j.jmarsys.2008.05.006 (this issue)
28. Gruber N., Frenzel H., Doney S.C., Marchesiello P., McWilliams J.C., Moisan J.R., Oram J.J., Plattner G.K., and Stolzenbach K.D. Eddy-resolving simulation of plankton ecosystem dynamics in the California Current System. Deep-Sea Research I (2006) 1483-1516
29. Gustafson E.J. Quantifying landscape spatial pattern: what is the state of the art?. Ecosystems 1 (1998) 143-156
30. Hagen-Zanker A., Straatman B., and Uljee I. Further development of a fuzzy set map comparison approach. International Journal of Geographical Information Science 19 (2005) 769-785
31. Hashioka T., and Yamanaka Y. Seasonal and regional variations of phytoplankton groups by top-down and bottom-up controls obtained by a 3D ecosystem model. Ecological Modelling 202 (2007) 68-80
32. Hedgpeth J.W. Models and muddles. Helgoland Marine Research 30 (1977) 92-104
33. Hoeting J.A., Madigan D., Raftery A.E., and Volinsky C.T. Bayesian model averaging: a tutorial. Statistical Science 14 (1999) 382-401
34. Jolliff J.K., Kindle J.C., Shulman I., Penta B., Friedrichs M.A.M., Helber R., and Arnone R.A. Summary diagrams for coupled hydrodynamic-ecosystem model skill assessment. J. Mar. Syst. 76 (2009) 64-82 (this volume)
35. Kalnay E. Atmospheric Modeling, Data Assimilation and Predictability (2003), Cambridge University Press, Cambridge UK 341 pp.
36. In: Kasibhatla P., Heimann M., Rayner P., Mahowald N., Prinn R.G., and Hartley D.E. (Eds). Inverse Methods in Global Biogeochemical Cycles. Geophysical Monograph Series vol. 114 (2000), American Geophysical Union, Washington DC 324 pp.
37. Kass R.E., and Raftery A.E. Bayes factors. Journal of the American Statistical Association 90 (1995) 773-795
38. Katz R.W., and Ehrendorfer M. Bayesian approach to decision making using ensemble weather forecasts. Weather and Forecasting 21 (2006) 220-231
39. Kaufman S. At Home in the Universe (1995), Oxford University Press, NY
40. Krzanowski W.J. Principles of Multivariate Analysis: a User's Perspective (1990), Clarendon Press, Oxford 563 pp.
41. Leggett R.W., and Williams L.R. A reliability index for models. Ecological Modelling 13 (1981) 303-312
42. Lillesand T.M., Kiefer R.W., and Chipman J.W. Remote Sensing and Image Interpretation (2004), John Wiley and Sons, New York
43. Loague K., and Green R.E. Statistical and graphical methods for evaluating solute transport models: overview and application. Journal of Contaminant Hydrology 7 (1991) 51-73
44. Nash J.E., and Sutcliffe J.V. River flow forecasting through conceptual models, part 1 - a discussion of principles. Journal of Hydrology 10 (1970) 282-290
45. Phillips J.D. Spatial structures and scale in categorical maps. Geographical and Environmental Modelling 6 (2002) 41-57
46. Pilkey O.H., and Pilkey-Jarvis L. Useless Arithmetic: Why Environmental Scientists Can't Predict the Future (2007), Columbia University Press, NY
47. Powell T.M., Lewis C.V.W., Curchitser E.N., Haidvogel D.B., Hermann A.J., and Dobbins E.L. Results from a three-dimensional, nested biological-physical model of the California Current System and comparisons with statistics from satellite imagery. Journal of Geophysical Research 111 (2006) doi:10.1029/2004JC002506
48. Press W.H., Flannery B.P., Teukolsky S.A., and Vetterling W.T. Numerical Recipes (1986) 818 pp.
49. Raick C., Alvera-Azcarate A., Barth A., Brankart J.M., Soetaert K., and Gregoire M. Application of a SEEK filter to a 1D biogeochemical model of the Ligurgian Sea: twin experiments and real in-situ data assimilation. Journal of Marine Systems 65 (2007) 561-583
50. Reckhow K.H. Importance of scientific uncertainty in decision-making. Environmental Management 18 (1994) 161-166
51. Reckhow K.H., Clements J.T., and Dodd R.C. Statistical evaluation of mechanistic water-quality models. Journal of Environmental Engineering-ASCE 116 (1990) 250-268
52. Rose K.A., Roth B.M., and Smith E.P. Skill assessment of spatial maps for oceanographic modeling. J. Mar. Syst. 76 (2009) 34-48 10.1016/j.jmarsys.2008.05.013 (this issue)
53. Schrum C., St. John M., and Alekseeva I. ECOSMO, a coupled ecosystem model of the North Sea and Baltic Sea: part II. Spatial-seasonal characteristics in the North Sea as revealed by EOF analysis. Journal of Marine Systems 61 (2006) 100-113
54. Smith E.P., and Rose K.A. Model goodness-of-fit analysis using regression and related techniques. Ecological Modelling 77 (1995) 49-64
55. Stow C.A., Roessler C., Borsuk M.E., Bowen J.D., and Reckhow K.H. A comparison of estuarine water quality models for TMDL development in the Neuse River Estuary. Journal of Water Resources Planning and Management 129 (2003) 307-314
56. Taylor K.E. Summarizing multiple aspects of model performance in a single diagram. Journal of Geophysical Research 106 (2001) 7183-7192
57. Villars M., de Vries I., Bokhorst M., Ferreira J., Gellers-Barkman S., Kelly-Gerreyn B., Lancelot C., Menesguen A., Moll A., Patsch J., Radach G., Skogen M., Soiland H., Svendsen E., and Vested H.J. Report of the ASMO Modelling Workshop on Eutrophication Issues, 5-8 November 1996 (1998), The Hague, The Netherlands 90 pp.
58. Wiggert J.D., Murtugudde R.G., and Christian J.R. Annual ecosystem variability in the tropical Indian Ocean: results of a coupled bio-physical ocean general circulation model. Deep-Sea Research II 53 (2006) 644-676
59. Wunsch C. The Ocean Circulation Inverse Problem (1996), Cambridge University Press, Cambridge UK 442 pp.