When “data” are not data: The pitfalls of post hoc analyses that use stock assessment model output

Canadian Journal of Fisheries and Aquatic Sciences

Volumn 72, Issue 4, 2015, Pages 634-641

  Brooks, Elizabeth N ^a   Deroba, Jonathan J ^a  

^a NORTHEAST FISHERIES SCIENCE CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION ANALYSIS; MANAGEMENT PRACTICE; PITFALL TRAP; SENSITIVITY ANALYSIS; STOCK ASSESSMENT; UNCERTAINTY ANALYSIS;

EID: 84925662982     PISSN: 0706652X     EISSN: 12057533     Source Type: Journal    
DOI: 10.1139/cjfas-2014-0231     Document Type: Article

References (70)

1. Anderson, D.R. 2008. Model based inference in the life sciences. Springer, New York.
2. Anthony, V.C., and Fogarty, M.J. 1985. Environmental effects on recruitment, growth, and vulnerability of Atlantic herring in the Gulf of Maine region. Can. J. Fish. Aquat. Sci. 42(S1): 158–173. doi:10.1139/f85-271.
3. Arrizabalaga, H., López-Rodas, V., Costas, E., and González-Garcés, A. 2007. Use of genetic data to assess the uncertainty in stock assessments due to the assumed stock structure: the case of albacore (Thunnus alalunga) from the Atlantic Ocean. Fish. Bull. 105: 140–146.
4. Baker, R., Buckland, A., and Sheaves, M. 2014. Fish gut content analysis: robust measures of diet composition. Fish Fish. 15: 170–177. doi:10.1111/faf.12026.
5. Bakun, A., Babcock, E.A., and Santora, C. 2009. Regulating a complex adaptive system via its wasp-waist: grappling with ecosystem-based management of the New England herring fishery. ICES J. Mar. Sci. 66: 1768–1775. doi:10.1093/ icesjms/fsp073.
6. Brander, K., Neuheimer, A., Andersen, K.H., and Hartvig, M. 2013. Overconfidence in model projections. ICES J. Mar. Sci. 70: 1065–1068. doi:10.1093/icesjms/ fst055.
7. Brewer, G.D. 1984. The wider dimensions of management uncertainty in world fisheries. In Exploitation of Marine Communities: Dahlem Workshop Report. Vol. 32. Edited by R.M. May. Springer-Verlag, New York. pp. 275–286.
8. Brooks, E.N. 2013. Effects of variable reproductive potential on reference points for fisheries Management. Fish. Res. 138: 152–158. doi:10.1016/j.fishres.2012.06.003.
9. Brooks, E.N., and Apostolaki, P. 2006. Implications of Swordfish (Xiphias gladius) stock structure and mixing on management benchmarks. Col. Vol. Sci. Pap. ICCAT, 61: 61–78.
10. Churchill, J.H., Runge, J., and Chen, C. 2011. Processes controlling retention of spring-spawned Atlantic cod (Gadus morhua) in the western Gulf of Maine and their relationship to an index of recruitment success. Fish. Oceanogr. 20(1): 32–46. doi:10.1111/j.1365-2419.2010.00563.x.
11. Collie, J., Minto, C., Worm, B., and Bell, R. 2013. Predation on prerecruits can delay rebuilding of depleted cod stocks. Bull. Mar. Sci. 89(1): 107–122. doi:10.5343/bms.2011.1134.
12. Curti, K.L., Collie, J.S., Legault, C.M., and Link, J.S. 2013. Evaluating the performance of a multispecies statistical catch-at-age model. Can. J. Fish. Aquat. Sci. 70(3): 470–484. doi:10.1139/cjfas-2012-0229.
13. de Valpine, P., and Hastings, A. 2002. Fitting population models incorporating process noise and observation error. Ecol. Monogr. 72(1): 57–76. doi:10.1890/ 0012- 9615(2002)072[0057:FPMIPN]2.0.CO;2.
14. Deroba, J.J. 2014. Evaluating the consequences of adjusting fish stock assessment estimates of biomass for retrospective patterns using Mohn’s Rho. N. Am. J. Fish. Manage. 34: 380–390. doi:10.1080/02755947.2014.882452.
15. Deroba, J.J., and Schueller, A.M. 2013. Performance of stock assessments with misspecified age- and time-varying natural mortality. Fish. Res. 146: 27–40. doi:10.1016/j.fishres.2013.03.015.
16. Deroba, J.J., Butterworth, D.S., Methot, R.D., Jr., De Oliveira, J.A.A., Fernandez, C., Nielsen, A., Cadrin, S.X., et al. 2014. Simulation testing the robustness of stock assessment models to error: some results from the ICES Strategic Initiative on stock assessment methods. ICES J. Mar. Sci. doi:10.1093/icesjms/ fst237.
17. Dickey-Collas, M., Hintzen, N.T., Nash, R.D.M., Schön, P.-J., and Payne, M.R. 2014. Quirky patterns in the time-series of estimates of recruitment could be artefacts. ICES J. Mar. Sci. doi:10.1093/icesjms/fsu022.
18. Dorn, M.W. 2002. Advice on West Coast rockfish harvest rates from Bayesian meta-analysis of stock–recruit relationships. N. Am. J. Fish. Manage. 22: 280– 300. doi:10.1577/1548-8675(2002)022<0280:AOWCRH>2.0.CO;2.
19. Efron, B. 1979. Bootstrap methods: another look at the jackknife. Ann. Stat. 7(1): 1–26. doi:10.1214/aos/1176344552.
20. Francis, R.I.C.C. 2006. Measuring the strength of environment–recruitment relationships: the importance of including predictor screening within crossvalidations. ICES J. Mar. Sci. 63(4): 594–599. doi:10.1016/j.icesjms.2006.01.001.
21. Gelman, A., Carlin, J.B., Stern, H.S., and Rubin, D.B. 2004. Bayesian data analysis. 2nd ed. Chapman and Hall, New York.
22. Gilbert, D.J. 1997. Towards a new recruitment paradigm for fish stocks. Can. J. Fish. Aquat. Sci. 54(4): 969–977. doi:10.1139/f96-272.
23. Hare, J.A., Brooks, E.N., Palmer, M.C., and Churchill, J.H. Re-evaluating the effect of wind on recruitment in Gulf of Maine Atlantic Cod (Gadus morhua) using an environmentally-explicit stock recruitment model. Fish. Oceanogr. [In press.]
24. Hoeting, J.A., Madigan, D., Raftery, A.E., and Volinsky, C.T. 1999. Bayesian model averaging: a tutorial. Stat. Sci. 14(4): 382–401.
25. Hollowed, A.B., Ianelli, J.N., and Livingston, P.A. 2000. Including predation mortality in stock assessments: a case study for Gulf of Alaska walleye pollock. ICES J. Mar. Sci. 57: 279–293. doi:10.1006/jmsc.1999.0637.
26. ICES. 2012. Report of the Benchmark Workshop on Pelagic Stocks (WKPELA), 13–17 February 2012. Copenhagen, Denmark. CM 2012/ACOM:47.
27. Kinzey, D., and Punt, A.E. 2009. Multispecies and single-species models of fish population dynamics: comparing parameter estimates. Nat. Res. Model. 22(1): 67–104. doi:10.1111/j.1939-7445.2008.00030.x.
28. Legault, C.M. 2009. Report of the Retrospective Working Group, 14–16 January 2008, Woods Hole, Massachusetts [online]. National Oceanic and Atmospheric Administration, National Marine Fisheries Service, Northeast Fisheries Science Center Reference Document 09-01, Woods Hole, Mass. Available from www.nefsc.noaa.gov/publications/crd/crd0901/index.html [February 2014].
29. Legault, C.M., and Restrepo, V.R. 1999. A flexible forward age-structured assessment program. ICCAT Coll. Vol. Sci. Pap. 49(2): 246–253.
30. Link, W.A. 1999. Modeling pattern in collections of parameters. J. Wildl. Manage. 63: 1017–1027. doi:10.2307/3802817.
31. Ludwig, D., and Walters, C.J. 1981. Measurement errors and uncertainty in parameter estimates for stock and recruitment. Can. J. Fish. Aquat. Sci. 38(6): 711–720. doi:10.1139/f81-094.
32. Magnusson, A., Punt, A.E., and Hilborn, R. 2013. Measuring uncertainty in fisheries stock assessment: the delta method, bootstrap, and MCMC. Fish Fish. 14: 325–342. doi:10.1111/j.1467-2979.2012.00473.x.
33. Maunder, M.N., and Punt, A.E. 2013. A review of integrated analysis in fisheries stock assessment. Fish. Res. 142: 61–74. doi:10.1016/j.fishres.2012.07.025.
34. Mayo, R.K., and Terceiro, M. (Editors). 2005. Assessment of 19 Northeast groundfish stocks through 2004 [online]. 2005 Groundfish Assessment Review Meeting (2005 GARM). US Dept. of Commerce, Northeast Fisheries Science Center Reference Document 05-13. Available from http://www.nefsc.noaa.gov/publications/crd/crd0513/crd0513.pdf.
35. Michielsens, C.G.J., and McAllister, M.K. 2004. A Bayesian hierarchical analysis of stock-recruit data: quantifying structural and parameter uncertainties. Can. J. Fish. Aquat. Sci. 61(6): 1032–1047. doi:10.1139/f04-048.
36. Miller, R.G. 1974. The jackknife — a review. Biometrika, 61(1): 1–15. doi:10.1093/biomet/61.1.1.
37. Mohn, R. 1999. The retrospective problem in sequential population analysis: an investigation using cod fishery and simulated data. ICES J. Mar. Sci. 56: 473– 488. doi:10.1006/jmsc.1999.0481.
38. Moustahfid, H., Link, J.S., Overholtz, W.J., and Tyrrell, M.C. 2009. The advantage of explicitly incorporating predation mortality into age-structured stock assessment models: an application for Atlantic mackerel. ICES J. Mar. Sci. 66: 445–454. doi:10.1093/icesjms/fsn217.
39. Myers, R.A. 1998. When do environment–recruit correlations work? Rev. Fish Biol. Fish. 8: 285–305. doi:10.1023/A:1008828730759.
40. Myers, R.A., and Barrowman, N.J. 1996. Is fish recruitment related to spawner abundance? Fish. Bull. 94: 707–724.
41. Myers, R.A., Bowen, K.G., and Barrowman, N.J. 1999. Maximum reproductive rate of fish at low population sizes. Can. J. Fish. Aquat. Sci. 56(12): 2404–2419. doi:10.1139/f99-201.
42. NEFSC. 2008. Assessment of 19 Northeast Groundfish Stocks through 2007: Report of the 3rd Groundfish Assessment Review Meeting (GARM III) [online]. US Dept. of Commerce, Northeast Fisheries Science Center, Reference Document 08-15. Available from http://www.nefsc.noaa.gov/publications/crd/ crd0815/.
43. NEFSC. 2012a. 53rd Northeast Regional Stock Assessment Workshop (53rd AW) Assessment Report [online]. US Dept. of Commerce, Northeast Fisheries Science Center Reference Document 12-05. Available from http://www.nefsc.noaa.gov/publications/crd/crd1205/.
44. NEFSC. 2012b. 54th Northeast Regional Stock Assessment Workshop (54th SAW) Assessment Report [online]. US Dept. of Commerce, Northeast Fisheries Science Center, Reference Document 12-18. Available from http://www.nefsc.noaa.gov/publications/crd/crd1218/.
45. NEFSC. 2012c. Assessment or Data Updates of 13 Northeast Groundfish Stocks through 2010 [online]. US Dept. of Commerce, Northeast Fisheries Science Center Reference Document 12-06. Available from http://nefsc.noaa.gov/publications/crd/crd1206/.
46. NEFSC. 2013. 55th Northeast Regional Stock Assessment Workshop (55th SAW) Assessment Report [online]. US Department of Commerce, Northeast Fisheries Science Center, Reference Document 13-11. Available from http://nefsc.noaa.gov/publications/crd/crd1311/.
47. Nieland, J.L, Hansen, M.J., Seider, M.J., and Deroba, J.J. 2008. Modeling the sustainability of lake trout fisheries in eastern Wisconsin waters of Lake Superior. Fish. Res. 94: 304–314. doi:10.1016/j.fishres.2008.07.005.
48. Nielsen, A., and Berg, C.W. 2014. Estimation of time-varying selectivity in stock assessments using state-space models. Fish. Res. 158: 96–101. doi:10.1016/j.fishres.2014.01.014.
49. Overholtz, W.J., and Link, J.S. 2007. Consumption impacts by marine mammals, fish, and seabirds on the Gulf of Maine – Georges Bank Atlantic herring complex during the years 1977-2002. ICES J. Mar. Sci. 64: 83–96. doi:10.1093/ icesjms/fsl009.
50. Overholtz, W.J., Jacobson, L.D., and Link., J.S. 2008. An ecosystem approach for assessment advice and biological reference points for the Gulf of Maine – Georges Bank Atlantic herring complex. N. Am. J. Fish. Manage. 28: 247–257. doi:10.1577/M06-267.1.
51. Prager, M.H., and Shertzer, K.W. 2010. Deriving acceptable biological catch from the overfishing limit: implications for assessment models. N. Am. J. Fish. Manage. 30: 289–294. doi:10.1577/M09-105.1.
52. Punt, A.E., and Dorn, M. 2014. Comparisons of meta-analytic methods for deriving a probability distribution for the steepness of the stock–recruitment relationship. Fish. Res. 149: 43–54. doi:10.1016/j.fishres.2013.09.015.
53. Punt, A.E., Smith, D.C., and Smith, A.D.M. 2011. Among-stock comparisons for improving stock assessments of data-poor stocks: the “Robin Hood” approach. ICES J. Mar. Sci. 68: 972–981. doi:10.1093/icesjms/fsr039.
54. Ralston, S., Punt, A.E., Hamel, O.S., DeVore, J.D., and Conser, R.J. 2011. A metaanalytic approach to quantifying scientific uncertainty in stock assessments. Fish. Bull. 109: 217–231.
55. Ricard, D., Minto, C., Jensen, O.P., and Baum, J.K. 2012. Examining the knowledge base and status of commercially exploited marine species with the RAM Legacy Stock Assessment Database. Fish Fish. 13: 380–398. doi:10.1111/j.1467- 2979.2011.00435.x.
56. Richardson, D.E., Hare, J.A., Fogarty, M.J., and Link, J.S. 2011. Role of egg predation by haddock in the decline of an Atlantic herring population. Proc. Natl. Acad. Sci. U.S.A. 108(33): 13606–13611. doi:10.1073/pnas.1015400108. PMID: 21825166.
57. Shertzer, K.W., and Conn, P.B. 2012. Spawner–recruit relationships of demersal marine fishes: prior distributions of steepness. Bull. Mar. Sci. 88: 39–50.
58. Steele, J.H. 1984. Kinds of variability and uncertainty affecting fisheries. In Exploitation of Marine Communities: Dahlem Workshop Report. Vol. 32. Edited by R.M. May. Springer-Verlag, New York. pp 245–262.
59. Stefansson, G. 2003. Issues in multispecies models. Nat. Res. Model. 16(4): 415– 437. doi:10.1111/j.1939-7445.2003.tb00121.x.
60. Stefansson, G., and Palsson, O.K. 1997. Statistical evaluation and modelling of the stomach contents of Icelandic cod (Gadus morhua). Can. J. Fish. Aquat. Sci. 54(1): 169–181. doi:10.1139/f96-246.
61. Stocker, M., Haist, V., and Fournier, D. 1985. Environmental variation and recruitment of Pacific herring (Clupea harengus pallasi) in the Strait of Georgia.Can. J. Fish. Aquat. Sci. 42(S1): 174–180. doi:10.1139/f85-272.
62. Szuwalski, C.S., Vert-Pre, K.A., Punt, A.E., Branch, T.A., and Hilborn, R. 2014. Examining common assumptions about recruitment: a meta-analysis of recruitment dynamics for worldwide marine fisheries. Fish Fish. 157: 28–40.doi:10.1111/faf.12083.
63. Thorson, J.T., Cope, J.M., Kleisner, K.M., Samhouri, J.F., Shelton, A.O., and Ward, E.J. 2013. Giants’ shoulders 15 years later: lessons, challenges, and guidelines in fisheries meta-analysis. Fish Fish. doi:10.1111/faf.12061.
64. TRAC. 2006. Benchmark review of stock assessment models for Gulf of Maine and Georges Bank herring [online]. Transboundary Resources Assessment Committee Proceedings 2006/01. Available from http://www.mar.dfompo. gc.ca/science/TRAC/trac.html.
65. TRAC. 2009. Gulf of Maine/Georges Bank Atlantic herring stock assessment update [online]. Transboundary Resources Assessment Committee Status Report 2009/04. Available from http://www2.mar.dfo-mpo.gc.ca/science/trac/ rd.html.
66. Van Kirk, K.F., Quinn, T.J., and Collie, J.S. 2010. A multispecies age-structured assessment model for the Gulf of Alaska. Can. J. Fish. Aquat. Sci. 67(7): 1135– 1148. doi:10.1139/F10-053.
67. Walters, C.J. 1984. Managing fisheries under biological uncertainty. In Exploitation of Marine Communities: Dahlem Workshop Report. Vol. 32. Edited by R.M. May. Springer-Verlag, New York. pp. 263–274.
68. Walters, C.J., and Ludwig, D. 1981. Effects of measurement errors on the assessment of stock–recruitment relationships. Can. J. Fish. Aquat. Sci. 38(6): 704– 710. doi:10.1139/f81-093.
69. Williams, B.K. 1997. Approaches to the management of waterfowl under uncertainty. Wildlife Soc. Bull. 25: 714–720.
70. Zhou, S. 2007. Discriminating alternative stock–recruitment models and evaluating uncertainty in model structure. Fish. Res. 86: 268–279. doi:10.1016/j. fishres.2007.06.026.