Strong inference in mathematical modeling: A method for robust science in the twenty-first century

Frontiers in Microbiology

Volumn 7, Issue JUL, 2016, Pages

  Ganusov, Vitaly V ^a  

^a UNIVERSITY OF TENNESSEE   (United States)

Author keywords

Immunology; Mathematical modeling; Microbiology; Public health; Robust science; Scientific method

Indexed keywords

BIOLOGY; EXPERIMENTAL MODEL; HUMAN; HUMAN EXPERIMENT; IMMUNOLOGY; MICROBIOLOGY; PREDICTION; PUBLIC HEALTH; PUBLICATION; STATISTICAL MODEL;

EID: 84983081973     PISSN: None     EISSN: 1664302X     Source Type: Journal    
DOI: 10.3389/fmicb.2016.01131     Document Type: Article

References (88)

1. Ahmed, R., and Gray, D. (1996). Immunological memory and protective immunity: understanding their relation. Science 272, 54-60. doi: 10.1126/science.272.5258.54
2. Anderson, R. M., Jackson, H. C., May, R. M., and Smith, A. M. (1981). Population dynamics of fox rabies in Europe. Nature 289, 765-771. doi: 10.1038/289765a0
3. Antia, R., Ganusov, V. V., and Ahmed, R. (2005). The role of models in understanding CD8+ T-cell memory. Nat. Rev. Immunol. 5, 101-111. doi: 10.1038/nri1550
4. Baker, M. (2015). Reproducibility crisis: blame it on the antibodies. Nature 521, 274-276. doi: 10.1038/521274a
5. Baker, M. (2016). Is there a reproducibility crisis? Nature 533, 452-454. doi: 10.1038/533452a
6. Bates, D. M., and Watts, D. G. (1988). Nonlinear Regression Analysis and Its Applications. Hoboken, NJ: John Wiles & Sons, Inc. doi: 10.1002/9780470316757
7. Bauer, P., Thorpe, A., and Brunet, G. (2015). The quiet revolution of numerical weather prediction. Nature 525, 47-55. doi: 10.1038/nature14956
8. Begley, C. G., and Ellis, L. M. (2012). Drug development: raise standards for preclinical cancer research. Nature 483, 531-533. doi: 10.1038/483531a
9. Bocharov, G., Klenerman, P., and Ehl, S. (2001). Predicting the dynamics of antiviral cytotoxic T-cell memory in response to different stimuli: cell population structure and protective function. Immunol. Cell. Biol. 79, 74-86. doi: 10.1046/j.1440-1711.2001.00985.x
10. Boulesteix, A.-L. (2015). Ten simple rules for reducing overoptimistic reporting in methodological computational research. PLoS Comput. Biol. 11:e1004191. doi: 10.1371/journal.pcbi.1004191
11. Boulesteix, A.-L., Stierle, V., and Hapfelmeier, A. (2015). Publication bias in methodological computational research. Cancer Inform. 14(Suppl 5), 11-19. doi: 10.4137/CIN.S30747
12. Brauer, F., and Castillo-Chávez, C. (2001). Mathematical Models in Population Biology and Epidemiology, Texts in Applied Mathematics. New York, NY: Springer.
13. Bru, A., and Cardona, P.-J. (2010). Mathematical modeling of tuberculosis bacillary counts and cellular populations in the organs of infected mice. PLoS ONE 5:e12985. doi: 10.1371/journal.pone.0012985
14. Bucy, R. P., Hockett, R. D., Derdeyn, C. A., Saag, M. S., Squires, K., Sillers, M., et al. (1999). Initial increase in blood CD4(+) lymphocytes after HIV antiretroviral therapy reflects redistribution from lymphoid tissues. J. Clin. Invest. 103, 1391-1398. doi: 10.1172/JCI5863
15. Burnham, K. P., and Anderson, D. R. (2002). Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. New York, NY: Springer-Verlag.
16. Butler, D. (2014). Models overestimate Ebola cases. Nature 515:18. doi: 10.1038/515018a
17. Casarett, D. (2016). The science of choosing wisely-overcoming the therapeutic illusion. New Engl. J. Med. 374, 1203-1205. doi: 10.1056/NEJMp1516803
18. Castillo, M. (2014). The fraud and retraction epidemic. AJNR Am. J. Neuroradiol. 35, 1653-1654. doi: 10.3174/ajnr.A3835
19. Chamberlin, T. C. (1890). The method of multiple working hypotheses: with this method the dangers of parental affection for a favorite theory can be circumvented. Science 15, 92-96.
20. Christley, R. M., Mort, M., Wynne, B., Wastling, J. M., Heathwaite, A. L., Pickup, R., et al. (2013). "Wrong, but useful": negotiating uncertainty in infectious disease modelling. PLoS ONE 8:e76277. doi: 10.1371/journal.pone.0076277
21. Cilfone, N. A., Ford, C. B., Marino, S., Mattila, J. T., Gideon, H. P., Flynn, J. L., et al. (2015). Computational modeling predicts IL-10 control of lesion sterilization by balancing early host immunity-mediated antimicrobial responses with caseation during Mycobacterium tuberculosis infection. J. Immunol. 194, 664-677. doi: 10.4049/jimmunol.1400734
22. Collaboration, O. S. (2015). PSYCHOLOGY. Estimating the reproducibility of psychological science. Science 349:aac4716. doi: 10.1126/science.aac4716
23. Danielson, D., and Graney, C. M. (2014). The case against Copernicus. Sci. Am. 310, 72-77. doi: 10.1038/scientificamerican0114-72
24. De Boer, R. J. (2012). Which of our modeling predictions are robust? PLoS Comput. Biol. 8:e1002593. doi: 10.1371/journal.pcbi.1002593
25. De Boer, R. J., Oprea, M., Antia, R., Murali-Krishna, K., Ahmed, R., and Perelson, A. S. (2001). Recruitment times, proliferation, and apoptosis rates during the CD8(+) T-cell response to lymphocytic choriomeningitis virus. J. Virol. 75, 10663-10669. doi: 10.1128/JVI.75.22.10663-10669.2001
26. Ditlev, J. A., Mayer, B. J., and Loew, L. M. (2013). There is more than one way to model an elephant. Experiment-driven modeling of the actin cytoskeleton. Biophys. J. 104, 520-532. doi: 10.1016/j.bpj.2012.12.044
27. Drake, J. M., Kaul, R. B., Alexander, L. W., O'Regan, S. M., Kramer, A. M., Pulliam, J. T., et al. (2015). Ebola cases and health system demand in Liberia. PLoS Biol. 13:e1002056. doi: 10.1371/journal.pbio.1002056
28. Ebersole, C. R., Axt, J. R., and Nosek, B. A. (2016). Scientists' reputations are based on getting it right, not being right. PLoS Biol. 14:e1002460. doi: 10.1371/journal.pbio.1002460
29. Editorial (2015). Let's think about cognitive bias. Nature 526:163. doi: 10.1038/526163a
30. Ehlers, M. D. (2016). Lessons from a recovering academic. Cell 165, 1043-1048. doi: 10.1016/j.cell.2016.05.005
31. Eisinger, D., and Thulke, H.-H. (2008). Spatial pattern formation facilitates eradication of infectious diseases. J. Appl. Ecol. 45, 415-423. doi: 10.1111/j.1365-2664.2007.01439.x
32. Elliott, L. P., and Brook, B. W. (2007). Revisiting chamberlin: multiple working hypotheses for the 21st century. BioScience 57, 608-614. doi: 10.1641/B570708
33. Ellis, G., and Silk, J. (2014). Defend the integrity of physics. Nature 516, 321-323. doi: 10.1038/516321a
34. Ellner, S. P., and Guckenheimer, J. (2006). Dynamics Models in Biology. Princeton, NJ: Princeton University Press.
35. Evans, M. R., Grimm, V., Johst, K., Knuuttila, T., de Langhe, R., Lessells, C. M., et al. (2013). Do simple models lead to generality in ecology? Trends Ecol. Evol. 28, 578-583. doi: 10.1016/j.tree.2013.05.022
36. Fanelli, D. (2013). Why growing retractions are (mostly) a good sign. PLoS Med. 10:e1001563. doi: 10.1371/journal.pmed.1001563
37. Fang, F. C., and Casadevall, A. (2012). Reforming science: structural reforms. Infect. Immun. 80, 897-901. doi: 10.1128/IAI.06184-11
38. Fang, F. C., Steen, R. G., and Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proc. Natl. Acad. Sci. U.S.A 109, 17028-17033. doi: 10.1073/pnas.1212247109
39. Fearon, D., Carr, J., Telaranta, A., Carrasco, M., and Thaventhiran, J. (2006). The rationale for the IL-2-independent generation of the self-renewing central memory CD8+ T cells. Immunol. Rev. 211, 104-118. doi: 10.1111/j.0105-2896.2006.00390.x
40. Freedman, L. P., Cockburn, I. M., and Simcoe, T. S. (2015). The economics of reproducibility in preclinical research. PLoS Biol. 13:e1002165. doi: 10.1371/journal.pbio.1002165
41. Freedman, L. P., and Gibson, M. C. (2015). The impact of preclinical irreproducibility on drug development. Clin. Pharmacol. Ther. 97, 16-18. doi: 10.1002/cpt.9
42. Ganusov, V. V. (2007). Discriminating between different pathways of memory CD8+ T cell differentiation. J. Immunol. 179, 5006-5013. doi: 10.4049/jimmunol.179.8.5006
43. Goodman, S. N., Fanelli, D., and Ioannidis, J. P. A. (2016). What does research reproducibility mean? Sci. Transl. Med. 8, 1-6. doi: 10.1126/scitranslmed.aaf5027
44. Grieneisen, M. L., and Zhang, M. (2012). A comprehensive survey of retracted articles from the scholarly literature. PLoS ONE 7:e44118. doi: 10.1371/journal.pone.0044118
45. Grueber, C. E., Nakagawa, S., Laws, R. J., and Jamieson, I. G. (2011). Multimodel inference in ecology and evolution: challenges and solutions. J. Evol. Biol. 24, 699-711. doi: 10.1111/j.1420-9101.2010.02210.x
46. Harris, T. H., Banigan, E. J., Christian, D. A., Konradt, C., Wojno, E. D. T., Norose, K., et al. (2012). Generalized Lévy walks and the role of chemokines in migration of effector CD8+ T cells. Nature 486, 545-548. doi: 10.1038/nature11098
47. Hilborn, R., and Mangel, M. (1997). The Ecological Detective: Confronting Models with Data. Princeton NJ: Princeton University Press.
48. Ho, D., Neumann, A., Perelson, A., Chen, W., Leonard, J., and Markowitz, M. (1995). Rapid turnover of plasma virions and CD4 lymphocytes in HIV-1 infection. Nature 373, 123-126. doi: 10.1038/373123a0
49. Hobbs, N. T., and Hilborn, R. (2006). Alternatives to statistical hypothesis testing in ecology: a guide to self teaching. Ecol. Appl. 16, 5-19. doi: 10.1890/04-0645
50. Hoeting, J. A., Madigan, D., Raftery, A. E., and Volinsky, C. T. (1999). Bayesian model averaging: a tutorial. Stat. Sci. 14, 382-401.
51. Ioannidis, J. P. A., Allison, D. B., Ball, C. A., Coulibaly, I., Cui, X., Culhane, A. C., et al. (2009). Repeatability of published microarray gene expression analyses. Nat. Genet. 41, 149-155. doi: 10.1038/ng.295
52. Jewett, D. L. (2005). What's wrong with single hypotheses?: why it is time for strong-inference-PLUS. Scientist 19:10.
53. Jin, X., Bauer, D. E., Tuttleton, S. E., Lewin, S., Gettie, A., Blanchard, J., et al. (1999). Dramatic rise in plasma viremia after CD8(+) T cell depletion in simian immunodeficiency virus-infected macaques. J. Exp. Med. 189, 991-998. doi: 10.1084/jem.189.6.991
54. Johnson, J. B., and Omland, K. S. (2004). Model selection in ecology and evolution. Trends Ecol. Evol. 19, 101-108. doi: 10.1016/j.tree.2003.10.013
55. Kaech, S. M., and Cui, W. (2012). Transcriptional control of effector and memory CD8+ T cell differentiation. Nat. Rev. Immunol. 12, 749-761. doi: 10.1038/nri3307
56. Kaptchuk, T. J. (2003). Effect of interpretive bias on research evidence. BMJ 326, 1453-1455. doi: 10.1136/bmj.326.7404.1453
57. Kirk, P. D. W., Babtie, A. C., and Stumpf, M. P. H. (2015). SYSTEMS BIOLOGY. Systems biology (un)certainties. Science 350, 386-388. doi: 10.1126/science.aac9505
58. Kot, M. (2001). Elements of Mathematical Ecology. Cambridge, UK: Cambridge University Press. doi: 10.1017/cbo9780511608520
59. Lofgren, E. T., Halloran, M. E., Rivers, C. M., Drake, J. M., Porco, T. C., Lewis, B., et al. (2014). Opinion: mathematical models: a key tool for outbreak response. Proc. Natl. Acad. Sci. U.S.A. 111, 18095-18096. doi: 10.1073/pnas.1421551111
60. MacCoun, R., and Perlmutter, S. (2015). Blind analysis: hide results to seek the truth. Nature 526, 187-189. doi: 10.1038/526187a
61. Marino, S., and Kirschner, D. (2004). The human immune response to Mycobacterium tuberculosis in lung and lymph node. J. Theor. Biol. 227, 463-486. doi: 10.1016/j.jtbi.2003.11.023
62. Mayer, J., Khairy, K., and Howard, J. (2010). Drawing an elephant with four complex parameters. Am. J. Phys. 78, 648-649. doi: 10.1119/1.3254017
63. Meshkat, N., Eisenberg, M., and Distefano, III. J. J. (2009). An algorithm for finding globally identifiable parameter combinations of nonlinear ODE models using Gröbner Bases. Math. Biosci. 222, 61-72. doi: 10.1016/j.mbs.2009.08.010
64. Moher, D., and Altman, D. G. (2015). Four proposals to help improve the medical research literature. PLoS Med. 12:e1001864. doi: 10.1371/journal.pmed.1001864
65. Mooney, D., and Swift, R. (1999). A Course in Mathematical Modeling. Mathematical Association of America.
66. Noecker, C., Schaefer, K., Zaccheo, K., Yang, Y., Day, J., and Ganusov, V. V. (2015). Simple mathematical models do not accurately predict early SIV dynamics. Viruses 7, 1189-1217. doi: 10.3390/v7031189
67. Nuzzo, R. (2015). How scientists fool themselves-and how they can stop. Nature 526, 182-185. doi: 10.1038/526182a
68. O'Donohue, W., and Buchanan, J. A. (2001). The weaknesses of strong inference. Behav. Philos. 29, 1-20.
69. Oreskes, N., Shrader-Frechette, K., and Belitz, K. (1994). Verification, validation, and confirmation of numerical models in the Earth sciences. Science 263, 641-646. doi: 10.1126/science.263.5147.641
70. Pabst, R., and Rosenberg, Y. J. (1998). Interpreting data on lymphocyte subsets in the blood of HIV patients-organ distribution, proliferation and migration kinetics are critical factors. Pathobiology 66, 117-122. doi: 10.1159/000028006
71. Pandey, A., Atkins, K. E., Medlock, J., Wenzel, N., Townsend, J. P., Childs, J. E., et al. (2014). Strategies for containing Ebola in West Africa. Science 346, 991-995. doi: 10.1126/science.1260612
72. Percus, J. (2012). Mathematical Methods in Immunology, Courant Lecture Notes Series. American Mathematical Soc.
73. Platt, J. R. (1964). Strong inference: certain systematic methods of scientific thinking may produce much more rapid progress than others. Science 146, 347-353. doi: 10.1126/science.146.3642.347
74. Popper, K. (2002). The logic of scientific discovery. New York, NY: Routledge Classics. Taylor & Francis.
75. Prinz, F., Schlange, T., and Asadullah, K. (2011). Believe it or not: how much can we rely on published data on potential drug targets? Nat. Rev. Drug Discov. 10:712. doi: 10.1038/nrd3439-c1
76. Quinn, F. (2012). A revolution in mathematics? What really happened a century ago and why it matters today. Notices AMS 59, 31-37. doi: 10.1090/noti787
77. Raue, A., Kreutz, C., Maiwald, T., Bachmann, J., Schilling, M., Klingmüller, U., et al. (2009). Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics 25, 1923-1929. doi: 10.1093/bioinformatics/btp358
78. Scudellari, M. (2015). The science myths that will not die. Nature 528, 322-325. doi: 10.1038/528322a
79. Segel, L. A. (1984). Modeling Dynamic Phenomena in Molecular and Cellular Biology. New York, NY: Cambridge University Press.
80. Silberzahn, R., and Uhlmann, E. L. (2015). Crowdsourced research: many hands make tight work. Nature 526, 189-191. doi: 10.1038/526189a
81. Simberloff, D. (2007). An angry indictment of mathematical modeilng. Bioscience 57, 884-885.
82. Steen, R. G., Casadevall, A., and Fang, F. C. (2013). Why has the number of scientific retractions increased? PLoS ONE 8:e68397. doi: 10.1371/journal.pone.0068397
83. Vries, G. D., Hillen, T., Lewis, M., and Schõnfisch, B. (2006). A Course in Mathematical Biology: Quantitative Modeling with Mathematical and Computational (Monographs on Mathematical Modeling and Computation). SIAM. doi: 10.1137/1.9780898718256
84. Wang, S., Hottz, P., Schechter, M., and Rong, L. (2015). Modeling the slow CD4+ T cell decline in HIV-infected individuals. PLoS Comput. Biol. 11:e1004665. doi: 10.1371/journal.pcbi.1004665
85. Wei, X., Ghosh, S., Taylor, M., Johnson, V., Emini, E., Deutsch, P., et al. (1995). Viral dynamics in human immunodeficiency virus type 1 infection. Nature 373, 117-122. doi: 10.1038/373117a0
86. Wodarz, D., May, R. M., and Nowak, M. A. (2000). The role of antigen-independent persistence of memory cytotoxic T lymphocytes. Int. Immunol. 12, 467-477. doi: 10.1093/intimm/12.4.467
87. Wodarz, D., and Nowak, M. A. (2002). Mathematical models of HIV pathogenesis and treatment. Bioessays 24, 1178-1187. doi: 10.1002/bies.10196
88. Zhang, Z., Tao, Y., and Li, Z. (2007). Factors affecting harelynx dynamics in the classic time series of the Hudson Bay Company, Canada. Clim. Res. 34, 83-89. doi: 10.3354/cr034083