Great challenges with few subjects: Statistical strategies for neuroscientists

Neuroscience and Biobehavioral Reviews

Volumn 35, Issue 3, 2011, Pages 462-473

  Nakagawa, Shinichi ^a   Hauber, Mark E ^b  

^a UNIVERSITY OF OTAGO   (New Zealand)

^b CITY UNIVERSITY OF NEW YORK   (United States)

Author keywords

Markov chain Monte Carlo; Meta analysis; Mixed effects modelling; Model averaging; Multiple imputation; Pseudoreplication; Sample size

Indexed keywords

ANALYSIS OF VARIANCE; ANALYTIC METHOD; ANALYTICAL ERROR; BAYES THEOREM; DATA BASE; DATA MINING; INFORMATION PROCESSING; MATHEMATICAL ANALYSIS; MEDICAL INFORMATION; MEDICAL RESEARCH; METHODOLOGY; MONTE CARLO METHOD; NERVOUS SYSTEM ELECTROPHYSIOLOGY; NEUROANATOMY; NEUROPHYSIOLOGY; NEUROPSYCHOLOGY; NEUROSCIENCE; NULL HYPOTHESIS; PRIORITY JOURNAL; PROBABILITY; RESEARCH SUBJECT; REVIEW; SCIENTIST; SIMULATION; STATISTICAL ANALYSIS; STATISTICAL MODEL; STATISTICAL PARAMETERS; STATISTICAL SIGNIFICANCE; STUDENT T TEST;

BIOMETRY; HUMANS; META-ANALYSIS AS TOPIC; MODELS, STATISTICAL; NEUROSCIENCES; RESEARCH DESIGN; STATISTICS AS TOPIC;

EID: 78649907710     PISSN: 01497634     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neubiorev.2010.06.003     Document Type: Review

References (110)

1. Airey D.C., Castillo-Juarez H., Casella G., Pollak E.J., DeVoogd T.J. Variation in the volume of zebra finch song control nuclei is heritable: developmental and evolutionary implications. Proc. R. Soc. Lond. B 2000, 267:2099-2104.
2. Allison P.D. Missing Data 2002, Sage, Thousand Oaks, CA.
3. Altman D.G., Schulz K.F., Moher D., Egger M., Davidoff F., Elbourne D., Gotzsche P.C., Lang T. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann. Intern. Med. 2001, 134:663-694.
4. Anderson D.R. Model Based Inference in The Life Sciences: A Primer on Evidence 2008, Springer, New York, NY.
5. Anderson D.R., Burnham K.P. Avoiding pitfalls when using information-theoretic methods. J. Wildlife Manage. 2002, 66:912-918.
6. Arnqvist G., Wooster D. Meta-analysis: synthesizing research findings in ecology and evolution. Trends Ecol. Evol. 1995, 10:236-240.
7. Ashmore R.C., Wild J.M., Schmidt M.F. Brainstem and forebrain contributions to the generation of learned motor behaviors for song. J. Neurosci. 2005, 25:8543-8554.
8. Blood E.A., Cabral H., Heeren T., Cheng D.M. Performance of mixed effects models in the analysis of mediated longitudinal data. BMC Med. Res. Methodol. 2010, 10:16.
9. Bolhuis J.J., Zijlstra G.G.O., den Boer-Visser A.M., Van der Zee E.A. Localized neuronal activation in the zebra finch brain is related to the strength of song learning. Proc. Natl. Acad. Sci. U.S.A. 2000, 97:2282-2285.
10. Bolker B.M., Brooks M.E., Clark C.J., Geange S.W., Poulsen J.R., Stevens M.H.H., White J.S.S. Generalized linear mixed models: a practical guide for ecology and evolution. Trends Ecol. Evol. 2009, 24:127-135.
11. Borenstein M., Hedges L.V., HigginS J.P.I., Rothstein H.R. Introduction to Meta-analysis 2009, Wiley, Chichester.
12. Borst A., Theunissen F.E. Information theory and neural coding. Nat. Neurosci. 1999, 2:947-957.
13. Boyce M.S. Statistics as viewed by biologists. J. Agric. Biol. Environ. Stat. 2002, 7:306-312.
14. Burnham K.P., Anderson D.R. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach 2002, Springer-Verlag, Berlin. 2nd ed.
15. Claeskens G., Hiort N.L. Model Selection and Model Averaging 2009, Cambridge University Press, Cambridge.
16. Clark J.S. Why environmental scientists are becoming Bayesians. Ecol. Lett. 2005, 8:2-14.
17. Clark J.S., Gelfand A.E. A future for models and data in environmental science. Trends Ecol. Evol. 2006, 21:375-380.
18. Cnotka J., Gunturkun O., Rehkamper G., Gray R.D., Hunt G.R. Extraordinary large brains in tool-using New Caledonian crows (Corvus moneduloides). Neurosci. Lett. 2008, 433:241-245.
19. Cohen J. Statistical Power Analysis for the Beahvioral Sciences 1988, Lawrence Erlbaum, Hillsdale, New Jersey. 2nd ed.
20. Cohen J. Things I have learned (so far). Am. Psycol. 1990, 45:1304-1312.
21. Cohen J. The earth is round (p<.05). Am. Psychol. 1994, 49:997-1003.
22. Congdon P. Bayesian Statistical Modelling 2007, Wiley, Chichester. 2nd ed.
23. Cooper H., Hedges L.V., Valentine J.C. The Handbook of Research Synthesis and Meta-Analysis 2009, Russell Sage Foundation, New York. 2nd ed.
24. Corfield J.R., Wild J.M., Hauber M.E., Parsons S., Kubke M.F. Evolution of brain size in the palaeognath lineage, with an emphasis on New Zealand ratites. Brain Behav. Evol. 2008, 71:87-99.
25. Crawley M.J. The R Book 2007, Wiley, Chichester.
26. Cumming G., Finch S. A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educ. Psychol. Meas. 2001, 61:532-574.
27. Egger M., Smith G.D., Altman D.G. Systematic Reviews in Health Care: Meta-Analysis in Context 2001, BMJ, London. 2nd ed.
28. Faraway J.J. Extending the Linear Model with R 2006, CRC, Boca Raton, FL.
29. Fidler F., Cumming G., Burgman M., Thomason N. Statistical reform in medicine, psychology and ecology. J. Socio-Econ. 2004, 33:615-630.
30. Forstmeier W., Coltman D.W., Birkhead T.R. Maternal effects influence the sexual behavior of sons and daughters in the zebra finch. Evolution 2004, 58:2574-2583.
31. Freckleton, R.P. Dealing with collinearity in behavioural and ecological data: model averaging and the problems with measurement errors. Behav. Ecol. Sociobiol., in press.
32. Frick R.W. Accepting the null hypothesis. Mem. Cognit. 1995, 23:132-138.
33. Galwey Introduction to Mixed Modelling: Beyond Regression and Analysis of Variance 2006, Wiley, Chichester.
34. Gamerman D., Lopes H.F. Markov Chain Monte Carlo: Stochastic Simultionn for Bayesian Inference 2006, CRC, Boca Raton, FL.
35. Garamszegi L.Z. Comparing effect sizes across variables: generalization without the need for Bonferroni correction. Behav. Ecol. 2006, 17:682-687.
36. Garamszegi L.Z., Calhim S., Dochtermann N., Hegyi G., Hurd P.L., Jorgensen C., Kutsukake N., Lajeunesse M.J., Pollard K.A., Schielzeth H., Symonds M.R.E., Nakagawa S. Changing philosophies and tools for statistical inference in behavioral ecology. Behav. Ecol. 2009, 20:1363-1375.
37. Gelman A. Teaching Bayes to graduate students in political science, sociology, public health, education, economics. Am. Stat. 2008, 62:202-205.
38. Gelman A., Carlin J.B., Stern H.S., Rubin D.B. Bayesian Data Analysis 1995, Chapman & Hall, London.
39. Gelman A., Hill J. Data analysis Using Regression and Multilevel/Hierarchical Models 2007, Cambridge University Press, Cambridge.
40. Gelman A., Jakulin A., Pittau M.G., Su Y.S. A weakly informative default prior distribution for logistic and other regression models. Ann. Appl. Stat. 2008, 2:1360-1383.
41. Gelman A., Stern H. The difference between "significant" and "not significant" is not itself statistically significant. Am. Stat. 2006, 60:328-331.
42. Gentner T.Q., Margoliash D. Neuronal populations and single cells representing learned auditory objects. Nature 2003, 424:669-674.
43. Gil D., Naguib M., Riebel K., Rutstein A., Gahr M. Early condition, song learning, and the volume of song brain nuclei in the zebra finch (Taeniopygia guttata). J. Neurobiol. 2006, 66:1602-1612.
44. Gilks W.R., Richardson S., Spiegelhalter D. Markov Chain Monte Carlo In Practice: Interdisciplinary Statistics 1995, CRC, Boca Raton, FL.
45. Gill J. Bayesian Methods: A Social and Behavioral Sciences Approach 2007, CRC, Boca Raton, FL.
46. Graham, J.W., Schafer, J.L., 1999. On the performance of multiple imputation for multivariate data with small sample size. In: Hoyle, R. (Ed.), Statistical Strategies for Small Sample Research. Sage, Thousand Oaks, CA.
47. Griffin A.S., West S.A. Kin discrimination and the benefit of helping in cooperatively breeding vertebrates. Science 2003, 302:634-636.
48. Grissom R.J., Kim J.J. Effect sizes for Research: A Broad Practical Approach 2005, Lawrence Erlbaum Associates, Mahwah, NJ.
49. Hadfield J.D. MCMC methods for multi-response Generalised Linear Mixed Models: the MCMCglmm R package. J. Stat. Soft. 2010, 33:1-22.
50. Hadfield J.D., Nakagawa S. General quantitative genetic methods for comparative biology: phylogenies, taxonomies and multi-trait models for continuous and categorical characters. J. Evol. Biol. 2010, 23:494-508.
51. Harlow L.L., Mulaik S.A., Steiger J.H. What if There Were no Significance Tests? 1997, Erlbaum, Mahwah, New Jersey.
52. Harvey P.H., Pagel M.D. The Comparative Method in Evolutionary Biology 1991, Oxford University Press, Oxford.
53. Hauber M.E., Clayton N.S., Kacelnik A., Reboreda J.C., DeVoogd T.J. Sexual dimorphism and species differences in HVC volumes of cowbirds. Behav. Neurosci. 1999, 113:1095-1099.
54. Hauber M.E., Cassey P., Woolley S.M.N., Theunissen F.E. Neurophysiological response selectivity for conspecific songs over synthetic sounds in the auditory forebrain of non-singing female songbirds. J. Comp. Physiol. A 2007, 193:765-774.
55. Hauber M.E., Woolley S.M.N., Theunissen F.E. Experience-dependence of neural responses to social versus isolate conspecific songs in the forebrain of female Zebra Finches. J. Ornithol. 2007, 148:S231-S239.
56. Horton N.J., Kleinman K.P. Much ado about nothing: a comparison of missing data methods and software to fit incomplete data regression models. Am. Stat. 2007, 61:79-90.
57. Hsu A., Woolley S.M.N., Fremouw T.E., Theunissen F.E. Modulation power and phase spectrum of natural sounds enhance neural encoding performed by single auditory neurons. J. Neurosci. 2004, 24:9201-9211.
58. Hunt M. How Science Takes Stock: The Story of Meta-analysis 1997, Russell Sage, New York.
59. Janusonis S. Comparing two small samples with an unstable, treatment-independent baseline. J. Neurosci. Meth. 2009, 179:173-178.
60. Jarvis E.D., Ribeiro S., da Silva M.L., Ventura D., Vielliard J., Mello C.V. Behaviourally driven gene expression reveals song nuclei in hummingbird brain. Nature 2000, 406:628-632.
61. Johnson D.H. Statistical sirens-the allure of nonparametrics. Ecology 1995, 76:1998-2000.
62. Johnson J.B., Omland K.S. Model selection in ecology and evolution. Trends Ecol. Evol. 2004, 19:101-108.
63. Kline R.B. Beyond Significance Testing 2004, American Psychological Association, Washington, DC.
64. Lipsey M.W., Wilson D.B. Practical Meta-analysis 2001, Sage, Beverly Hills, CA.
65. Little R.J.A. A test of missing completely at random for multivariate data with missing values. J. Am. Stat. Assoc. 1988, 83:1198-1202.
66. Little R.J.A., Rubin D.B. Statistical Analysis With Missing Data 2002, Wiley, New York. 2nd ed.
67. Lukacs P.M., Burnham K.P., Anderson D.R. Model selection bias and Freedman's paradox. Ann. I. Stat. Math. 2010, 62:117-125.
68. Lunn D.J., Thomas A., Best N., Spiegelhalter D. WinBUGS-a Bayesian modelling framework: concepts, structure, and extensibility. Stat. Comp. 2000, 10:325-337.
69. MacDougall-Shackleton S.A., Ball G.F. Comparative studies of sex differences in the song-control system of songbirds. Trends Neurosci. 1999, 22:432-436.
70. McCarthy M.A. Bayesian Methods for Ecology 2007, Cambridge University Press, Cambridge.
71. McCulloch C.E., Searle S.R. Generalized, Linear and Mixed Models 2002, Wiley, Chichester.
72. McKnight P.E., McKnight K.M., Sidani S., Figueredo A.J. Missing Data: A Gentle Introduction 2007, The Guilford Press, New York, NY.
73. Miller E.R., Pastor-Barriuso R., Dalal D., Riemersma R.A., Appel L.J., Guallar E. Meta-analysis: high-dosage vitamin E supplementation may increase all-cause mortality. Ann. Intern. Med. 2005, 142:37-46.
74. Mogie M. In support of null hypothesis significance testing. Proc. R. Soc. Lond. B 2004, 271:S82-S84.
75. Moller A.P., Jennions M.D. Testing and adjusting for publication bias. Trends Ecol. Evol. 2001, 16:580-586.
76. Nakagawa S., Cuthill I.C. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol. Rev. 2007, 82:591-605.
77. Nakagawa S., Freckleton R.P. Missing inaction: the dangers of ignoring missing data. Trends Ecol. Evol. 2008, 23:592-596.
78. Nakagawa, S., Freckleton, R.P. Model averaging, missing data and multiple imputation: a case study for behavioral ecolgy. Behav. Ecol. Sociobiol., in press.
79. Nakagawa, S., Schielzeth, H. Repeatability for Gaussian and non-Gaussian data: a practical guide for biologists. Biol. Rev., in press, doi:10.1111/j.1469-185X.2010.00141.x.
80. Nicholas P. Statistics for Epidemiology 2003, CRC, Boca Raton, FL.
81. Nickerson R.S. Null hypothesis significance testing: a review of an old and continuing controversy. Psychol. Methods 2000, 5:241-301.
82. O'Hara R.B. How to make models add up-a primer on GLMMs. Ann. Zool. Fenn. 2009, 46:124-137.
83. Paterson S., Lello J. Mixed models: getting the best use of parasitological data. Trends Parasitol. 2003, 19:370-375.
84. Pilz K.M., Smith H.G. Egg yolk androgen levels increase with breeding density in the European Starling Sturnus vulgaris. Funct. Ecol. 2004, 18:58-66.
85. Pinheiro J.C., Bates D.M. Mixed-Effects Models in S and S-PLUS 2000, Springer, New York.
86. R Development Core Team, 2009. R: A Language and Environment For Statistical Computing, 2.8.2 ed. R Foundation for Statistical Computing, Vienna, Austria.
87. Rasbash, J., Browne, W., Goldstein, H., Yang, M., Plewis, I., Healy, M., Woodhouse, G., Draper, D., Langford, I., Lewis, T., 2002. A User's Guide to MLwiN. Centre for Multilevel Modeling, Institute of Education, University of London, London.
88. Raudenbush S.W., Bryk A.S. Hierarchical Linear Models: Appplication and Data Analysis Methods 2002, Sage, Thousand Oaks.
89. Rosenthal R. The "file drawer problem" and tolerance for null results. Psychol. Bull. 1979, 86:638-641.
90. Rothstein H.R., Sutton A.J., Borenstein M. Publication Bias in Meta-Analysis 2005, Wiley, Chichester.
91. Rubin D.B. Inference and missing data. Biometrika 1976, 63:581-590.
92. Rubin D.B. Multiple Imputation for Nonresponse in Surveys 1987, J. Wiley & Sons, New York, NY.
93. Ruxton G.D. The unequal variance t-test is an underused alternative to Student's t-test and the Mann-Whitney U test. Behav. Ecol. 2006, 17:688-690.
94. Schafer J.L. Analysis of Incomplete Multivariate Data 1997, Chapman & Hall, London.
95. Schafer J.L. Multiple imputation: a primer. Stat. Methods Med. Res. 1999, 8:3-15.
96. Schafer J.L., Graham J.W. Missing data: our view of the state of the art. Psychol. Methods 2002, 7:147-177.
97. Schwabl H. Yolk is a source of maternal testosterone for developing birds. Proc. Natl. Acad. Sci. U.S.A. 1993, 90:11446-11450.
98. Shank S.S., Margoliash D. Sleep and sensorimotor integration during early vocal learning in a songbird. Nature 2009, 458:73-74.
99. Smith M.L., Glass G.V. Meta-analysis of psychotherapy outcome studies. Am. Psychol. 1977, 32:752-760.
100. Stephens P.A., Buskirk S.W., del Rio C.M. Inference in ecology and evolution. Trends Ecol. Evol. 2007, 22:192-197.
101. Stephens P.A., Buskirk S.W., Hayward G.D., Del Rio C.M. Information theory and hypothesis testing: a call for pluralism. J. Appl. Ecol. 2005, 42:4-12.
102. Stewart G. Meta-analysis in applied ecology. Biol. Lett. 2010, 6:78-81.
103. Sutton A.J., HigginS J.P.I. Recent developments in meta-analysis. Stat. Med. 2008, 27:625-650.
104. Symonds, M.R.E., Moussalli, A. Model selection, multimodel inference and model averaging using Akaike's information criterion: an introduction for statistically terrified behavioural ecologists. Behav. Ecol. Sociobiol., in press.
105. Tomaszycki M.L., Peabody C., Replogle K., Clayton D.F., Tempelman R.J., Wade J. Sexual differentiation of the zebra finch song system: potential roles for sex chromosome genes. BMC Neurosci. 2009, 10:1-14.
106. Thompson B. What future quantitative social science research could look like: confidence intervals for effect sizes. Educ. Res. 2002, 31:25-32.
107. Tobias M.L., Viswanathan S.S., Kelley D.B. Rapping, a female receptive call, initiates male-female duets in the South African clawed frog. Proc. Natl. Acad. Sci. U.S.A. 1998, 95:1870-1875.
108. Tonra C.M., Johnson M.D., Heath S.K., Hauber M.E. Does nesting habitat predict hatch synchrony between brood parasitic brown-headed cowbirds (Molothrus ater) and two host species?. Ecography 2009, 32:497-503.
109. Woodworth G.G. Biostatistics: A Bayesian Introduction 2004, Wiley, Hoboken, NJ.
110. Zuur A.F., Ieno E.N., Walker N.J., Saveliev A.A., Smith G.M. Mixed Effects Models and Extensions in Ecology with R 2009, Springer, New York.