Non-parametric combination and related permutation tests for neuroimaging

Human Brain Mapping

Volumn 37, Issue 4, 2016, Pages 1486-1511

  Winkler, Anderson M ^a   Webster, Matthew A ^a   Brooks, Jonathan C ^b   Tracey, Irene ^a   Smith, Stephen M ^a   Nichols, Thomas E ^a,c  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b UNIVERSITY OF BRISTOL   (United Kingdom)

^c UNIVERSITY OF WARWICK   (United Kingdom)

Author keywords

Conjunctions; General linear model; Multiple testing; Non parametric combination; Permutation tests

Indexed keywords

ALGORITHM; ARTICLE; BRAIN MAPPING; HUMAN; META ANALYSIS (TOPIC); NEUROIMAGING; NONPARAMETRIC TEST; PAIN ASSESSMENT; PERMUTATION TEST; PRIORITY JOURNAL; STATISTICAL MODEL; THEORY; BRAIN CORTEX; IMAGE PROCESSING; NUCLEAR MAGNETIC RESONANCE IMAGING; PAIN MEASUREMENT; PATHOPHYSIOLOGY; PHYSIOLOGY; PROCEDURES;

ALGORITHMS; CEREBRAL CORTEX; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; NEUROIMAGING; PAIN MEASUREMENT; STATISTICS, NONPARAMETRIC;

EID: 84959020010     PISSN: 10659471     EISSN: 10970193     Source Type: Journal    
DOI: 10.1002/hbm.23115     Document Type: Article

References (103)

1. Abou Elseoud A, Nissilä J, Liettu A, Remes J, Jokelainen J, Takala T, Aunio A, Starck T, Nikkinen J, Koponen H, Zang YF, Tervonen O, Timonen M, Kiviniemi V (2014): Altered resting-state activity in seasonal affective disorder. Hum Brain Mapp 35:161-172.
2. Anderson TW (2003): An Introduction to Multivariate Statistical Analysis. Hoboken, NJ: Wiley.
3. Benjamini Y, Hochberg Y (1995): Controlling the false discovery rate: A practical and powerful approach to multiple testing. J R Stat Soc Ser B 57:289-300.
4. Benjamini Y, Heller R (2008): Screening for partial conjunction hypotheses. Biometrics 64:1215-1222.
5. Berger RL (1982): Multiparameter hypothesis testing and acceptance sampling. Technometrics 24:295-300.
6. Berk RH, Cohen A (1979): Asymptotically optimal methods of combining tests. J Am Stat Assoc 74:812-814.
7. Bhandary M, Zhang X (2011): Comparison of several tests for combining several independent tests. J Modern Appl Stat Meth 10:436-446.
8. Birnbaum A (1954): Combining independent tests of significance. J Am Stat Assoc 49:559-574.
9. Blair RC, Higgins JJ, Karniski W, Kromrey JD (1994): A study of multivariate permutation tests which may replace Hotelling's T2 test in prescribed circumstances. Multivariate Behav Res 29:141-163.
10. Bland JM, Altman DG (1986): Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 327:307-310.
11. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2009): Introduction to Meta-Analysis. West Sussex, UK: Wiley.
12. Brombin C, Midena E, Salmaso L (2013): Robust non-parametric tests for complex-repeated measures problems in ophthalmology. Stat Meth Med Res 22:643-660.
13. Brooks JCW, Zambreanu L, Godinez A, Craig ADB, Tracey I (2005): Somatotopic organisation of the human insula to painful heat studied with high resolution functional imaging. NeuroImage 27:201-209.
14. Brown MB (1975): A method for combining non-independent, one-sided tests of significance. Biometrics 31:987-992.
15. Calhoun VD, Sui J (2016): Multimodal fusion of brain imaging data: A key to finding the missing link(s) in complex mental illness. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging (in press). doi:10.1016/j.bpsc.2015.12.005.
16. Chang L-C, Lin H-M, Sibille E, Tseng GC (2013): Meta-analysis methods for combining multiple expression profiles: Comparisons, statistical characterization and an application guideline. BMC Bioinformatics 14:368.
17. Chen G, Adleman NE, Saad ZS, Leibenluft E, Cox RW (2014): Applications of multivariate modeling to neuroimaging group analysis: A comprehensive alternative to univariate general linear model. NeuroImage 99:571-588.
18. Chen Z (2011): Is the weighted z-test the best method for combining probabilities from independent tests? J Evol Biol 24:926-930.
19. Christensen R (2001): Advanced Linear Modelling, 2nd ed. New York, USA: Springer.
20. Darlington RB, Hayes AF (2000): Combining independent p values: Extensions of the stouffer and binomial methods. Psychol Meth 5:496-515.
21. David FN (1934): On the Pλn test for randomness: Remarks, further illustration, and table of Pλn for given values of -log10λn. Biometrika 26:1. 1-11.
22. Draper D, Gaver DP, Goel PK, Greenhouse JB, Hedges LV, Morris CN, Waternaux C (1992): Combining information: Statistical issues and opportunities for research. Washington, DC: National Academy Press.
23. Dudbridge F, Koeleman BPC (2003): Rank truncated product of P-values, with application to genomewide association scans. Gene Epidemiol 25:360-366.
24. Eaton JW Bateman D, Hauberg S, Wehbring R (2015): GNU Octave: A highlevel interactive language for numerical computations. Samurai Media Ltd, Hong Kong, PRC. Available at: https://www.gnu.org/software/octave/octave.pdf.
25. Edgington ES (1972): An additive method for combining probability values from independent experiments. J Psychol 80:351-363.
26. Efron B (2004): Large-scale simultaneous hypothesis testing. J Am Stat Assoc 99:96-104.
27. Fisher RA (1932): Statistical Methods for Research Workers, 4th ed. Edinburgh: Oliver; Boyd.
28. Fox PT, Mintun MA, Reiman EM, Raichle ME (1988): Enhanced detection of focal brain responses using intersubject averaging and change-distribution analysis of subtracted PET images. J Cerebral Blood Flow Metab 8:642-653.
29. Friston KJ, Frith CD, Liddle PF, Frackowiak RS (1991): Comparing functional (PET) images: The assessment of significant change. J Cereb Blood Flow Metab 11:690-699.
30. Friston KJ, Penny WD, Glaser DE (2005): Conjunction revisited. NeuroImage 25:661-667.
31. Genovese CR, Lazar NA, Nichols T (2002): Thresholding of statistical maps in functional neuroimaging using the false discovery rate. NeuroImage 15:870-878.
32. Good IJ (1955): On the weighted combination of significance tests. J R Stat Soc Series B 17:264-265.
33. Hall P, Wilson SR (1991): Two guidelines for bootstrap hypothesis testing. Biometrics 47:757-762.
34. Hayasaka S, Du A-T, Duarte A, Kornak J, Jahng G-H, Weiner MW, Schuff N (2006): A non-parametric approach for co-analysis of multi-modal brain imaging data: Application to alzheimer's disease. NeuroImage 30:768-779.
35. Hayasaka S, Nichols TE (2004): Combining voxel intensity and cluster extent with permutation test framework. NeuroImage 23:54-63.
36. Hayter AAJ (1986): The maximum familywise error rate of Fisher's least significant difference test. J Am Stat Assoc 81:1000-1004.
37. Hochberg Y, Tamhane AC (1987): Multiple Comparison Procedures. New York, NY: Wiley.
38. Holm S (1979): A simple sequentially rejective multiple test procedure. Scand J Stat 6:65-70.
39. Hotelling H (1951): A generalized T test and measure of multivariate dispersion. In: Neyman, J, editor. Proceedings of the second berkeley symposium on mathematical statistics and probability. Berkeley: University of California Press. 042 pp 23-41.
40. Hotelling H (1931): The generalization of Student's ratio. Ann Math Stat 2:360-378.
41. Hsu JC (1996): Multiple Comparison: Theory and Methods. Boca Raton, FL: Chapman & Hall/CRC.
42. Jiang B, Zhang X, Zuo Y, Kang G (2011): A powerful truncated tail strength method for testing multiple null hypotheses in one dataset. J Theoretical Biol 277:67-73.
43. Johnson RA, Wichern DW (2007): Applied Multivariate Statistical Analysis, 6th ed. Upper Sadle River, NJ: Pearson Prentice Hall.
44. Kost JT, McDermott MP (2002): Combining dependent p-values. Stat Probab Lett 60:183-190.
45. Kuhfeld WF (1986): A note on Roy's largest root. Psychometrika 51:479-481.
46. Lancaster HO (1961): The combination of probabilities: An application of orthonormal functions. Aus J Stat 3:20-33.
47. Lawley DN (1938): A generalization of Fisher's z test. Biometrika 30:180-187.
48. Lazar NA, Luna B, Sweeney JA, Eddy WF (2002): Combining brains: A survey of methods for statistical pooling of information. NeuroImage 16:538-550.
49. Lehmann EL, Romano JP (2005): Testing Statistical Hypotheses, 3rd ed. New York, NY: Springer.
50. Li J, Tseng GC (2011): An adaptively weighted statistic for detecting differential gene expression when combining multiple transcriptomic studies. Ann Appl Stat 5:994-1019.
51. Licata SC, Nickerson LD, Lowen SB, Trksak GH, MacLean RR, Lukas SE (2013): The hypnotic zolpidem increases the synchrony of BOLD signal fluctuations in widespread brain networks during a resting paradigm. NeuroImage 70:211-222.
52. Lipták T (1958): On the combination of independent tests. A Magyar Tudományos Akadémia Matematikai Kutató Intézetének Közlémenyei 3:171-197.
53. Loughin T (2004): A systematic comparison of methods for combining p-values from independent tests. Comput Stat Data Anal 47:467-485.
54. Marcus R, Peritz E, Gabriel KR (1976): On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63:655.
55. Meier U (2006): A note on the power of Fisher's least significant difference procedure. Pharm Stat 5:253-263.
56. Mudholkar GS, George EO (1979): The logit statistic for combining probabilities. In: Rustagi, J, editor. Symposium on Optimizing Methods in Statistics. New York: Academic Press. pp 345-366.
57. Nichols T (2012): Multiple testing corrections, nonparametric methods, and random field theory. NeuroImage 62:811-815.
58. Nichols T, Brett M, Andersson J, Wager T, Poline J-B (2005): Valid conjunction inference with the minimum statistic. NeuroImage 25:653-660.
59. Nichols T, Hayasaka S (2003): Controlling the familywise error rate in functional neuroimaging: A comparative review. Stat Meth Med Res 12:419-446.
60. Oosterhoff J (1969): Combination of One-Sided Statistical Tests. Amsterdam, The Netherlands: Mathematisch Centrum.
61. Owen AB (2009): Karl Pearson's meta-analysis revisited. Ann Stat 37:3867-3892.
62. Pantazis D, Nichols TE, Baillet S, Leahy RM (2005): A comparison of random field theory and permutation methods for the statistical analysis of MEG data. Neuroimage 25:383-394.
63. Pearson K (1933): On a method of determining whether a sample of size n supposed to have been drawn from a parent population having a known probability integral has probably been drawn at random. Biometrika 25:379-410.
64. Pesarin F (1990): On a nonparametric combination method for dependent permutation tests with applications. Psychother Psychosom 54:172-179.
65. Pesarin F (1992): A resampling procedure for nonparametric combination of several dependent tests. J Italian Stat Soc 1:87-101.
66. Pesarin F (2001): Multivariate Permutation Tests, with Applications in Biostatistics. West Sussex, England, UK: Wiley.
67. Pesarin F, Salmaso L (2010a): Permutation Tests for Complex Data: Theory, Applications and Software. West Sussex, England, UK: Wiley.
68. Pesarin F, Salmaso L (2010b): Finite-sample consistency of combination-based permutation tests with application to repeated measures designs. J Nonparametr Stat 22:669-684.
69. Petrovic P, Kalso E, Petersson KM, Ingvar M (2002): Placebo and opioid analgesia-Imaging a shared neuronal network. Science 295:1737-1740.
70. Pillai KCS (1955): Some new test criteria in multivariate analysis. The Annals of Mathematical Statistics 26:117-121.
71. Reynolds DV (1969): Surgery in the rat during electrical analgesia induced by focal brain stimulation. Science 164:444-445.
72. Rosenthal R (1978): Combining results of independent studies. Psychol Bull 85:185-193.
73. Roy M, Shohamy D, Daw N, Jepma M, Wimmer GE, Wager TD (2014): Representation of aversive prediction errors in the human periaqueductal gray. Nat Neurosci 17:1607-1612.
74. Roy SN (1953): On a heuristic method of test construction and its use in multivariate analysis. Ann Math Stat 24:220-238.
75. Scheffé H (1959): The Analysis of Variance. New York: Wiley.
76. Shaffer JP (1986): Modified sequentially rejective multiple test procedures. J Am Stat Assoc 81:826-831.
77. Smith SM, Nichols TE (2009): Threshold-free cluster enhancement: Addressing problems of smoothing, threshold dependence and localisation in cluster inference. Neuroimage 44:83-98.
78. Stouffer SA, Suchman EA, DeVinney LC, Star SA Jr, Robin MW (1949): The American Soldier: Adjustment During Army Life (Vol. 1). Princeton, NJ: Princeton University Press.
79. Šidák Z (1967): Rectangular confidence regions for the means of multivariate normal distributions. J Am Stat Assoc 62:626-633.
80. Taylor J, Tibshirani R (2006): A tail strength measure for assessing the overall univariate significance in a dataset. Biostatistics 7:167-181.
81. The MathWorks Inc. (2013): MATLAB version 8.1 (r2013a). Natick, Massachusetts.
82. Thomas AG, Dennis A, Rawlings NB, Stagg CJ, Matthews L, Morris M, Kolind SH, Foxley S, Jenkinson M, Nichols TE, Dawes H, Bandettini PA, Johansen-Berg H (2015): Multi-modal characterization of rapid anterior hippocampal volume increase associated with aerobic exercise. NeuroImage (in press).
83. Timm NH (2002): Applied Multivariate Analysis. New York: Springer.
84. Tippett LHC (1931): The Methods of Statistics. London: Williams; Northgate.
85. Tracey I, Ploghaus A, Gati JS, Clare S, Smith S, Menon RS, Matthews PM (2002): Imaging attentional modulation of pain in the periaqueductal gray in humans. J Neurosci 22:2748-2752.
86. Tukey JW (1949): Comparing individual means in the analysis of variance. Biometrics 5:99-114.
87. Uludağ K, Roebroeck A (2014): General overview on the merits of multimodal neuroimaging data fusion. NeuroImage 102:3-10.
88. Westberg M (1985): Combining independent statistical tests. Statistician 34:287-296.
89. Westfall PH, Troendle JF (2008): Multiple testing with minimal assumptions. Biom J 50:745-755.
90. Westfall PH, Young SS (1993): Resampling-Based Multiple Testing: Examples and Methods for P-Value Adjustment. New York: Wiley.
91. Whitlock MC (2005): Combining probability from independent tests: The weighted z-method is superior to Fisher's approach. J Evol Biol 18:1368-1373.
92. Wilkinson B (1951): A statistical consideration in psychological research. Psychol Bull 48:156-158.
93. Wilks SS (1932): Certain generalizations in the analysis of variance. Biometrika 24:471-494.
94. Wilson EB (1927): Probable inference, the law of succession, and statistical inference. J Am Stat Assoc 22:209-212.
95. Winer BJ (1962): Statistical Principles in Experimental Design. New York: McGraw-Hill.
96. Winkler AM, Ridgway GR, Webster MA, Smith SM, Nichols TE (2014): Permutation inference for the general linear model. NeuroImage 92:381-397.
97. Won S, Morris N, Lu Q, Elston RC (2009): Choosing an optimal method to combine p-values. Stat Med 28:1537-1553.
98. Woo C-W, Krishnan A, Wager TD (2014): Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations. NeuroImage 91:412-419.
99. Wu SS (2006): Combining univariate tests for multivariate location problem. Commun Stat 35:1483-1494.
100. Zaykin DV, Zhivotovsky LA, Westfall PH, Weir BS (2002): Truncated product method for combining p-values. Genetic Epidemiol 22:170-185.
101. Zaykin DV (2011): Optimally weighted z-test is a powerful method for combining probabilities in meta-analysis. J Evol Biol 24:1836-1841.
102. Zhu D, Zhang T, Jiang X, Hu X, Chen H, Yang N, Lv J, Han J, Guo L, Liu T (2014): Fusing DTI and fMRI data: A survey of methods and applications. NeuroImage 102:184-191.
103. Zwet W, van Oosterhoff J (1967): On the combination of independent test statistics. Ann Math Stat 38:659-680.