Permutation inference for the general linear model

NeuroImage

Volumn 92, Issue , 2014, Pages 381-397

  Winkler, Anderson M ^a,b,c   Ridgway, Gerard R ^d   Webster, Matthew A ^a   Smith, Stephen M ^a   Nichols, Thomas E ^a,e  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b GLAXOSMITHKLINE   (United Kingdom)

^c YALE UNIVERSITY SCHOOL OF MEDICINE   (United States)

^d UNIVERSITY COLLEGE LONDON   (United Kingdom)

^e UNIVERSITY OF WARWICK   (United Kingdom)

Author keywords

General linear model; Multiple regression; Permutation inference; Randomise

Indexed keywords

ARTICLE; COMPUTER SIMULATION; CONCEPTUAL FRAMEWORK; CONTROLLED STUDY; EXPERIMENTAL DESIGN; FALSE POSITIVE RESULT; IMAGE PROCESSING; INTERMETHOD COMPARISON; LEARNING ALGORITHM; NEUROIMAGING; NONPARAMETRIC TEST; PERMUTATION INFERENCE; PRIORITY JOURNAL; SAMPLE SIZE; STATISTICAL CONCEPTS; STATISTICAL DISTRIBUTION; STATISTICAL MODEL; SYSTEM ANALYSIS; ALGORITHM; ANIMAL; BRAIN; BRAIN MAPPING; HUMAN; METHODOLOGY; NERVE CELL NETWORK; PHYSIOLOGY; PROCEDURES; STATISTICAL ANALYSIS;

ALGORITHMS; ANIMALS; BRAIN; BRAIN MAPPING; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; HUMANS; LINEAR MODELS; NERVE NET; RESEARCH DESIGN;

EID: 84896520627     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2014.01.060     Document Type: Article

References (94)

1. Anderson M.J., Legendre P. An empirical comparison of permutation methods for tests of partial regression coefficients in a linear model. J. Stat. Comput. Simul. 1999, 62(3):271-303.
2. Anderson M.J., Robinson J. Permutation tests for linear models. Aust. N.Z. J. Stat. 2001, 43(1):75-88.
3. Anderson M., ter Braak C.J.F. Permutation tests for multi-factorial analysis of variance. J. Stat. Comput. Simul. 2003, 73(2):85-113.
4. Arndt S., Cizadlo T., Andreasen N.C., Heckel D., Gold S., O'Leary D.S. Tests for comparing images based on randomization and permutation methods. J. Cereb. Blood Flow Metab. 1996, 16(6):1271-1279.
5. Aspin A.A., Welch B.L. Tables for use in comparisons whose accuracy involves two variances, separately estimated. Biometrika 1949, 36(3):290-296.
6. Beaton A.E. Salvaging experiments: interpreting least squares in non-random samples. Computer Science and Statistics: Tenth Annual Symposium of the Interface 1978, 137-145. United States Department of Commerce, Gaithersburg, Maryland. D. Hogben, D. Fife (Eds.).
7. Beckmann C.F., Jenkinson M., Smith S.M. General multi-level linear modelling for group analysis in FMRI. Tech. rep. 2001, University of Oxford, Oxford.
8. Belmonte M., Yurgelun-Todd D. Permutation testing made practical for functional magnetic resonance image analysis. IEEE Trans. Med. Imaging 2001, 20(3):243-248.
9. Benjamini Y., Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B 1995, 57(1):289-300.
10. 2 test in prescribed circumstances. Multivar. Behav. Res. 1994, 29(2):141-163.
11. Box G.E.P., Andersen S.L. Permutation theory in the derivation of robust criteria and the study of departures from assumption. J. R. Stat. Soc. Ser. B 1955, 17(1):1-34.
12. Brammer M.J., Bullmore E.T., Simmons A., Williams S.C., Grasby P.M., Howard R.J., Woodruff P.W., Rabe-Hesketh S. Generic brain activation mapping in functional magnetic resonance imaging: a nonparametric approach. Magn. Reson. Imaging 1997, 15(7):763-770.
13. Breakspear M., Brammer M.J., Bullmore E.T., Das P., Williams L.M. Spatiotemporal wavelet resampling for functional neuroimaging data. Hum. Brain Mapp. 2004, 23(1):1-25.
14. Brown B.M., Maritz J.S. Distribution-free methods in regression. Aust. J. Stat. 1982, 24(3):318-331.
15. Brunner E., Munzel U. The nonparametric Behrens-Fisher problem: asymptotic theory and a small-sample approximation. Biom. J. 2000, 42(1):17-25.
16. Bullmore E., Brammer M., Williams S.C., Rabe-Hesketh S., Janot N., David A., Mellers J., Howard R., Sham P. Statistical methods of estimation and inference for functional MR image analysis. Magn. Reson. Med. 1996, 35(2):261-277.
17. Bullmore E.T., Suckling J., Overmeyer S., Rabe-Hesketh S., Taylor E., Brammer M.J. Global, voxel, and cluster tests, by theory and permutation, for a difference between two groups of structural MR images of the brain. IEEE Trans. Med. Imaging 1999, 18(1):32-42.
18. Bullmore E., Long C., Suckling J., Fadili J., Calvert G., Zelaya F., Carpenter T.A., Brammer M. Colored noise and computational inference in neurophysiological (fMRI) time series analysis: resampling methods in time and wavelet domains. Hum. Brain Mapp. 2001, 12(2):61-78.
19. Button K.S., Ioannidis J.P.A., Mokrysz C., Nosek B.A., Flint J., Robinson E.S.J., Munafò M.R. Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 2013, 14(5):365-376.
20. Cade B.S., Richards J.D. Permutation tests for least absolute deviation regression. Biometrics 1996, 52(3):886-902.
21. Christensen R. Plane Answers to Complex Questions: The Theory of Linear Models 2002, Springer, New York.
22. Chung J.H., Fraser D.A.S. Randomization tests for a multivariate two-sample problem. J. Am. Stat. Assoc. 1958, 53(283):729-735.
23. Dekker D., Krackhardt D., Snijders T.A.B. Sensitivity of MRQAP tests to collinearity and autocorrelation conditions. Psychometrika 2007, 72(4):563-581.
24. Draper N.R., Stoneman D.M. Testing for the inclusion of variables in linear regression by a randomisation technique. Technometrics 1966, 8(4):695-699.
25. Dwass M. Modified randomization tests for nonparametric hypotheses. Ann. Math. Stat. 1957, 28(1):181-187.
26. Edgington E.S. Approximate randomization tests. J. Psychol. 1969, 72(2):143-149.
27. Edgington E.S. Randomization Tests 1995, Marcel Dekker, New York.
28. Efron B. Computers and the theory of statistics: thinking the unthinkable. SIAM Rev. 1979, 21(4):460-480.
29. Ernst M.D. Permutation methods: a basis for exact inference. Stat. Sci. 2004, 19(4):676-685.
30. Fisher R.A. The Design of Experiments 1935, Oliver and Boyd, Edinburgh.
31. Fisher R.A. The fiducial argument in statistical inference. Ann. Eugen. 1935, 6(4):391-398.
32. Freedman D., Lane D. A nonstochastic interpretation of reported significance levels. J. Bus. Econ. Stat. 1983, 1(4):292-298.
33. Gail M.H., Tan W.Y., Piantadosi S. Tests for no treatment effect in randomized clinical trials. Biometrika 1988, 75(1):57-64.
34. Genovese C.R., Lazar N.A., Nichols T. Thresholding of statistical maps in functional neuroimaging using the false discovery rate. NeuroImage 2002, 15(4):870-878.
35. Gonzalez L., Manly B.F.J. Analysis of variance by randomization with small data sets. Environmetrics 1998, 9(1):53-65.
36. Good P. Extensions of the concept of exchangeability and their applications. J. Mod. Appl. Stat. Methods 2002, 1(2):243-247.
37. Good P. Permutation, Parametric, and Bootstrap Tests of Hypotheses 2005, Springer, New York.
38. Guttman I. Linear Models: An Introduction 1982, Wiley, New York.
39. Hall P., Wilson S.R. Two guidelines for bootstrap hypothesis testing. Biometrics 1991, 47(2):757-762.
40. Hayasaka S., Phan K.L., Liberzon I., Worsley K.J., Nichols T.E. Nonstationary cluster-size inference with random field and permutation methods. NeuroImage 2004, 22(2):676-687.
41. Holmes A.P., Blair R.C., Watson J.D., Ford I. Nonparametric analysis of statistic images from functional mapping experiments. J. Cereb. Blood Flow Metab. 1996, 16(1):7-22.
42. Horn S.D., Horn R.A., Duncan D.B. Estimating heteroscedastic variances in linear models. J. Am. Stat. Assoc. 1975, 70(350):380-385.
43. Huh M.H., Jhun M. Random permutation testing in multiple linear regression. Commun. Stat. Theory Methods 2001, 30(10):2023-2032.
44. James G. The comparison of several groups of observations when the ratios of the population variances are unknown. Biometrika 1951, 38(3):324-329.
45. Jung B.C., Jhun M., Song S.H. A new random permutation test in ANOVA models. Stat. Pap. 2006, 48(1):47-62.
46. Kempthorne O. The randomization theory of experimental inference. J. Am. Stat. Assoc. 1955, 50(271):946-967.
47. Kennedy P.E. Randomization tests in econometrics. J. Bus. Econ. Stat. 1995, 13(1):85-94.
48. Kennedy P.E., Cade B.S. Randomization tests for multiple regression. Commun. Stat. Simul. 1996, 25:923-936.
49. Kherad-Pajouh S., Renaud O. An exact permutation method for testing any effect in balanced and unbalanced fixed effect ANOVA. Comput. Stat. Data Anal. 2010, 54(7):1881-1893.
50. Knuth D.E. The art of computer programming. Fascicle 2 2005, vol. 4. Addison-Wesley.
51. Laird A.R., Rogers B.P., Meyerand M.E. Comparison of Fourier and wavelet resampling methods. Magn. Reson. Med. 2004, 51(2):418-422.
52. Lehmann E., Stein C. On the theory of some non-parametric hypotheses. Ann. Math. Stat. 1949, 20(1):28-45.
53. Levin B., Robbins H. Urn models for regression analysis, with applications to employment discrimination studies. Law Contemp. Probl. 1983, 46(4):247-267.
54. Locascio J.J., Jennings P.J., Moore C.I., Corkin S. Time series analysis in the time domain and resampling methods for studies of functional magnetic resonance brain imaging. Hum. Brain Mapp. 1997, 5(3):168-193.
55. Ludbrook J., Dudley H. Why permutation tests are superior to t and F tests in biomedical research. Am. Stat. 1998, 52(2):127-132.
56. Manly B.F.J. Randomization and regression methods for testing for associations with geographical, environmental and biological distances between populations. Res. Popul. Ecol. 1986, 28(2):201-218.
57. Manly B.F.J. Randomization, Bootstrap and Monte Carlo Methods in Biology 2007, Chapman & Hall, London. 3rd edition.
58. Marroquin J.L., Biscay R.J., Ruiz-Correa S., Alba A., Ramirez R., Armony J.L. Morphology-based hypothesis testing in discrete random fields: a non-parametric method to address the multiple-comparison problem in neuroimaging. NeuroImage 2011, 56(4):1954-1967.
59. Nichols T.E., Holmes A.P. Nonparametric permutation tests for functional neuroimaging: a primer with examples. Hum. Brain Mapp. 2002, 15(1):1-25.
60. Nichols T.E., Ridgway G.R., Webster M.G., Smith S.M. GLM permutation: nonparametric inference for arbitrary general linear models. NeuroImage 2008, 41(S1):S72.
61. O'Gorman T.W. The performance of randomization tests that use permutations of independent variables. Commun. Stat. Simul. Comput. 2005, 34(4):895-908.
62. Oja H. On permutation tests in multiple regression and analysis of covariance problems. Aust. J. Stat. 1987, 29(1):91-100.
63. Pantazis D., Nichols T.E., Baillet S., Leahy R.M. A comparison of random field theory and permutation methods for the statistical analysis of MEG data. NeuroImage 2005, 25(2):383-394.
64. Pearson E.S. Some aspects of the problem of randomization. Biometrika 1937, 29(1/2):53-64.
65. Peirce C.S., Jastrow J. On small differences of sensation. Mem. Natl. Acad. Sci. 1884, 3:75-83.
66. Pesarin F. A new solution for the generalized Behrens-Fisher problem. Statistica 1995, 55(2):131-146.
67. Pesarin F. Multivariate Permutation Tests: With Applications in Biostatistics 2001, John Wiley and Sons, West Sussex, England, UK.
68. Pesarin F., Salmaso L. Permutation Tests for Complex Data: Theory, Applications and Software 2010, John Wiley and Sons, West Sussex, England, UK.
69. Phipson B., Smyth G.K. Permutation P-values should never be zero: calculating exact P-values when permutations are randomly drawn. Stat. Appl. Genet. Mol. Biol. 2010, 9(1). (Article39).
70. Pitman E.J.G. Significance tests which may be applied to samples from any populations. Suppl. J. R. Stat. Soc. 1937, 4(1):119-130.
71. Pitman E.J.G. Significance tests which may be applied to samples from any populations. II. The correlation coefficient test. Suppl. J. R. Stat. Soc. 1937, 4(2):225-232.
72. Pitman E.J.G. Significance tests which may be applied to samples from any populations: III. The analysis of variance test. Biometrika 1938, 29(3/4):322-335.
73. Press W.H., Teukolsky S.A., Vetterling W.T., Flannery B.P. Numerical Recipes in C 1992, Cambridge University Press, Cambridge, UK.
74. Ridgway G.R. Statistical Analysis for Longitudinal MR Imaging of Dementia 2009, University College London, (Ph.D. thesis).
75. Rorden C., Bonilha L., Nichols T.E. Rank-order versus mean based statistics for neuroimaging. NeuroImage 2007, 35(4):1531-1537.
76. Salimi-Khorshidi G., Smith S.M., Nichols T.E. Adjusting the effect of nonstationarity in cluster-based and TFCE inference. NeuroImage 2011, 54(3):2006-2019.
77. Scheffé H. Statistical inference in the non-parametric case. Ann. Math. Stat. 1943, 14(4):305-332.
78. Scheffé H. The Analysis of Variance 1959, John Wiley and Sons, New York.
79. Searle S.R. Linear Models 1971, John Wiley and Sons, New York.
80. Sen P.K. Estimates of the regression coefficient based on Kendall's tau. J. Am. Stat. Assoc. 1968, 63(324):1379-1389.
81. Smith S.M., Nichols T.E. Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localisation in cluster inference. NeuroImage 2009, 44(1):83-98.
82. Smith S., Jenkinson M., Beckmann C., Miller K., Woolrich M. Meaningful design and contrast estimability in FMRI. NeuroImage 2007, 34(1):127-136.
83. Still A.W., White A.P. The approximate randomization test as an alternative to the F test in analysis of variance. Br. J. Math. Stat. Psychol. 1981, 34(2):243-252.
84. Suckling J., Bullmore E. Permutation tests for factorially designed neuroimaging experiments. Hum. Brain Mapp. 2004, 22(3):193-205.
85. ter Braak C.J.F. Permutation versus bootstrap significance tests in multiple regression and ANOVA. Bootstrapping and Related Techniques. No. 1989 1992, 79-86. Springer-Verlag, Berlin. K.-H. Jöckel, G. Rothe, W. Sendler (Eds.).
86. Theil H. A rank-invariant method for linear and polynomial regression. I. II. III. Proceedings of the Section of Sciences, Koninklijke Akademie van Wetenschappen te Amsterdam 1950, 53:386-392. (521-525, 1397-1412).
87. Trotter H.F., Tukey J.W. Conditional Monte Carlo techniques in a complex problem about normal samples. Symposium on Monte Carlo Methods 1956, 64-79. Wiley, New York. H.A. Meyer (Ed.).
88. Welch B.L. On the comparison of several mean values: an alternative approach. Biometrika 1951, 38(3):330-336.
89. Welch W.J. Construction of permutation tests. J. Am. Stat. Assoc. 1990, 85(411):693-698.
90. Westfall P.H., Troendle J.F. Multiple testing with minimal assumptions. Biom. J. 2008, 50(5):745-755.
91. Westfall P.H., Young S.S. Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment 1993, John Wiley and Sons, New York.
92. Wilson E.B. Probable inference, the law of succession, and statistical inference. J. Am. Stat. Assoc. 1927, 22(158):209-212.
93. Winkler A.M., Sabuncu M.R., Yeo B.T.T., Fischl B., Greve D.N., Kochunov P., Nichols T.E., Blangero J., Glahn D.C. Measuring and comparing brain cortical surface area and other areal quantities. NeuroImage 2012, 61(4):1428-1443.
94. Yekutieli D., Benjamini Y. Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics. J. Stat. Plann. Infer. 1999, 82(1-2):171-196.