Permutation methods

Wiley Interdisciplinary Reviews: Computational Statistics

Volumn 3, Issue 6, 2011, Pages 527-542

  Berry, Kenneth J ^a   Johnston, Janis E ^b   Mielke, Paul W ^a  

^a Veterinary Dentistry and Oral Surgery   (United States)

^b U S DEPARTMENT OF AGRICULTURE   (United States)

Author keywords

Euclidean distance; Exact tests; Permutation tests; Resampling

Indexed keywords

EUCLIDEAN DISTANCE; EXACT TESTS; MOMENT APPROXIMATION; PERMUTATION TESTS; POPULATION MODEL; RESAMPLING; SQUARED EUCLIDEAN DISTANCE; STATISTICAL TESTING;

WIRE;

STATISTICAL TESTS;

EID: 84874974881     PISSN: 19395108     EISSN: 19390068     Source Type: Journal    
DOI: 10.1002/wics.177     Document Type: Article

References (226)

1. Bakeman R, Robinson BF, Quera V. Testing sequential association: estimating exact p values using sampled permutations. Psychol Methods 1996, 1:4-15.
2. Kempthorne O. Some aspects of experimental inference. J Am Stat Assoc 1966, 61:11-34.
3. Read TRC, Cressie NAC. Goodness-of-Fit for Discrete Multivariate Data. New York: Springer-Verlag; 1988.
4. Kennedy PE. Randomization tests in econometrics. J Bus Econ Stat 1995, 13:85-94.
5. Ludbrook J. Advantages of permutation (randomization) tests in clinical and experimental pharmacology and physiology. Clin Exp Pharmacol Physiol 1994, 21:673-686.
6. Neyman J, Pearson ES. On the use and interpretation of certain test criteria for purposes of statistical inference: part I. Biometrika 1928, 20A:175-240.
7. Neyman J, Pearson ES. On the use and interpretation of certain test criteria for purposes of statistical inference: part II. Biometrika 1928, 20A:263-294.
8. Fisher RA. Statistical Methods for Research Workers. Edinburgh: Oliver and Boyd; 1925.
9. Geary RC. Some properties of correlation and regression in a limited universe. Metron Riv Int Stat 1927, 7:83-119.
10. Eden T, Yates F. On the validity of Fisher's z test when applied to an actual example of non-normal data. J Agric Sci 1933, 23:6-17.
11. Pitman EJG. Significance tests which may be applied to samples from any populations. Suppl J R Stat Soc 1937, 4:119-130.
12. Pitman EJG. Significance tests which may be applied to samples from any populations: II. The correlation coefficient test. Suppl J R Stat Soc 1937, 4:225-232.
13. Pitman EJG. Significance tests which may be applied to samples from any populations: III. The analysis of variance test. Biometrika 1938, 29:322-335.
14. Dwass M. Modified randomization tests for nonparametric hypotheses. Ann Math Stat 1957, 28:181-187.
15. Hope ACA. A simplified Monte Carlo significance test procedure. J R Stat Soc B 1968, 30:582-598.
16. Gabriel KR, Hall WJ. Rerandomization inference on regression and shift effects: computationally feasible methods. J Am Stat Assoc 1983, 78:827-836.
17. Berry KJ, Mielke PW, Mielke HW. The Fisher-Pitman permutation test: an attractive alternative to the F test. Psychol Rep 2002, 90:495-502.
18. Edgington ES, Onghena P. Randomization Tests. 4th ed. Boca Raton, FL: Chapman & Hall/CRC; 2007.
19. Bear G. Computationally intensive methods warrant reconsideration of pedagogy in statistics. Behav Res Methods Instrum Comput 1995, 27:144-147.
20. Frick RW. Interpreting statistical testing: process and propensity, not population and random sampling. Behav Res Methods Instrum Comput 1998, 30:527-535.
21. Ludbrook J, Dudley H. Why permutation tests are superior to t and F tests in biomedical research. Am Stat 1998, 52:127-132.
22. Holford TR. Editorial: exact methods for categorical data. Stat Methods Med Res 2003, 12:1.
23. Mielke PW, Berry KJ. Permutation Methods: A Distance Function Approach. 2nd ed New York: Springer-Verlag; 2007.
24. Mielke PW, Berry KJ, Johnston JE. Robustness without rank order statistics. J Appl Stat 2011, 38:207-214.
25. Romano JP. Bootstrap and randomization tests of some nonparametric hypotheses. Ann Stat 1989, 17:141-159.
26. Efron B, Tibshirani RJ. An Introduction to the Bootstrap. Boca Raton, FL: Chapman & Hall/CRC; 1993.
27. Hall P, Wilson SR. Two guidelines for bootstrap hypothesis testing. Biometrics 1991, 47:757-762.
28. Westfall PH, Young SS. Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment. New York: John Wiley & Sons; 1993.
29. Splawa-Neyman J. Próba uzasadnienia zastosowań rachunku prawdopodobieństwa do doświadczeń polowych (On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Rocz Nauk Rolniczych 1923, 10:1-51. Translated by Dabrowska DM and Speed TP in Stat Sci 1990, 5:465-472.
30. Fisher RA. The logic of inductive inference. J R Stat Soc 1935, 98:39-82.
31. Fisher RA. The Design of Experiments. Edinburgh: Oliver and Boyd; 1935.
32. Conover WJ. Practical Nonparametric Statistics. 3rd ed. New York: John Wiley & Sons; 1999.
33. Kempthorne O. The randomization theory of experimental inference. J Am Stat Assoc 1955, 50:946-967.
34. Kruskal WH, Wallis WA. Use of ranks in one-criterion variance analysis. J Am Stat Assoc 1952, 47:583-621. Erratum in: J Am Stat Assoc 1953, 48:907-911.
35. Wald A, Wolfowitz J. Statistical tests based on permutations of the observations. Ann Math Stat 1944, 15:358-372.
36. Box JF. R. A. Fisher: The Life of a Scientist. New York: John Wiley & Sons; 1978.
37. Salsburg D. The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century. New York: Henry Holt; 2001.
38. Darwin C. The Effects of Cross and Self Fertilization in the Vegetable Kingdom. London: John Murray; 1876.
39. Fisher RA. 'The coefficient of racial likeness' and the future of craniometry. J R Anthropol Inst 1936, 66:57-63.
40. Hotelling H, Pabst MR. Rank correlation and tests of significance involving no assumption of normality. Ann Math Stat 1936, 7:29-43.
41. Welch BL. On the z-test in randomized blocks and Latin squares. Biometrika 1937, 29:21-52.
42. Pearson ES. Some aspects of the problem of randomization. Biometrika 1937, 29:53-64.
43. Edgington ES. Randomization Tests. 3rd ed. New York: Marcel Dekker; 1995.
44. Welch BL. On tests for homogeneity. Biometrika 1938, 30:149-158.
45. McCarthy MD. On the application of the z-test to randomized blocks. Ann Math Stat 1939, 10:337-359.
46. Kendall MG. A new measure of rank correlation. Biometrika 1938, 30:81-93.
47. Kendall MG, Kendall SFH, Babington Smith B. The distribution of Spearman's coefficient of rank correlation in a universe in which all rankings occur an equal number of times. Biometrika 1939, 30:251-273.
48. Olds EG. Distribution of sums of squares of rank differences for small numbers of individuals. Ann Math Stat 1938, 9:133-148.
49. Spearman C. The proof and measurement of association between two things. Am J Psychol 1904, 15:72-101.
50. Spearman C. 'Footrule' for measuring correlation. Br J Psychol 1906, 2:89-108.
51. Kendall MG. Rank Correlation Methods. London: Griffin; 1948.
52. Friedman M. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 1937, 32:675-701.
53. Friedman M. A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 1940, 11:86-92.
54. Kendall MG, Babington Smith B. The problem of m rankings. Ann Math Stat 1939, 10:275-287.
55. Savage IR. Nonparametric statistics. J Am Stat Assoc 1957, 52:331-344.
56. Wilcoxon F. Individual comparisons by ranking methods. Biometrics Bull 1945, 1:80-83.
57. Mann HB, Whitney DR. On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 1947, 18:50-60.
58. Festinger L. The significance of differences between means without reference to the frequency distribution function. Psychometrika 1946, 11:97-105.
59. Wald A, Wolfowitz J. On a test whether two samples are from the same population. Ann Math Stat 1940, 11:147-162.
60. Jonckheere AR. A distribution-free k-sample test against ordered alternatives. Biometrika 1954, 41:133-145.
61. Terpstra TJ. The asymptotic normality and consistency of Kendall's test against trend, when ties are present in one ranking. Indagationes Math 1952, 14:327-333.
62. Mann HB. Nonparametric tests against trend. Econometrica 1945, 13:245-259.
63. Mood AM. On the asymptotic efficiency of certain nonparametric two-sample tests. Ann Math Stat 1954, 25:514-522.
64. Swed FS, Eisenhart C. Tables for testing randomness of grouping in a sequence of alternatives. Ann Math Stat 1943, 14:66-87.
65. Finney DJ. The FisherYates test of significance in 2×2 contingency tables. Biometrika 1948, 35:145-156.
66. David ST, Kendall MG, Stuart A. Some questions of distribution in the theory of rank correlation. Biometrika 1951, 38:131-140.
67. Wilcoxon F. Probability tables for individual comparisons by ranking methods. Biometrics 1947, 3:119-122.
68. Fix E, Hodges JL. Significance probabilities of the Wilcoxon test. Ann Math Stat 1955, 26:301-312.
69. White C. The use of ranks in a test of significance for comparing two treatments. Biometrics 1952, 8:33-41.
70. Van der Reyden D. A simple statistical significance test. Rhod Agric J 1952, 49:96-104.
71. Auble D. Extended tables for the Mann-Whitney statistic. Bull Inst Educ Res 1953, 1:1-39.
72. Scheffé H. Statistical inference in the non-parametric case. Ann Math Stat 1943, 14:305-332.
73. Wald A, Wolfowitz J. An exact test for randomness in the non-parametric case based on serial correlation. Ann Math Stat 1943, 14:378-388.
74. Pitman EJG. Lecture notes on nonparametric statistical inference. Unpublished notes; 1948.
75. Wolfowitz J. Non-parametric statistical inference. In: Neyman J, ed. Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability. Berkeley, CA: University of California Press; 1949, 93-113.
76. Freeman GH, Halton JH. Note on an exact treatment of contingency, goodness of fit and other problems of significance. Biometrika 1951, 38:141-149.
77. Hoeffding W. The large-sample power of tests based on permutations of observations. Ann Math Stat 1952, 23:169-192.
78. Silvey SD. The equivalence of asymptotic distributions under randomisation and normal theories. Proc Glasgow Math Assoc 1955, 1:139-147.
79. Tompkins CB. Machine attacks on problems whose variables are permutations. In: Curtiss JH, ed. Numerical Analysis, volume 6 of Proceedings of Symposia in Applied Mathematics. New York: McGraw-Hill; 1956, 195-211.
80. Gregory RJ. A FORTRAN computer program for the Fisher exact probability test. Educ Psychol Meas 1973, 33:697-700.
81. Tritchler DL, Pedrini DT. A computer program for Fisher's exact probability test. Educ Psychol Meas 1975, 35:717-719.
82. Agresti A, Wackerly D. Some exact conditional tests of independence for R×C cross-classification tables. Psychometrika 1977, 42:111-125.
83. Agresti A, Wackerly D, Boyett JM. Exact conditional tests for cross-classifications: approximation of attained significance levels. Psychometrika 1979, 44:75-83.
84. Fleishman AI. A program for calculating the exact probability along with explorations of M by N contingency tables. Educ Psychol Meas 1977, 37:799-803.
85. Howell DC, Gordon LR. Computing the exact probability of an r by c contingency table with fixed marginal totals. Behav Res Methods Instrum 1976, 8:317.
86. March DL. Exact probabilities for R×C contingency tables (Algorithm 434). Commun ACM 1972, 15:991-992.
87. Arbuckle J, Aiken LS. A program for Pitman's permutation test for differences in location. Behav Res Methods Instrum 1975, 7:381.
88. Baker FB, Collier RO. Some empirical results on variance ratios under permutation in the completely randomized design. J Am Stat Assoc 1966, 61:813-820.
89. Collier RO, Baker FB. Some Monte Carlo results on power of F tests under permutation in simple randomized block design. Biometrika 1966, 53:199-203.
90. 2 test of goodness of fit. J Am Stat Assoc 1975, 70:811-813.
91. Klotz J, Teng J. One-way layout for counts and the exact enumeration of the Kruskal-Wallis H distribution with ties. J Am Stat Assoc 1977, 72:165-169.
92. Mielke PW. Asymptotic behavior of two-sample tests based on powers of ranks for detecting scale and location alternatives. J Am Stat Assoc 1972, 67:850-854.
93. Mielke PW, Berry KJ. An extended class of matched pairs tests based on powers of ranks. Psychometrika 1976, 41:89-100.
94. Edgington ES. Statistical Inference: The Distribution-free Approach. New York: McGraw-Hill; 1969.
95. Boyett JM. R×C tables with given row and column totals (Algorithm 144). Appl Stat 1979, 28:329-332.
96. Mielke PW, Berry KJ, Johnson ES. Multi-response permutation procedures for a priori classifications. Commun Stat Theory Methods 1976, 5:1409-1424.
97. Mielke PW. Multiresponse permutation procedures. In: Encyclopedia of Statistical Sciences. Vol. V. New York: John Wiley & Sons, 1985, 724-727.
98. Mielke PW. On asymptotic non-normality of null distributions of MRPP statistics. Commun Stat Theory Methods 1979, 8:1541-1550. Errata in: Commun Stat Theory Methods 1981, 10:1795 and 1982, 11:847.
99. Tukey JW. Some sampling simplified. J Am Stat Assoc 1950, 45:501-519.
100. Mielke PW, Sen PK. On asymptotic non-normal null distributions for locally most powerful rank test statistics. Commun Stat Theory Methods 1981, 10:1079-1094.
101. Edgington ES. Randomization Tests. New York: Marcel Dekker; 1980.
102. Edgington ES. Randomization Tests. 2nd ed. New York: Marcel Dekker; 1987.
103. Hubert L. Assignment Methods in Combinatorial Data Analysis. New York: Marcel Dekker; 1987.
104. Good PI. Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses. New York: Springer-Verlag; 1994.
105. Good PI. Permutation, Parametric and Bootstrap Tests of Hypotheses. New York: Springer-Verlag; 1994.
106. Manly BFJ. Randomization and Monte Carlo Methods in Biology. London: Chapman & Hall; 1991.
107. Manly BFJ. Randomization and Monte Carlo Methods in Biology. 2nd ed. London: Chapman & Hall; 1997.
108. Weerahandi S. Exact Statistical Methods for Data Analysis. New York: Springer-Verlag; 1995.
109. Good PI. Resampling Methods: A Practical Guide to Data Analysis. Boston, MA: Birkhäuser; 1999.
110. Verbeek A, Kroonenberg PM. A survey of algorithms for exact distributions of test statistics in r × c contingency tables with fixed margins. Comput Stat Data Anal 1985, 3:159-185.
111. Balmer DW. Recursive enumeration of r×c tables for exact likelihood evaluation (Algorithm 236). Appl Stat 1988, 37:290-301.
112. Dallal GE. PITMAN: a FORTRAN program for exact randomization tests. Comput Biomed Res 1988, 21:9-15.
113. Thakur AK, Berry KJ, Mielke PW. A FORTRAN program for testing trend and homogeneity in proportions. Comput Programs Biomed 1985, 19:229-233.
114. Berry KJ, Mielke PW. Goodman and Kruskal's tau-b statistic: a nonasymptotic test of significance. Sociol Methods Res 1985, 13:543-550.
115. Berry KJ, Mielke PW. Exact chi-square and Fisher's exact probability test for 3 by 2 cross-classification tables. Educ Psychol Meas 1987, 47:631-636.
116. Berry KJ, Mielke PW. Exact cumulative probabilities for the multinomial distribution. Educ Psychol Meas 1995, 55:769-772.
117. Berry KJ, Mielke PW, Helmericks SG. Exact confidence limits for proportions. Educ Psychol Meas 1988, 48:713-716.
118. Berry KJ, Mielke PW, Helmericks SG. An algorithm to generate discrete probability distributions: binomial, hypergeometric, negative binomial, inverse hypergeometric, and Poisson. Behav Res Methods Instrum Comput 1994, 26:366-367.
119. Mehta CR, Patel NR. A network algorithm for the exact treatment of the 2×k contingency table. Commun Stat Simul Comput 1980, 9:649-664.
120. Pagano M, Taylor Halvorsen K. An algorithm for finding the exact significance levels of r×c contingency tables. J Am Stat Assoc 1981, 76:931-934.
121. Mehta CR, Patel NR. A network algorithm for performing Fisher's exact test in r × c contingency tables. J Am Stat Assoc 1983, 78:427-434.
122. Berry KJ, Mielke PW. Goodman and Kruskal's tau-b statistic: a FORTRAN-77 subroutine. Educat Psychol Meas 1986, 46:646-649.
123. Berry KJ, Mielke PW. R by C chi-square analyses of small expected cell frequencies. Educ Psychol Meas 1986, 46:169-173.
124. Berry KJ, Mielke PW. Monte Carlo comparisons of the asymptotic chi-square and likelihood-ratio tests with the nonasymptotic chi-square test for sparse R by C tables. Psychol Bull 1988, 103:256-264.
125. Patefield WM. An efficient method of generating random r×c tables with given row and column totals (Algorithm 159). Appl Stat 1981, 30:91-97.
126. Saunders IA. Enumeration of R×C tables with repeated row totals (Algorithm 205). Appl Stat 1984, 33:340-352.
127. Mielke PW, Berry KJ. Non-asymptotic inferences based on the chi-square statistic for r by c contingency tables. J Stat Plann Infer 1985, 12:41-45.
128. Mielke PW, Berry KJ. Fisher's exact probability test for cross-classification tables. Educ Psychol Meas 1992, 52:97-101.
129. Baglivo J, Olivier D, Pagano M. Methods for the analysis of contingency tables with large and small cell counts. J Am Stat Assoc 1988, 83:106-113.
130. Kreiner S. Analysis of multidimensional contingency tables by exact conditional tests: techniques and strategies. Scand J Stat 1987, 14:97-112.
131. Mielke PW, Berry KJ. Cumulant methods for analyzing independence of r-way contingency tables and goodness-of-fit frequency data. Biometrika 1988, 75:790-793.
132. Mielke PW, Berry KJ. Nonasymptotic probability values for Cochran's Q statistic: a FORTRAN 77 program. Percept Mot Skills 1996, 82:303-306.
133. Mielke PW, Berry KJ. Exact probabilities for first-order, second-order, and third-order interactions in 2×2×2×2 contingency tables. Percept Mot Skills 1998, 86:760-762.
134. Berry KJ, Mielke PW. Analyzing independence in r-way contingency tables. Educ Psychol Meas 1989, 49:605-607.
135. Mielke PW, Berry KJ, Zelterman D. Fisher's exact test of mutual independence for 2×2×2 cross-classification tables. Educ Psychol Meas 1994, 54:110-114.
136. Zelterman D, Chan IS, Mielke PW. Exact tests of significance in higher dimensional tables. Am Stat 1995, 49:357-361.
137. Berry KJ, Mielke PW. Moment approximations as an alternative to the F test in analysis of variance. Br J Math Stat Psychol 1983, 36:202-206.
138. Berry KJ, Mielke PW. A rapid FORTRAN subroutine for the Fisher exact probability test. Educ Psychol Meas 1983, 43:167-171.
139. Berry KJ, Mielke PW. Subroutines for computing exact chi-square and Fisher's exact probability tests. Educ Psychol Meas 1985, 45:153-159.
140. Mielke PW, Berry KJ. Asymptotic clarifications, generalizations, and concerns regarding an extended class of matched pairs tests based on powers of ranks. Psychometrika 1983, 48:483-485.
141. Mielke PW, Berry KJ. An exact solution to an occupancy problem: a useful alternative to Cochran's Q test. Percept Mot Skills 1996, 82:91-95.
142. Zimmermann H. Exact calculation of permutational distributions for two dependent samples. Biom J 1985, 3:349-352.
143. Zimmermann H. Exact calculation of permutational distributions for two independent samples. Biom J 1985, 4:431-434.
144. Mielke PW, Berry KJ, Eighmy JL. A permutation procedure for comparing archaeomagnetic polar directions. In: Eighmy JL, Sternberg RS, eds. Archaeomagnetic Dating. Tucson, AZ: University of Arizona Press; 1991, 102-108.
145. Berry KJ, Mielke PW, Kvamme KL. Efficient permutation procedures for analysis of artifact distributions. In: Hietala HJ, ed. Intrasite Spatial Analysis in Archaeology. Cambridge: Cambridge University Press; 1984, 54-74.
146. Zimmerman GM, Goetz H, Mielke PW. Use of an improved statistical method for group comparisons to study effects of prairie fire. Ecology 1985, 66:606-611.
147. Biondini ME, Mielke PW, Berry KJ. Data-dependent permutation techniques for the analysis of ecological data. Vegetatio 1988, 75:161-168.
148. Mielke PW, Berry KJ, Brier GW. Application of multi-response permutation procedures for examining seasonal changes in monthly mean sea-level pressure patterns. Mon Weather Rev 1981, 109:120-126.
149. Mielke PW. Meteorological applications of permutation techniques based on distance functions. In: Krishnaiah PR, Sen PK, eds. Handbook of Statistics. Vol. 4. Amsterdam: North-Holland; 1984, 813-830.
150. Mielke PW. Geometric concerns pertaining to applications of statistical tests in the atmospheric sciences. J Atmos Sci 1985, 42:1209-1212.
151. Wong RKW, Chidambaram N, Mielke PW. Application of multi-response permutation procedures and median regression for covariate analyses of possible weather modification effects on hail responses. Atmosphere-Ocean 1983, 21:1-13.
152. Tucker DF, Mielke PW, Reiter ER. The verification of numerical models with multivariate randomized block permutation procedures. Meteorol Atmos Phys 1989, 40:181-188.
153. Lee TJ, Pielke RA, Mielke PW. Modeling the clear-sky surface energy budget during FIFE 1987. J Geophys Res 1995, 100:25585-25593.
154. Kelly FP, Vonder Haar TH, Mielke PW. Imagery randomized block analysis (IRBA) applied to the verification of cloud edge detectors. J Atmos Oceanic Technol 1989, 6:671-679.
155. Cotton WR, Thompson G, Mielke PW. Realtime mesoscale prediction on workstations. Bull Am Meteorol Soc 1994, 75:349-362.
156. Mielke PW, Berry KJ, Landsea CW, Gray WM. Artificial skill and validation in meteorological forecasting. Weather Forecast 1996, 11:153-169.
157. Mielke PW, Berry KJ, Landsea CW, Gray WM. A single-sample estimate of shrinkage in meteorological forecasting. Weather Forecast 1997, 12:847-858.
158. Mehta CR, Patel NR. A hybrid algorithm for Fisher's exact test in unordered r×c contingency tables. Commun Stat Theory Methods 1986, 15:387-403.
159. Mehta CR, Patel NR. FEXACT: a FORTRAN subroutine for Fisher's exact test on unordered r×c contingency tables (Algorithm 643). ACM Trans Math Softw 1986, 12:154-161.
160. Joe H. Extreme probabilities for contingency tables under row and column independence with application to Fisher's exact test. Commun Stat Theory Methods 1988, 17:3677-3685.
161. Zar JH. A fast and efficient algorithm for the Fisher exact test. Behav Res Methods Instrum Comput 1987, 19:413-414.
162. Romesburg HC, Marshall K, Mauk TP. FITEST: a computer program for "exact chi-square" goodness-of-fit significance tests. Comput Geosci 1981, 7:47-58.
163. Mielke PW, Berry KJ. Exact goodness-of-fit probability tests for analyzing categorical data. Educat Psychol Meas 1993, 53:707-710.
164. Tritchler DL. An algorithm for exact logistic regression. J Am Stat Assoc 1984, 79:709-711.
165. Vollset SE, Hirji KF. A microcomputer program for exact and asymptotic analysis of several 2 × 2 tables. Epidemiology 1991, 2:217-220.
166. Vollset SE, Hirji KF, Elashoff RM. Fast computation of exact confidence limits for the common odds ratio in a series of 2×2 tables. J Am Stat Assoc 1991, 86:404-409.
167. Berry KJ, Mielke PW. Simulated power comparisons of the asymptotic and nonasymptotic Goodman and Kruskal tau tests for sparse R by C tables. In: Srivastava JN, ed. Probability and Statistics: Essays in Honor of Franklin A. Graybill. Amsterdam: North-Holland; 1988, 9-19.
168. Berry KJ, Mielke PW. A generalization of Cohen's kappa agreement measure to interval measurement and multiple raters. Educ Psychol Meas 1988, 48:921-933.
169. Hirji KF, Mehta CR, Patel NR. Computing distributions for exact logistic regression. J Am Stat Assoc 1987, 82:1110-1117.
170. Baker RD, Tilbury JB. Rapid computation of the permutation paired and grouped t-tests (Algorithm 283). Appl Stat 1993, 42:432-441.
171. Chen RS, Dunlap WP. SAS procedures for approximate randomization tests. Behav Res Methods Instrum Comput 1993, 25:406-409.
172. Edgington ES, Khuller PLV. A randomization test computer program for trends in repeated-measures data. Educ Psychol Meas 1992, 52:93-95.
173. Mielke PW, Berry KJ. Nonasymptotic inferences based on Cochran's Q test. Percept Mot Skills 1995, 81:319-322.
174. Mehta CR, Patel NR, Senchaudhuri P. Exact power and sample-size computations for the Cochran-Armitage trend test. Biometrics 1998, 54:1615-1621.
175. Berry KJ, Mielke PW. Computation of exact and approximate probability values for a matched-pairs permutation test. Commun Stat Simul Comput 1985, 14:229-248.
176. Berry KJ, Mielke PW. Analysis of multivariate matched-pairs data: a FORTRAN 77 program. Percept Mot Skills 1996, 83:788-790.
177. Brockwell PJ, Mielke PW. Asymptotic distributions of matched-pairs permutation statistics based on distance measures. Aust J Stat 1984, 26:30-38.
178. Mielke PW, Berry KJ. A extended class of permutation techniques for matched pairs. Commun Stat Theory Methods 1982, 11:1197-1207.
179. 2 test. Psycholo Rep 1996, 78:1003-1008.
180. Berry KJ, Kvamme KL, Mielke PW. A permutation technique for the spatial analysis of artifacts into classes. Am Antiq 1980, 45:55-59.
181. Berry KJ, Kvamme KL, Mielke PW. Improvements in the permutation test for the spatial analysis of the distribution of artifacts into classes. Am Antiq 1983, 48:547-553.
182. Berry KJ, Mielke PW. Computation of finite population parameters and approximate probability values for multi-response permutation procedures (MRPP). Commun Statis Simul Comput 1983, 12:83-107.
183. Berry KJ, Mielke PW. Computation of exact probability values for multi-response permutation procedures (MRPP). Commun Stat Simul Comput 1984, 13:417-432.
184. Berry KJ, Mielke PW. Least absolute regression residuals: analyses of split-plot designs. Psychol Rep 1999, 85:445-453.
185. O'Reilly FJ, Mielke PW. Asymptotic normality of MRPP statistics from invariance principles of U-statistics. Commun Stat Theory Methods 1980, 9:629-637.
186. Brockwell PJ, Mielke PW, Robinson J. On non-normal invariance principles for multi-response permutation procedures. Aust J Stat 1982, 24:33-41.
187. Mielke PW. The application of multivariate permutation methods based on distance functions in the earth sciences. Earth Sci Rev 1991, 31:55-71.
188. Mielke PW. Non-metric statistical analyses: some metric alternatives. J Stat Plann Infer 1986, 13:377-387.
189. Mielke PW, Berry KJ, Brockwell PJ, Williams JS. A class of nonparametric tests based on multiresponse permutation procedures. Biometrika 1981, 68:720-724.
190. Mielke PW, Berry KJ. Multivariate tests for correlated data in completely randomized designs. J Educ Behav Stat 1999, 24:109-131.
191. Berry KJ, Mielke PW. Least absolute regression residuals: analyses of block designs. Psychol Rep 1998, 83:923-929.
192. Mielke PW, Iyer HK. Permutation techniques for analyzing multi-response data from randomized block experiments. Commun Stat Theory Methods 1982, 11:1427-1437.
193. Berry KJ, Mielke PW. A family of multivariate measures of association for nominal independent variables. Educ Psychol Meas 1992, 52:41-55.
194. Berry KJ, Mielke PW. A measure of association for nominal independent variables. Educ Psychol Meas 1992, 52:895-898.
195. Mielke PW. Goodman-Kruskal tau and gamma. In: Kotz S, Johnson NL, eds. Encyclopedia of Statistical Sciences. Vol. III. New York: John Wiley & Sons; 1983, 446-449.
196. Berry KJ, Mielke PW. Nonasymptotic goodness-of-fit tests for categorical data. Educ Psychol Meas 1994, 54:676-679.
197. Mielke PW, Berry KJ. Permutation covariate analyses of residuals based on Euclidean distance. Psychol Rep 1997, 81:795-802.
198. Mielke PW, Berry KJ. Permutation-based multivariate regression analysis: the case for least sum of absolute deviations regression. Ann Oper Res 1997, 74:259-268.
199. Berry KJ, Mielke PW. A FORTRAN program for permutation covariate analyses of residuals based on Euclidean distance. Psychol Rep 1998, 82:371-375.
200. Berry KJ, Mielke PW. Least sum of absolute deviations regression: distance, leverage, and influence. Percept Mot Skills 1998, 86:1063-1070.
201. Berry KJ, Mielke PW. Least absolute regression residuals: analyses of randomized designs. Psychol Rep 1999, 84:947-954.
202. Berry KJ, Mielke PW. A generalized agreement measure. Educ Psychol Meas 1990, 50:123-125.
203. Berry KJ, Mielke PW. Agreement measure comparisons between two independent sets of raters. Educ Psychol Meas 1997, 57:360-364.
204. Keller-McNulty S, Higgins JJ. Effect of tail weight and outliers and power and type-I error of robust permutation tests for location. Commun Stat Comput Simul 1987, 16:17-35.
205. Bailer AJ. Testing variance equality with randomization tests. J Stat Comput Simul 1989, 31:1-8.
206. Kim MJ, Nelson CR, Startz R. Mean revision in stock prices? A reappraisal of the empirical evidence. Rev Econ Stud 1991, 58:515-528.
207. McQueen G. Long-horizon mean-reverting stock priced revisited. J Financ Quant Anal 1992, 27:1-17.
208. Noreen EW. Computer-Intensive Methods For Testing Hypotheses: An Introduction. New York: John Wiley & Sons; 1989.
209. Johnston JE, Berry KJ, Mielke PW. Permutation tests: precision in estimating probability values. Percept Mot Skills 2007, 105:915-920.
210. Lunneborg CE. Data Analysis by Resampling: Concepts and Applications. Pacific Grove, CA: Duxbury; 2000.
211. Good PI. Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses. 2nd ed. New York: Springer-Verlag; 2000.
212. Good PI. Permutation, Parametric and Bootstrap Tests of Hypotheses. 2nd ed. New York: Springer-Verlag; 2000.
213. Good PI. Resampling Methods: A Practical Guide to Data Analysis. 2nd ed. Boston, MA: Birkhäuser; 2001.
214. Mielke PW, Berry KJ. Permutation Methods: A Distance Function Approach. New York: Springer-Verlag; 2001.
215. Pesarin F. Multivariate Permutation Tests: With Applications in Biostatistics. Chichester: John Wiley & Sons; 2001.
216. Lahiri SN. Resampling Methods for Dependent Data. New York: Springer-Verlag; 2003.
217. Good PI. Permutation, Parametric and Bootstrap Tests of Hypotheses. 3rd ed. New York: Springer-Verlag; 2005.
218. Good PI. Resampling Methods: A Practical Guide to Data Analysis. 3rd ed. Boston, MA: Birkhäuser; 2006.
219. Hirji KF. Exact Analysis of Discrete Data. Boca Raton, FL: Chapman & Hall/CRC; 2006.
220. Manly BFJ. Randomization, Bootstrap and Monte Carlo Methods in Biology. 3rd ed. Boca Raton, FL: Chapman & Hall/CRC; 2007.
221. Bergman R, Ludbrook J, Spooren WPJM. Different outcomes of the Wilcoxon-Mann-Whitney test from different statistics packages. Am Stat 2000, 54:72-77.
222. Hilbe J. Statistical computing software reviews: section editor's notes. Am Stat 2000, 54:71.
223. Fisher RA. Statistical Methods and Scientific Inference. 2nd ed. New York: Hafner; 1959.
224. Goodman SN, discussants. p values, hypothesis tests, and likelihood: implications for epidemiology of a neglected historical debate. Am J Epidemiol 1993, 137:485-501.
225. Hubbard R. Alphabet soup: blurring the distinctions between p's and α's in psychological research. Theory Psychol 2004, 14:295-327.
226. Hubbard R, Bayarri MJ, discussants. Confusion over measures of evidence (p's) versus errors (α's) in classical statistical testing. Am Stat 2003, 57:171-182.