Precis of statistical significance: Rationale, validity, and utility

Behavioral and Brain Sciences

Volumn 21, Issue 2, 1998, Pages 169-239

  Chow, Siu L ^a,b,c  

^a UNIVERSITY OF REGINA   (Canada)

^b UNIVERSITY OF ADELAIDE   (Australia)

^c UNIVERSITY OF TORONTO   (Canada)

Author keywords

Bayesianism; Effect size; Null hypothesis; Statistical hypothesis testing; Statistical significance; Theory corroboration

Indexed keywords

BAYES THEOREM; META ANALYSIS; METHODOLOGY; PROBABILITY; RELIABILITY; REVIEW; SAMPLING; STATISTICAL ANALYSIS; THEORY; VALIDATION PROCESS; REPRODUCIBILITY; RESEARCH; STATISTICS;

REPRODUCIBILITY OF RESULTS; RESEARCH; STATISTICS;

EID: 0031829312     PISSN: 0140525X     EISSN: None     Source Type: Journal    
DOI: 10.1017/S0140525X98001162     Document Type: Review

References (284)

1. Abelson, R. P. (1995) Statistics as principled argument. Erlbaum. [JFK]
2. (1997) On the surprising longevity of flogged horses: Why there is a case for the significance test. Psychological Science 8:12-15. [JFK]
3. Acree, M. C. (1978) Theories of statistical inference in psychological research: A historicocritical study. University Microfilms International (University Microfilms No. H790 H7000). [GG]
4. Allport, D. A. (1975) Critical notice: The state of cognitive psychology. Quarterly Journal of Experimental Psychology 27:141-52. [KJV]
5. Anderson, N. H. & Cuneo, D. O. (1978) The height + width rule in children's judgements of quantity. Journal of Experimental Psychology: General 107:335-78. [GG]
6. Anthony, T. & Mullen, B. (1991) Conceptual rigor mortis: Or, why people are supporting pornography when they don't send money to televangelists. Theory and Psychology 1:361-67. [SL]
7. Ashby, D., ed. (1993) Papers from the Conference on Methodological and Ethical Issues in Clinical Trials, 27-28 June 1991, special issue. Statistics in Medicine 12:1373-534. [JP]
8. Bakan, D. (1966) The test of significance in psychological research. Psychological Bulletin 66:423-37. [CFB, aSLC]
9. Banaji, M. R. & Crowder, R. G. (1991) Some everyday thoughts on ecologically valid methods. American Psychologist 46:78-79. [KJV]
10. Barnett, V. (1982) Comparative statistical inference (2nd edition). Wiley. [DR]
11. Berger, J. O. (1985) Statistical decision theory and Bayesian analysis, second edition. Springer-Verlag. [NGW]
12. Berger, J. O. & Berry, D. A. (1988) Statistical analysis and the illusion of objectivity. American Scientist 76:159-65. [PS, NGW]
13. Bernardo, J. M. & Smith, A. F. M. (1994) Bayesian theory. Wiley. [RAMG, JP]
14. Bernieri, F. J. (1991) Rigor is rigor: But rigor is not necessarily science. Theory and Psychology 1:369-73. [SL]
15. Berry, D. A. & Lindgren, B. W. (1996) Statistics: Theory and methods, second edition. Duxbury Press. [NGW]
16. Black, M. (1969) Some half-baked thoughts about induction. In: Philosophy, science, and method, ed. S. Morgenbesser, P. Suppes & M. White. St. Martins Press. [PS]
17. Bolles, R. C. (1962) The difference between statistical hypotheses. Psychological Reports 11:639-45. [NGW]
18. Bookstein, F. L. (1997) Biometrics and brain maps: The promise of the morphometric synthesis. In: Neuroinformatics: An overview of the human brain project. Progress in neuroinformatics, vol. 1., ed. S. Koslow & M. Huerta. Erlbaum. [FLB]
19. Bookstein, F. L., Streissguth, A., Sampson, P. & Barr, H. (1996) Exploiting redundant measurement of dose and behavioral outcome: New methods from the teratology of alcohol. Developmental Psychology 32:404-15. [FLB]
20. Boring, E. G. (1919) Mathematical vs. scientific significance. Psychological Bulletin 16:335-38. [HJS]
21. (1920) The logic of the normal law of error in mental measurement. American Journal of Psychology 31:1-33. [HJS]
22. (1954) The nature and history of experimental control. American Journal of Psychology 67:573-89. [aSLC]
23. (1963) The nature and history of experimental control (reprinted). In: History, psychology, and science: Selected papers by Edwin G. Boring, ed. R. I. Watson & D. T. Campbell. Wiley. [LGT]
24. (1969) Perspective: Artifact and control. In: Artifacts in behavioral research, ed. R. Rosenthal & R. L. Rosnow. Academic Press. [aSLC]
25. Box, G. E. P. (1980) Sampling and Bayes' inference in scientific modeling and robustness. Journal of the Royal Statistical Society A 143:383-430. [ZGS]
26. Box, G. E. P. & Tiao, G. C. (1973) Bayesian inference in statistical analysis. Addison-Wesley. [RAMG, HR]
27. Camilli, G. (1990) The test of homogeneity for 2X2 contingency tables: A review of and some opinion on the controversy. Psychological Bulletin 108:135-45. [JRV]
28. Campbell, D. T. (1969) Prospective: Artifact and control. In: Artifacts in behavioral research, ed. R. Rosenthal & R. L. Rosnow. Academic Press. [aSLC]
29. Campbell, D. T. & Stanley, J. C. (1963) Experimental and quasi-experimental designs for research. Rand McNally. [aSLC]
30. Carlin, B. P. & Loui, T. A. (1996) Bayes and empirical Bayes methods for data analysis. Chapman and Hall. [RAMG]
31. 2. Educational and Psychological Measurements 35:541-54. [SL]
32. Carver, R. P. (1978) The case against statistical significance testing. Harvard Educational Review 48:378-99. [CFB]
33. Chomsky, N. (1957) Syntactic structures. Mouton. [aSLC]
34. (1965) Aspects of the theory of syntax. MIT Press. [HJS]
35. Chow, S. L. (1987a) Experimental Psychology: Rationale, procedures and issues. Detselig. [aSLC]
36. (1987b) Some reflections on Harris and Rosenthal's thirty-one meta-analyses. Journal of Psychology 121:95-100. [aSLC]
37. (1987c) Meta-analysis of pragmatic and theoretical research: A critique. Journal of Psychology 121:259-71. [arSLC]
38. (1988) Significance test or effect size? Psychological Bulletin 103:105-10. [aSLC, JG, JRV]]
39. (1989) Significance tests and deduction: Reply to Folger (1989). Psychological Bulletin 106:161-65. [arSLC]
40. (1991a) Conceptual rigor versus practical impact. Theory and Psychology 1:337-60. [aSLC, HJS]
41. (1991b) Rigor and logic: A response to comments on "Conceptual Rigor." Theory and Psychology 1:389-400. [aSLC]
42. (1991c) Some reservations about statistical power. American Psychologist 46:1088-89. [aSLC]
43. (1992) Research methods in psychology: A primer. Detselig. [aSLC]
44. (1996) Statistical significance. Rationale, validity, and utility. Sage Publications. [CFB, JFK, HEK, DM, JSR, LGT, JRV, BDZ]
45. Church, R. M., Crystal, J. D. & Collyer, C. E. (1996) Correction of errors in scientific research. Behavior Research Methods, Instrumentation, and Computers 28:305-10. [LGT]
46. Clark-Carter, D. (1997) The account taken of statistical power in research published in the British Journal of Psychology. British Journal of Psychology 88:71-83.
47. Cohen, J. (1962) The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology 65:145-53. [GG]
48. (1965) Some statistical issues in psychological research. In: Handbook of clinical psychology, ed. B. B. Wolman. McGraw-Hill. [aSLC]
49. (1987) Statistical power analysis for the behavioral sciences (Revised edition). Academic Press. [aSLC, DM]
50. (1990) Things I have learned (so far). American Psychologist 45: 1304-12. [aSLC, LEK, BRL]
51. (1992a) Statistical power analysis. Current Directions in Psychological Science 1:98-105. [aSLC]
52. (1992b) A power primer. Psychological Bulletin 112:155-59. [aSLC, JFK]
53. (1994) The earth is round (p < .05). American Psychologist 49: 997-1003. [CFB, aSLC, RWF, JEH, NGW]
54. Cohen, M. R. & Nagel, E. (1934) An introduction to logic and scientific method. Routledge & Kegan Paul. [aSLC]
55. Coltheart, M. (1980) Iconic memory and visible persistence. Perception and Psychophysics 27:183-228. [aSLC]
56. Cook, T. D. & Campbell, D. T. (1979) Quasi-experimentation: Design and analysis issues for field settings. Rand McNally. [aSLC, LGT]
57. Cook, T. D. & Leviton, L. C. (1980) Reviewing the literature: A comparison of traditional methods with meta-analysis. Journal of Personality 48:449-72. [aSLC]
58. Cooper, H. M. (1979) Statistically combining independent studies: A meta-analysis of sex differences in conformity research. Journal of Personality and Social Psychology 37:131-46. [aSLC]
59. Cooper, H. M. & Rosenthal, R. (1980) Statistical versus traditional procedures for summarizing research findings. Psychological Bulletin 87:442-49. [aSLC]
60. Copi, I. (1982). Symbolic logic (6th edition). Macmillan. [aSLC]
61. Cowles, M. (1989) Statistics in psychology: An historical perspective. Erlbaum. [LEK]
62. Cowles, M. & Davis, C. (1982) On the origins of the .05 level of statistical significance. American Psychologist 37:553-58. [JFK]
63. Cox, D. R. & Hinkley, D. V. (!974) Theoretical statistics. Chapman and Hall. [ZGS]
64. Cox, R. T. (1946) Probability, frequency and reasonable expectation. American Journal of Physics 14:1-13. [RAMG]
65. Dale, A. I. (1991) A history of inverse probability. Springer-Verlag. [RAMG]
66. Danziger, K. (1987) Statistical method and the historical development of research practice in American psychology. In: The probabilistic revolution, vol. 2.: Ideas in the sciences, ed. L. Krüger, G. Gigerenzer & M. Morgan. MIT Press. [HJS]
67. (1990) Constructing the subject: Historical origins of psychological research. Cambridge University Press. [aSLC, GG, HJS]
68. Dar, R. (1987) Another look at the Meehl, Lakatos, and the scientific practices of psychologists. American Psychologist 42:145-51. [RD]
69. De Finetti, B. (1974) Theory of probability. Wiley. [RAMG]
70. De Groot, A. D. (1990) Unifying psychology: A European view. New Ideas in Psychology 3:309-20. [KJV]
71. DeGroot, M. H. (1970) Optimal statistical decisions. McGraw-Hill. [NGW]
72. Draper, D., Hodges, J. S., Mallows, C. L. & Pregibon, D. (1993) Exchangeability and data analysis. Journal of the Royal Statistical Society A 156:9-37. [JRV]
73. Earman, J. (1992) Bayes or bust? A critical examination of Bayesian confirmation theory. The MIT Press. [aSLC]
74. Ebbinghaus, H. (1885/1964) Memory: A contribution to experimental psychology. Dover. [KJV]
75. Edgington, E. S. (1966) Statistical inference and nonrandom samples. Psychological Bulletin 66:485-87. [AFH, JRV]
76. (1995) Randomization tests (3rd edition). Dekker. [AFH, JRV]
77. Edwards, A. W. F. (1972) Likelihood. Cambridge University Press. [BDZ]
78. Erwin, E. (1996) A final accounting: Philosophical and empirical issues in Freudian psychology. MIT Press. [EE]
79. (1997) Philosophy and psychotherapy: Razing the troubles of the brain. Sage. [EE]
80. Estes, W. K. (1997) Significance testing in psychological research: Some persisting issues. Psychological Science 8:18-20. [JFK]
81. Eysenck, H. J. (1975) Who needs a random sample? Bulletin of the British Psychological Society 28:195-98. [JRV]
82. (1978) An exercise in mega-silliness. American Psychologist 33:517. [aSLC]
83. Falk, R. & Greenbaum, C. W. (1995) Significance tests die hard: The amazing persistence of a probabilistic misconception. Theory and Psychology 5:75-98. [aSLC, HJS]
84. Ferguson, T. (1967) Mathematical statistics: A decision theoretic approach. Academic Press. [NGW]
85. Feynman, R. P. (1985) "Surely you're joking, Mr. Feynman!": Adventures of a curious character. Bantam. [KJV]
86. Fillmore, C. J. (1968) The case for case. In: Universals in linguistic theory, ed. E. Bach & R. T. Harms. Holt, Rinehart, and Winston. [aSLC]
87. Fisher, R. A. (1925, 1963) Statistical methods for research workers. Hafner. [HEK]
88. (1935a) The design and analysis of experiments. Oliver and Boyd. [GG, JFK, JRV]
89. (1935b) The fiducial argument in statistical inference. Annals of Eugenics 6:391-98. [JP]
90. (1955) Statistical methods and scientific induction. Journal of the Royal Statistical Society (B) 17:69-77. [GG]
91. (1956/1990) Statistical methods and scientific inference. Oliver and Boyd. (3rd edition 1973 reprinted, Oxford University Press, 1990), [GG, HEK, JP]
92. (1959/1973) Statistical methods and scientific inference (2nd edition). Hafner Publishing Co. [aSLC, NGW]
93. (1962) Some examples of Bayes's method of the experimental determination of probabilities a priori. Journal of the Royal Statistical Society B 24:118-24. [JP]
94. Folger, R. (1989) Significance tests and the duplicity of binary decisions. Psychological Bulletin 106:155-60. [rSLC, JG]
95. Frick, R. W. (1995) Accepting the null hypothesis. Memory and Cognition 23:132-38. [RWF, LEK]
96. (1996) The appropriate use of null hypothesis testing. Psychological Methods 1:379-90. [RWF, JRV]
97. (in press a) A better stopping rule of conventional statistical tests. Behavior Research Methods, Instruments, and Computers. [RWF]
98. (in press b) Interpreting statistical testing: Processes, not populations and random sampling. Behavior Research Methods, Instruments, and Computers. [RWF]
99. Gallo, P. S., Jr. (1978) Meta-analysis - A mixed meta-phor? American Psychologist 33:515-17. [aSLC]
100. Garner, W. R., Hake, H. W. & Eriksen, C. W. (1956) Operationalism and the concept of perception. Psychological Review 63:149-59. [aSLC]
101. Gergen, K. J. (1991) Emerging challenges for theory and psychology. Theory and Psychology 1:13-35. [aSLC)
102. Gibson, J. J. (1967/1982) James J. Gibson: Autobiography. In: Reasons for realism: Selected essays of James J. Gibson, ed. E. Reed & R. Jones. Erlbaum. [KJV]
103. Gigerenzer, G. (1987) Probabilistic thinking and the fight against subjectivity. In: The probabilistic revolution, Vol.2. Ideas in the sciences, ed. L. Krüger, G. Gigerenzer & M. S. Morgan. MIT Press. [GG, HJS]
104. (1993) The superego, the ego, and the id in statistical reasoning. In: A handbook for data analysis in the behavioral sciences: Methodological issues, ed. G. Keren & C. Lewis. Erlbaum. [arSLC, GG, JG, LEK, JP, HJS]
105. Gigerenzer, G. & Murray, D. J. (1987) Cognition as intuitive statistics. Erlbaum. [GG, LEK, HJS]
106. Gigerenzer, G. & Richter, H. R. (1990) Context effects and their interaction with development: Area judgements. Cognitive Developments 5:235-64. [GG]
107. Gigerenzer, G., Swijtink, Z., Porter, T., Daston, L., Beatty, J. & Krüger, L. (1989) The empire of chance. How probability changed science and everyday life. Cambridge University Press. [GG, HJS, ZGS]
108. Glass, G. V. (1976) Primary, secondary and meta-analysis of research. Educational Researcher 5:3-8. [aSLC]
109. (1978) Integrating findings: The meta-analysis of research. Review of Research in Education 5:351-79. [aSLC]
110. Glass, G. V. & Kliegl, R. M. (1983) An apology for research integration in the study of psychotherapy. Journal of Consulting and Clinical Psychology 51:28-41. [aSLC]
111. Glass, G. V., McGaw, B. & Smith, M. L. (1981) Meta-analysis in social research. Sage. [arSLC]
112. Godambe, V. P. & Sprott, D. A., eds. (1971) Foundation of statistical inference. Holt Rinehart and Winston. [HEK]
113. Gonzalez, R. (1994) The statistics ritual in psychological research. Psychological Science 5:325-28. [RAMG, JFK]
114. Good, R. & Fletcher, H. J. (1981) Reporting explained variance. Journal of Research on Science Teaching 18:1-7. [BAT]
115. Green, D. M. & Swets, J. A. (1966) Signal detection: Theory and psychophysics. John Wiley. [JP]
116. Greenwald, A. G. (1975) Consequences of prejudice against the null hypothesis. Psychological Bulletin 82:1-20. [LEK]
117. Greenwald, A. G., Gonzalez, R., Harris, R. J. & Guthrie, D. (1996) Effect sizes and p-values: What should be reported and what should be replicated? Psychophysiology 33:175-83. [JRV]
118. Guttman, L. (1977) What is not what in statistics. Statistician 26:81-107. [CFB]
119. Haber, R. N. (1983) The impending demise of the icon: A critique of the concept of iconic storage in visual information processing. Behavioral and Brain Sciences 6:1-11. [aSLC]
120. Hacking, I. (1965) Logic of statistical inference. Cambridge University Press. [GG]
121. Hagen, R. L. (1997) In praise of the null hypothesis statistical test. American Psychologist 52:15-24. [aSLC, JRV]
122. Hahn, G. J. & Meeker, W. O. (1993) Assumptions for statistical inference. The American Statistician 47:1-11. [JRV]
123. Hammond, G. (1996) The objections to null hypothesis testing as a means of analysing psychological data. Australian Journal of Psychology 48:104-06. [SL]
124. Hammond, K. R., Hamm, R. M. & Grassia, J. (1986) Generalizing over conditions by combining the multitrait-multimethod matrix and the representative design of experiments. Psychological Bulletin 100:257-69. [KJV]
125. Hanson, N. R. (1958) New books. Mind LXVII:272-75. [RAMG]
126. Harris, M. J. (1991) Significance tests are not enough: The role of effect-size estimation in theory corroboration. Theory and Psychology 1:375-82. [SL]
127. Harris, M. J. & Rosenthal, R. (1985) Mediation of interpersonal expectancy effects: 31 meta-analyses. Psychological Bulletin 97:363-86. [aSLC]
128. Harris, R. J. (1997a) Reforming significance testing via three-valued logic. In: What if there were no significance tests?, ed. L. Harlow, S. Mulaik & J. Steiger. Erlbaum. [RJH]
129. (1997b) Significance tests have their place. Psychological Science 8:8-11. [RJH, JFK]
130. Hartigan, J. A. (1983) Bayes theory. Springer-Verlag. [RAMG]
131. Hedges, L. V. (1987) How hard is hard science, how soft is soft science?: The empirical cumulativeness of research. American Psychologist 42:443-55. [NGW]
132. Heidbreder, G. R. (1996) Maximum entropy and Bayesian methods. Kluwer Academic. [RAMG]
133. Hogben, L. (1957) Statistical theory: The relationship of probability, credibility and error. W. W. Norton. [aSLC, LEK]
134. Hogg, R. V. & Tanis, E. A. (1993) Probability and statistical inference, fourth edition. Macmillan. [NGW]
135. Holton, G. (1988) Thematic origins of scientific thought: Kepler to Einstein (revised edition). Harvard University Press. [KJV]
136. Howson, C. & Urbach, P. (1993) Scientific reasoning: The Bayesian approach, second edition. Open Court. [PS]
137. (1994) Probability, uncertainty and the practice of statistics. In: Subjective probability, ed. G. Wright & P. Ayton. Wiley. [RAMG]
138. Hubbard, R., Parsa, R. A. & Luthy, M. R. (1997) The spread of statistical significance testing in psychology: The case of the Journal of Applied Psychology, 1917-1994. Theory and Psychology 7:545-54. [HJS]
139. Hudson, W. W., Thyer, B. A. & Stocks, J. T. (1985) Assessing the importance of experimental outcomes. Journal of Social Service Research 8(4):87-98. [BAT]
140. Hunter, J. E. (1997) Needed: A ban on the significance test. Psychological Science 8:3-7. [CFB, EE, JEH, JFK, LGT]
141. Hunter, J. E. & Schmidt, F. L. (1990) Methods of meta-analysis: Correcting error and bias in research findings. Sage. [arSLC]
142. Hunter, M. A. & May, R. B. (1993) Some myths concerning parametric and nonparametric tests. Canadian Psychology 34:384-89. [JRV]
143. Inman, H. F. (1994) Karl Pearson and R. A. Fisher on statistical tests: A 1935 exchange from Nature. The American Statistician 48:2-11. [LEK]
144. Jacobson, N. S. & Christensen, A. (1996) Studying the effectiveness of psychotherapy: How well can clinical trials do the job? American Psychologist 51:1031-39. [JFK]
145. Jacobson, N. S., Follette, W. C. & Revenstorf, D. (1984) Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. Behavior Therapy 15:336-52. [JFK]
146. Jacobson, N. S. & Truax, P. (1991) Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology 59:12-19. [JFK]
147. Jeffreys, H. (1939) Theory of probability. Oxford University Press. [HEK]
148. Kaiser, H. F. (1960) Directional statistical decisions. Psychological Review 67:160-67. [RJH]
149. Kass, R. E. & Raftery, A. E. (1995) Bayes factors and model uncertainty. Journal of the American Statistical Association 90:773-95. [NGW]
150. Kelley, T. L. (1923) The principles and techniques of mental measurement. American Journal of Psychology 34:408-32. [HJS]
151. Kempthorne, O. (1955) The randomization theory of experimental inference. Journal of the American Statistical Association 50:946-67. [JRV]
152. Keppel, G. & Underwood, B. J. (1962) Proactive inhibition in short-term retention of single items. Journal of Verbal Learning and Verbal Behavior 1:153-61. [aSLC]
153. Kirk, R. E. (1984) Basic statistics (2nd edition). Brooks/Cole. [aSLC]
154. (1996) Practical significance: A concept whose time has come. Educational and Psychological Measurement 56:746-59. [JEH]
155. Kraemer, H. C. & Thiemann, S. (1987) How many subjects? Statistical power analysis in research. Sage. [aSLC]
156. Kreutzer, M. A., Leonard, C. & Flavell, J. H. (1975) An interview study of children's knowledge about memory. Monographs of the Society for Research in Child Development 40, Serial No. 159. [KJV]
157. Kuhn, T. S. (1970) The structure of scientific revolutions, second edition. University of Chicago Press. [JG]
158. Kyburg, H. E., Jr. (1974) The logical foundations of statistical inference. Reidel. [HEK]
159. Lad, F. & Deely, J. (1994) Experimental design from a subjective utilitarian viewpoint. In: Aspects of uncertainty, ed. P. R. Freeman & A. F. M. Smith. Wiley. [ZGS]
160. Lakatos, I. (1978) Falsification and the methodology of scientific research programmes. In: The methodology of scientific research programs: Imre Lakatos' philosophical papers, vol. 1, ed. J. Worral & G. Currie. Cambridge University Press. [RD] Also in: Criticism and the growth of knowledge, ed. I. Lakatos & A. Musgrave. Cambridge University Press. [JG]
161. Lakatos, I. (1978) Falsification and the methodology of scientific research programmes. In: The methodology of scientific research programs: Imre Lakatos' philosophical papers, vol. 1, ed. J. Worral & G. Currie. Cambridge University Press. [RD] Also in: Criticism and the growth of knowledge, ed. I. Lakatos & A. Musgrave. Cambridge University Press. [JG]
162. Langley, P. W., Simon, H. A., Bradshaw, G. L. & Zytkow, J. M. (1987) Scientific discovery: An account of the creative processes. MIT Press. [MGS]
163. Lapin, L. L. (1994) Quantitative methods for business decisions (sixth edition). The Dryden Press. [MGS]
164. Lashley, B. R. & Bond, C. F., Jr. (1997) Significance testing for round robin data. Psychological Methods 2:278-91. [BRL]
165. Lashley, B. R. & Kenny, D. A. (submitted) Power estimation in social relations analyses. University of Connecticut Press. [BRL]
166. Lecoutre, B., Derzko, G. & Grouin, J.-M. (1995) Bayesian predictive approach for inference about proportions. Statistics in Medicine 14:1057-63. [JP]
167. LeLorier, J., Grégoire, G., Benhaddad, A., Lapierre, J. & Derderian, F. (1997) Discrepancies between meta-analyses and subsequent large randomized, controlled trials. New England Journal of Medicine 337:536-42. [rSLC]
168. Leventhal, L. (1994) Nudging aside Meehl's paradox. Canadian Psychology 35:283-98. [SL]
169. Leviton, L. C. & Cook, T. D. (1981) What differentiates meta-analysis from other forms of review. Journal of Personality 49:231-36. [aSLC]
170. Lewin, K. (1946) Action research and minority problems. The Journal of Social Issues 2:34-46. [LGT]
171. Lindley, D. V. (1965) Introduction to probability and statistics. Part 2: Inference. Cambridge University Press. [RAMG]
172. (1972) Bayesian statistics: A review. Siam, Regional Conference Series in Applied Mathematics, Philadelphia. [JP]
173. Lipsey, M. W. & Wilson, D. B. (1993) The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist 48:1181-1209. [EE, NGW]
174. Loftus, G. R. (1996) Psychology will be a much better science when we change the way we analyze data. Current Directions in Psychological Science 5:161-71. [CFB, JEH, JFK, KJV]
175. Loftus, G. R. & Masson, M. J. (1994) Using confidence intervals in within-subject designs. Psychonomic Bulletin and Review 1:476-90. [RWF]
176. Lorenz, K. Z. (1973) The fashionable fallacy of dispensing with description. Die Naturwissenschaften 60:1-9. [KJV]
177. Lykken, D. T. (1968) Statistical significance in psychological research. Psychological Bulletin 70:151-59. [RD]
178. (1991) What's wrong with psychology anyway? In: Thinking clearly about psychology, vol. 1: Matters of public interest, ed. D. Cicchetti & W. M. Grove. University of Minnesota Press. [NGW]
179. Macdonald, R. R. (1997) On statistical testing in psychology. British Journal of Psychology 88:333-47. [JRV]
180. Manicas, P. T. & Secord, P. F. (1983) Implications for psychology of the new philosophy of science. American Psychologist 38:399-413. [aSLC]
181. May, R. B. & Hunter, M. A. (1993) Some advantages of permutation tests. Canadian Psychology 34:401-07. [AFH]
182. May, R. B., Masson, M. E. J. & Hunter, M. A. (1990) Application of statistics in behavioral research. Harper & Row. [JRV]
183. Mayo, D. (1996) Error and the growth of experimental knowledge. The University of Chicago Press. [DM]
184. Meehl, P. E. (1967) Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science 34:103-15. [aSLC, RD, KJV, NGW]
185. (1978) Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology 46:806-34. [aSLC, RD, JG, JSR, KJV, NGW]
186. (1990) Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry 1:108-41. [aSLC, ZGS, NGW]
187. Mill, J. S. (1973) A system of logic: Ratiocinative and inductive. University of Toronto Press. [arSLC, HEK]
188. Miller, G. A. (1956) The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review 63:81-97. [aSLC, WSV]
189. (1962) Some psychological studies of grammar. American Psychologist 17:748-62. [aSLC]
190. Mintz, J. (1983) Integrating research evidence: A commentary on meta-analysis. Journal of Consulting and Clinical Psychology 51:71-75.
191. Mook, D. G. (1983) In defense of external invalidity. American Psychologist 38:379-87. [aSLC, JRV]
192. Morrison, D. E. & Henkel, R. E., eds. (1970) The significance test controversy: A reader. Aldine. [CFB, aSLC, JP]
193. Mosteller, F. & Bush, R. R. (1954) Selected quantitative techniques. In: Handbook of social psychology: Vol. 1.-Theory and method, ed. G. Lindzey. Addison-Wesley. [aSLC]
194. Mosteller, F. & Tukey, J. W. (1977) Data analysis and regression: A second course in statistics. Addison-Wesley. [BDZ]
195. Nagel, E. (1961) The structure of science. Harcourt, Brace and World. [HEK]
196. Neisser, U. (1967) Cognitive psychology. Appleton-Century-Croft. [aSLC]
197. (1976) Cognition and reality: Principles and implications of cognitive psychology. Freeman. [KJV]
198. Newell, A. (1973) You can't play 20 questions with Nature and win: Projective comments on the papers of this symposium. In: Visual information processing, ed. W. G. Chase. Academic Press. [KJV]
199. Neyman, J. (1942) Basic ideas and some recent results of the theory of testing statistical hypotheses. Journal of the Royal Statistical Society 105:292-327. [HEK]
200. (1950) First course in probability and statistics. Holt. [JP]
201. (1952) Lectures and conferences on mathematical statistics and probability. (2nd edition) Graduate School, U.S. Department of Agriculture. [JP]
202. (1957) "Inductive behavior" as a basic concept of philosophy of science. International Statistical Review 25:7-22. [ZGS]
203. Neyman, J. & Pearson, E. S. (1928) On the use and interpretation of certain test criteria for purposes of statistical inferences (Part I). Biometrika 20A:175-240. [arSLC, JFK, HEK, JP, ZGS]
204. (1933a) On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London, Series A 231:289-337. [HEK, JP, ZGS]
205. (1933b) The testing of statistical hypotheses in relation to probabilities a priori. Proceedings of the Cambridge Philosophical Society 29:492-510. [JP]
206. Oakes, M. (1986) Statistical inference: A commentary for the social and behavioral sciences. Wiley. [aSLC, GG, LEK, DR]
207. Oakes, W. F. (1975) On the alleged falsity of the null hypothesis. Psychological Record 25:265-72. [SL]
208. O'Hagan, A. (1994) The advanced theory of statistics: Vol. 2B, Bayesian inference. Edward Arnold. [RAMG]
209. Ohlsson, S. & Jewett, J. J. (1997) Simulation models and the power law of learning. In: Proceedings of the Nineteenth Annual Conference of the Cognitive Science Society, ed. M. G. Shafto & P. Langley. Erlbaum. [MGS]
210. Pagano, R. (1994) Understanding statistics in the behavioral sciences, 4th edition. West Publishing Company. [EE]
211. Phillips, L. D. (1973) Bayesian statistics for social scientists. Nelson. [aSLC]
212. Pitman, E. J. G. (1937a) Significance tests which may be applied to samples from any populations. Journal of the Royal Statistical Society (Series B) 4:119-30. [JRV]
213. (1937b) Significance tests which may be applied to samples from any population II. The correlation coefficient test. Journal of the Royal Statistical Society (Series B) 4:225-32. [JRV]
214. (1937c) Significance tests which may be applied to samples from any populations III. The analysis of variance test. Biometrika 29:322-35. [JRV]
215. Platt, J. C. (1964) Strong inference. Science 146:347-53. [LGT]
216. Polya, G. (1954) Patterns of plausible reasoning. Princeton University Press. [PS]
217. Popper, K. R. (1968a) The logic of scientific discovery (originally published in 1959). Harper & Row. [arSLC]
218. (1968b) Conjectures and refutations (originally published in 1962). Harper & Row. [arSLC]
219. Prentice, D. A. & Miller, D. T. (1992) When small effects are impressive. Psychological Bulletin 112:160-64. [AFH]
220. Presby, S. (1978) Overly broad categories obscure important differences between therapies. American Psychologist 33:524-615. [aSLC]
221. Rachman, S. & Wilson, G. T. (1980) The effects of psychological therapy. Pergamon Press. [aSLC]
222. Ramsey, F. L. & Schafer, D. W. (1997) The statistical sleuth. A course in methods of data analysis. Duxbury Press. [CFB]
223. Rindskopf, D. (1997) Testing "small," not null, hypotheses: Classical and Bayesian approaches. In: What if there were no significance tests?, ed. L. L. Harlow, S. A. Mulaik & J. H. Steiger. Erlbaum. [DR].
224. Robert, C. P. (1994) The Bayesian choice: A decision-theoretic motivation. Springer-Verlag. [RAMG, JP]
225. Rogers, J. I., Howard, K. I. & Vessey, J. T. (1993) Using significance tests to evaluate equivalence between two experimental groups. Psychological Bulletin 113:553-65. [rSLC, LEK]
226. Rosenthal, R. (1983) Assessing the statistical and social importance of the effects of psychotherapy. Journal of Consulting and Clinical Psychology 51:4-13. [aSLC]
227. (1984) Meta-analytic procedures for social research. Sage. [aSLC, NGW]
228. Rosenthal, R. & Rubin, D. B. (1979) A note on percent variance explained as a measure of the importance of effects. Journal of Applied Social Psychology 9:395-96. [aSLC]
229. (1982) A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology 74:166-69. [aSLC]
230. Rosnow, R. L. & Rosenthal, R. (1989) Statistical procedures and the justification of knowledge in psychological science. American Psychologist 44:1276-84. [aSLC, BM, KJV]
231. Rossi, J. S. (1990) Statistical power of psychological research: What have we gained in 20 years? Journal of Consulting and Clinical Psychology 58:646-56. [JSR]
232. (1997) A case study in the failure of psychology as a cumulative science: The spontaneous recovery of verbal learning. In: What if there were no significance tests?, ed. L. L. Harlow, S. A. Mulaik & J. Steiger. Erlbaum. [JSR]
233. Rouanet, H. (1996) Bayesian methods for assessing importance of effects. Psychological Bulletin 119:149-58. [JP, HR]
234. Rouanet, H., Bernard, J. M., Bert, M. C., Lecoutre, B. & Lecoutre, M. P. (1991) L'Inference statistique dans la démarche du chercheur (Statistical inference in the strategy of the researcher). Peter Lang. (English version in preparation). [HR]
235. Rouanet, H., Bernard, J. M. & Lecoutre, B. (1986) Nonprobabilistic statistical inference: A set-theoretic approach. The American Statistician 40:60-65. [JRV]
236. Rozeboom, W. W. (1960) The fallacy of the null-hypothesis significance-test. Psychological Bulletin 57:416-28. [CFB, aSLC, JP, BDZ]
237. (1991) Conceptual rigor: Where is it? Theory and Psychology 1:383-88. [SL]
238. Savage, L. J. (1954) Foundations of statistics. Wiley. [HEK]
239. Savin, H. B. & Perchonock, E. (1965) Grammatical structure and the immediate recall of English sentences. Journal of Verbal Learning and Verbal Behavior 4:348-53. [aSLC, HJS]
240. Scarr, S. (1997) Rules of evidence: A larger context for the statistical debate. Psychological Science 8:16-17. [JFK]
241. Schmidt, F. L. (1992) What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. American Psychologist 47:1173-81. [aSLC]
242. (1996) Statistical significance testing and cumulative knowledge in psychology: Implications for the training of researchers. Psychological Methods 1:115-29. [arSLC, RWF, JEH, JFK]
243. Schmidt, F. L., Hunter, J. E. & Urry, V. E. (1976) Statistical power in criterion-related validation studies. Journal of Applied Psychology 61:473-85. [JEH]
244. Schmidt, F. L., Ocasio, B. P., Hillery, J. M. & Hunter, J. E. (1985) Further within-setting empirical tests of the situational specificity hypothesis in personnel selection. Personnel Psychology 105:309-16. [rSLC]
245. Schneider, W. & Shiffrin, R. M. (1977) Controlled and automatic human information processing: I. Detection, search, and attention. Psychological Review 84:1-66. [arSLC]
246. Sedlmeier, P. & Gigerenzer, G. (1989) Do studies of statistical power have an effect on the power of studies? Psychological Bulletin 105:309-16. [EE, GG, JG]
247. Shapiro, S. (1997) Is meta-analysis a valid approach to the evaluation of small effects in observational studies? Journal of Clinical Epidemiology 50:223-29. [EE]
248. Shaver, J. P. (1993) What statistical significance testing is, and what it is not. Journal of Experimental Education 61(4):293-316. [CFB]
249. Shrout, P. E. (1997) Should significance tests be banned? Introduction to a special section exploring the pros and cons. Psychological Science 8:1-2. [EE, JFK]
250. Siegel, S. (1956) Non-parametric statistics for the behavioral sciences. McGraw-Hill. [aSLC]
251. Siegmund, D. (1985) Sequential analysis. Springer-Verlag. [NGW]
252. Silvey, S. D. (1975) Statistical inference. Chapman and Hall. [DR]
253. Simon, H. A. (1969) The sciences of the artificial. MIT Press. [JFK]
254. (1979) Models of thought. Yale University Press. [MGS]
255. Simonton, D. K. (1997) Creative productivity: A predictive and explanatory model of career trajectories and landmarks. Psychological Review 104:66-89. [KJV]
256. Sivia, D. S. (1996) Data analysis. Clarendon Press. [RAMG]
257. Slonim, M. J. (1960) Sampling. Simon and Schuster. [rSLC]
258. Smith, A. F. M. & Roberts, G. O. (1993) Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, Series B 55:3-23. [RAMG]
259. Smith, M., Glass, G. & Miller, T. (1980) The benefits of psychotherapy. Johns Hopkins University Press. [EE]
260. Sohn, D. (1980) Critique of Cooper's meta-analytic assessment of the findings of sex differences in conformity behavior. Journal of Personality and Social Psychology 39:1215-21. [aSLC]
261. Sperling, G. (1960) The information available in brief visual presentations. Psychological Monographs 74:11 (Whole No. 498). [aSLC, JSR]
262. Spiegelhalter, D. J., Freedman, L. S. & Parmar, M. K. B. (1994)
263. Sterling, T. D. (1959) Publication decisions and their possible effects on inferences drawn from tests of significance - or vice versa. Journal of the American Statistical Association 54:30-34. [HJS]
264. Sternberg, S. (1969) Memory-scanning: Mental processes revealed by reaction-time experiments. American Scientist 57:421-57. [rSLC]
265. Stevenson, W. J. (1992) Introduction to management science (second edition). Irwin. [MGS]
266. Stocks, J. T. (1987) Estimating proportion of explained variance for selected analysis of variance designs. Journal of Social Service Research 11(1):77-91. [BAT]
267. Stout, D. A. (1987) Statistics in American Psychology: The social construction of experimental and correlational psychology, 1900-1930. Unpublished Doctoral Dissertation, University of Edinburgh. [HJS]
268. Sun Li Hsu, J. S. J., Guttman, I. & Leonard, T. (1996) Bayesian methods for variance component models. Journal of the American Statistical Association 91:743-52. [RAMG]
269. Thompson, B. (1996) AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher 25(2):26-30. [CFB, aSLC]
270. Townsend, J. T. (1994) Methodology and statistics in the behavioural sciences. Psychological Science 5:321-25. [RAMG]
271. Tukey, J. W. (1960) Conclusions vs. decisions. Technometrics 2:1-11. [aSLC]
272. (1969) Analyzing data: Sanctification or detective work. American Psychologist 24:83-91. [CFB]
273. Tversky, A. & Kahneman, D. (1971) Belief in the law of small numbers. Psychological Bulletin 76:105-10. [CFB, JG]
274. Utts, J. M. (1996) Seeing through statistics. Duxbury Press. [NGW]
275. Van Fraassen, B. (1989) Laws and symmetry. Oxford University Press. [EE]
276. Vitouch, O. & Glück, J. (1997) "Small group PETting:" Sample sizes in brain mapping research. Human Brain Mapping 5:74-77. [JG]
277. Walizer, M. H. & Wiener, P. L. (1978) Research methods and analysis: Searching for relationships. Harper and Row. [LGT]
278. Waller, N. G., Tellegen, A., McDonald, R. & Lykken, D. T. (1996) Exploring nonlinear models in personality assessment: Development and preliminary validation of a negative emotionality scale. Journal of Personality 64:545-76. [NGW]
279. Walley, P. (1991) Statistical reasoning with imprecise probabilities. Chapman and Hall. [RAMG]
280. Wilson, G. T. & Rachman, S. J. (1983) Meta-analysis and the evaluation of psychotherapy outcome: Limitations and liabilities. Journal of Consulting and Clinical Psychology 51:54-64. [aSLC]
281. Winston, W. L. (1991) Operations research: Applications and algorithms (second edition). PWS-Kent. [MGS]
282. Yates, F. (1951) The influence of Statistical methods for research workers on the development of the science of statistics. Journal of the American Statistical Association 46:19-34. [MRN]
283. Yngve, V. (1960) A model and an hypothesis for language structure. Proceedings of the American Philosophical Society 104:444-66. [aSLC]
284. Zechmeister, E. B. & Nyberg, S. E. (1982) Human memory: An introduction to research and theory. Brooks/Cole. [JSR]