The empirical benefits of conceptual rigor: Systematic articulation of conceptual hypotheses can reduce the risk of non-replicable results (and facilitate novel discoveries too)

Journal of Experimental Social Psychology

Volumn 66, Issue , 2016, Pages 107-115

  Schaller, Mark ^a  

^a NONE

Author keywords

Effect size; False negatives; False positives; Generalizability; Hypotheses; Replication; Research practices

Indexed keywords

EID: 84978139614     PISSN: 00221031     EISSN: 10960465     Source Type: Journal    
DOI: 10.1016/j.jesp.2015.09.006     Document Type: Article

References (60)

1. Abelson, R.P., A variance explained paradox: When a little is a lot. Psychological Bulletin 97 (1985), 129–133.
2. Abelson, R.P., Attitude extremity. Petty, R.E., Krosnick, J.A., (eds.) Attitude strength: Antecedents and consequences, 1995, Erlbaum, Mahwah NJ, 25–42.
3. Abelson, R.P., Prentice, D.A., Beliefs as possessions: A functional perspective. Pratkanis, A.R., Breckler, S.J., Greenwald, A.G., (eds.) Attitude structure and function, 1989, Erlbaum, Hillsdale NJ, 361–381.
4. Asendorpf, J.B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J.J.A., Fiedler, K., et al. Recommendations for increasing replicability in psychology. European Journal of Personality 27 (2013), 108–119.
5. Bargh, J.A., The four horsemen of automaticity: Awareness, intention, efficiency, and control in social cognition. Wyer, R.S., Srull, T.K., (eds.) Handbook of social cognition, 2nd ed., 1994, Erlbaum, Hillsdale, NJ, 1–40.
6. Bargh, J.A., Chen, M., Burrows, L., Automaticity of social behavior: Direct effects of trait construct and stereotype activation on action. Journal of Personality and Social Psychology 71 (1996), 230–244.
7. Bem, D.J., Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology 100 (2011), 407–425.
8. Brandt, M.J., IJzerman, H., Dijksterhuis, A., Farach, F.J., Geller, J., Giner-Sorolla, R., et al. The replication recipe: What makes for a convincing replication?. Journal of Experimental Social Psychology 50 (2014), 217–224.
9. Brewer, M.B., Dull, V., Lui, L., Perceptions of the elderly: Stereotypes as prototypes. Journal of Personality and Social Psychology 41 (1981), 656–670.
10. Brown, S.D., Furrow, D., Hill, D.F., Gable, J.C., Porter, L.P., Jacobs, W.J., The duty to describe: Better the devil you know than the devil you don't. Perspectives on Psychological Science 9 (2014), 626–640.
11. Buckels, E.E., Beall, A.T., Hofer, M.K., Lin, E., Zhou, Z., Schaller, M., Individual differences in activation of the parental care motivational system: Assessment, prediction, and implications. Journal of Personality and Social Psychology 108 (2015), 497–514.
12. Cartwright, D., Determinants of scientific progress: The case of research on the risky shift. American Psychologist 28 (1973), 222–231.
13. Cesario, J., Priming, replication, and the hardest science. Perspectives on Psychological Science 9 (2014), 40–48.
14. Cesario, J., Plaks, J.E., Hagiwara, N., Navarrete, C.D., Higgins, E.T., The ecology of automaticity: How situational contingencies shape action semantics and social behavior. Psychological Science 21 (2010), 1311–1317.
15. Cesario, J., Plaks, J.E., Higgins, E.T., Automatic social behavior as motivated preparation to interact. Journal of Personality and Social Psychology 90 (2006), 893–910.
16. Chamberlin, T.C., The method of multiple working hypotheses. Science 148 (1890/1965), 754–759 Originally published in Science in 1890.
17. Cialdini, R.B., Darby, B.L., Vincent, J.E., Transgression and altruism: A case for hedonism. Journal of Experimental Social Psychology 9 (1973), 502–516.
18. Dienes, Z., Using Bayes to get the most out of non-significant results. Frontiers in Psychology, 5, 2014, 781, 10.3389/fpsyg.2014.00781.
19. Dijksterhuis, A., Welcome back, theory!. Perspectives on Psychological Science 9 (2014), 72–75.
20. Doyen, S., Klein, O., Pichon, C.-L., Cleeremans, A., Behavioral priming: It's all in the mind, but whose mind?. PloS One, 7, 2012, e29081, 10.1371/journal.pone.0029081.
21. Eich, E., Business not as usual. Psychological Science 25 (2014), 3–6.
22. Festinger, L., A theory of social comparison processes. Human Relations 7 (1954), 117–140.
23. Feyerabend, P., Against method. 1975, New Left, London UK.
24. Fiedler, K., Kutzner, F., Krueger, J.I., The long way from α-error control to validity proper: Problems with a short-sighted false-positive debate. Perspectives on Psychological Science 7 (2012), 661–669.
25. Finkel, E.J., Eastwick, P.W., Reis, H.T., Best research practices in psychology: Illustrating epistemological and pragmatic considerations with the case of relationship science. Journal of Personality and Social Psychology 108 (2015), 275–297.
26. Funder, D.C., Levine, J.M., Mackie, D.M., Morf, C.C., Vazire, S., West, S.G., Improving the dependability of research in personality and social psychology: Recommendations for research and educational practice. Personality and Social Psychology Review 18 (2014), 3–12.
27. García Márquez, G. (1995). Of love and other demons. New York: Knopf. (Originally published as Del amor y otros demonios. Barcelona, Spain: Mondadori. Translated by Edith Grossman.).
28. Gawronski, B., Bodenhausen, G.V., Theory and explanation in social psychology. 2015, Guilford Press, New York.
29. Gawronski, B., Cesario, J., Of mice and men: What animal research can tell us about context effects on automatic responses in humans. Personality and Social Psychology Review 17 (2013), 187–215.
30. Glass, D.J., Hall, N., A brief history of the hypothesis. Cell 134 (2008), 378–381.
31. Greenwald, A.G., Pratkanis, A.R., Leippe, M.R., Baumgardner, M.H., Under what conditions does theory obstruct research progress?. Psychological Review 93 (1986), 216–229.
32. Higgins, E.T., Value from regulatory fit. Current Directions in Psychological Science 14 (2005), 209–213.
33. Klatzky, R.L., Creswell, J.D., An intersensory interaction account of priming effects—and their absence. Perspectives on Psychological Science 9 (2014), 49–58.
34. Klein, S.B., What can recent replication failures tell us about the theoretical commitments of psychology?. Theory and Psychology 24 (2014), 326–338.
35. Kruglanski, A.W., Pierro, A., Higgins, E.T., Experience of time by people on the go: A theory of the locomotion–temporality interface. Personality and Social Psychology Review, 2015, 10.1177/1088868315581120 (in press).
36. Kruglanski, A.W., That ‘vision thing’: The state of theory in social and personality psychology at the edge of the new millennium. Journal of Personality and Social Psychology 80 (2001), 871–875.
37. Kunda, Z., The case for motivated reasoning. Psychological Bulletin 108 (1990), 480–498.
38. Loersch, C., Payne, B.K., The situated inference model: An integrative account of the effects of primes on perception, behavior, and motivation. Perspectives on Psychological Science 6 (2011), 234–252.
39. MacCoun, R.J., Biases in the intrepretation and use of research results. Annual Review of Psychology 49 (1998), 259–287.
40. Maner, J.K., Let's put our money where our mouth is: If authors are to change their ways, reviewers (and editors) must change with them. Perspectives on Psychological Science 9 (2014), 343–351.
41. Maner, J.K., Into the wild: Field research can increase both replicability and real-world impact. Journal of Experimental Social Psychology 66 (2016), 100–106 (in this issue).
42. Morewedge, C.K., Shu, L.L., Gilbert, D.T., Wilson, T.D., Bad riddance or good rubbish: Ownership and not loss aversion cause the endowment effect. Journal of Experimental Social Psychology 45 (2009), 947–951.
43. Murayama, K., Pekrun, R., Fiedler, K., Research practices that can prevent an inflation of false positive rates. Personality and Social Psychology Review 18 (2014), 107–118.
44. Nickerson, R.S., Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology 2 (1998), 175–220.
45. Open Science Collaboration. An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science 7 (2012), 657–660.
46. Open Science Collaboration. Estimating the reproducibility of psychological science. Science, 349(6251), 2015, 10.1126/science.aac4716.
47. Pashler, H., Harris, C.R., Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science 7 (2012), 531–536.
48. Platt, J.R., Strong inference. Science 146 (1964), 347–353.
49. Popper, K., The logic of scientific discovery. 1959/2005, Routledge, New York Originally published in 1934 as Logik der Forschung. Vienna, Austria: Verlag von Julius Springer.
50. Prentice, D.A., Miller, D.T., When small effects are impressive. Psychological Bulletin 112 (1992), 160–164.
51. Rosnow, R.L., Rosenthal, R., Rubin, D.B., Contrasts and correlations in effect-size estimation. Psychological Science 11 (2000), 446–453.
52. Sakaluk, J.K., Exploring small, confirming big: An alternative system to The New Statistics for advancing cumulative and replicable psychological research. Journal of Experimental Social Psychology 66 (2016), 47–54 (in this issue).
53. Schaller, M., Crandall, C.S., On the purposes served by psychological research and its critics. Theory and Psychology 8 (1998), 205–212.
54. Schaller, M., Crandall, C.S., Stangor, C., Neuberg, S.L., “What kinds of social psychology experiments are of value to perform?” Comment on Wallach and Wallach (1994). Journal of Personality and Social Psychology 69 (1995), 611–618.
55. Schmidt, F.L., Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods 1 (1996), 115–129.
56. Schweinsberg, M., Madan, N., Vianello, M., Sommer, S.A., Jordan, J., et al. The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline. Journal of Experimental Social Psychology 66 (2016), 55–67 (in this issue).
57. Simmons, J.P., Nelson, L.D., Simonsohn, U., False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22 (2011), 1359–1366.
58. Stroebe, W., Strack, F., The alleged crisis and the illusion of exact replication. Perspectives on Psychologial Science 9 (2014), 59–71.
59. Tesser, A., Self-generated attitude change. Advances in Experimental Social Psychology 11 (1978), 289–338.
60. Wallach, L., Wallach, M.A., Gergen versus the mainstream: Are hypotheses in social psychology subject to empirical test?. Journal of Personality and Social Psychology 67 (1994), 233–242.