Lack of group-to-individual generalizability is a threat to human subjects research

Proceedings of the National Academy of Sciences of the United States of America

Volumn 115, Issue 27, 2018, Pages E6106-E6115

  Fisher, Aaron J ^a   Medaglia, John D ^b,c   Jeronimus, Bertus F ^d  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b DREXEL UNIVERSITY COLLEGE OF MEDICINE   (United States)

^c UNIVERSITY OF PENNSYLVANIA   (United States)

^d UNIVERSITY OF GRONINGEN   (Netherlands)

Author keywords

Ecological fallacy; Generalizability; Idiographic science; Replicability; Research methodology

Indexed keywords

ARTICLE; BIOLOGICAL VARIATION; COGNITIVE THERAPY; FEMALE; GENERALIZED ANXIETY DISORDER; GROUP TO INDIVIDUAL GENERALIZABILITY; HUMAN; MAJOR CLINICAL STUDY; MAJOR DEPRESSION; MALE; MEDICAL RESEARCH; MEDICINE; METHODOLOGY; MINDFULNESS; PERSONALITY DISORDER; PRIORITY JOURNAL; SOCIOLOGY; STATISTICAL ANALYSIS; STATISTICAL PARAMETERS; ANXIETY DISORDER; BIPOLAR DISORDER; CONTROLLED STUDY; GROUP THERAPY; RANDOMIZED CONTROLLED TRIAL;

ANXIETY DISORDERS; BIPOLAR DISORDER; FEMALE; HUMANS; MALE; PSYCHOTHERAPY, GROUP;

EID: 85049381357     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1711978115     Document Type: Article

References (49)

1. Molenaar PCM (2004) A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement 2:201–218.
2. Molenaar PCM, Campbell CG (2009) The new person-specific paradigm in psychology. Curr Dir Psychol Sci 18:112–117.
3. Quine WV (1969) Natural Kinds. Essays in Honor of Carl G. Hempel (Springer, New York), pp 5–23.
4. Wilson EO (1998) Consilience among the great branches of learning. Daedalus 127: 131–149.
5. Carrier M (2012) The Completeness of Scientific Theories: On the Derivation of Empirical Indicators Within a Theoretical Framework: The Case of Physical Geometry (Springer Science and Business Media, Dordrecht, The Netherlands).
6. Hamaker EL, Dolan CV, Molenaar PCM (2005) Statistical modeling of the individual: Rationale and application of multivariate stationary time series analysis. Multivariate Behav Res 40:207–233.
7. Salvatore S, Valsiner J (2010) Between the general and the unique. Theory Psychol 20: 817–833.
8. Medaglia JD, Ramanathan DM, Venkatesan UM, Hillary FG (2011) The challenge of non-ergodicity in network neuroscience. Network 22:148–153.
9. Molenaar PCM (2008) On the implications of the classical ergodic theorems: Analysis of developmental processes has to focus on intra-individual variation. Dev Psychobiol 50:60–69.
10. Lamiell JT (1981) Toward an idiothetic psychology of personality. Am Psychol 36: 276–289.
11. Castro-Schilo L, Ferrer E (2013) Comparison of nomothetic versus idiographic-oriented methods for making predictions about distal outcomes from time series data. Multivariate Behav Res 48:175–207.
12. Simpson EH (1951) The interpretation of interaction in contingency tables. J R Stat Soc B 13:238–241.
13. Robinson WS (1950) Ecological correlations and the behavior of individuals. Am Sociol Rev 15:351–357.
14. Hamaker E (2012) Why researchers should think “within-person”: A paradigmatic rationale. Handbook of Research Methods for Studying Daily Life (The Guilford Press, New York), pp 43–61.
15. Beltz AM, Wright AGC, Sprague BN, Molenaar PCM (2016) Bridging the nomothetic and idiographic approaches to the analysis of clinical data. Assessment 23:447–458.
16. Fisher AJ (2015) Toward a dynamic model of psychological assessment: Implications for personalized care. J Consult Clin Psychol 83:825–836.
17. Pearl J (2009) Causality: Models, Reasoning, and Inference (Cambridge Univ Press, Cambridge, UK).
18. Pearl J (1999) Simpson’s paradox: An anatomy (UCLA Cognitive Systems Laboratory, Los Angeles), Technical Report R-264.
19. Boltzmann L (1877) On the relation between the second law of the mechanical theory of heat and the probability calculus with respect to the theorems on thermal equilibrium. Kais Akad Wiss Wien Math Natumiss Classe 76:373–435.
20. Fisher AJ, Reeves JW, Lawyer G, Medaglia JD, Rubel JA (2017) Exploring the idiographic dynamics of mood and anxiety via network analysis. J Abnorm Psychol 126: 1044–1056.
21. American Psychiatric Association (2013) The Diagnostic and Statistical Manual of Mental Disorders: DSM-5 (American Psychiatric Association Publishing, Washington, DC).
22. Fisher AJ, Woodward SH (2014) Cardiac stability at differing levels of temporal analysis in panic disorder, post-traumatic stress disorder, and healthy controls. Psychophysiology 51:80–87.
23. Wright AG, Hopwood CJ, Simms LJ (2015) Daily interpersonal and affective dynamics in personality disorder. J Pers Disord 29:503–525.
24. Wright AG, Simms LJ (2016) Stability and fluctuation of personality disorder features in daily life. J Abnorm Psychol 125:641–656.
25. Watson D, Clark LA, Tellegen A (1988) Development and validation of brief measures of positive and negative affect: The PANAS scales. J Pers Soc Psychol 54: 1063–1070.
26. Barge-Schaapveld DQCM, Nicolson NA (2002) Effects of antidepressant treatment on the quality of daily life: An experience sampling study. J Clin Psychiatry 63: 477–485.
27. Snippe E, et al. (2017) The impact of treatments for depression on the dynamic network structure of mental states: Two randomized controlled trials. Sci Rep 7:46523.
28. Geschwind N, Peeters F, Drukker M, van Os J, Wichers M (2011) Mindfulness training increases momentary positive emotions and reward experience in adults vulnerable to depression: A randomized controlled trial. J Consult Clin Psychol 79:618–628.
29. van der Krieke L, et al. (2016) HowNutsAreTheDutch (HoeGekIsNL): A crowdsourcing study of mental symptoms and strengths. Int J Methods Psychiatr Res 25:123–144.
30. Matthews JN, Altman DG, Campbell MJ, Royston P (1990) Analysis of serial measurements in medical research. BMJ 300:230–235.
31. Beck N, Katz JN, Tucker R (1998) Taking time seriously: Time-series-cross-section analysis with a binary dependent variable. Am J Pol Sci 42:1260–1288.
32. DiPrete TA, Forristal JD (1994) Multilevel models: Methods and substance. Annu Rev Sociol 20:331–357.
33. Molenaar PCM (2005) Rejoinder to Rogosa’s commentary on “A manifesto on psychology as idiographic science.” Measurement 3:116–119.
34. Durbin J, Watson GS (1950) Testing for serial correlation in least squares regression. I. Biometrika 37:409–428.
35. Kessler RC, et al. (2008) Co-morbid major depression and generalized anxiety disorders in the National Comorbidity Survey follow-up. Psychol Med 38:365–374.
36. Craske MG, Waters AM (2005) Panic disorder, phobias, and generalized anxiety disorder. Annu Rev Clin Psychol 1:197–225.
37. Cacioppo JT, Tassinary LG, Berntson G (2007) Handbook of Psychophysiology (Cambridge Univ Press, Cambridge, UK).
38. Watson D (2005) Rethinking the mood and anxiety disorders: A quantitative hierarchical model for DSM-V. J Abnorm Psychol 114:522–536.
39. Kievit RA, Frankenhuis WE, Waldorp LJ, Borsboom D (2013) Simpson’s paradox in psychological science: A practical guide. Front Psychol 4:513.
40. Tyekucheva S, Parmigiani G (2017) Bioinformatic analysis of epidemiological and pathological data. Pathology and Epidemiology of Cancer, eds Loda M, Mucci LA, Mittelstadt ML, Van Hemelrijck M, Cotter MB (Springer International Publishing, Cham, Switzerland), pp 91–104.
41. Anderson DA, Aitkin M (1985) Variance component models with binary response: Interviewer variability. J R Stat Soc B 47:203–210.
42. Kang HM, et al. (2010) Variance component model to account for sample structure in genome-wide association studies. Nat Genet 42:348–354.
43. Snijders TA (2011) Multilevel analysis. International Encyclopedia of Statistical Science, ed Lovric M (Springer, Berlin), pp 879–882.
44. Kim J, Zhu W, Chang L, Bentler PM, Ernst T (2007) Unified structural equation modeling approach for the analysis of multisubject, multivariate functional MRI data. Hum Brain Mapp 28:85–93.
45. Wei WW (2006) Time series analysis. The Oxford Handbook of Quantitative Methods in Psychology (Oxford Univ Press, Oxford), Vol 2.
46. Maxwell SE, Lau MY, Howard GS (2015) Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? Am Psychol 70:487–498.
47. Harris R (2017) Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions (Hachette Book Group, New York).
48. Bernau C, et al. (2014) Cross-study validation for the assessment of prediction algorithms. Bioinformatics 30:i105–i112.
49. Monajemi H, Donoho DL, Stodden V (2016) Making massive computational experiments painless. 2016 IEEE International Conference on Big Data (Big Data) (IEEE, New York), pp 2368–2373.