On Using Bayesian Methods to Address Small Sample Problems

Structural Equation Modeling

Volumn 23, Issue 5, 2016, Pages 750-773

  McNeish, Daniel ^a  

^a UTRECHT UNIVERSITY   (Netherlands)

Author keywords

Bayes; prior distribution; small sample

Indexed keywords

BAYESIAN NETWORKS;

BAYES; BAYESIAN ESTIMATE; EMPIRICAL STUDIES; FREQUENTIST METHODS; PRIOR DISTRIBUTION; SMALL SAMPLE DATUM; SMALL SAMPLE SIZE; SMALL SAMPLES;

FISHER INFORMATION MATRIX;

EID: 84974808077     PISSN: 10705511     EISSN: 15328007     Source Type: Journal    
DOI: 10.1080/10705511.2016.1186549     Document Type: Article

References (76)

1. S.Ahn, A.J.Ames, & N.D.Myers, (2012). A review of meta-analyses in education:Methodological strengths and weaknesses. Review of Educational Research, 82, 436–476. doi:10.3102/0034654312458162
2. T.Asparouhov, & B.Muthén, (2010). Bayesian analysis of latent variable models using Mplus (Tech. Rep.). Retrieved from http://www.statmodel2.com/download/BayesAdvantages18.pdf
3. S.A.Baldwin, & G.W.Fellingham, (2013). Bayesian methods for the analysis of small sample multilevel data with a complex variance structure. Psychological Methods, 18, 151–164. doi:10.1037/a0030642
4. M.S.Bartlett, (1950). Tests of significance in factor analysis. British Journal of Statistical Psychology, 3, 77–85. doi:10.1111/bmsp.1950.3.issue-2
5. D.J.Bauer, & P.J.Curran, (2003). Distributional assumptions of growth mixture models:Implications for overextraction of latent trajectory classes. Psychological Methods, 8, 338–363. doi:10.1037/1082-989X.8.3.338
6. P.M.Bentler, & K.H.Yuan, (1999). Structural equation modeling with small samples:Test statistics. Multivariate Behavioral Research, 34, 181–197. doi:10.1207/S15327906Mb340203
7. J.V.Bradley, (1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31, 144–152. doi:10.1111/bmsp.1978.31.issue-2
8. W.J.Browne, & D.Draper, (2006). A comparison of Bayesian and likelihood-based methods for fitting multilevel models. Bayesian Analysis, 1, 473–514. doi:10.1214/06-BA117
9. M.W.L.Cheung, (2008). A model for integrating fixed-, random-, and mixed-effects meta-analyses into structural equation modeling. Psychological Methods, 13, 182–202. doi:10.1037/a0013163
10. G.Cumming, (2013). The new statistics why and how. Psychological Science, 25, 7–29. doi:10.1177/0956797613504966
11. P.J.Curran, (2003). Have multilevel models been structural equation models all along? Multivariate Behavioral Research, 38, 529–569. doi:10.1207/s15327906mbr3804_5
12. P.J.Curran, & D.J.Bauer, (2007). Building path diagrams for multilevel models. Psychological Methods, 12, 283–297. doi:10.1037/1082-989X.12.3.283
13. S.Depaoli, (2013). Mixture class recovery in GMM under varying degrees of class separation:Frequentist versus Bayesian estimation. Psychological Methods, 18, 186–219. doi:10.1037/a0031609
14. S.Depaoli, (2014). The impact of inaccurate “informative” priors for growth parameters in Bayesian growth mixture modeling. Structural Equation Modeling, 21, 239–252. doi:10.1080/10705511.2014.882686
15. S.Depaoli, & J.P.Clifton, (2015). A Bayesian approach to multilevel structural equation modeling with continuous and dichotomous outcomes. Structural Equation Modeling, 22, 327–351. doi:10.1080/10705511.2014.937849
16. S.Depaoli, & R.van de Schoot, (2015). Improving transparency and replication in Bayesian statistics:The WAMBS-Checklist. Psychological Methods. Advance online publication. doi:10.1037/met0000065
17. J.Doron, & P.Gaudreau, (2014). A point-by-point analysis of performance in a fencing match:Psychological processes associated with winning and losing streaks. Journal of Sport & Exercise Psychology, 36, 3–13. doi:10.1123/jsep.2013-0043
18. D.B.Dunson, (2000). Bayesian latent variable models for clustered mixed outcomes. Journal of the Royal Statistical Society:Series B, 62, 355–366. doi:10.1111/1467-9868.00236
19. D.B.Dunson, (2001). Commentary:Practical advantages of Bayesian analysis of epidemiologic data. American Journal of Epidemiology, 153, 1222–1226. doi:10.1093/aje/153.12.1222
20. S.M.Eldridge, D.Ashby, G.S.Feder, A.R.Rudnicka, & O.C.Ukoumunne, (2004). Lessons for cluster randomized trials in the twenty-first century:A systematic review of trials in primary care. Clinical Trials, 1, 80–90. doi:10.1191/1740774504cn006rr
21. C.K.Enders, & D.Tofighi, (2007). Centering predictor variables in cross-sectional multilevel models:A new look at an old issue. Psychological Methods, 12, 121–138. doi:10.1037/1082-989X.12.2.121
22. J.M.Ferron, B.A.Bell, M.R.Hess, G.Rendina-Gobioff, & S.T.Hibbard, (2009). Making treatment effect inferences from multiple-baseline data:The utility of multilevel modeling approaches. Behavior Research Methods, 41, 372–384. doi:10.3758/BRM.41.2.372
23. A.Gelman, (2006). Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper). Bayesian Analysis, 1, 515–534. doi:10.1214/06-BA117A
24. A.Gelman, J.Hill, & M.Yajima, (2012). Why we (usually) don’t have to worry about multiple comparisons. Journal of Research on Educational Effectiveness, 5, 189–211. doi:10.1080/19345747.2011.618213
25. A.Gelman, & D.B.Rubin, (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472. doi:10.1214/ss/1177011136
26. H.Goldstein, M.Yang, R.Omar, R.Turner, & S.Thompson, (2000). Meta-analysis using multilevel models with an application to the study of class size effects. Applied Statistics, 49, 399–412.
27. D.A.Harville, (1977). Maximum likelihood approaches to variance component estimation and to related problems. Journal of the American Statistical Association, 72, 320–338. doi:10.1080/01621459.1977.10480998
28. L.V.Hedges, & E.C.Hedberg, (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29, 60–87. doi:10.3102/0162373707299706
29. J.J.Hoogland, & A.Boomsma, (1998). Robustness studies in covariance structure modeling:An overview and a meta-analysis. Sociological Methods & Research, 26, 329–367. doi:10.1177/0049124198026003003
30. J.Hox, (2010). Multilevel analysis:Techniques and applications. New York, NY:Routledge.
31. J.Hox, R.van de Schoot, & S.Matthijsse, (2012). How few countries will do? Comparative survey analysis from a Bayesian perspective. Survey Research Methods, 6, 87–93.
32. S.Jackman, (2009). Bayesian analysis for the social sciences. New York, NY:Wiley.
33. M.Jordan, (2011). What are the open problems in Bayesian statistics. The ISBA Bulletin, 18, 1–4.
34. J.B.Kadane, (2015). Bayesian methods for prevention research. Prevention Science, 16, 1017–1025. doi:10.1007/s11121-014-0531-x
35. M.G.Kenward, & J.H.Roger, (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53, 983–997. doi:10.2307/2533558
36. M.G.Kenward, & J.H.Roger, (2009). An improved approximation to the precision of fixed effects from restricted maximum likelihood. Computational Statistics & Data Analysis, 53, 2583–2595. doi:10.1016/j.csda.2008.12.013
37. S.Kliem, C.Kröger, & J.Kosfelder, (2010). Dialectical behavior therapy for borderline personality disorder:A meta-analysis using mixed-effects modeling. Journal of Consulting and Clinical Psychology, 78, 936–951. doi:10.1037/a0021015
38. J.K.Kruschke, (2010). Doing Bayesian data analysis:A tutorial with R and BUGS. Burlington, MA:Academic.
39. P.C.Lambert, A.J.Sutton, P.R.Burton, K.R.Abrams, & D.R.Jones, (2005). How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS. Statistics in Medicine, 24, 2401–2428. doi:10.1002/(ISSN)1097-0258
40. S.Y.Lee, & X.Y.Song, (2004). Evaluation of the Bayesian and maximum likelihood approaches in analyzing structural equation models with small sample sizes. Multivariate Behavioral Research, 39, 653–686. doi:10.1207/s15327906mbr3904_4
41. R.Levy, (2009). The rise of Markov chain Monte Carlo estimation for psychometric modeling. Journal of Probability and Statistics, 2009, 1–18. doi:10.1155/2009/537139
42. R.Levy, & J.Choi, (2013). Bayesian structural equation modeling. In G.R.Hancock & R.O.Mueller (Eds.), Structural equation modeling:A second course (2nd ed., pp. 563–623). Charlotte, NC:Information Age.
43. H.Liu, Z.Zhang, & K.J.Grimm, (2016). Comparison of inverse Wishart and separation-strategy priors for Bayesian estimation of covariance parameter matrix in growth curve analysis. Structural Equation Modeling, 23, 354–367. doi:10.1080/10705511.2015.1057285
44. R.C.MacCallum, & J.T.Austin, (2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51, 201–226. doi:10.1146/annurev.psych.51.1.201
45. Y.C.McNab, D.C.Malloy, T.Hadjistavropoulos, P.R.Sevigny, E.F.McCarthy, M.Murakami, … P.L.Liu, (2010). Idealism and relativism across cultures:A cross-cultural examination of physicians’ responses on the Ethics Position Questionnaire. Journal of Cross-Cultural Psychology, 42, 1272–1278. doi:10.1177/0022022110383313
46. D.M.McNeish, (2016). Using data-dependent priors to mitigate small sample bias in latent growth models:A discussion and illustration using Mplus. Journal of Educational and Behavioral Statistics, 41, 27–56. doi:10.3102/1076998615621299
47. D.McNeish, & L.M.Stapleton, (2014). The effect of small sample size on two level model estimates:A review and illustration. Educational Psychology Review, 28, 295–314. doi:10.1007/s10648-014-9287-x
48. D.McNeish, & L.M.Stapleton, (2016). Modeling clustered data with very few clusters. Multivariate Behavioral Research. Advance online publication. doi:10.1080/00273171.2016.1167008
49. T.M.Moore, S.P.Reise, S.Depaoli, & M.G.Haviland, (2015). Iteration of partially specified target matrices:Applications in exploratory and Bayesian confirmatory factor analysis. Multivariate Behavioral Research, 50, 149–161. doi:10.1080/00273171.2014.973990
50. B.Muthén, (2010). Bayesian analysis in Mplus:A brief introduction (Unpublished manuscript). Retrieved from https://www.statmodel.com/download/IntroBayesVersion%203.pdf
51. B.Muthén, & T.Asparouhov, (2012). Bayesian structural equation modeling:A more flexible representation of substantive theory. Psychological Methods, 17, 313–335. doi:10.1037/a0026802
52. A.J.O’Malley, & A.M.Zaslavsky, (2005). Variance-covariance functions for domain means of ordinal survey items. Survey Methodology, 31, 169–182.
53. N.G.Polson, & J.G.Scott, (2012). On the half-Cauchy prior for a global scale parameter. Bayesian Analysis, 7, 887–902. doi:10.1214/12-BA730
54. R.F.Potthoff, & S.N.Roy, (1964). A generalized multivariate analysis of variance model useful especially for growth curve problems. Biometrika, 51, 313–326. doi:10.1093/biomet/51.3-4.313
55. L.R.Price, (2012). Small sample properties of Bayesian multivariate autoregressive time series models. Structural Equation Modeling, 19, 51–64. doi:10.1080/10705511.2012.634712
56. S.W.Raudenbush, & A.S.Bryk, (2002). Hierarchical linear models:Applications and data analysis methods. Thousand Oaks, CA:Sage.
57. D.W.Russell, (2002). In search of underlying dimensions:The use (and abuse) of factor analysis in Personality and Social Psychology Bulletin. Personality and Social Psychology Bulletin, 28, 1629–1646. doi:10.1177/014616702237645
58. R.Scheines, H.Hoijtink, & A.Boomsma, (1999). Bayesian estimation and testing of structural equation models. Psychometrika, 64, 37–52. doi:10.1007/BF02294318
59. J.A.Schoeneberger, (2015). The impact of sample size and other factors when estimating multilevel logistic models. The Journal of Experimental Education, 84, 3.
60. S.S.Skene, & M.G.Kenward, (2010a). The analysis of very small samples of repeated measurements I:An adjusted sandwich estimator. Statistics in Medicine, 29, 2825–2837. doi:10.1002/sim.v29:27
61. S.S.Skene, & M.G.Kenward, (2010b). The analysis of very small samples of repeated measurements II:A modified Box correction. Statistics in Medicine, 29, 2838–2856. doi:10.1002/sim.v29:27
62. T.M.Soares, F.B.Gonçalves, & D.Gamerman, (2009). An integrated Bayesian model for DIF analysis. Journal of Educational and Behavioral Statistics, 34, 348–377. doi:10.3102/1076998609332752
63. V.S.Staggs, (2015). Comparison of naïve, Kenward–Roger, and parametric bootstrap interval approaches to small-sample inference in linear mixed models. Communications in Statistics:Simulation and Computation. Advance online publication. doi:10.1080/03610918.2015.1019002
64. D.Stegmueller, (2013). How many countries for multilevel modeling? A comparison of frequentist and Bayesian approaches. American Journal of Political Science, 57, 748–761. doi:10.1111/ajps.2013.57.issue-3
65. A.Stenling, A.Ivarsson, U.Johnson, & M.Lindwall, (2015). Bayesian structural equation modeling in sport and exercise psychology. Journal of Sport & Exercise Psychology, 37, 410–420. doi:10.1123/jsep.2014-0330
66. A.J.Swain, (1975). Analysis of parametric structures for variance matrices (Unpublished doctoral dissertation). Department of Statistics, University of Adelaide, Adelaide, Australia.
67. R.van de Schoot, (2016). 25 years of Bayes in psychology. Paper presented at the 7th Mplus Users’ Meeting, Utrecht, The Netherlands. Retrieved from http://mplus.fss.uu.nl/wp-content/uploads/sites/24/2012/07/opening-review-short.pptx
68. R.van de Schoot, J.J.Broere, K.H.Perryck, M.Zondervan-Zwijnenburg, & N.E.van Loey, (2015). Analyzing small data sets using Bayesian estimation:The case of posttraumatic stress symptoms following mechanical ventilation in burn survivors. European Journal of Psychotraumatology, 6. doi:10.3402/ejpt.v6.25216
69. R.van de Schoot, D.Kaplan, J.Denissen, J.B.Asendorpf, F.J.Neyer, & M.A.Aken, (2014). A gentle introduction to Bayesian analysis:Applications to developmental research. Child Development, 85, 842–860. doi:10.1111/cdev.12169
70. R.van de Schoot, O.Ryan, S.Winter, M.A.J.Zondervan-Zwijenburg, & S.Depaoli, (under review). A systematic review of empirical Bayesian applications in psychology.
71. W.van den Noortgate, & P.Onghena, (2003). Multilevel meta-analysis:A comparison with traditional meta-analytical procedures. Educational and Psychological Measurement, 63, 765–790. doi:10.1177/0013164403251027
72. S.B.Wanless, S.E.Rimm-Kaufman, T.Abry, R.A.Larsen, & C.L.Patton, (2015). Engagement in training as a mechanism to understanding fidelity of implementation of the responsive classroom approach. Prevention Science, 16, 1107–1116. doi:10.1007/s11121-014-0519-6
73. K.H.Yuan, (2005). Fit indices versus test statistics. Multivariate Behavioral Research, 40, 115–148. doi:10.1207/s15327906mbr4001_5
74. K.H.Yuan, & P.M.Bentler, (1997). Mean and covariance structure analysis:Theoretical and practical improvements. Journal of the American Statistical Association, 92, 767–774. doi:10.1080/01621459.1997.10474029
75. K.H.Yuan, & P.M.Bentler, (1999). F tests for mean and covariance structure analysis. Journal of Educational and Behavioral Statistics, 24, 225–243. doi:10.2307/1165323
76. Z.Zhang, F.Hamagami, L.L.Wang, J.R.Nesselroade, & K.J.Grimm, (2007). Bayesian analysis of longitudinal data using growth curve models. International Journal of Behavioral Development, 31, 374–383. doi:10.1177/0165025407077764