Sample Size Considerations in Prevention Research Applications of Multilevel Modeling and Structural Equation Modeling

Prevention Science

Volumn 16, Issue 7, 2015, Pages 987-996

  Hoyle, Rick H ^a   Gottfredson, Nisha C ^b  

^a DUKE UNIVERSITY   (United States)

^b UNIVERSITY OF NORTH CAROLINA   (United States)

Author keywords

Multilevel modeling; Sample size; Structural equation modeling

Indexed keywords

ORGANIZATION AND MANAGEMENT; PREVENTIVE HEALTH SERVICE; SAMPLE SIZE; THEORETICAL MODEL;

MODELS, THEORETICAL; PREVENTIVE HEALTH SERVICES; SAMPLE SIZE;

EID: 84942500263     PISSN: 13894986     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11121-014-0489-8     Document Type: Article

References (56)

1. Aitkin, M. A., & Longford, N. T. (1986). Statistical modeling issues in school effectiveness studies. Journal of the Royal Statistical Society, 149, 1–26. Retrieved from http://www.jstor.org/stable/2981882.
2. Anderson, T. W. (1957). Maximum likelihood estimates for a multivariate normal distribution when some observations are missing. Journal of the American Statistical Association, 52, 200–203. Retrieved from http://www.jstor.org/stable/2280845.
3. Austin, P. C. (2010). Estimating multilevel logistic regression models when the number of clusters is low: A comparison of different statistical software procedures. The International Journal of Biostatistics, 6, 1–18. doi:10.2202/1557-4679.1195.
4. Bandalos, D. L., & Gagné, P. (2012). Simulation methods in structural equation modeling. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (pp. 92–108). New York: Guildford Press.
5. Bauer, D. J. (2003). Estimating multilevel linear models as structural equation models. Journal of Educational and Behavioral Statistics, 28, 135–167. doi:10.3102/10769986028002135.
6. Bauer, D. J., & Sterba, S. K. (2011). Fitting multilevel models with ordinal outcomes: Performance of alternative specifications and methods of estimation. Psychological Methods, 16, 373–390. doi:10.1037/a0025813.
7. Bearden, W. O., Sharma, S., & Teel, J. E. (1982). Sample size effects on chi square and other statistics used in evaluating causal models. Journal of Marketing Research, 19, 425–430. doi:10.2307/3151716.
8. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246. doi:10.1037/0033-2909.107.2.238.
9. Bollen, K. A. (1990). Overall fit in covariance structure models: Two types of sample size effects. Psychological Bulletin, 107, 256–259. doi:10.1037/0033-2909.107.2.256.
10. Bollen, K. A., & Curran, P. J. (2006). Latent curve models: A structural equation approach. Hoboken: Wiley.
11. Bollen, K. A., & Noble, M. D. (2011). Structural equation models and the quantification of behavior. Proceedings of the National Academy of Sciences of the United States of America, 108, 15639–15646. doi:10.1073/pnas.1010661108.
12. Breslow, N. E., & Clayton, D. G. (1993). Approximate inference in generalized linear mixed models. Journal of the American Statistical Association, 88, 9–25. Retrieved from http://www.jstor.org/stable/2290687.
13. Curran, P. J. (2003). Have multilevel models been structural equation models all along? Multivariate Behavioral Research, 38, 529–569. doi:10.1207/s15327906mbr3804_5.
14. Curran, P. J., Lee, T., Howard, A. L., Lane, S., & MacCallum, R. (2012). Disaggregating within-person and between-person effects in multilevel and structural equation growth models. In J. R. Harring & G. R. Hancock (Eds.), Advances in longitudinal models in the social and behavioral sciences (pp. 217–253). Charlotte: Information Age Publishing.
15. Demidenko, E. (2004). Mixed models: Theory and applications. Hoboken: Wiley.
16. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 44, 1–38. Retrieved from http://www.jstor.org/stable/2984875?origin=JSTOR-pdf.
17. Duncan, S. C., Duncan, T. E., & Strycker, L. A. (2002). A multilevel analysis of neighborhood context and youth alcohol and drug problems. Prevention Science, 3, 125–133. doi:10.1023/A:1015483317310.
18. Fan, X., Thompson, B., & Wang, L. (1999). Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling, 6, 56–83. doi:10.1080/10705519909540119.
19. Gagné, P., & Hancock, G. R. (2006). Measurement model quality, sample size, and solution propriety in confirmatory factor models. Multivariate Behavioral Research, 41, 65–83. doi:10.1207/s15327906mbr4101_5.
20. Gillmore, M. R., Hawkins, J. D., Catalano, R. F., Jr., Day, L. E., Moore, M., & Abbott, R. (1991). Structure of problem behaviors in preadolescence. Journal of Consulting and Clinical Psychology, 59, 499–506. doi:10.1037/0022-006X.59.4.499.
21. Goldstein, H. (1986). Multilevel mixed linear model analysis using iterative generalized least squares. Biometrika, 73, 43–56. doi:10.1093/biomet/73.1.43.
22. Hopkin, C. R., Hoyle, R. H., & Gottfredson, N. C. (2013). Maximizing the yield of small samples in prevention research: A review of general strategies and best practices. Manuscript submitted for publication.
23. Hoyle, R. H. (2011). Structural equation modeling for social and personality psychology. London: Sage Publications.
24. Hoyle, R. H., & Kenny, D. A. (1999). Sample size, reliability, and tests of statistical mediation. In R. H. Hoyle (Ed.), Statistical strategies for small sample research (pp. 195–222). Thousand Oaks: Sage Publications.
25. Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118.
26. Jackson, D. L., Voth, J., & Frey, M. P. (2013). A note on sample size and solution propriety for confirmatory factor analytic models. Structural Equation Modeling, 20, 86–97. doi:10.1080/10705511.2013.742388.
27. Kaplan, D. (1995). Statistical power in structural equation modeling. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 100–117). Newbury Park: Sage Publications.
28. Kim, K. H. (2005). The relation among fit indexes, power, and sample size in structural equation modeling. Structural Equation Modeling, 12, 368–390. doi:10.1207/s15328007sem1203_2.
29. Kim, S.-Y. (2012). Sample size requirements in single- and multiphase growth mixture models: A Monte Carlo simulation. Structural Equation Modeling, 19, 457–476. doi:10.1080/10705511.2012.687672.
30. Kreft, I. G. G., & de Leeuw, J. (1988). Introducing multilevel modeling. Thousand Oaks: Sage Publications.
31. Kreft, I. G. G., de Leeuw, J., & Aiken, L. S. (1995). The effect of different forms of centering in hierarchical linear models. Multivariate Behavioral Research, 30, 1–21. doi:10.1207/s15327906mbr3001_1.
32. Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13, 203–229. doi:10.1037/a0012869.
33. Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1, 85–91. doi:10.1027/1614-1881.1.3.86.
34. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149. Retrieved from http://doi.apa.org/journals/met/1/2/130.pdf.
35. MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7, 19–40. doi:10.1037/1082-989X.7.1.19.
36. Marsh, H. W., Hau, K.-T., Balla, J. R., & Grayson, D. (1998). Is more ever too much? The number of indicators per factor in confirmatory factor analysis. Multivariate Behavioral Research, 33, 181–220. doi:10.1207/s15327906mbr3302_1.
37. McArdle, J. J., & Nesselroade, J. R. (2003). Growth curve analysis in contemporary psychological research. In J. Schinka & W. Velicer (Eds.), Comprehensive handbook of psychology: Research methods in psychology (pp. 447–480). New York: Wiley. doi:10.1002/0471264385.
38. Meredith, W., & Tisak, J. (1990). Latent curve analysis. Psychometrika, 55, 107–122. doi:10.1007/BF02294746.
39. Noh, M., & Lee, Y. (2007). REML estimation for binary data in GLMMs. Journal of Multivariate Analysis, 98, 896–915. doi:10.1016/j.jmva.2006.11.009.
40. Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2002). Reliable estimation of generalized linear mixed models using adaptive quadrature. The Stata Journal, 2, 1–21. Retrieved from http://www.stata-journal.com/article.html?article=st0005.
41. Raudenbush, S. W. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2, 173–185. doi:10.1037/1082-989X.2.2.173.
42. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks: Sage Publications.
43. Raudenbush, S. W., & Liu, X. (2000). Statistical power and optimal design for multisite randomized trials. Psychological Methods, 5, 199–213. doi:10.1037//1082-989X.5.2.199.
44. Raudenbush, S. W., Bryk, A. S., & Congdon, R. (2004). HLM 6 for Windows [Computer software]. Lincolnwood: Scientific Software.
45. Raudenbush, S. W., Spybrook, J., Congdon, R., Liu, X., & Martinez, A. (2011). Optimal Design Software for multi-level and longitudinal research (version 3.01) [Software]. Available from www.wtgrantfoundation.org.
46. Rodríguez, G., & Goldman, N. (2001). Improved estimation procedures for multilevel models with binary response: A case study. Journal of the Royal Statistical Society: Series A (Statistics in Society), 164, 339–355. doi:10.1111/1467-985X.00206.
47. Schabenberger, O. (2005). Introducing the GLIMMIX procedure for generalized linear mixed models. Cary: SAS Institute. Retrieved from http://www2.sas.com/proceedings/sugi30/196-30.pdf.
48. Schwarz, G. E. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464. Retrieved from http://www.jstor.org/stable/2958889.
49. Shiyko, M. P., Lanza, S. T., Tan, X., Li, R., & Shiffman, S. (2012). Using the time-varying effect model (TVEM) to examine dynamic associations between negative affect and self-confidence on smoking urges: Differences between successful quitters and relapsers. Prevention Science, 13, 288–299. doi:10.1007/s11121-011-0264-z.
50. Simon, T. R., Ikeda, R. M., Smith, E. P., Reese, L. E., Rabiner, D. L., Miller-Johnson, S., et al. (2008). The multisite violence prevention project: Impact of a universal school-based violence prevention program on social-cognitive outcomes. Prevention Science, 9, 231–244. doi:10.1007/s11121-008-0101-1.
51. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press.
52. Snijders, T. A. B., & Bosker, R. J. (2004). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). London: Sage Publications.
53. Steiger, J. H., & Lind, J. C. (1980). Statistically based tests for the number of common factors. Iowa City: Paper presented at the Meeting of the Psychometric Society.
54. Tanaka, J. S. (1987). “How big is big enough?”: Sample size and goodness of fit in structural equation models with latent variables. Child Development, 58, 134–146. Retrieved from http://www.jstor.org/stable/1130296.
55. Weston, R., & Gore, P. A., Jr. (2006). A brief guide to structural equation modeling. The Counseling Psychologist, 34, 719–751. doi:10.1177/0011000006286345.
56. Whiteside, S. P., & Lynam, D. R. (2001). The five factor model and impulsivity: Using a structural model of personality to understand impulsivity. Personality and Individual Differences, 30, 669–689. doi:10.1016/S0191-8869(00)00064-7.