Effects of Single-Sex Schooling in the Final Years of High School: A Comparison of Analysis of Covariance and Propensity Score Matching

Sex Roles

Volumn 69, Issue 7-8, 2013, Pages 404-422

  Nagengast, Benjamin ^a   Marsh, Herbert W ^b,c,d   Hau, Kit Tai ^e  

^a UNIVERSITY OF TÜBINGEN   (Germany)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c UNIVERSITY OF WESTERN SYDNEY   (Australia)

^d KING SAUD UNIVERSITY   (Saudi Arabia)

^e CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Causal inference; High School and Beyond study; Propensity score; Single sex schooling

Indexed keywords

EID: 84884820734     PISSN: 03600025     EISSN: 15732762     Source Type: Journal    
DOI: 10.1007/s11199-013-0261-8     Document Type: Article

References (100)

1. Angrist, J. D., & Pischke, J. S. (2009). Mostly harmless econometrics: An empiricist's companion. Princeton: Princeton University Press.
2. Austin, P. C., Grootendorst, P., & Anderson, G. M. (2007). A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: A Monte Carlo study. Statistics in Medicine, 26, 734-753. doi: 10. 1002/sim. 2580.
3. Baker, D. B., Riordan, C., & Schaub, M. (1995). The effects of sex-grouped schooling on achievement: The role of national context. Comparative Education Review, 39, 468-482.
4. Bigler, R. S., & Signorella, M. L. (2011). Single-sex education: New perspectives and evidence on a continuing controversy. Sex Roles, 65, 659-669. doi: 10. 1007/s11199-011-0046-x.
5. Billger, S. M. (2009). On reconstructing school segregation: The efficacy and equity of single-sex schooling. Economics of Education Review, 28, 393-402. doi: 10. 1016/j. econedurev. 2007. 08. 005.
6. Burgess, S., Greaves, E., Vignoles, A., & Wilson, D. (2009). Parental choice of primary school in England: what 'type' of school do parents choose? (Working Paper No. 09/224). Bristol, UK: The centre for market and public organisation. Retrieved from www. bristol. ac. uk/cmpo/publications/papers/2009/wp224. pdf.
7. Carpenter, P., & Hayden, M. (1987). Girls' academic achievements: Single-sex versus coeducational schools in Australia. Sociology of Education, 60, 156-167. doi: 10. 2307/2112273.
8. Caspi, A. (1995). Puberty and the gender organization of schools: How biology and social context shape the adolescent experience. In L. J. Crockett & A. C. Crouter (Eds.), Pathways through adolescence: Individual development in relation to social contexts (pp. 57-74). Mahwah: Erlbaum.
9. Cochran, W. G., & Rubin, D. B. (1973). Controlling bias in observational studies: A review. Sankhya-A, 35, 417-446.
10. Daly, P. (1996). The effects of single-sex and coeducational secondary schooling on girls' achievement. Research Papers in Education, 11, 289-306. doi: 10. 1080/0267152960110306.
11. Daly, P., & Shuttleworth, I. (1997). Determinants of public examination entry and attainment in mathematics: Evidence on gender and gender-type of school from the 1980s and 1990s in Northern Ireland. Evaluation and Research in Education, 11, 91-101. doi: 10. 1080/09500799708666919.
12. Dehejia, R. H., & Wahba, S. (1999). Propensity score matching methods for nonexperimental causal studies: Re-evaluating the evaluation of training programs. Journal of the American Statistical Association, 94, 1053-1062. doi: 10. 1162/003465302317331982.
13. R Development Core Team (2010). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from http://www. R-project. org.
14. Diamond, A., & Sekhon, J. S. (2006). Genetic matching for estimating causal effects: A general multivariate matching method for achieving balance in observational studies. (Working paper). University of California, Berkeley. Retrieved from http://sekhon. berkeley. edu/papers/GenMatch. pdf.
15. Eliot, L. (2011). Single-sex education and the brain. Sex Roles, this issue. doi: 10. 1007/s11199-011-0037-y.
16. Foster, E. M. (2010). Causal inference and developmental psychology. Developmental Psychology, 46, 1454-1480. doi: 10. 1037/a0020204.
17. Gitelman, A. I. (2005). Estimating causal effects from multilevel group-allocation data. Journal of Educational and Behavioral Statistics, 30, 397-412. doi: 10. 3102/10769986030004397.
18. Hansen, B. B. (2004). Full matching in an observational study of coaching for the SAT. Journal of the American Statistical Association, 99, 609-618. doi: 10. 1198/016214504000000647.
19. Hansen, B. B., & Klopfer, S. O. (2006). Optimal full matching and related designs via network flows. Journal of Computational and Graphical Statistics, 15, 609-627. doi: 10. 1198/106186006X137047.
20. Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15, 234-249. doi: 10. 1037/a0019623.
21. Harker, R. (2000). Achievement, gender, and the single-sex/coed debate. British Journal of Sociology of Education, 21, 203-218. doi: 10. 1080/713655349.
22. Hattie, J. (2008). Visible learning. A synthesis of over 800 meta-analyses relating to achievement. London: Routledge.
23. Heckman, J. J., & Vytlacil, E. J. (2007a). Econometric evaluation of social programs, part I: Causal models, structural models and econometric policy evaluation. In J. J. Heckman & E. E. Leamer (Eds.), Handbook of econometrics (Vol. 6, pt. 2, pp. 4779-4874). Amsterdam, the Netherlands: North-Holland.
24. Hill, J. L., Weiss, C., & Zhai, F. (2011). Challenges with propensity score strategies in a high-dimensional setting and a potential alternative. Multivariate Behavioral Research, 46, 477-513. doi: 10. 1080/00273171. 2011. 570161.
25. Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis, 15, 199-236. doi: 10. 1093/pan/mpl013.
26. Ho, D. E., Imai, K., King, G. & Stuart, E. A. (2011). MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software, 42, (8). Retrieved from http://www. jstatsoft. org/v42/i08/paper.
27. Hoffnung, M. (2011). Career and family outcomes for women graduates of single-sex versus coed colleges. Sex Roles, 65, 680-692. doi: 10. 1007/s11199-010-9914-z.
28. Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945-960. doi: 10. 1080/01621459. 1986. 10478354.
29. Hong, G., & Raudenbush, S. W. (2006). Evaluating kindergarten retention policy: A case study of causal inference for multilevel observational data. Journal of the American Statistical Association, 101, 901-910. doi: 10. 1198/016214506000000447.
30. Imbens, G. W., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142, 615-635. doi: 10. 1016/j. jeconom. 2007. 05. 001.
31. Jencks, C. (1985). How much do high school students learn? Sociology of Education, 58, 128-135. doi: 10. 2307/2112252.
32. Kang, J. D. Y., & Schafer, J. L. (2007). Demystifying double-robustness: A comparison of alternative strategies for estimating a population mean from incompleted data. Statistical Science, 22, 523-539. doi: 10. 1214/07-STS227.
33. LaLonde, R. J. (1986). Evaluating the econometric evaluations of training programs with experimental data. American Economic Review, 76, 604-620. doi: 10. 2307/1806062.
34. Lee, V. E. (1998). Is single-sex secondary schooling a solution to the problem of gender inequity? In American Association for University Women Educational Foundation (Ed.). Separated by sex. A critical look at single-sex education for girls (pp. 41-52). Washington, D. C.: American Association for University Women Educational Foundation.
35. Lee, V. E., & Bryk, A. S. (1986). Effects of single-sex schools on student achievement and attitudes. Journal of Educational Psychology, 78, 381-395. doi: 10. 1037/0022-0663. 78. 5. 381.
36. Lee, V. E., & Bryk, A. S. (1989). Effects of single-sex schools: Response to Marsh. Journal of Educational Psychology, 81, 647-650. doi: 10. 1037/0022-0663. 81. 4. 647.
37. Lee, V. E., & Lockheed, M. E. (1990). The effects of single-sex schooling on achievement and attitudes in Nigeria. Comparative Education Review, 34, 209-231.
38. Lee, V. E., & Marks, H. M. (1990). Sustained effects of the single-sex secondary school experience on attitudes, behaviors, and values in college. Journal of Educational Psychology, 82, 578-592. doi: 10. 1037/0022-0663. 82. 3. 578.
39. LePore, P. C., & Warren, J. R. (1997). A comparison of single-sex and coeducational Catholic secondary schooling: Evidence from the National Educational Longitudinal Study of 1988. American Educational Research Journal, 34, 485-511. doi: 10. 3102/00028312034003485.
40. Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New York: John Wiley.
41. Lumley, T. (2010). Survey: analysis of complex survey samples [Computer software]. R package version 3. 22-4.
42. Mael, F., Alonso, A., Gibson, D., Rogers, K., & Smith, M. (2005). Single-sex versus coeducational schooling: A systematic review. Doc# 2005-01. Washington: Department of Education. Office of Planning, Evaluation and Policy Development.
43. Manski, C. F. (2010). Identification of treatment response with social interactions. Working paper. Evanston: Northwestern University, Department of Economics and Institute for Policy Research.
44. Marsh, H. W. (1989a). Effects of attending single-sex and coeducational high schools on achievement, attitudes, behaviors, and sex differences. Journal of Educational Psychology, 81, 70-85. doi: 10. 1037/0022-0663. 81. 1. 70.
45. Marsh, H. W. (1989b). Effects of single-sex and coeducational schools. A reponse to Lee and Bryk. Journal of Educational Psychology, 81, 651-653. doi: 10. 1037/0022-0663. 81. 4. 651.
46. Marsh, H. W. (1991). Public, Catholic single-sex and Catholic coeducational high schools: Their effect on achievement, affect, and behaviors. American Journal of Education, 99, 320-356.
47. Marsh, H. W., & Hau, K.-T. (2007). Applications of latent-variable models in educational psychology: The need for methodological-substantive synergies. Contemporary Educational Psychology, 32, 151-170. doi: 10. 1016/j. cedpsych. 2006. 10. 008.
48. Marsh, H. W., Smith, I. D., Marsh, M. R., & Owens, L. (1988). The transition from single-sex to coeducational high schools: Effects on multiple dimensions of self-concept and on academic achievement. American Educational Research Journal, 25, 237-269. doi: 10. 3102/00028312025002237.
49. Marsh, H. W., Owens, L., Marsh, M. R., & Smith, I. D. (1989). The transition from single-sex to coeducational high schools: Teacher perceptions, academic achievement, and self-concept. British Journal of Educational Psychology, 59, 155-173. doi: 10. 1111/j. 2044-8279. 1989. tb03088. x.
50. Morgan, S. L., & Winship, C. (2007). Counterfactuals and causal inference: Methods and principles for social research. Cambridge: Cambridge University Press.
51. Muthén, L. K. & Muthén, B. O. (1998-2010). Mplus User's Guide. Sixth Edition. Los Angeles, CA: Muthén & Muthén.
52. Nagengast, B. (2009). Causal inference in multilevel designs. Unpublished doctoral dissertation. School of Social and Behavioural Sciences. Germany: Friedrich-Schiller-Universität Jena.
53. National Center for Educational Statistics. (1986). High school and beyond, 1980: sophomore cohort second follow-up (1984). Data file user's manual. Ann Arbor: Inter-university Consortium for Political and Social Research.
54. Neyman, J. (1923/1990). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science, 5, 465-480.
55. OECD. (2010). PISA 2009 results: Overcoming social background. Equity in learning opportunities and outcomes. Volume 2. Paris: Author.
56. Park, H., Behrman, J. R., & Choi, J. (2010). Causal effects of single-sex schools on college attendance: Random assignment in Korean high schools. PSC Working Paper Series, 15.
57. Pohl, S., Steiner, P. M., Eisermann, J., Soellner, R., & Cook, T. D. (2009). Unbiased causal inference from an observational study: Results of a within-study comparison. Educational Evaluation and Policy Analysis, 31, 463-479. doi: 10. 3102/0162373709343964.
58. Raudenbush, S. W. (2004). What are value-added models estimating and what does this imply for statistical practice? Journal of Educational and Behavioral Statistics, 29, 121-129. doi: 10. 3102/10769986029001121.
59. Raudenbush, S. W., & Willms, J. D. (1995). The estimation of school effects. Journal of Educational and Behavioral Statistics, 20, 307-335. doi: 10. 3102/10769986020004307.
60. Riordan, C. (1990). Girls and boys in school: Together or separate? New York: Teachers College Press.
61. Riordan, C. (1994). Single-gender schools: Outcomes for African and Hispanic Americans. Research in Sociology of Education and Socialization, 10, 177-205.
62. Riordan, C. (1998). The future of single-sex schools. In AAUW Educational Foundation (Ed.), Separated by sex (pp. 53-62). Washington, DC: American Association for University Women Educational Foundation. Retrieved from http://www. aauw. org/research/upload/SeparatedBySex. pdf.
63. Rosenbaum, P. R. (1984). The consequences of adjustment for a concomitant variable that has been affected by the treatment. Journal of the Royal Statistical Society-Series A, 147, 656-666. doi: 10. 2307/2981697.
64. Rosenbaum, P. R. (1991). A characterization of optimal designs for observational studies. Journal of the Royal Statistical Society-Series B, 53, 597-610.
65. Rosenbaum, P. R. (2002). Covariance adjustment in randomized experiments and observational studies. Statistical Science, 17, 286-304. doi: 10. 1214/ss/1042727942.
66. Rosenbaum, P. R., & Rubin, D. B. (1983a). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55. doi: 10. 1093/biomet/70. 1. 41.
67. Rosenbaum, P. R., & Rubin, D. B. (1983b). Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. Journal of the Royal Statistical Society-Series B, 45, 212-218. doi: 10. 2307/2345524.
68. Rosenbaum, P. R., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. The American Statistician, 39, 33-38. doi: 10. 1080/00031305. 1985. 10479383.
69. Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66, 688-701. doi: 10. 1037/h0037350.
70. Rubin, D. B. (1977). Assignment to treatment group on the basis of a covariate. Journal of Educational Statistics, 2, 1-26. doi: 10. 3102/10769986002001001.
71. Rubin, D. B. (1978). Bayesian-inference for causal effects: The role of randomization. The Annals of Statistics, 6, 34-58. doi: 10. 2307/2958688.
72. Rubin, D. B. (1980). Bias reduction using Mahalanobis metric matching. Biometrics, 36, 293-298. doi: 10. 2307/2529981.
73. Rubin, D. B. (1986). Statistics and causal inference-which ifs have causal answers. Journal of the American Statistical Association, 81, 961-962. doi: 10. 1080/01621459. 1986. 10478355.
74. Rubin, D. B. (1990a). Formal modes of statistical-inference for causal effects. Journal of Statistical Planning and Inference, 25, 279-292. doi: 10. 1016/0378-3758(90)90077-8.
75. Rubin, D. B. (1990b). Neyman (1923) and causal inference in experiments and observational studies. Statistical Science, 5, 472-480.
76. Rubin, D. B. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services & Outcomes Research Methodology, 2, 169-188. doi: 10. 1023/A: 1020363010465.
77. Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100, 322-331. doi: 10. 1198/016214504000001880.
78. Rubin, D. B. (2008). Comment: The design and analysis of gold standard randomized experiments. Journal of the American Statistical Association, 103, 1350-1353. doi: 10. 1198/016214508000001011.
79. Rubin, D. B., & Thomas, N. (1996). Matching using estimated propensity scores, relating theory to practice. Biometrics, 52, 249-264. doi: 10. 2307/2533160.
80. Rubin, D. B., & Thomas, N. (2000). Combining propensity score matching with additional adjustments for prognostic covariates. Journal of the American Statistical Association, 95, 573-585. doi: 10. 1080/01621459. 2000. 10474233.
81. Schafer, J. L., & Kang, J. D. Y. (2008). Average causal effects from nonrandomized studies: a practical guide and simulated example. Psychological Methods, 13, 279-313. doi: 10. 1037/a0014268.
82. Senn, S., Graf, E., & Caputo, A. (2007). Stratification for the propensity score compared with linear regression techniques to assess the effect of treatment or exposure. Statistics in Medicine, 26, 5529-5544. doi: 10. 1002/sim. 3133.
83. Shadish, W. R., & Cook, T. D. (2009). The renaissance of field experimentation in evaluating interventions. Annual Review of Psychology, 60, 607-629. doi: 10. 1146/annurev. psych. 60. 110707. 163544.
84. Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random to nonrandom assignment. Journal of the American Statistical Association, 103, 1334-1343. doi: 10. 1198/016214508000000733.
85. Smithers, A., & Robinson, P. (2006). The paradox of single-sex and coeducational schooling. Buckingham: Carmichael Press.
86. Sobel, M. E. (2006). What do randomized studies of housing mobility demonstrate?: Causal inference in the face of interference. Journal of the American Statistical Association, 101, 1398-1407. doi: 10. 1198/016214506000000636.
87. Spielhofer, T., Benton, T., & Schagen, S. (2004). A study of the effects of school size and single-sex education in English schools. Research Papers in Education, 19, 133-159. doi: 10. 1080/02671520410001695407.
88. Steyer, R., von Davier, A. A., Nachtigall, C., & Buhl, T. (2000). Causal regression models I: Individual and average causal effects. Methods of Psychological Research Online, 5, 39-71.
89. Steyer, R., Nachtigall, C., Wüthrich-Martone, O., & Kraus, K. (2002). Causal regression models III: Covariates, conditional, and unconditional average causal effects. Methods of Psychological Research Online, 7, 41-68.
90. Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25, 1-21. doi: 10. 1214/09-STS313.
91. Stuart, E. A., & Green, K. M. (2008). Using full matching to estimate causal effects in non-experimental studies: Examining the relationship between adolescent marijuana use and adult outcomes. Developmental Psychology, 44, 395-406. doi: 10. 1037/0012-1649. 44. 2. 395.
92. Sullivan, A. (2009). Academic self-concept, gender and single-sex schooling. British Educational Research Journal, 35, 259-288. doi: 10. 1080/01411920802042960.
93. Sullivan, A., Joshi, H., & Leonhard, D. (2010). Single-sex schooling and academic attainment at school and through the lifecourse. American Educational Research Journal, 47, 6-36. doi: 10. 3102/0002831209350106.
94. Thoemmes, F., & Kim, E. S. (2011). A systematic review of propensity score methods in the social sciences. Multivariate Behavioral Research, 46, 90-118. doi: 10. 1080/00273171. 2011. 540475.
95. U. S. Department of Education (2006). Nondiscrimination on the basis of sex in education programs or activities receiving financial assistance: final rule, Federal Register, 34 CFR Part 106, 25 October.
96. VanderWeele, T. J. (2008). Ignorability and stability assumptions in neighborhood effects research. Statistics in Medicine, 27, 1934-1943. doi: 10. 1002/sim. 3139.
97. Watson, C. M., Quatman, T., & Edler, E. (2002). Career aspirations of adolescent girls: Effects of achievement level, grade, and single-sex school environment. Sex Roles, 46, 323-335. doi: 10. 1023/A: 1020228613796.
98. West, S. G., & Thoemmes, F. (2010). Campbell's and Rubin's perspectives on causal inference. Psychological Methods, 15, 18-37. doi: 10. 1037/a0015917.
99. Woodward, L. J., Fergusson, D. M., & Horwood, L. J. (1999). Effects of single-sex and coeducational secondary schooling on children's academic achievement. Australian Journal of Education, 43, 142-156.
100. Wooldridge, J. (2005). Fixed-effects and related estimators for correlated random-coefficient and treatment-effect panel data models. The Review of Economics and Statistics, 87, 385-390. doi: 10. 1162/0034653053970320.