A meta-analysis of blended learning and technology use in higher education: From the general to the applied

Journal of Computing in Higher Education

Volumn 26, Issue 1, 2014, Pages 87-122

  Bernard, Robert M ^a   Borokhovski, Eugene ^a   Schmid, Richard F ^a   Tamim, Rana M ^b   Abrami, Philip C ^a  

^a Systematic Reviews Project Manager Centre for the Study of Learning and Performance   (Canada)

^b ZAYED UNIVERSITY   (United Arab Emirates)

Author keywords

Bended learning; Higher education; Meta analysis; Technology use

Indexed keywords

EID: 84897820241     PISSN: 10421726     EISSN: 18671233     Source Type: Journal    
DOI: 10.1007/s12528-013-9077-3     Document Type: Article

References (78)

1. Note: A list of studies included in the meta-analysis is available upon request from the authors.
2. Abrami, P. C. (2010). On the nature of support in computer supported collaborative learning using gStudy. Computers in Human Behavior, 26(5), 835-839. doi: 10. 1016/j. chb. 2009. 04. 007.
3. Abrami, P. C., & Bernard, R. M. (2006). Research on distance education: In defense of field experiments. Distance Education, 27(1), 5-26.
4. Abrami, P. C., & Bernard, R. M. (2012). Statistical control versus classification of study quality in meta-analysis. Effective Education, 4(1), 43-72. doi: 10. 1080/19415532. 2012. 761889.
5. Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M., Tamim, R. M., et al. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage one meta-analysis. Review of Educational Research, 78(4), 1102-1134. doi: 10. 3102/0034654308326084.
6. Abrami, P. C., Bernard, R. M., Bures, E. M., Borokhovski, E., & Tamim, R. (2011). Interaction in distance education and online learning: Using evidence and theory to improve practice. Journal of Computing in Higher Education, 23(2/3), 82-103. doi: 10. 1007/s12528-011-9043-x.
7. Albrecht, B. (2006). Enriching student experience through blended learning. ECAR Research Bulletin, 12.
8. Allen, M., Bourhis, J., Burrell, N., & Mabry, E. (2002). Comparing student satisfaction with distance education to traditional classrooms in higher education: A meta-analysis. American Journal of Distance Education, 16(2), 83-97. doi: 10. 1207/S15389286AJDE1602_3.
9. Allen, M., Bourhis, J., Mabry, E., Burrell, N., Timmerman, E., & Titsworth, S. (2006). Comparing distance education to face-to-face methods of education. In B. Gayle, R. Preiss, N. Burrell, & M. Allen (Eds.), Classroom and communication education research: Advances through meta-analysis (pp. 229-241). Hillsdale, NJ: Erlbaum.
10. Allen, M., Mabry, E., Mattrey, M., Bourhis, J., Titsworth, S., & Burrell, N. (2004). Evaluating the effectiveness of distance learning: A comparison using meta-analysis. Journal of Communication, 54(3), 402-420. doi: 10. 1111/j. 1460-2466. 2004. tb02636. x.
11. Anderson, T. (2003). Getting the mix right again: An updated and theoretical rationale for interaction. International Review of Research in Open and Distance Learning, 4(2), 9-14. Retrieved from http://www. irrodl. org/index. php/irrodl/article/view/149.
12. Arabasz, P., & Baker, M. (2003). Respondent Summary: Evolving campus support models for e-learning courses. EDUCAUSE Center for Applied Research. http://www. educause. edu/ir/library/pdf/EKF/ekf0303. pdf.
13. Azevedo, R., & Bernard, R. M. (1995). A meta-analysis of the effects of feedback in computer-based instruction. Journal of Educational Computing Research, 13(2), 111-127. doi: 10. 2190/9LMD-3U28-3A0G-FTQT.
14. Bele, J. L., & Rugelj, J. (2007). Blended learning-an opportunity to take the best of both worlds. International Journal of Emerging Technologies in Learning, 2(3). doi: 10. 3991%2Fijet. v2i3. 133.
15. Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, C. A., Tamim, R. M., Surkes, M. A., et al. (2009). A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research, 79(3), 1243-1289. doi: 10. 3102/0034654309333844.
16. Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., et al. (2004). How does distance education compare to classroom instruction? A Meta-analysis of the empirical literature. Review of Educational Research, 74(3), 379-439. doi: 10. 3102/00346543074003379.
17. Bernard, R. M., Rojo de Rubalcava, B., & St-Pierre, D. (2000). Collaborative online distance education: Issues for future practice and research. Distance Education, 21(2), 260-277. doi: 10. 1080/0158791000210205.
18. Bernard, R. M., Zhang, D., Abrami, P. C., Sicoly, F., Borokhovski, E., & Surkes, M. (2008). Exploring the structure of the Watson-Glaser critical thinking appraisal: One scale or many subscales? Thinking Skills and Creativity, 3, 15-22. doi: 10. 1016/j. tsc. 2007. 11. 001.
19. Bliuc, A. M., Goodyear, P., & Ellis, R. A. (2007). Research focus and methodological choices in studies into students' experiences of blended learning in higher education. The Internet and Higher Education, 10(4), 231-244. doi: 10. 1016/j. iheduc. 2007. 08. 001.
20. Bonk, C. J., & Graham, C. R. (Eds.). (2006). The handbook of blended learning: Global perspectives, local designs. San Francisco, CA: Pfeiffer.
21. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. (2005). Comprehensive meta-analysis version 2. 2. 048. Englewood, NJ: Biostat.
22. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. (2009). Introduction to meta-analysis. Chichester, UK: Wiley.
23. Borokhovski, E., Tamim, R. M., Bernard, R. M., Abrami, P. C., & Sokolovskaya, A. (2012). Are contextual and design student-student interaction treatments equally effective in distance education? A follow-up meta-analysis of comparative empirical studies. Distance Education, 33(3), 311-329. doi: 10. 1080/01587919. 2012. 723162.
24. Campbell, D., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago, IL: Rand McNally.
25. Cavanaugh, C. S. (2001). The effectiveness of interactive distance education technologies in K-12 learning: A meta-analysis. International Journal of Educational Telecommunications, 7(1), 73-88. Norfolk, VA: AACE. Retrieved March 13, 2007 from http://www. editlib. org/p/8461.
26. Cavanaugh, C., Gillan, K. J., Kromey, J., Hess, M., & Blomeyer, R. (2004). The effects of distance education on K-12 student outcomes: A meta-analysis. Naperville, IL: Learning Point Associates.
27. Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445-459. doi: 10. 3102/00346543053004445.
28. Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21-29. doi: 10. 1007/BF02299088.
29. Clark, R. E., Yates, K., Early, S., & Moulton, K. (2009). An analysis of the failure of electronic media and discovery-based learning: Evidence for the performance benefits of guided training methods. In K. H. Silber & R. Foshay (Eds.), Handbook of training and improving workplace performance, Volume I: Instructional design and training delivery. Washington, DC: ISPI.
30. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.
31. Cook, D. A. (2009). The failure of e-learning research to inform educational practice and what we can do about it. Medical Teacher, 31(2), 158-162. doi: 10. 1080/01421590802691393.
32. Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008). Internet-based learning in the health professions: A meta-analysis. Journal of the American Medical Association, 300(10), 1181-1196. doi: 10. 1001/jama. 300. 10. 1181.
33. Cooper, H. M. (2010). Research synthesis and meta-analysis: A step-by-step approach (4th ed.). Los Angeles, CA: SAGE Publications.
34. Cooper, H. M., Hedges, L. V., & Valentine, J. C. (Eds.). (2009). The handbook of research synthesis and meta-analysis (2nd ed.). New York, NY: Russell Sage Foundation.
35. Driscoll, M., & Carliner, S. (2005). Advanced web-based training strategies. Blended learning as a curriculum design strategy (pp. 87-116). New York, NY: ASTD Press.
36. Duval, S., & Tweedie, R. (2004). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455-463. doi: 10. 1111/j. 0006-341X. 2000. 00455. x.
37. Garrison, D. R., & Vaughan, N. D. (2008). Blended learning in higher education: Framework, principles, and guidelines. San Francisco, CA: Jossey-Bass.
38. Glass, G. V. (1976). Primary, secondary and meta-analysis. Educational Researcher, 5(10), 3-8.
39. Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
40. Graham, C. R. (2005). Blended learning systems. In C. J. Bonk, & C. R. Graham (Eds.), The handbook of blended learning: Global perspectives, local designs. Chichester: Wiley (originally Pfeiffer).
41. Hammerstrøm, K., Wade, C. A. & Jørggensen, A.-M. K. (2010). Searching the literature: A guide to information retrieval for campbell systematic reviews 2010: Supplement 1. Oslo, Norway: The Campbell Collaboration. doi: 10. 4073/csrs. 2010. 1.
42. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
43. Hedges, L. V., Shymansky, J. A. & Woodworth, G. (1989). A practical guide to modern methods of meta-analysis. (Stock Number PB-52). Washington, DC: National Science Teachers Association. (ERIC Document Reproduction Service No. ED309952).
44. Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analysis. British Medical Journal, 327, 557-560. doi: 10. 1136/bmj. 327. 7414. 557.
45. Hunt, M. (1997). How science takes stock: The story of meta-analysis. New York, NY: Russell Sage Foundation.
46. Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: SAGE Publications.
47. Jahng, N., Krug, D. & Zhang, Z. (2007). Student achievement in online education compared to face-to-face education. European Journal of Open, Distance and E-Learning. Retrieved from http://www. eurodl. org/materials/contrib/2007/Jahng_Krug_Zhang. htm.
48. Johnson, D. W., & Johnson, R. T. (2009). An educational psychology success story: Social interdependence theory and cooperative learning. Educational Researcher, 368(5), 365-379. doi: 10. 3102/0013189X09339057.
49. Keegan, D. (1996). Foundations of distance education (3rd ed.). London: Routledge.
50. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. New York, NY: Sage Publications.
51. Lou, Y., Abrami, P. C., & D'Appollonia, S. (2001). Small group and individual learning with technology: A meta-analysis. Review of Educational Research, 71, 449-521. doi: 10. 3102/00346543071003449.
52. Lou, Y., Bernard, R. M., & Abrami, P. C. (2006). Media and pedagogy in undergraduate distance education: A theory-based meta-analysis of empirical literature. Educational Technology Research and Development, 54, 141-176. doi: 10. 1007/s11423-006-8252-x.
53. Machtmes, K., & Asher, J. W. (2000). A meta-analysis of the effectiveness of telecourses in distance education. American Journal of Distance Education, 14(1), 27-46. doi: 10. 1080/08923640009527043.
54. Marquis, C. (2004). WebCT survey discovers a blend of online learning and classroom-based teaching is the most effective form of learning today. WebCT. com. Retrieved from http://www. webct. com/service/ViewContent?contentID=19295938.
55. Means, B, Toyama, Y., Murphy, R. F., & Baki, M. (2013). The effectiveness of online and blended learning: A meta-analysis of the empirical literature, Teachers College Record, 115(3), 1-47. Retrieved from http://www. tcrecord. org/library/content. asp?contentid=16882.
56. Means, B., Toyama, Y., Murphy, R., Bakia, M,. & Jones, K. (2010). Evaluation of evidence-based practices in online Learning: A meta-analysis and review of online learning studies. Technical Report. U. S. Department of Education, Washington, DC.
57. Moore, M. G. (1989). Editorial: Three types of interaction. The American Journal of Distance Education, 3(2), 1-6. doi: 10. 1080/08923648909526659.
58. Pettigrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Oxford, UK: Blackwell Publishing.
59. Piggot, T. D. (2012). Advances in meta-analysis. New York, NY: Springer.
60. Pintrich, P. R. (2003). A Motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology, 95(4), 667-686. doi: 10. 1037/0022-0663. 95. 4. 667.
61. Rosen, Y., & Salomon, G. (2007). The differential learning achievements of constructivist technology-intensive learning environments as compared with traditional ones: A meta-analysis. Journal of Educational Computing Research, 36(1), 1-14. doi: 10. 2190/R8M4-7762-282U-554J.
62. Ross, S. M., & Morrison, G. R. (1989). In search of a happy medium in instructional technology research: Issues concerning external validity, media replications, and learner control. Educational Technology Research and Development, 37(1), 19-33. doi: 10. 1007/BF02299043.
63. Ross, S. M., Morrison, G. R., & Lowther, D. L. (2010). Educational technology research past and present: Balancing rigor and relevance to impact school learning. Contemporary Educational Technology, 1(1), 17-35. Retrieved from http://www. cedtech. net/articles/112. pdf.
64. Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2005). Publication bias in meta-analysis: Prevention, assessment and adjustments. Chichester, UK: Wiley.
65. Scammacca, N., Roberts, G., & Stuebing, K. K. (2013). Meta-analysis with complex research designs: Dealing with dependence from multiple measures and multiple group comparisons. Review of Educational Research. Published online 13 September 2013. doi: 10. 3102/0034654313500826.
66. Schmid, R. F., Bernard, R. M., Borokhovski, E., Tamim, R. M., Abrami, P. C., Surkes, M. A., et al. (2014). The effects of technology use in postsecondary education: A meta-analysis of classroom applications. Computers & Education, 72, 271-291. doi: 10. 1016/j. compedu. 2013. 11. 002.
67. Shachar, M., & Neumann, Y. (2003). Differences between traditional and distance education academic performances: A meta-analytical approach. International Review of Research in Open and Distance Education, 4(2). Retrieved from http://www. irrodl. org/index. php/irrodl/article/view/153/704.
68. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
69. Sitzmann, T., Kraiger, K., Stewart, D., & Wisher, R. (2006). The comparative effectiveness of web-based and classroom instruction: A meta-analysis. Personnel Psychology, 59(3), 623-664. doi: 10. 1111/j. 1744-6570. 2006. 00049. x.
70. Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research., 81(3), 4-28. doi: 10. 3102/0034654310393361.
71. Ungerleider, C. S., & Burns, T. C. (2003). Information and communication technologies in elementary and secondary education: A state of the art review. International Journal of Educational Policy, Research & Practice, 3(4), 27-54.
72. Valentine, J. C., & Cooper, H. (2003). Effect size substantive interpretation guidelines: Issues in the interpretation of effect sizes. Washington, DC: What Works Clearinghouse.
73. Valentine, J. C., & Cooper, H. (2008). A systematic and transparent approach for assessing the methodological quality of intervention effectiveness research: The Study Design and Implementation Assessment Device (Study DIAD). Psychological Methods, 13(2), 130-149. doi: 10. 1037/1082-989X. 13. 2. 130.
74. Watson, G., & Glaser, E. M. (1980). Watson-Glaser critical thinking appraisal: Forms A and B. San Antonio, TX: PsychCorp.
75. Williams, S. L. (2006). The effectiveness of distance education in allied health science programs: A meta-analysis of outcomes. American Journal of Distance Education, 20(3), 127-141. doi: 10. 1207/s15389286ajde2003_2.
76. Zhao, Y., Lei, J., Yan, B., & Tan, S. (2005). What makes the difference? A practical analysis of research on the effectiveness of distance education. Retrieved from http://ott. educ. msu. edu/literature/report. pdf.
77. Zhoa, Y., & Breslow, L. (2013). Literature review on hybrid/blended learning. Unpublished manuscript. Retrieved from http://tll. mit. edu/sites/default/files/library/Blended_Learning_Lit_Reveiw. pdf.
78. Zimmerman, B. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaertzs & P. R. Pintrich (Eds.), Handbook of self-regulation (pp. 13-39). New York, NY: Academic Press.