In Search of Quality: Using Quality Matters to Analyze the Quality of Massive, Open, Online Courses (MOOCs)

International Review of Research in Open and Distributed Learning

Volumn 16, Issue 5, 2015, Pages 83-101

  Lowenthal, Patrick R ^a   Hodges, Charles B ^b  

^a BOISE STATE UNIVERSITY   (United States)

^b GEORGIA SOUTHERN UNIVERSITY   (United States)

Author keywords

Course Quality; Instructional Design; Massive Open Online Courses; MOOCs; Online Learning; Quality Assurance; Quality Matters

Indexed keywords

EID: 84945930434     PISSN: None     EISSN: 14923831     Source Type: Journal    
DOI: 10.19173/irrodl.v16i5.2348     Document Type: Article

References (72)

1. Adair, D., Alman, S. W., Budzick, D., Grishman, L. M., Mancini, M. E., & Thackaberry, S. (2014). Many shades of MOOCs. Internet Learning, 3(1), 53-72.
2. Allen, I. E., Seaman, J., Lederman, D., & Jaschik, S. (2012). Conflicted: Faculty and online education. Inside Higher Ed, Babson Survey Research Group, and Quahog Research Group. Retrieved from http://www.insidehighered.com/sites/default/server_files/files/IHE-BSRG-Conflict.pdf
3. Allen, M. W. (2003). Michael Allen's guide to e-learning: Building interactive, fun, and effective learning programs for any company. Hoboken, NJ: John Wiley & Sons.
4. Anderson, T. (2013). Promise and/or peril: MOOCs and open and distance education. Commonwealth of Learning. Retrieved from http://www.ethicalforum.be/sites/default/files/MOOCsPromisePeril.pdf
5. Batchelder, L. (2009, October). Do learners really need learning objectives? Retrieved from http://www.bottomlineperformance.com/do-learners-really-need-learning-objectives/
6. Bean, C. (2009, July). Our objection to learning objectives. Retrieved from http://www.kineo.com/us/resources/top-tips/learning-strategy-and-design/our-objection-to-learning-objectives
7. Bejerano, A. R. (2008). Face-to-face or online instruction? Face-to-face is better. Communications Current, 3(3). Retrieved from https://www.natcom.org/CommCurrentsArticle.aspx?id=884
8. Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., Wallet, P. A., Fiset, M., & Huang, B. (2004). How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of educational research, 74(3), 379-439.
9. Bidwell, A. (2013, September). Employers, students remain skeptical of online education. US News. Retrieved from http://www.usnews.com/news/articles/2013/09/20/employers-students-remain-skeptical-of-online-education
10. California State Faculty Association. (2013, May). Massive virtual fires engulf San Jose State University. Retrieved from http://chronicle.com/article/Document-San-Jose-StateUs/139139/
11. Carey, D. (2013, December). Pay no attention to supposed low MOOC completion rates. New America: EdCentral. Retrieved from http://www.edcentral.org/pay-attention-supposedly-low-mooc-completion-rates/
12. Carr, N. (2012, September). The crisis in higher education. MIT Technology Review. Retrieved from http://www.technologyreview.com/featuredstory/429376/the-crisis-in-higher-education/
13. Conole, G. (2013). MOOCs as disruptive technologies: Strategies for enhancing the learner experience and quality MOOCs. Revista de Educación a Distancia, 39. Retrieved from http://www.um.es/ead/red/39/conole.pdf
14. Coursera. (2013, February 7). Five courses receive college credit recommendations. Retrieved from http://blog.coursera.org/post/42486198362/five-courses-receive-college-credit-recommendations
15. DeBoer, J., Ho, A. D., Stump, G. S., & Breslow, L. (2014). Changing "course": Reconceptualizing educational variables for massive open online courses. Educational Researcher, 43(2), 74-84.
16. Delbanco, A. (2013, March). MOOCs of hazard. New Republic. Retrieved from http://www.newrepublic.com/article/112731/moocs-will-online-education-ruin-university-experience
17. Downes, S. (2011, January). 'Connectivism' and connective knowledge. Huffington Post. Retrieved from http://www.huffingtonpost.com/stephen-downes/connectivism-and-connecti_b_804653.html
18. Duffy, T. M., & Kirkley, J. R. (Eds.) (2004). Learners-centered theory and practice in distance education: Cases from higher education. Mahwah, NJ: Lawrence Erlbaum.
19. Edmundson, M. (2012, July). The trouble with online education. New York Times. Retrieved from http://www.nytimes.com/2012/07/20/opinion/the-trouble-with-online-education.html
20. EDUCAUSE. (2012). What campus leaders need to know about MOOCs: An EDUCAUSE executive briefing. Retrieved from http://net.educause.edu/ir/library/pdf/PUB4005.pdf
21. Ertmer, P. A., & Newby, T. J. (1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 6(4), 50-72. doi:10.1111/j.1937-8327.1993.tb00605.x
22. Fain, P. (2013, January). As California goes? Inside Higher Ed. Retrieved from http://www.insidehighered.com/news/2013/01/16/california-looks-moocs-online-push
23. Gabriel, T. (2011, April). More pupils are learning online, fueling debate on quality. The New York Times. Retrieved from http://www.nytimes.com/2011/04/06/education/06online.html
24. Glass, G. V., & Smith, M. L. (1979). Meta-analysis of research on class size and achievement. Educational Evaluation and Policy Analysis, 1(1), 2-16.
25. Inglis, A. (2005). Quality improvement, quality assurance, and benchmarking: comparing two frameworks for managing quality processes in open and distance learning. The International Review of Research in Open and Distributed Learning, 6(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/221/304
26. Jaggars, S. S. (2011). Online learning: Does it help low-income and underprepared students? (CCRC Working Paper No. 26, Assessment of Evidence Series). New York, NY: Columbia University, Teachers College, Community College Research Center.
27. Jaggars, S., & Bailey, T. R. (2010). Effectiveness of fully online courses for college students: Response to a Department of Education meta-analysis. Retrieved from http://academiccommons.columbia.edu/download/fedora_content/download/ac:172121/CONTENT/effectiveness-online-response-meta-analysis.pdf
28. Jaggars, S. S., & Xu, D. (2010). Online learning in the Virginia Community College system. New York, NY: Columbia University, Teachers College, Community College Research Center.
29. Jaschik, S., & Lederman, D. (2014). The 2014 Inside Higher Ed Survey of Faculty Attitudes on Technology: A study by Gallup and Inside Higher Ed. Inside Higher Ed. Retrieved from http://www.insidehighered.com/download/form.php?width=500&height=550&iframe=true&title=Survey%20of%20Faculty%20Attitudes%20on%20Technology&file=IHE-FacTechSurvey2014%20final.pdf
30. Jordan, K. (2014). Initial trends in enrolment and completion of massive open online courses. The International Review of Research in Open and Distributed Learning, 15(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/viewFile/1651/2813
31. Kilgore, W., & Lowenthal, P. R. (2015). The Human Element MOOC: An experiment in social presence. In R. D. Wright (Ed.), Student-teacher interaction in online learning environments (pp. 389-407). Hershey, PA: IGI Global.
32. Kolowich, S. (2013, February). American council on higher education recommends 5 MOOCs for credit. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/American-Council-on-Education/137155/
33. Kroll, K. (2013). Where have all the faculty gone? Inside HigherEd. Retrieved from https://www.insidehighered.com/views/2013/05/09/faculty-who-teach-online-are-invisible-campuses-essay
34. Leddy, T. (2013, June). Are MOOCs good for students? Boston Review. Retrieved from http://bostonreview.net/us/are-moocs-good-students
35. Lederman, D., & Barwick, D. W. (2007). Does class size matter? Inside HigherEd. Retrieved from https://www.insidehighered.com/views/2007/12/06/barwick
36. Lockee, B. B., Moore, M., & Burton, J. (2001). Old concerns with new distance education research. EDUCAUSE Quarterly, 2, 60-62.
37. Lytle, R. (2012, July). College professors fearful of online education growth. US News & World Reports. Retrieved from http://www.usnews.com/education/online-education/articles/2012/07/06/college-professors-fearful-of-online-education-growth
38. Markoff, J. (2011). Virtual and artificial, but 58,000 want course. The New York Times. Retrieved from http://www.nytimes.com/2011/08/16/science/16stanford.html
39. Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of Massive Open Online Courses (MOOCs). Computers & Education, 80, 77-83.
40. Mazoue, J. (2013, October). Five myths about MOOCs. EDUCAUSE Review Online. Retrieved from http://www.educause.edu/ero/article/five-myths-about-moocs
41. McDonald, J. (2002). "As good as face-to-face" as good as it gets? Journal of Asynchronous Learning Networks, 6(2). 10-23.
42. Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. US Department of Education.
43. Meyer, K. A. (2002). Quality in distance education: Focus on on-line learning (ASHE-ERIC Higher Education Report). San Francisco, CA: Jossey-Bass.
44. Meyer, K. A. (2004). Putting the distance learning comparison study in perspective: Its role as personal journey research. Online Journal of Distance Learning Administration, 7(1). Retrieved from http://www.westga.edu/~distance/ojdla/spring71/meyer71.pdf
45. Miyazoe, T., & Anderson, T. (2013). Interaction Equivalency in an OER, MOOCS and Informal Learning Era. Journal of Interactive Media in Education, 2013(2). Retrieved from http://jime.open.ac.uk/articles/10.5334/2013-09/
46. Moore, C. (2007, December). Makeover: Turn objectives into motivators [Web log post]. Retrieved from http://blog.cathy-moore.com/2007/12/makeover-turn-objectives-into-motivators/
47. Mulryan-Kyne, C. (2010). Teaching large classes at college and university level: Challenges and opportunities. Teaching in Higher Education, 15(2), 175-185.
48. Palloff, R. M., & Pratt, K. (1999). Building learning communities in cyberspace. San Francisco, CA: Jossey-Bass.
49. Pappano, L. (2012, November). The year of the MOOC. The New York Times. Retrieved from http://www.nytimes.com/2012/11/04/education/edlife/massive-open-online-courses-are-multiplying-at-a-rapid-pace.html
50. Parker, K., Lenhart, A., & Moore, K. (2011). The digital revolution and higher education: College presidents, public differ on value of online learning. Pew Internet & American Life Project. Retrieved from http://www.pewsocialtrends.org/files/2011/08/online-learning.pdf
51. Phipps, R., & Merisotis, J. (1999). What's the difference? A review of contemporary research on effectiveness of distance learning in higher education. Washington, DC: Institute for Higher Education.
52. Pretz, K. (2014). Low completion rates for MOOCs. IEEE Roundup. Retrieved from http://theinstitute.ieee.org/ieee-roundup/opinions/ieee-roundup/low-completion-rates-for-moocs
53. Public Agenda. (2013, September) Not yet sold: What employers and community college students think about online education: A report by Public Agenda. Retrieved from http://www.publicagenda.org/files/NotYetSold_PublicAgenda_2013.pdf
54. Quality Matters. (2013). Higher Ed Program Rubric. Retrieved from https://www.qualitymatters.org/rubric
55. Quality Matters. (2014). Introduction to the Quality Matters Program. Retrieved from https://www.qualitymatters.org/sites/default/files/Introduction%20to%20the%20Quality%20Matters%20Program%20HyperlinkedFinal2014.pdf
56. Ralston-Berg, P. (2014). Surveying student perspectives of quality: Value of QM rubric items. Internet Learning, 3(1), 117-126.
57. Rees, J. (2013, July). The MOOC racket. Slate. Retrieved from http://www.slate.com/articles/technology/future_tense/2013/07/moocs_could_be_disastrous_for_students_and_professors.html
58. Reich, J. (2014). MOOC completion and retention in the context of student intent. EDUCAUSE Review Online. Retrieved from http://www.educause.edu/ero/article/mooc-completion-and-retention-context-student-intent
59. Rosen, R. J. (2012, July). Overblown-claims-of-failure watch: How not to gauge the success of online courses. The Atlantic. Retrieved from http://www.theatlantic.com/technology/archive/2012/07/overblown-claims-of-failure-watch-how-not-to-gauge-the-success-of-online-courses/260159/
60. Rushkoff, D. (2013, January). Online courses need human element to educate. CNN. Retrieved from http://www.cnn.com/2013/01/15/opinion/rushkoff-moocs/index.html
61. Russell, T. L. (1999). The "no significant difference" phenomenon. Raleigh: North Carolina University.
62. Salmon, G. (2000). E-moderating: The key to teaching and learning online. London: Kogan Page.
63. Samuels, B. (2013). Being present. Inside Higher Ed. Retrieved from https://www.insidehighered.com/views/2013/01/24/essay-flaws-distance-education
64. Sharma, G. (2013, July). A MOOC delusion: Why visions to educate the world are absurd. The Chronicle of Higher Education. Retrieved from http://chronicle.com/blogs/worldwise/a-mooc-delusion-why-visions-to-educate-the-world-are-absurd/32599
65. Shattuck, K. (2007). Quality Matters: Collaborative program planning at a state level. Online Journal of Distance Learning Administration, 10(3).
66. Shattuck, K. (2012, May). What we're learning from Quality Matters-focused research: Research, practice, continuous improvement. Quality Matters. Retrieved from https://www.qualitymatters.org/what-were-learning-paperfinalmay-18-2012dec2012kspdf/download/What%20we're%20learning%20paper_FINAL_May%2018,%202012_Dec2012ks.pdf
67. Shattuck, K., Zimmerman, W. A., & Adair, D. (2014). Continuous improvement of the QM rubric and review processes: Scholarship of integration and application. Internet Learning, 3(1), 25-34.
68. Wiley, D. (2002). The Disney debacle: Online instruction versus face-to-face instruction. TechTrends, 46(6), 72.
69. Wilson, B. G., & Christopher, L. (2008). Hyper versus reality on campus: Why e-learning isn't likely to replace a professor any time soon. In S. Carliner & P. Shank (Eds.), The e-learning handbook: Past promises, present challenges (pp. 55-76). San Francisco, CA: Pfeiffer/Wiley.
70. Xu, D., & Jaggars, S. S. (2011). Online and hybrid course enrollment and performance in Washington State Community and Technical Colleges (CCRC Working Paper No. 31). Community College Research Center, Columbia University. Retrieved from http://files.eric.ed.gov/fulltext/ED517746.pdf
71. Yang, D., Sinha, T., Adamson, D., & Rose, C. P. (2013, December). "Turn on, tune in, drop out": Anticipating student dropouts in massive open online courses. Proceedings from the 2013 NIPS Data-Driven Education Workshop, Lake Tahoe, NV. Retrieved from http://lytics.stanford.edu/datadriveneducation/papers/yangetal.pdf
72. You, J., Hochlberg, S. A., Ballard, P., Xiao, M., & Walters, A. (2014). Measuring online course design: A comparative analysis. Internet Learning, 3(1), 35-52.