The development of a content analysis model for assessing students' cognitive learning in asynchronous online discussions

Educational Technology Research and Development

Volumn 59, Issue 1, 2011, Pages 43-70

  Yang, Dazhi ^a   Richardson, Jennifer C ^b   French, Brian F ^c   Lehman, James D ^b  

^a BOISE STATE UNIVERSITY   (United States)

^b PURDUE UNIVERSITY   (United States)

^c WASHINGTON STATE UNIVERSITY   (United States)

Author keywords

Assessment; Asynchronous online discussions; Cognitive learning; Content analysis; Mixed methods

Indexed keywords

EID: 79551690035     PISSN: 10421629     EISSN: 15566501     Source Type: Journal    
DOI: 10.1007/s11423-010-9166-1     Document Type: Article

References (52)

1. Allen, E., & Seaman, J. (2008). Staying the course: Online education in the United States, 2008. Retrieved December 29, 2008, from http://sloan-c. org/publications/survey/pdf/staying_the_course. pdf.
2. Anderson, T. (2008). Teaching in an online learning context. In T. Anderson (Ed.), The theory and practice of online learning (2nd ed., pp. 343-365). Athabasca: Athabasca University, AU.
3. Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Longman.
4. Anderson, J. G. (n. d.). Measurement models: Exploratory and confirmatory factor analysis. Retrieved March 28, 2008, from http://web. ics. purdue. edu/~janders1/Soc681/Measurement%20Models. ppt.
5. Armour-Thomas, E. (1986). Toward a qualitative analysis of standardized tests using an information processing model. ERIC Document Reproduction Service No. ED286933.
6. Barbera, E. (2006). Collaborative knowledge construction in highly structured virtual discussions. Quarterly Review of Distance Education, 7(1), 1-12.
7. Bloom, B. S. (1956). Taxonomy of educational objectives. Handbook I: The cognitive domain. New York: David McKay.
8. Brown, J. D. (1996). Testing in language programs. Upper Saddle River, NJ: Prentice Hall Regents.
9. Clariana, R. B., Wallace, P. E., & Godshalk, V. M. (2009). Deriving and measuring group knowledge structure from essays: The effects of anaphoric reference. Educational Technology Research and Development. Retrieved April 12, 2009, from http://www. springerlink. com/content/x4t20q1936521447/fulltext. pdf.
10. Cognitive Definition. (2010). LearningRX. com. Inc. Available from http://www. learningrx. com/cognitive-definition-faq. htm.
11. Corich, S., Kinshuk, & Hunt, L. M. (2006). Measuring critical thinking within discussion forums using a computerised content analysis tool. Retrieved November 12, 2006, from http://www. networkedlearningconference. org. uk/abstracts/pdfs/P07%20Corich. pdf.
12. Council for Higher Education Accreditation (CHEA). (2002). Student learning outcomes workshop. CHEA Chronicle, 5(2), 1-3. Retrieved November 18, 2006, from http://www. chea. org/Chronicle/vol5/no2/Chron-vol5-no2. pdf (Electronic Version).
13. DeCoster, J. (1998). Overview of factor analysis. Retrieved March 31, 2008, from http://www. stat-help. com/factor. pdf.
14. Facione, P. A., Facione, N. C., & Giancarlo, C. A. (2000). The disposition toward critical thinking: Its character, measurement, and relationship to critical thinking skill. Informal Logic, 20(1), 61-84.
15. Facione, P. A., Giancarlo, C. A., Facione, N. C., & Gainen, J. (1995). The disposition toward critical thinking. Journal of General Education, 44, 1-25.
16. Fahy, P. J. (2003). Indicators of support in online interaction. International Review of Research in Open and Distance Learning. Available from http://www. irrodl. org/content/v4. 1/fahy. html.
17. Garrison, D. R. (1992). Critical thinking and self-directed learning in adult education: An analysis of responsibility and control issues. Adult Education Quarterly, 42(3), 136-148.
18. Garrison, D. R. (2003). Cognitive presence for effective asynchronous online learning: The role of reflective inquiry, self-direction and metacognition. In J. Bourne & J. C. Moore (Eds.), Elements of quality online education: Practice and direction (pp. 47-58). Needham, MA: Sloan Center for Online Education.
19. Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105.
20. Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7-23.
21. Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago: Aldine.
22. Golafshani, N. (2003). Understanding reliability and validity in qualitative research. The Qualitative Report, 8(4), 597-606.
23. Gunawardena, C. N., Lowe, C. A., & Anderson, T. (1997). Analysis of global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397-431.
24. Hanson, W. E., Creswell, J. W., Plano-Clark, V. L., Petska, K. S., & Creswell, J. D. (2005). Mixed methods research designs in counseling psychology. Journal of Counseling Psychology, 52, 224-235.
25. Henri, F. (1992). Computer conferencing and content analysis. In A. Kaye (Ed.), Collaborative learning through computer conferencing: The Najaden papers (pp. 117-136). London: Springer-Verlag.
26. Hu, L., & Bentler, P. M. (1999). Cut-off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55.
27. Joreskog, K. G., & Sorbom, D. (2001). LISREL 8: Users' reference guide. Chicago: Scientific Software International.
28. Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17-64). Westport: American Council on Education and Praeger.
29. Marra, R. (2006). A review of research methods for assessing content of computer-mediated discussion forums. Journal of Interactive Learning Research, 17(3), 243-267.
30. Marra, R. M., Moore, J. L., & Klimczack, A. K. (2004). Content analysis of online discussion forums: A comparative analysis of protocols. Educational Technology Research and Development, 52(2), 23-40.
31. Mazur, J. (2004). Conversation analysis for educational technologists: Theoretical and methodological issues for researching the structures, processes and meaning of on-line talk. In D. H. Jonassen (Ed.), Handbook for research in educational communications and technology (2nd ed., pp. 1073-1099). Mahwah, NJ: Lawrence Erlbaum Associates.
32. McLoughlin, C., & Luca, J. (2000). Cognitive engagement and higher order thinking through computer conferencing: We know why but do we know how? Retrieved October 2, 2006, from http://lsn. curtin. edu. au/tlf/tlf2000/mcloughlin. html.
33. McPeck, J. (1981). Critical thinking and education. New York: St. Martin's.
34. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage.
35. Moallem, M. (2005). Designing and managing student assessment in online learning environment. In P. Comeaux (Ed.), Assessing online learning (pp. 18-34). Bolton, MA: Anker.
36. Moore, M. G. (1989). Three types of interaction. The American Journal of Distance Education, 3(2). Retrieved January 24, 2005, from http://www. ajde. com/Contents/vol3_2. htm#editorial (Electronic version).
37. Moskal, B. M., & Leydens, J. A. (2000). Scoring rubric development: Validity and reliability. Practical Assessment, Research and Evaluation, 7(10). Retrieved May 31, 2007, from http://PAREonline. net/getvn. asp?v=7& n=10.
38. Newman, D. R., Webb, B., & Cochrane, C. (1996). A content analysis method to measure critical thinking in face-to-face and computer supported group learning. Retrieved October 5, 2006, from http://www. qub. ac. uk/mgt/papers/methods/contpap. html.
39. Pallant, J. (2007). SPSS survival manual (3rd ed.). Maidenhead: McGraw Hill/Open University.
40. Pandit, N. R. (1996). The creation of theory: a recent application of the grounded theory method. Qualitative Report, 2(4). http://www. nova. edu/ssss/QR/QR2-4/pandit. html.
41. Pirnay-Dummer, P., Ifenthaler, D., & Spector, J. M. (2009). Highly integrated model assessment technology and tools. Educational Technology Research and Development. Retrieved April 12, 2009, from http://www. springerlink. com/content/6005268510618175/fulltext. pdf.
42. Reeves, T. C. (2000). Alternative assessment approaches for online learning environments in higher education. Journal of Educational Computing Research, 23(1), 101-111.
43. Riffe, D., Lacy, S., & Fico, F. (1998). Analyzing media messages: Quantitative content analysis. New Jersey: Lawrence Erlbaum.
44. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12, 8-22.
45. Savenye, W. C. (2004). Alternatives for assessing learning in Web-based courses. Distance Learning, 1(1), 29-35.
46. Schumacher, B. K., West, D. V., & Angell, L. R. (1997, November). Assessing knowledge and cognitive skills. Paper presented at the Annual Meeting of the National Communication Association, Chicago, IL. ERIC Document Reproduction Service No. ED417429.
47. Scientific Software International. (2008). Retrieved June 12, 2008, from http://www. ssicentral. com/.
48. Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd ed.). Thousand Oaks, CA: Sage.
49. Swan, K., Schenker, J., Arnold, S., & Kuo, C.-L. (2007). Shaping online discussion: Assessment matters. E-mentor, 1(18). Retrieved December 12, 2006, from http://www. e-mentor. edu. pl/_xml/wydania/18/390. pdf.
50. Swan, K., Shen, J., & Hiltz, S. R. (2006). Assessment and collaboration in online learning. Journal of Asynchronous Learning Networks, 10(1), 45-62. Retrieved November 12, 2006, from http://www. sloan-c-wiki. org/JALN/v10n1/pdf/v10n1_5swan. pdf (Electronic Version).
51. Wijekumar, K., Ferguson, L., & Wagoner, D. (2006). Problems with assessment validity and reliability in Web-based distance learning environments and solutions. Journal of Educational Multimedia and Hypermedia, 15(2), 199-215.
52. Yang, Y. C. (2008). A catalyst for teaching critical thinking in a large university class in Taiwan: Asynchronous online discussions with the facilitation of teaching assistants. Educational Technology Research and Development, 56, 241-264.