Using postgraduate students' evaluations of research experience to benchmark departments and faculties: Issues and challenges

British Journal of Educational Psychology

Volumn 79, Issue 3, 2009, Pages 577-598

  Ginns, Paul ^a   Marsh, Herbert W ^b   Behnia, Masud ^a   Cheng, Jacqueline H S ^b   Scalas, L Francesca ^b  

^a UNIVERSITY OF SYDNEY   (Australia)

^b UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ACADEMIC ACHIEVEMENT; ADULT; ARTICLE; ATTITUDE; AUSTRALIA; CURRICULUM; EDUCATION; FEMALE; HUMAN; MALE; PSYCHOMETRY; QUALITY CONTROL; QUESTIONNAIRE; RESEARCH; UNIVERSITY;

ADULT; ATTITUDE; AUSTRALIA; BENCHMARKING; CURRICULUM; EDUCATION, GRADUATE; FACULTY; FEMALE; HUMANS; MALE; PSYCHOMETRICS; QUESTIONNAIRES; RESEARCH;

EID: 68249134143     PISSN: 00070998     EISSN: None     Source Type: Journal    
DOI: 10.1348/978185408X394347     Document Type: Article

References (55)

1. Abrami, P. C., d'Apollonia, S., & Cohen, P. A. (1990). Validity of student ratings of instruction: What we know and what we do not. Journal of Educational Psychology, 82, 219-231.
2. Ainley, J. (2001). The 1999 Postgraduate Research Experience Questionnaire. Canberra: Department of Education, Training and Youth Affairs.
3. American Association of Colleges of Nursing (2002). Indicators of quality in research-focused doctoral programs in nursing. Journal of Professional Nursing, 18, 289-294.
4. Bair, C. R., & Haworth, J. G. (2004). Doctoral student attrition and persistence: A meta-synthesis of research. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 19). Dordrecht: Kluwer.
5. Barrie, S. C., Ginns, P., & Prosser, M. (2005). Early impact and outcomes of an institutionally aligned, student focused learning perspective on teaching quality assurance. Assessment and Evaluation in Higher Education, 30, 641-656.
6. Biggs, J. (2003). Teaching for quality learning at university. Camberwell: ACER.
7. Bliese, P. (2000). Within-group agreement, non-independence, and reliability. In K. Klein & S. Kozlowski (Eds.), Multi-level theory, research, and methods in organizations (pp.349-381). San Francisco, CA: Jossey-Bass.
8. Cashin, W. E. (1988). Student ratings of teaching. A summary of research. (IDEA paper No. 20). Kansas State University, Division of Continuing Education.
9. Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245-276.
10. Centra, J. A. (1993). Reflective faculty evaluation. San Francisco, CA: Jossey-Bass.
11. Cohen, P. A. (1980). Effectiveness of student-rating feedback for improving college instruction: A meta-analysis. Research in Higher Education, 13, 321-341.
12. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 35, 297-334.
13. d'Appollonia, S., & Abrami, P. C. (1997). Navigating student ratings of instruction. American Psychologist, 52, 1198-1208.
14. Ehrenberg, R. G., & Hurst, P. J. (1998). The 1995 ratings of doctoral programs: A hedonic model. Economics of Education Review, 17, 137-148.
15. Feldman, K. A. (1989a). Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers. Research in Higher Education, 30, 137-194.
16. Feldman, K. A. (1989b). Association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies. Research in Higher Education, 30, 583-645.
17. Feldman, K. A. (1997). Identifying exemplary teachers and teaching: Evidence from student ratings. In R. P. Perry & J. C. Smart (Eds.), Effective teaching in higher education: Research and practice (pp.368-395). New York: Agathon Press.
18. Feldman, K. A. (1998). Reflections on the effective study of college teaching and student ratings: One continuing quest and two unresolved issues. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (pp.35-74). New York: Agathon Press.
19. Gilmore, G. M., Kane, M. T., & Naccarato, R. W. (1978). The generalizability of student ratings of instruction: Estimates of teacher and course components. Journal of Educational Measurement, 15, 1-13.
20. Ginns, P., Ramsden, P., & Conrad, L. (2005). Associations between research students' experiences and learning outcomes. In C. Rust (Ed.), Improving student learning: Diversity and inclusivity (pp.315-327). Oxford: Oxford Centre for Staff and Learning Development.
21. Goldstein, H. (2003). Multilevel statistical models. London: Edward Arnold.
22. Greenwald, A. G., & Gillmore, G. M. (1997). No pain, no gain? The importance of measuring course workload in student ratings of instruction. Journal of Educational Psychology, 89, 743-751.
23. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
24. Jöreskog, K., & Sörbom, D. (1996-2001). LISREL version 8.72.
25. Lorenzo-Seva, U., & Ferrando, P. J. (2006). FACTOR: a computer program to fit the exploratory factor analysis model. Behavior Research Methods, 38, 88-91.
26. Lovitts, B. E. (2001). Leaving the ivory tower: The causes and consequences of departure from doctoral study. New York: Rowman and Littlefield.
27. Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13, 203-229.
28. Marsh, H. W. (1980). The influence of student, course and instructor characteristics on evaluations of university teaching. American Educational Research Journal, 17, 219-237.
29. Marsh, H. W. (1982). SEEQ: A reliable, valid, and useful instrument for collecting students' evaluations of university teaching. British Journal of Educational Psychology, 52, 77-95.
30. Marsh, H. W. (1984). Students' evaluations of university teaching: Dimensionality, reliability, validity, potential biases and utility. Journal of Educational Psychology, 76, 707-754.
31. Marsh, H. W. (1987). Students' evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11, 253-388.
32. Marsh, H. W. (1991). Multidimensional students' evaluations of teaching effectiveness: A test of alternative higher-order structures. Journal of Educational Psychology, 83, 285-296.
33. Marsh, H. W. (2007). Students' evaluations of university teaching: A multidimensional perspective. In R. P. Perry & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective. New York: Springer.
34. Marsh, H. W., & Bailey, M. (1993). Multidimensionality of students' evaluations of teaching effectiveness: A profile analysis. Journal of Higher Education, 64, 1-18.
35. Marsh, H. W., & Dunkin, M. J. (1997). Students' evaluations of university teaching: A multidimensional perspective. Effective teaching in higher education: Research and practice (pp.241-320). New York: Agathon Press.
36. Marsh, H. W., & Roche, L. A. (1997). Making students' evaluations of teaching effectiveness effective. American Psychologist, 52, 1187-1197.
37. Marsh, H. W., Balla, J. R., & Hau, K. T. (1996). An evaluation of incremental fit indices: A clarification of mathematical and empirical processes. In G. A. Marcoulides & R. E. Schumacker (Eds.), Advanced structural equation modelling techniques (pp.315-353). Hillsdale, NJ: Erlbaum.
38. Marsh, H. W., Balla, J. R., & McDonald, R. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103, 391-410.
39. Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis testing approaches to setting cutoff values for fit indexes and dangers in overgeneralising Hu & Bentler's (1999) findings. Structural Equation Modelling, 11, 320-341.
40. Marsh, H. W., Rowe, K. J., & Martin, A. (2002). PhD students evaluation of research supervision: Issues, complexities, and challenges in a nationwide Australian experiment in benchmarking universities. Journal of Higher Education, 73, 313-348.
41. Marsh, H. W., Rowe, K. J., & Martin, A. (2003). PhD students' evaluation of research supervision: Issues, complexities, and challenges in a nationwide experiment in benchmarking universities. Paper presented at the Joint AARE-NZARE (Australian and New Zealand Associations of Research in Education) Meeting, Auckland, NZ, 30 November 3, 2003.
42. McKeachie, W. J. (1979). Student ratings of faculty: A reprise. Academe, 65, 384-397.
43. McKeachie, W. J. (1997). Student ratings: The validity of use. American Psychologist, 52, 1218-1225.
44. Pearson, M. (1999). The changing environment for doctoral education in Australia: Implications for quality management, improvement and innovation. Higher Education Research and Development, 18, 269-287.
45. Pearson, M., & Kayrooz, C. (2004). Enabling critical reflection on research supervisory practice. International Journal for Academic Development, 9, 99-116.
46. Prosser, M., & Trigwell, K. (1999). Understanding learning and teaching: The experience in higher education. Buckingham: Open University Press.
47. Ramsden, P. (1991). A performance indicator of teaching quality in higher education: The Course Experience Questionnaire. Studies in Higher Education, 16, 129-150.
48. Ramsden, P., Conrad, L., Ginns, P., & Prosser, M. (2003). Students' and supervisors' experiences of the context for postgraduate study. Paper presented at the Tenth European Conference for Research on Learning and Instruction, Padua, Italy, 26-30 August 2003.
49. Rasbash, J., Steele, F., Browne, W., & Prosser, B. (2004). A user's guide to MLwiN version 2.0. University of Bristol.
50. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models (2nd ed.). Thousand Oaks, CA: Sage.
51. Schafer, J. L., & Olsen, M. K. (1998). Multiple imputation for multivariate missing-data problems: A data analyst's perspective. Multivariate Behavioral Research, 33, 545-571.
52. Snijders, T., & Bosker, R. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage Publications.
53. Trigwell, K., & Dunbar-Goddett, H. (2005). The research experience of postgraduate students at the University of Oxford. Oxford: Institute for the Advancement of University Learning. Retrieved 29 January 2007, from http://www.learning.ox.ac.uk/files/PGRreport.pdf.
54. Wijesundera, S., Hicks, O., & Mann, S. (1996). Student and supervisor perceptions of postgraduate supervision. Paper presented at the National Confference 'Quality in Postgraduate Research - Is it happening?', 18-19 April, Adelaide.
55. Wilson, K. L., Lizzio, A., & Ramsden, P. (1997). The development, validation and application of the course experience questionnaire. Studies in Higher Education, 22, 33-53.