National student survey metrics: where is the room for improvement?

Higher Education

Volumn 78, Issue 6, 2019, Pages 1075-1089

  Langan, A Mark ^a   Harris, W Edwin ^a  

^a MANCHESTER METROPOLITAN UNIVERSITY   (United Kingdom)

Author keywords

Customer; National Student Survey; Output metrics; Student satisfaction

Indexed keywords

EID: 85064954409     PISSN: 00181560     EISSN: 1573174X     Source Type: Journal    
DOI: 10.1007/s10734-019-00389-1     Document Type: Article

References (66)

1. Ball, S. J. (2017). The education debate (third ed.). Bristol: Policy Press.
2. Bell, A. R., & Brooks, C. (2018). What makes students satisfied? A discussion and analysis of the UK’s national student survey. Journal of Further and Higher Education, 42(8), 1118–1142.
3. Bennett, R., & Kane, S. (2014). Students’ interpretations of the meanings of questionnaire items in the National Student Survey. Quality in Higher Education, 20(2), 129–164. 10.1080/13538322.2014.924786.
4. Birnbaum, R. (2000). The life cycle of academic management fads. The Journal of Higher Education, 71(1), 1–16. 10.1080/00221546.2000.11780813.
5. Boehler, M. L., Rogers, D. A., Schwind, C. J., Mayforth, R., Quin, J., Williams, R. G., & Dunnington, G. (2006). An investigation of medical student reactions to feedback: a randomised controlled trial. Medical Education, 40(8), 746–749. 10.1111/j.1365-2929.2006.02503.x.
6. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. 10.1023/A:1010933404324.
7. Brown, S. (2011). Bringing about positive change in the higher education student experience: a case study. Quality Assurance in Education, 19(3), 195–207. 10.1108/09684881111158027.
8. Burgess, A., Senior, C., & Moores, E. (2018). A 10-year case study on the changing determinants of university student satisfaction in the UK. PLoS One, 13(2), e0192976. 10.1371/journal.pone.0192976.
9. Callender, C., Ramsden, P., & Griggs, J. (2014). Review of the “National Student Survey”: report to the UK higher education funding bodies by NatCen Social Research, the Institute of Education, University of London and the Institute for Employment Studies.
10. Cheng, J. H. S., & Marsh, H. W. (2010). National Student Survey: are differences between universities and courses reliable and meaningful? Oxford Review of Education, 36(6), 693–712. 10.1080/03054985.2010.491179.
11. Clayson, D. E. (2009). Student evaluations of teaching: are they related to what students learn?: a meta-analysis and review of the literature. Journal of Marketing Education, 31(1), 16–30. 10.1177/0273475308324086.
12. Dean, A., & Gibbs, P. (2015). Student satisfaction or happiness? A preliminary rethink of what is important in the student experience. Quality Assurance in Education, 23(1), 5–19. 10.1108/QAE-10-2013-0044.
13. Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: a cross-national analysis of university ranking systems. Higher Education, 49(4), 495–533. 10.1007/s10734-004-1746-8.
14. Douglas, A., Douglas, J., & Barnes, B. (2006). Measuring student satisfaction at a UK university. Quality Assurance in Education, 14(3), 251–267. 10.1108/09684880610678568.
15. Douglas, J. A., Douglas, A., McClelland, R. J., & Davies, J. (2015). Understanding student satisfaction and dissatisfaction: an interpretive study in the UK higher education context. Studies in Higher Education, 40(2), 329–349. 10.1080/03075079.2013.842217.
16. Eley, M. (2001). The course experience questionnaire: altering question format and phrasing could improve the CEQ’s effectiveness. Higher Education Research and Development, 20(3), 293–312. 10.1080/07294360127208.
17. Elliott, K. M., & Shin, D. (2002). Student satisfaction: an alternative approach to assessing this important concept. Journal of Higher Education Policy and Management, 24(2), 197–209. 10.1080/1360080022000013518.
18. Fielding, A., Dunleavy, P. J., & Langan, A. M. (2010). Interpreting context to the UK’s National Student (satisfaction) Survey data for science subjects. Journal of Further and Higher Education, 34(3), 347–368. 10.1080/0309877X.2010.484054.
19. García-Aracil, A. (2008). European graduates’ level of satisfaction with higher education. Higher Education, 57(1), 1. 10.1007/s10734-008-9121-9.
20. Genuer, R., Poggi, J.-M., & Tuleau-Malot, C. (2010). Variable selection using random forests. Pattern Recognition Letters, 31(14), 2225–2236. 10.1016/j.patrec.2010.03.014.
21. Grömping, U. (2009). Variable importance assessment in regression: linear regression versus random forest. The American Statistician, 63, 308–319.
22. Grove, J. (2013). Global gender index. Times Higher Education. http://www.timeshighereducation.co.uk/features/globalgender-index-2013/2003517.article. Accessed 15 April 2019.
23. Gunn, A. (2018). Metrics and methodologies for measuring teaching quality in higher education: developing the Teaching Excellence Framework (TEF). Educational Review, 70(2), 129–148. 10.1080/00131911.2017.1410106.
24. Hardman, J., Paucar-Caceres, A., & Fielding, A. (2013). Predicting students’ progression in higher education by using the random forest algorithm. Systems Research and Behavioral Science, 30, 194–203. 10.1002/sres.2130.
25. Hartman, D. E., & Schmidt, S. L. (1995). Understanding student/alumni satisfaction from a consumer’s perspective: the effects of institutional performance and program outcomes. Research in Higher Education, 36(2), 197–217. 10.1007/BF02207788.
26. Hazelkorn, E. (2007). The impact of league tables and ranking systems on higher education decision making. Higher Education Management and Policy, 19(2), 1–24. 10.1787/hemp-v19-art12-en.
27. Hazelkorn, E. (2015). Rankings and the reshaping of higher education: the battle for world-class excellence. Springer.
28. HEFCE. (2013). UK review of the provision of information about higher education: National Student Survey results and trends analysis 2005–2013 (p. 60).
29. Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher Education, 38, 758–773. 10.1080/03075079.2011.598505.
30. Kuh, G. D. (2003). What we’re learning about student engagement from NSSE: benchmarks for effective educational practices. Change: The Magazine of Higher Learning, 35(2), 24–32. 10.1080/00091380309604090.
31. Kuh, G. D. (2009). The national survey of student engagement: conceptual and empirical foundations. New Directions for Institutional Research, 141, 5–20. 10.1002/ir.283.
32. Langan, A. M., Dunleavy, P., & Fielding, A. (2013). Applying models to national surveys of undergraduate science students: what affects ratings of satisfaction? Educational Sciences, 3(2), 193–207. 10.3390/educsci3020193.
33. Langan, A. M., Harris, W. E., Barrett, N., Hamshire, C. H., & Wibberley, C. (2016). Benchmarking factor selection and sensitivity: a case study with nursing courses. Studies in Higher Education, 43(9), 1586–1596. 10.1080/03075079.2016.1266613.
34. Langan, A. M., Scott, N., Partington, S., & Oczujda, A. (2017). Coherence between text comments and the quantitative ratings in the UK’s National Student Survey. Journal of Further and Higher Education, 41(1), 16–29. 10.1080/0309877X.2014.1000281.
35. Lee, J. W., Lee, J. B., Park, M., & Song, S. H. (2005). An extensive comparison of recent classification tools applied to microarray data. Computational Statistics & Data Analysis, 48(4), 869–885.
36. Lenton, P. (2015). Determining student satisfaction: an economic analysis of the National Student Survey. Economics of Education Review, 47, 118–127. 10.1016/j.econedurev.2015.05.001.
37. Letcher, D. W., & Neves, J. S. (2010). Determinants of undergraduate business student satisfaction. Research in Higher Education Journal, 6, 1.
38. Liaw, A., & Wiener, M. (2002). Classification and regression by randomForest. R News, 2/3(December), 18–22.
39. Locke, W. (2014). Shifting academic careers: implications for enhancing professionalism in teaching and supporting learning (Report). York: Higher Education Academy https://www.heacademy.ac.uk/resource/shifting-academic-careers-implications-enhancing-professionalism-teaching-and-supported. Accessed 13 September 2018.
40. Marsh, H. W. (2007). Students’ evaluations of university teaching: dimensionality, reliability, validity, potential biases and usefulness. In R. P. Perry & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education: an evidence-based perspective (pp. 319–383). Dordrecht: Springer Netherlands. 10.1007/1-4020-5742-3_9.
41. Molesworth, M., Nixon, E., & Scullion, R. (2009). Having, being and higher education: the marketisation of the university and the transformation of the student into consumer. Teaching in Higher Education, 14(3), 277–287. 10.1080/13562510902898841.
42. Murtonen, M., Gruber, H., & Lehtinen, E. (2017). The return of behaviourist epistemology: a review of learning outcomes studies. Educational Research Review, 22, 114–128. 10.1016/j.edurev.2017.08.001.
43. Neuendorf, K. A. (2016). The content analysis guidebook. Sage.
44. Nisbett, R. E., & Wilson, T. D. (1977). The halo effect: evidence for unconscious alteration of judgments. Journal of Personality and Social Psychology, 35, 250–256. 10.1037/0022-3514.35.4.250.
45. O’Neill, M. A., & Palmer, A. (2004). Importance-performance analysis: a useful tool for directing continuous quality improvement in higher education. Quality Assurance in Education, 12(1), 39–52. 10.1108/09684880410517423.
46. Palacio, M. A., Díaz Meneses, G., & Pérez Pérez, P. J. (2002). The configuration of the university image and its relationship with the satisfaction of students. Journal of Educational Administration, 40, 486–505. 10.1108/09578230210440311.
47. Payne, S. L., Kleine, K. L. M., Purcell, J., & Carter, G. R. (2005). Evaluating academic challenge beyond the NSSE. Innovation Higher Education, 30, 129–146. 10.1007/s10755-005-5015-2.
48. Peterson, R. A., & Wilson, W. R. (1992). Measuring customer satisfaction: fact and artifact. Journal of the Academy of Marketing Science, 20(1), 61. 10.1007/BF02723476.
49. Prasad, A. M., Iverson, L. R., & Liaw, A. (2006). Newer classification and regression tree techniques: bagging and random forests for ecological prediction. Ecosystems, 9, 181–199. 10.1007/s10021-005-0054-1.
50. R Core Team. (2018). R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing URL https://www.R-project.org/. Accessed 15 April 2019.
51. Radloff, A., & Coates, H. (2010). Doing more for learning: Enhancing engagement and outcomes - Australasian Student Engagement Report. Australian Council for Educational Research. https://research.acer.edu.au/cgi/viewcontent.cgi?article=1011&context=ausse.
52. Ramsden, P. (1991). A performance indicator of teaching quality in higher education: The Course Experience Questionnaire. Studies in Higher Education, 16(2), 129–150. 10.1080/03075079112331382944.
53. Richardson, J. T. E. (2005). Instruments for obtaining student feedback: a review of the literature. Assessment & Evaluation in Higher Education, 30(4), 387–415. 10.1080/02602930500099193.
54. Rienties, B. (2014). Understanding academics’ resistance towards (online) student evaluation. Assessment & Evaluation in Higher Education, 39, 987–1001. 10.1080/02602938.2014.880777.
55. Shah, M. (2012). Ten years of external quality audit in Australia: evaluating its effectiveness and success. Assessment & Evaluation in Higher Education, 37, 761–772. 10.1080/02602938.2011.572154.
56. Shah, M., Cheng, M., & Fitzgerald, R. (2017). Closing the loop on student feedback: the case of Australian and Scottish universities. Higher Education, 74(1), 115–129.
57. Sharpe, R. (2019). Evaluating the student experience: a critical review of the use of surveys to enhance the student experience. In K. Trimmer, T. Newman, & F. F. Padró (Eds.), Ensuring quality in professional education volume II: engineering pedagogy and international knowledge structures (pp. 29–45). Cham: Springer International Publishing.
58. Soh, K. (2017). The seven deadly sins of world university ranking: a summary from several papers. Journal of Higher Education Policy and Management, 39(1), 104–115. 10.1080/1360080X.2016.1254431.
59. Strobl, C., Malley, J., & Tutz, G. (2009). An introduction to recursive partitioning: rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychological Methods, 14, 323–348. 10.1037/a0016973.
60. Surridge, P. (2008). The National Student Survey 2005–2007: findings and trends (No. HEFCW/08/79 Annex A). Higher Education Funding Council for England.
61. Wilson, K. L., Lizzio, A., & Ramsden, P. (1997). The development, validation and application of the course experience questionnaire. Studies in Higher Education, 22(1), 33–53. 10.1080/03075079712331381121.
62. Yorke, M. (2009). ‘Student experience’ surveys: some methodological considerations and an empirical investigation. Assessment & Evaluation in Higher Education, 34(6), 721–739. 10.1080/02602930802474219.
63. Yorke, M. (2013). Surveys of ‘the student experience’ and the politics of feedback. In Reconceptualising feedback in higher education: developing dialogue with students (pp. 6–18). London: Routledge.
64. Yorke, M. (2015). “Student experience” surveys: theoretical, political and empirical dimensions. In Researching higher education: international perspectives on theory, policy and practice (pp. 209–226). London: Routledge.
65. Yorke, M., Orr, S., & Blair, B. (2014). Hit by a perfect storm? Art & design in the National Student Survey. Studies in Higher Education, 39(10), 1788–1810. 10.1080/03075079.2013.806465.
66. Zepke, N. (2014) Student engagement research in higher education: questioning an academic orthodoxy. Teaching in Higher Education, 19(6), 697–708. 10.1080/13562517.2014.901956.