Fitting a Mixture Rasch Model to English as a Foreign Language Listening Tests: The Role of Cognitive and Background Variables in Explaining Latent Differential Item Functioning

International Journal of Testing

Volumn 15, Issue 3, 2015, Pages 216-238

  Aryadoust, Vahid ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

artificial neural network; gender; item response theory; lexico grammatical knowledge; listening comprehension; metacognitive strategy awareness; mixture Rasch measurement

Indexed keywords

EID: 84938962020     PISSN: 15305058     EISSN: 15327574     Source Type: Journal    
DOI: 10.1080/15305058.2015.1004409     Document Type: Article

References (45)

1. T.A.Ackerman,, M.J.Gierl,, & C.M.Walker, (2003). Using multidimensional Item Response Theory to evaluation educational and psychological tests. Educational Measurement, Issues and Practice, 22, 37–50.
2. N.Alexeev,, J.L.Templin,, & A.S.Cohen, (2011). Spurious latent classes in the mixture Rasch model. Journal of Educational Measurement, 48(3), 313–332.
3. D.Andrich, (1978). A rating formulation for ordered response categories. Psychometrika, 43, 357–374.
4. V.Aryadoust, (2012). Differential item functioning in while-listening performance tests: The case of IELTS listening test. International Journal of Listening, 26, 40–60.
5. V.Aryadoust,, & C.C.M.Goh, (2014). Predicting listening item difficulty with language complexity measures: A comparative data mining study. CaMLA Working Papers, 2014–01. Ann Arbor, MI: CaMLA.
6. V.Aryadoust,, C.C.M.Goh,, & O.K.Lee, (2011). An investigation of differential item functioning in the MELAB listening test. Language Assessment Quarterly, 8, 361–385.
7. P.Baghaei,, & C.H.Carstensen, (2013). Fitting the mixed Rasch model to a reading comprehension test: Identifying reader types. Practical Assessment, Research & Evaluation, 18(5), 1–13.
8. G.D.Bodie,, & N.Crick, (2014). Listening, hearing, sensing: Three modes of being and the phenomenology of Charles Sanders Peirce. Communication Theory, 24, 105–123.
9. G.Buck, (2001). Assessing listening. Cambridge, UK: Cambridge University Press.
10. Y.-F.Chen,, & H.Jiao, (2014). Exploring the utility of background and cognitive variables in explaining latent differential item functioning: An example of the PISA 2009 reading assessment. Educational Assessment, 19, 77–96.
11. A.Cohen,, & D.M.Bolt, (2005). A mixture model analysis of differential item functioning. Journal of Educational Measurement, 42, 133–148.
12. Y.Dai, (2013). A mixture Rasch model with a covariate: A simulation study via Bayesian Markov Chain Monte Carlo estimation. Applied Psychological Measurement, 37, 375–396.
13. P.Dunkel,, G.Henning,, & C.Chaudron, (1993). The assessment of a listening comprehension construct: A tentative model for test specification and development. Modern Language Journal, 77, 180–191.
14. J.Field, (2009). A cognitive validation of the lecture-listening component of the IELTS listening paper. In L.Taylor (Ed.), IELTS research reports (Vol.). Canberra, Australia: IELTSAustralia, Pty Ltd & British Council.
15. J.H.Flavell,, P.H.Miller,, & S.A.Miller, (1993). Cognitive development (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
16. Frick, H., Strobl, C., & Zeileis, A. (in press). Rasch mixture models for DIF detection: A comparison of old and new score specifications. Educational and Psychological Measurement.
17. H.Fukuhara,, & A.Kamata, (2011). A bi-factor multidimensional item response theory model for differential item functioning analysis on testlet-based items. Applied Psychological Measurement, 35, 604–622.
18. C.C.M.Goh, (2000). A cognitive perspective on language learners’ listening comprehension problems. System, 28, 55–75.
19. C.C.M.Goh,, & G.Hu, (2014). Exploring the relationship between metacognitive awareness and listening performance with questionnaire data. Language Awareness, 23, 255–274.
20. S.Hong,, & S.-Y.Min, (2007). Mixed Rasch modeling of the self-rating depression scale: Incorporating latent class and Rasch rating scale models. Educational and Psychological Measurement, 67, 280–299.
21. X.Huang,, M.Wilson,, & L.Wang, (in press). Exploring plausible causes of differential item functioning in the PISA science assessment: Language, curriculum or culture. Educational Psychology: An International Journal of Experimental Educational Psychology.
22. M.Imhof,, & L.Janusik, (2006). Development and validation of the Imhof-Janusik Listening Concepts Inventory to measure listening conceptualization differences between cultures. Journal of Intercultural Communication Research, 35, 79–98.
23. L.Janusik, (2007). Building listening theory: The validation of the Conversational Listening Span. Communication Studies, 58, 139–156.
24. L.Janusik,, & S.A.Keaton, (2011). Listening metacognitions: Another key to teaching listening? Listening Education, 3(2), 35–44.
25. K.D.Kubinger, (2005). Psychological test calibration using the Rasch model: Some critical suggestions on traditional approaches. International Journal of Testing, 5, 377–394.
26. A.J.Kunnan, (2013). Fairness and justice in language assessment. In A.J.Kunnan (Ed.), The companion to language assessment (pp. 540–559). Boston, MA: Wiley-Blackwell.
27. F.Li,, A.S.Cohen,, S.H.Kim,, & S.J.Cho, (2009). Model selection methods for mixture dichotomous IRT models. Applied Psychological Measurement, 33, 353–373.
28. P.Y.Lin,, & Y.C.Lin, (2013). Examining student factors in sources of setting accommodation DIF. Educational and Psychological Measurement, 74, 759–794.
29. J.M.Linacre, (2014). A user's guide to WINSTEPS. Chicago, IL: Winsteps.com.
30. A.M.Maij-de Meij,, H.Kelderman,, & H.van der Flier, (2008). Fitting a mixture item response theory model to personality questionnaire data: Characterizing latent classes and investigating possibilities for improving prediction. Applied Psychological Measurement, 32, 611–631.
31. M.E.Oliveri,, K.Ercikan,, & B.D.Zumbo, (2013). Analysis of sources of latent class DIF in international assessments. International Journal of Testing, 13, 272–293.
32. F.Rijmen,, & P.De Boeck, (2005). A relation between a between-item multidimensional IRT model and the mixture Rasch model. Psychometrika, 70, 481–496.
33. J.Rost, (1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271–282.
34. Rost, J., Carstensen, C. H., & von Davier, M. (1997). Applying the mixed Rasch model to personality questionnaires. In J. Rost & R. Langeheine (Eds.), Applications of latent trait and latent class models in the social sciences (pp. 324–332). New York, NY: Waxmann.
35. J.Rost,, & M.von Davier, (1994). A conditional item fit index for Rasch models. Applied Psychological Measurement, 18, 171–182.
36. K.M.Samuelsen, (2005). Examining differential item functioning from a latent class perspective. Unpublished PhD dissertation, University of Maryland, College Park.
37. A.Smit,, H.Kelderman,, & H.van der Flier, (2000). The mixed Birnbaum model: Estimation using collateral information. Methods of Psychological Research—Online, 5(4), 31–43.
38. L.Vandergrift,, & C.C.M.Goh, (2012). Teaching and learning second language listening: Metacognition in action. New York, NY: Routledge.
39. L.Vandergrift,, C.C.M.Goh,, C.Mareschal,, & M.H.Tafaghodtari, (2006). The Metacognitive Awareness Listening Questionnaire (MALQ): Development and validation. Language Learning, 56, 431–462.
40. M.von Davier,. (2001a). WINMIRA [Computer Software]. Groningen, The Netherlands: ASC-Assessment Systems Corporation, USA and Science Plus Group.
41. M.von Davier,. (2001b). WINMIRA user manual. Groningen, The Netherlands: ASC-Assessment Systems Corporation, USA and Science Plus Group.
42. M.von Davier,. (2005a). A general diagnostic model applied to language testing data (Research Report RR-05-16). Princeton, NJ: Educational Testing Service.
43. M.von Davier,. (2005b). Mdltm: Software for the general diagnostic model and for estimating mixtures of multidimensional discrete latent traits models [Computer software]. Princeton, NJ: Educational Testing Service.
44. M.von Davier,. (2008). The mixture general diagnostic model. In G.R.Hancock & K.M.Samuelsen (Eds.), Advances in latent variable mixture models (pp. 1–24). Charlotte, NC: Information Age Publishing.
45. A.D.Wolvin,, & C.G.Coakley, (1994). Listening competency. International Journal of Listening, 8, 148–160.