Are self-assessments reliable indicators of topic knowledge?

Proceedings of the ASIST Annual Meeting

Volumn 47, Issue , 2010, Pages

  Cole, Michael J ^a   Zhang, Xiangmin ^a   Liu, Jinging ^a   Liu, Chang ^a   Belkin, Nicholas J ^a   Bierig, Ralf ^a   Gwizdka, Jacek ^a  

^a RUTGERS UNIVERSITY   (United States)

Author keywords

Knowledge representation; Methodology; User studied

Indexed keywords

EID: 84861420442     PISSN: None     EISSN: 15508390     Source Type: Journal    
DOI: 10.1002/meet.14504701285     Document Type: Conference Paper

References (32)

1. Armstrong, S., Gleitman, L., and Gleitman, H. (1983). What some concepts might not be. Cognition, 13(3), 263-308.
2. Belkin, N. J. (2008). Some(what) grand challenges for information retrieval. SIGIR Forum 42(1), 47-54.
3. Belkin, N.J., Oddy, R.N., & Brooks, H.M. (1982). ASK for information retrieval: Part I. Background and theory. Journal of Documentation 38(2), 61-71
4. Bierig, R., Cole, M.J., & Gwizdka, J. (2010). A data analysis and modelling framework for the evaluation of interactive information retrieval. In Proceedings of ECIR 2010, (Milton Keynes, UK 2009), Lecture Notes in Computer Science. Berlin: Springer.
5. Bond, T.G. & Fox, C.M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences. Mawah, NJ: Lawrence Erlbaum.
6. Borlund, P. (2003). The IIR evaluation model: A framework for evaluation of interactive information retrieval systems. Information Research, 8(3), Retrieved from http://informationr.net/ir/8-3/paper152.html
7. Boud, D. & Falchikov, N. (1989). Quantitative studies of student self-assessment in higher education: A critical analysis of findings. Higher Education, 18(5), 529-549.
8. Brantmeier, C. (2006). Advanced L2 learners and reading placement: Self-assessment, CBT, and subsequent performance. System, 34(1), 15-35.
9. Brantmeier, C. & Vanderplank, R. (2008). Descriptive and criterion-referenced self-assessment with L2 readers. System, 36(3), 456-477.
10. Burgess, C. (1998). From simple associations to the building blocks of language: Modeling meaning in memory with the HAL model. Behavior Research Methods, Instruments, & Computers, 30, 188-198.
11. Fodor, J. (1975). The language of thought. Cambridge, MA: Harvard University Press.
12. Harrington, M. & Carey, M. (2009). The on-line Yes/No test as a placement tool. System, 37(4), 614-626.
13. Heilman, M. & Eskenazi, M. (2008). Self-assessment in vocabulary tutoring. In Proceedings of Intelligent Tutoring Systems 9th International Conference, ITS 2008, Montreal, Canada, June 23-27, 2008, volume 5091/2008 of LNCS, 656-658, Berlin. Springer.
14. Hersh, W. & Voorhees, E. (2009). TREC genomics special issue overview. Information Retrieval, 12(1),:1-15.
15. Hsieh-Yee, I. (1993). Effects of search experience and subject knowledge on online search behavior: Measuring the search tactics of novice and experienced searchers. Journal of the American Society for Information Science, 44(3), 161-174.
16. Katz, J. (1972). Semantic theory. New York: Harper & Row.
17. Kelly, D. & Cool, C. (2002). The effects of topic familiarity on information search behavior. In Proceedings of the Second ACM/IEEE Joint Conference on Digital Libraries (JCDL '02), Portland, Oregon, 74-75. New York. ACM.
18. Kelly, D., Kantor, P., Morse, E., Scholtz, J., & Sun, Y. (2006). User-centered evaluation of interactive question answering systems. In Proceedings of the Interactive Question Answering Workshop at HLT-NAACL 2006 June 9, 2006, New York, New York, 49-56. Stroudsburg, PA. Association for Computational Linguistics.
19. Landauer, T.K., Laham, D., & Foltz, P. (2003). Automated scoring and annotation of essays with the intelligent essay assessor. In Shermis, M. and Burstein, J., editors, Automated Essay Scoring: A cross-disciplinary perspective, pp 87-112. Lawrence Erlbaum, Mawah, NJ.
20. Landauer, T.K. (2002). On the computational basis of learning and cognition: Arguments from LSA. In Ross, N., editor, The Psychology of Learning and Motivation, volume 41, pages 43-84.
21. Malabonga, V., Kenyon, D., & Carpenter, H. (2005). Self-assessment, preparation and response time on a computerized oral proficiency test. Language Testing, 22(1), 59-82.
22. Meara, P. & Buxton, B. (1987). An Alternative to Multiple choice tests. Language Testing, 4(2), 142-154.
23. Mitrovic, A. (2001). Investigating students' self-assessment skills. In Proceedings of the 8th International Conference for User Modeling, Sonthofen, Germany, July 13-17, 2001, volume 2109/2010 of LNCS, 247-250, Berlin. Springer.
24. Mitrovic, A. & Martin, B. (2007). Evaluating the Effect of Open Student Models on Self-Assessment. International Journal of Artificial Intelligence in Education, 17(2), 121-144.
25. Pirrone, R., Azevedo, R. & Biswas, G. (2009). Why metacognition in modern education systems? In Proceedings of AAAI Fall Symposium on Cognitive and Metacognitive Educational Systems, Alexandria, Virginia, USA, November 8, 2009. vii-viii Menlo Park, CA. AAAI Press
26. Roberts, P.M., Cohen, A.M., & Hersh, W.R. (2009). Tasks, topics and relevance judging for the TREC genomics track: Five years of experience evaluating biomedical text information retrieval systems. Information Retrieval, 12(1), 81-97.
27. Ross, J.A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment Research & Evaluation, 11(10), 1-13.
28. Toms, E., Mackenzie, T., Jordan, C., O'Brien, H., Freund, L., Toze, S., Dawe, E., & MacNutt, A. (2007). How task affects information search. In Workshop Pre-Proceedings in Initiative for the Evaluation of XML Retrieval (INEX). 337-341.
29. Wen, L., Ruthven, I., & Borlund, P. (2006). The effects on topic familiarity on online search behaviour and use of relevance criteria. In Advances in Information Retrieval, 28th European Conference on IR Research, ECIR 2006 (London, UK), 456-459, London. Springer
30. White, R.W., Dumais, S.T., & Teevan, J. (2009). Characterizing the influence of domain expertise on web search behavior. In Proceedings of the Second ACM International Conference on Web Search and Data Mining, pages 132-141. ACM New York, NY.
31. Wildemuth, B.M. (2004). The effects of domain knowledge on search tactic formulation. Journal of the American Society for Information Science and Technology, 55(3), 246-258.
32. Zhang, X., Anghelescu, H., & Yuan, X. (2005). Domain knowledge, search behaviour, and search effectiveness of engineering and science students: An exploratory study. Information Research, 10(2) Retrieved from http://informationr.net/ir/10-2/paper217.html