On the assessment of expertise profiles

Journal of the American Society for Information Science and Technology

Volumn 64, Issue 10, 2013, Pages 2024-2044

  Berendsen, Richard ^a   De Rijke, Maarten ^a   Balog, Krisztian ^b   Bogers, Toine ^c   Van Den Bosch, Antal ^d  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b UNIVERSITY OF STAVANGER   (Norway)

^c Birketinget 6   (Denmark)

^d RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

Author keywords

information retrieval; knowledge management

Indexed keywords

AUTOMATICALLY GENERATED; CONTENT ANALYSIS; EXPERT FINDING; EXPERT USERS; GAIN INSIGHT; GROUND TRUTH; KNOWLEDGE AREAS; TEST COLLECTION;

ALGORITHMS; KNOWLEDGE MANAGEMENT;

INFORMATION RETRIEVAL;

EID: 84883863475     PISSN: 15322882     EISSN: 15322890     Source Type: Journal    
DOI: 10.1002/asi.22908     Document Type: Article

References (51)

1. Allan, J., Carterette, B., &, Lewis, J., (2005). When will information retrieval be good enough? In 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 433-440). New York: ACM.
2. Azzopardi, L., &, Vinay, V., (2008). Retrievability: An evaluation measure for higher order information access tasks. In 17th ACM Conference on Information and Knowledge Management (pp. 561-570). New York: ACM.
3. Bailey, P., Craswell, N., Soboroff, I., &, de Vries, A., (2007). The CSIRO enterprise search test collection. ACM SIGIR Forum, 41 (2), 42-45.
4. Bailey, P., Craswell, N., de Vries, A.P., &, Soboroff, I., (2008). Overview of the TREC 2007 Enterprise Track. In The Sixteenth Text Retrieval Conference Proceedings (TREC 2007). NIST. Special Publication.
5. Balog, K., (2008, June). People search in the enterprise (PhD thesis). University of Amsterdam.
6. Balog, K., Azzopardi, L., &, de Rijke, M., (2009). A language modeling framework for expert finding. Information Processing and Management, 45 (1), 1-19.
7. Balog, K., Bogers, T., Azzopardi, L., de Rijke, M., &, van den Bosch, A., (2007). Broad expertise retrieval in sparse data environments. In 30th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 551-558). New York: ACM.
8. Balog, K., &, de Rijke, M., (2007). Determining expert profiles (with an application to expert finding). In Proceedings of the 20th international joint conference on Artificial intelligence (IJCAI07) (pp. 2657-2662). San Francisco, CA: Morgan Kaufmann Publishers, Inc.
9. Balog, K., Fang, Y., de Rijke, M., Serdyukov, P., &, Si, L., (2012). Expertise retrieval. Foundations and Trends in Information Retrieval, 6 (2-3), 127-256.
10. Balog, K., Soboroff, I., Thomas, P., Craswell, N., de Vries, A.P., &, Bailey, P., (2009). Overview of the TREC 2008 Enterprise Track. In The Seventeenth Text Retrieval Conference Proceedings (TREC 2008). NIST. Special Publication.
11. Bogers, T., &, Balog, K., (2006). The UvT expert collection. Retrieved from http://ilk.uvt.nl/uvt-expert-collection/
12. Broder, A., (2002). A taxonomy of web search. ACM SIGIR Forum, 36 (2), 3-10.
13. Buckley, C., &, Voorhees, E., (2004). Retrieval evaluation with incomplete information. In 27th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 25-32). New York: ACM.
14. Clarke, C., Kolla, M., Cormack, G., Vechtomova, O., Ashkan, A., Bttcher, S., &, MacKinnon, I., (2008). Novelty and diversity in information retrieval evaluation. In 31st annual international ACM SIGIR Conference on Research and development in information retrieval (pp. 659-666). New York: ACM.
15. Cohen, P., (1995). Empirical methods for artificial intelligence. Cambridge, MA: MIT Press.
16. Craswell, N., de Vries, A., &, Soboroff, I., (2006). Overview of the TREC-2005 enterprise track. In The Fourteenth Text REtrieval Conference Proceedings (TREC 2005). NIST. Special Publication.
17. Cross, R., Parker, A., &, Borgatti, S., (2002). A birds-eye view: Using social network analysis to improve knowledge creation and sharing. Knowledge Directions, 2 (1), 48-61.
18. Harman, D., (1995). Overview of the second text retrieval conference (TREC-2). Information Processing & Management, 31 (3), 271-289.
19. Harman, D., &, Buckley, C., (2009). Overview of the reliable information access workshop. Information Retrieval, 12 (6), 615-641.
20. Hertzum, M., &, Pejtersen, A.M., (2006). The information-seeking practices of engineers: Searching for documents as well as for people. Information Processing & Management, 36 (5), 761-778.
21. Hofmann, K., Balog, K., Bogers, T., &, de Rijke, M., (2010). Contextual factors for finding similar experts. Journal of the American Society for Information Science and Technology, 61 (5), 994-1014.
22. Hofmann, K., Whiteson, S., &, de Rijke, M., (2011). A probabilistic method for inferring preferences from clicks. In 20th ACM Conference on Information and Knowledge Management (pp. 249-258). New York: ACM.
23. Joachims, T., (2002). Evaluating retrieval performance using clickthrough data. In SIGIR Workshop on Mathematical/Formal Methods in Information Retrieval (pp. 12-15). New York: ACM.
24. Jones, K.S., &, van Rijsbergen, C.J., (1975). Report on the need for and the provision of an ideal information retrieval test collection. British Library Research and Development Report, 5266, 43. University Computer Laboratory, Cambridge, United Kingdom.
25. Jurczyk, P., &, Agichtein, E., (2007). Hits on question answer portals: exploration of link analysis for author ranking. In 30th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 845-846). New York: ACM.
26. Kendall, M., (1938). A new measure of rank correlation. Biometrika, 30 (1/2), 81-93, 1938. Oxford University Press, London, United Kingdom.
27. Lazar, J., Feng, J., &, Hochheiser, H., (2010). Research methods in human-computer interaction. New York: John Wiley & Sons Inc.
28. Liebregts, R., &, Bogers, T., (2009). Design and implementation of a university-wide expert search engine. In 31st European Conference on Information Retrieval (pp. 587-594). Berlin, Heidelberg, Germany: Springer Verlag.
29. Radlinski, F., &, Craswell, N., (2010). Comparing the sensitivity of information retrieval metrics. In 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 667-674). New York: ACM.
30. Radlinski, F., Kurup, M., &, Joachims, T., (2008). How does clickthrough data reflect retrieval quality? In 17th ACM Conference on Information and Knowledge Management (pp. 43-52). New York: ACM.
31. de Rijke, M., Balog, K., Bogers, T., &, van den Bosch, A., (2010). On the evaluation of entity profiles. In CLEF 2010: Conference on Multilingual and Multimodal Information Access Evaluation (pp. 94-99). Berlin, Heidelberg, Germany: Springer-Verlag.
32. Sanderson, M., (2010). Test collection based evaluation of information retrieval systems. Foundations and Trends in Information Retrieval, 4 (4), 247-375. Now Publishers, Delft, The Netherlands.
33. Sanderson, M., &, Soboroff, I., (2007). Problems with Kendall tau. In 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 839-840). New York: ACM.
34. Sanderson, M., &, Zobel, J., (2005). Information retrieval system evaluation: effort, sensitivity, and reliability. In 28th Annual International ACMSIGIR Conference on Research and Development in Information Retrieval (pp. 162-169). New York: ACM.
35. Seo, J., &, Croft, W., (2009). Thread-based expert finding. In SIGIR 2009 Workshop on Search in Socal Media. New York: ACM.
36. Sheskin, D., (2011). Handbook of parametric and nonparametric statistical procedures. Boca Raton, FL: CRC Press.
37. Smirnova, E., &, Balog, K., (2011). A user-oriented model for expert finding. In 33rd European Conference on Information Retrieval (pp. 580-592). Berlin, Heidelberg, Germany: Springer-Verlag.
38. Smith, C., &, Kantor, P., (2008). User adaptation: Good results from poor systems. In 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 147-154). New York: ACM.
39. Smucker, M., Allan, J., &, Carterette, B., (2007). A comparison of statistical significance tests for information retrieval evaluation. In 16th ACM conference on Information and Knowledge Management (pp. 623-632). New York: ACM.
40. Soboroff, I., de Vries, A., &, Craswell, N., (2007). Overview of the TREC-2006 Enterprise Track. In The Fifteenth Text REtrieval Conference Proceedings (TREC 2006). NIST. Special Publication.
41. Spärck Jones, K., (1974). Automatic indexing. Journal of Documentation, 30 (4), 393-432.
42. Su, L., (1992). Evaluation measures for interactive information retrieval. Information Processing & Management, 28 (4), 503-516.
43. Su, L., (1994). The relevance of recall and precision in user evaluation. Journal of the American Society for Information Science, 45 (3), 207-217.
44. Turpin, A., &, Scholer, F., (2006). User performance versus precision measures for simple search tasks. In 29th Annual International ACM SIGIR conference on research and development in information retrieval (pp. 11-18). New York: ACM.
45. Voorhees, E., (2000). Variations in relevance judgments and the measurement of retrieval effectiveness. Information Processing & Management, 36 (5), 697-716.
46. Voorhees, E., &, Buckley, C., (2002). The effect of topic set size on retrieval experiment error. In 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 316-323). New York: ACM.
47. Voorhees, E., &, Harman, D., (1996). Overview of the fifth Text REtrieval Conference (TREC-5). In The Fifth Text REtrieval Conference (TREC 1996). Special Publication.
48. Voorhees, E., &, Harman, D., (2000). Overview of the ninth Text REtrieval Conference (TREC-9). In The Ninth Text REtrieval Conference (TREC 2000). Special Publication.
49. W3C. (2005). The W3C test collection. Retrieved from http://research. microsoft.com/users/nickcr/w3c-summary.html
50. Welch, B., (1947). The generalization of Students problem when several different population variances are involved. Biometrika, 34 (1/2), 28-35.
51. Zobel, J., (1998). How reliable are the results of large-scale information retrieval experiments? In 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 307-314). New York: ACM.