Visual interactive failure analysis: Supporting users in information retrieval evaluation

IIiX 2012 - Proceedings 4th Information Interaction in Context Symposium: Behaviors, Interactions, Interfaces, Systems

Volumn , Issue , 2012, Pages 194-203

  Angelini, Marco ^a   Ferro, Nicola ^b   Santucci, Giuseppe ^a   Silvello, Gianmaria ^b  

^a SAPIENZA UNIVERSITY OF ROME   (Italy)

^b UNIVERSITY OF PADOVA   (Italy)

Author keywords

Best practices; Data test collection; Evaluation infrastructure; Experimental evaluation; Scientific data

Indexed keywords

BEST PRACTICES; EVALUATION INFRASTRUCTURE; EXPERIMENTAL EVALUATION; SCIENTIFIC DATA; TEST COLLECTION;

LEARNING ALGORITHMS; QUERY LANGUAGES; SEARCH ENGINES;

FAILURE ANALYSIS;

EID: 84867444620     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2362724.2362757     Document Type: Conference Paper

References (18)

1. Banks, D., Over, P., and Zhang, N.-F. (1999). Blind Men and Elephants: Six Approaches to TREC data. Information Retrieval, 1:7-34.
2. Berkhin, P. (2006). A Survey of Clustering Data Mining Techniques. In Kogan, J., Nicholas, C., and Teboulle, M., editors, Grouping Multidimensional Data, pages 25-71. Springer-Verlag, Heidelberg, Germany.
3. Derthick, M., Christel, M. G., Hauptmann, A. G., and Wactlar, H. D. (2003a). Constant density displays using diversity sampling. In Proceedings of InfoVis'03, pages 137-144, Washington, DC, USA. IEEE Computer Society.
4. Derthick, M., Christel, M. G., Hauptmann, A. G., and Wactlar, H. D. (2003b). Constant density displays using diversity sampling. In Proceedings of the IEEE Information Visualization, pages 137-144.
5. Freund, Y., Iyer, R., Schapire, R. E., and Singer, Y. (2003). An Efficient Boosting Algorithm for Combining Preferences. Journal of Machine Learning Research, 4(Nov):933-969.
6. Geng, X., Liu, T.-Y., Qin, T., and Li, H. (2007). Feature Selection for Ranking. In Kraaij, W., de Vries, A. P., Clarke, C. L. A., Fuhr, N., and Kando, N., editors, Proc. 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007), pages 407-414. ACM Press, New York, USA.
7. Harman, D. and Buckley, C. (2009). Overview of the reliable information access workshop. Information Retrieval, 12(6):615-641.
8. Järvelin, K. and Kekäläinen, J. (2002). Cumulated Gain-Based Evaluation of IR Techniques. ACM Transactions on Information System (TOIS), 20(4):422-446.
9. Keskustalo, H., Järvelin, K., Pirkola, A., and Kekäläinen, J. (2008). Intuition-Supporting Visualization of User's Performance Based on Explicit Negative Higher-Order Relevance. In Chua, T.-S., Leong, M.-K., Oard, D. W., and Sebastiani, F., editors, Proc. 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2008), pages 675-682. ACM Press, New York, USA.
10. Korfhage, R. R. (1997). Information Storage and Retrieval. Wiley Computer Publishing, John Wiley & Sons, Inc., USA.
11. Liu, T.-Y. (2009). Learning to Rank for Information Retrieval. Foundations and Trends in Information Retrieval, 3(3):225-331.
12. Liu, T.-Y. Y., Xu, J., Qin, T., Xiong, W., and Li, H. (2007). LETOR: Benchmark Dataset for Research on Learning to Rank for Information Retrieval. In Joachims, T., Li, H., Liu, T.-Y., and Zhai, C., editors, SIGIR 2007 Workshop on Learning to Rank for Information Retrieval.
13. Seo, J. and Shneiderman, B. (2004). A rank-by-feature framework for interactive exploration of multidimensional data. In Proceedings of the IEEE Information Visualization, pages 65-72.
14. Seo, J. and Shneiderman, B. (2005). A rank-by-feature framework for interactive exploration of multidimensional data. Information Visualization, 4:96-113.
15. Sormunen, E., Hokkanen, S., Kangaslampi, P., Pyy, P., and Sepponen, B. (2002). Query Performance Analyser - a Web-based tool for IR research and instruction. In Järvelin, K., Beaulieu, M., Baeza-Yates, R., and Hyon Myaeng, S., editors, Proc. 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002), page 450. ACM Press, New York, USA.
16. Teevan, J., Dumais, S. T., and Horvitz, E. (2010). Potential for personalization. ACM Transactions on Computer-Human Interaction (TOCHI), 17(1):1-31.
17. van Rijsbergen, C. J. (1979). Information Retrieval. Butterworths, London, England, 2nd edition.
18. Voorhees, E. and Harman, D. (1999). Overview of the Seventh Text REtrieval Conference (TREC-7). In NIST Special Publication 500-242: The Seventh Text REtrieval Conference (TREC 7). Springer-Verlag, Heidelberg, Germany.