ViewSer: Enabling large-scale remote user studies of web search examination and interaction

SIGIR'11 - Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2011, Pages 365-374

  Lagun, Dmitry ^a   Agichtein, Eugene ^a  

^a EMORY UNIVERSITY   (United States)

Author keywords

Remote user studies; Web search behavior; Web search evaluation

Indexed keywords

EYE MOVEMENTS; INFORMATION RETRIEVAL; SEARCH ENGINES; WEBSITES;

EYE-TRACKING STUDIES; NOVEL METHODOLOGY; REMOTE USERS; SEARCH BEHAVIOR; SEARCH ENGINE RESULTS; SEARCH RESULT QUALITIES; WEB SEARCH BEHAVIOR; WEB SEARCHES;

EYE TRACKING;

EID: 80052129511     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2009916.2009967     Document Type: Conference Paper

References (26)

1. E. Agichtein, E. Brill, and S. Dumais. Improving web search ranking by incorporating user behavior information. In Proceedings of the annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 19-26. SIGIR, 2006.
2. O. Alonso, D. Rose, and B. Stewart. Crowdsourcing for relevance evaluation. In ACM SIGIR Forum, volume 42, pages 9-15. SIGIR, 2008.
3. R. Bednarik and M. Tukiainen. Validating the restricted focus viewer: A study using eye-movement tracking. Behavior research methods, 39(2):274-282, 2007.
4. A. Blackwell, A. Jansen, and K. Marriott. Restricted focus viewer: a tool for tracking visual attention. Theory and Application of Diagrams, pages 575-588, 2000.
5. O. Chapelle and Y. Zhang. A dynamic bayesian network click model for web search ranking. In Proceedings of the International Conference on World Wide Web, pages 1-10. WWW, 2009.
6. E. Chi, P. Pirolli, and S. Lam. Aspects of augmented social cognition: Social information foraging and social search. In Proceedings of the International Conference on Online Communities and Social Computing, pages 60-69. Springer-Verlag, 2007.
7. C. Clarke, E. Agichtein, S. Dumais, and R. White. The influence of caption features on clickthrough patterns in web search. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 135-142. SIGIR, 2007. (Pubitemid 350164954)
8. E. Cutrell and Z. Guan. What are you looking for?: an eye-tracking study of information usage in web search. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pages 407-416. SIGCHI, 2007. (Pubitemid 47581626)
9. J. Friedman, T. Hastie, and R. Tibshirani. Special invited paper. additive logistic regression: A statistical view of boosting. The annals of statistics, 28(2):337-374, 2000.
10. Q. Guo and E. Agichtein. Towards predicting web searcher gaze position from mouse movements. In Proceeding of the annual ACM SIGCHI Conference Extended Abstracts on Human factors in Computing Systems, pages 3601-3606. SIGCHI, 2010.
11. J. Heer and M. Bostock. Crowdsourcing graphical perception: using mechanical turk to assess visualization design. In Proceedings of the ACM SIGCHI International Conference on Human Factors in Computing Systems, pages 203-212. SIGCHI, 2010.
12. J. Huang, R. W. White, and S. Dumais. No clicks, no problem: using cursor movements to understand and improve search. In Proceeding of the annual ACM SIGCHI Conference on Human factors in Computing Systems. SIGCHI, 2011.
13. A. Jansen, A. Blackwell, and K. Marriott. A tool for tracking visual attention: The restricted focus viewer. Behavior Research Methods, 35(1):57-69, 2003. (Pubitemid 37545049)
14. K. Jarvelin and J. Kekalainen. Cumulated gain-based evaluation of information retrieval techniques. ACM Transactions on Information Systems (TOIS), 20(4):422-446, 2002. (Pubitemid 44642296)
15. T. Joachims. Optimizing search engines using clickthrough data. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 133-142. SIGKDD, 2002.
16. T. Joachims. Training linear svms in linear time. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 217-226. SIGKDD, 2006. (Pubitemid 44535518)
17. T. Kanungo and D. Orr. Predicting the readability of short web summaries. In Proceedings of the Second ACM WSDM International Conference on Web Search and Data Mining, pages 202-211. WSDM, 2009.
18. G. Kazai, N. Milic-Frayling, and J. Costello. Towards methods for the collective gathering and quality control of relevance assessments. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 452-459. SIGIR, 2009.
19. D. Kelly and K. Gyllstrom. An examination of two delivery modes for interactive search system experiments: remote and laboratory. In Proceeding of the annual ACM SIGCHI Conference on Human factors in Computing Systems, pages 1531-1540. SIGCHI, 2011.
20. A. Kittur, E. Chi, and B. Suh. Crowdsourcing user studies with mechanical turk. In Proceeding of the annual ACM SIGCHI Conference on Human factors in Computing Systems, pages 453-456. CHI, 2008.
21. J. Le, A. Edmonds, V. Hester, and L. Biewald. Ensuring quality in crowdsourced search relevance evaluation. In Proceedings of the ACM SIGIR Workshop on Crowdsourcing for Search Evaluation (CSE 2010), pages 17-20. SIGIR, 2010.
22. L. Lorigo, M. Haridasan, H. Brynjarsdóttir, L. Xia, T. Joachims, G. Gay, L. Granka, F. Pellacini, and B. Pan. Eye tracking and online search: Lessons learned and challenges ahead. Journal of the American Society for Information Science and Technology, 59(7):1041-1052, 2008. (Pubitemid 351731797)
23. K. Rayner. Eye movements in reading and information processing: 20 years of research. Psychological bulletin, 124(3):372, 1998.
24. K. Wang, N. Gloy, and X. Li. Inferring search behaviors using partially observable markov (pom) model. In Proceedings of the ACM WSDM International Conference on Web Search and Data Mining, pages 211-220. WSDM, 2010.
25. Y. Yue, R. Patel, and H. Roehrig. Beyond position bias: examining result attractiveness as a source of presentation bias in clickthrough data. In Proceedings of the International Conference on World Wide Web, pages 1011-1018. WWW, 2010.
26. Z. Zhu, W. Chen, T. Minka, C. Zhu, and Z. Chen. A novel click model and its applications to online advertising. In Proceedings of the ACM WSDM International Conference on Web Search and Data Mining, pages 321-330. WSDM, 2010.