Understanding intrinsic diversity in web search: Improving whole-session relevance

ACM Transactions on Information Systems

Volumn 32, Issue 4, 2014, Pages 1-45

  Raman, Karthik ^a   Bennett, Paul N ^b   Collins Thompson, Kevyn ^c  

^a School of Operations Research and Information Engineering   (United States)

^b MICROSOFT RESEARCH   (United States)

^c UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Diversity; Proactive search; Search behavior; Search session analysis

Indexed keywords

SEARCH ENGINES; SIGNAL ANALYSIS; WEBSITES;

DIVERSITY; INTRINSIC DIVERSITY; MULTIPLE INTERACTIONS; MULTIPLE QUERIES; MULTIPLE USER; PROACTIVE SEARCH; SEARCH BEHAVIOR; SEARCH SESSIONS;

INFORMATION RETRIEVAL;

EID: 84909953299     PISSN: 10468188     EISSN: 15582868     Source Type: Journal    
DOI: 10.1145/2629553     Document Type: Article

References (77)

1. Eugene Agichtein, Ryen W. White, Susan T. Dumais, and Paul N. Bennett. 2012. Search, interrupted: Understanding and predicting search task continuation. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'12). ACM, New York, NY, 315-324. DOI: http://dx.doi.org/10.1145/2348283.2348328
2. Rakesh Agrawal, Sreenivas Gollapudi, Alan Halverson, and Samuel Ieong. 2009. Diversifying search results. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining (WSDM'09). ACM, New York, NY, 5-14. DOI: http://dx.doi.org/10.1145/1498759.1498766
3. Peter Bailey, Liwei Chen, Scott Grosenick, Li Jiang, Yan Li, Paul Reinholdtsen, Charles Salada, Haidong Wang, and Sandy Wong. 2012. User task understanding: A web search engine perspective. http://research.microsoft.com/apps/-pubs/default.aspx?id=180594.
4. Peter Bailey, Nick Craswell, Ryen W. White, Liwei Chen, Ashwin Satyanarayana, and S. M. M. Tahaghoghi. 2010. Evaluating whole-page relevance. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'10). ACM, New York, NY, 767-768. DOI: http://dx.doi.org/10.1145/1835449.1835606
5. Peter L. Bartlett, Michael I. Jordan, and Jon M. Mcauliffe. 2004. Large margin classifiers: Convex loss, low noise, and convergence rates. In Advances in Neural Information Processing Systems 16. MIT Press, 1173-1180. DOI: http://papers.nips.cc/paper/2416-large-margin-classifiers-convex-loss-low-noise-and-convergence-rates.pdf.
6. Paul N. Bennett, Krysta Svore, and Susan T. Dumais. 2010. Classification-enhanced ranking. In Proceedings of the 19th International Conference on World Wide Web (WWW'10). ACM, New York, NY, 111-120. DOI: http://dx.doi.org/10.1145/1772690.1772703
7. Christina Brandt, Thorsten Joachims, Yisong Yue, and Jacob Bank. 2011. Dynamic ranked retrieval. In Proceedings of the 4th ACM International Conference on Web Search and Data Mining (WSDM'11). ACM, New York, NY, 247-256. DOI: http://dx.doi.org/10.1145/1935826.1935872
8. Gruia Cəlinescu, Chandra Chekuri, Martin Pál, and Jan Vondrák. 2011. Maximizing amonotone submodular function subject to a matroid constraint. SIAM J. Comput. 40, 6 (2011), 1740-1766.
9. Gabriele Capannini, Franco Maria Nardini, Raffaele Perego, and Fabrizio Silvestri. 2011. Efficient diversification of web search results. Proc. VLDB Endow. 4, 7 (2011), 451-459. http://dl.acm.org/citation.cfm?id=1988776.1988781.
10. Jaime Carbonell and Jade Goldstein. 1998. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'98). ACM, New York, NY, 335-336. DOI: http://dx.doi.org/10.1145/290941.291025
11. Harr Chen and David R. Karger. 2006. Less is more: Probabilistic models for retrieving fewer relevant documents. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'06). ACM, New York, NY, 429-436. DOI: http://dx.doi.org/10.1145/1148170.1148245
12. Charles L. A. Clarke, Maheedhar Kolla, Gordon V. Cormack, Olga Vechtomova, Azin Ashkan, Stefan Büttcher, and Ian MacKinnon. 2008. Novelty and diversity in information retrieval evaluation. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'08). ACM, New York, NY, 659-666. DOI: http://dx.doi.org/10.1145/1390334.1390446
13. Charles L. Clarke, Maheedhar Kolla, and Olga Vechtomova. 2009. An effectiveness measure for ambiguous and underspecified queries. In Proceedings of the 2nd International Conference on Theory of Information Retrieval: Advances in Information Retrieval Theory (ICTIR'09). Springer-Verlag, Berlin, Heidelberg, 188-199. DOI: http://dx.doi.org/10.1007/978-3-642-04417-5-17
14. Wisam Dakka, Rishabh Dayal, and Panagiotis G. Ipeirotis. 2006. Automatic discovery of useful facet terms. In Proceedings of the SIGIR Workshop on Faceted Search.
15. Van Dang, Michael Bendersky, and W. Bruce Croft. 2010. Learning to rank query reformulations. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'10). ACM, New York, NY, 807-808. DOI: http://dx.doi.org/10.1145/1835449.1835626
16. Van Dang and Bruce W. Croft. 2013. Term level search result diversification. In Proceedings of the 36th International ACMSIGIR Conference on Research and Development in Information Retrieval (SIGIR'13). ACM, New York, NY, 603-612. DOI: http://dx.doi.org/10.1145/2484028.2484095
17. Van Dang and W. Bruce Croft. 2012. Diversity by proportionality: An election-based approach to search result diversification. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'12). ACM, New York, NY, 65-74. DOI: http://dx.doi.org/10.1145/2348283.2348296
18. Fabio De Bona, Stefan Riezler, Keith Hall, Massimiliano Ciaramita, Amac¸ Herdaʇdelen, and Maria Holmqvist. 2010. Learning dense models of query similarity from user click logs. In Proceedings of the NAACL Conference on Human Language Technologies (HLT'10). Association for Computational Linguistics, Stroudsburg, PA, 474-482. http://dl.acm.org/citation.cfm?id=1857999.1858070
19. Georges Dupret, Ricardo Zilleruelo-Ramos, and Sumio Fujita. 2010. Using related queries to improve Web search results ranking. In Proceedings of the 17th International Symposium on String Processing and Information Retrieval (SPIRE). Lecture Notes in Computer Science, Vol. 6393, Springer, Berlin, 213-224.
20. Henry Feild and James Allan. 2013. Task-aware query recommendation. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'13). ACM, New York, NY, 83-92. DOI: http://dx.doi.org/10.1145/2484028.2484069
21. M. L. Fisher, G. L. Nemhauser, and L. A. Wolsey. 1978. An analysis of approximations for maximizing submodular set functionsII. In Polyhedral Combinatorics, M. L. Balinski and A. J. Hoffman (Eds.), Mathematical Programming Studies, Vol. 8, Springer, Berlin Heidelberg, 73-87. DOI: http://dx.doi.org/10.1007/BFb0121195
22. Steve Fox, Kuldeep Karnawat, Mark Mydland, Susan Dumais, and Thomas White. 2005. Evaluating implicit measures to improve web search. ACM Trans. Inf. Syst. 23, 2 (2005), 147-168. DOI: http://dx.doi.org/10.1145/1059981.1059982
23. Sumio Fujita, Georges Dupret, and Ricardo A. Baeza-Yates. 2012. Learning to rank query recommendations by semantic similarities. CoRR abs/1204.2712 (2012).
24. Jianfeng Gao, Wei Yuan, Xiao Li, Kefeng Deng, and Jian-Yun Nie. 2009. Smoothing clickthrough data for web search ranking. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'09). ACM, New York, NY, 355-362. DOI: http://dx.doi.org/10.1145/1571941.1572003
25. Sreenivas Gollapudi, Samuel Ieong, Alexandros Ntoulas, and Stelios Paparizos. 2011. Efficient query rewrite for structured web queries. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management (CIKM'11). ACM, New York,NY, 2417-2420. DOI: http://dx.doi.org/10.1145/2063576.2063981
26. Dongyi Guan, Sicong Zhang, and Hui Yang. 2013. Utilizing query change for session search. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'13). ACM, New York, NY, 453-462. DOI: http://dx.doi.org/10.1145/2484028.2484055
27. Jiafeng Guo, Xueqi Cheng, Gu Xu, and Xiaofei Zhu. 2011. Intent-aware query similarity. In Proceedings of the ACM Conference on Information and Knowledge Management (CIKM'11). ACM, New York, NY, 259-268. DOI: http://dx.doi.org/10.1145/2063576.2063619
28. Ahmed Hassan, Yang Song, and Li-Wei He. 2011. A task level metric for measuring web search satisfaction and its application on improving relevance estimation. In Proceedings of the 20th ACM International Conference on Information and KnowledgeManagement (CIKM'11). ACM, New York, NY, 125-134. DOI: http://dx.doi.org/10.1145/2063576.2063599
29. Ahmed Hassan and Ryen W. White. 2012. Task tours: Helping users tackle complex search tasks. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management (CIKM'12). ACM, New York, NY, 1885-1889. DOI: http://dx.doi.org/10.1145/2396761.2398537
30. Jiyin He, Marc Bron, and Arjen P. de Vries. 2013. Characterizing stages of a multi-session complex search task through direct and indirect query modifications. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'13). ACM, New York, NY, 897-900. DOI: http://dx.doi.org/10.1145/2484028.2484178
31. Jiyin He, Vera Hollink, Corrado Boscarino, Arjen P. de Vries, and Roberto Cornacchia. 2011. CWI at TREC 2011: Session, Web, and Medical. In Proceedings of the 20th Text Retrieval Conference (TREC'11).
32. Kalervo Järvelin, Susan L. Price, Lois M. L. Delcambre, and Marianne Lykke Nielsen. 2008. Discounted cumulated gain based evaluation of multiple-query IR sessions. In Proceedings of the 30th European Conference on Advances in Information Retrieval (ECIR'08). Lecture Notes in Computer Science, Vol. 4956, Springer-Verlag, Berlin, Heidelberg, 4-15. http://dl.acm.org/citation.cfm?id=1793274.1793280
33. Thorsten Joachims. 1999. Making large-scale support vector machine learning practical. In Advances in Kernel Methods, Bernhard Schölkopf, Christopher J. C. Burges, and Alexander J. Smola (Eds.), MIT Press, 169-184. http://dl.acm.org/citation.cfm?id=299094.299104.
34. Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, and Geri Gay. 2007. Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search. ACM Trans. Inf. Syst. 25, 2, Article 7 (2007). DOI: http://dx.doi.org/10.1145/1229179.1229181
35. Rosie Jones and Kristina Lisa Klinkner. 2008. Beyond the session timeout: Automatic hierarchical segmentation of search topics in query logs. In Proceedings of the 17th ACM Conference on Information and Knowledge Management (CIKM'08). ACM, New York, NY, 699-708. DOI: http://dx.doi.org/10.1145/1458082.1458176
36. Evangelos Kanoulas, Ben Carterette, Paul D. Clough, and Mark Sanderson. 2010. Overview of the TREC 2010 session track. In Proceedings of the 19th Text Retrieval Conference (TREC).
37. Evangelos Kanoulas, Ben Carterette, Paul D. Clough, and Mark Sanderson. 2011a. Evaluating multiquery sessions. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'11). ACM, New York, NY, 1053-1062. DOI: http://dx.doi.org/10.1145/2009916.2010056
38. Evangelos Kanoulas, Mark M. Hall, Paul Clough, Ben Carterette, and Mark Sanderson. 2011b. Overview of the TREC 2011 session track. In Proceedings of the 20th Text Retrieval Conference (TREC).
39. Christian Kohlschutter, Paul-Alexandru Chirita, and Wolfgang Nejdl. 2006. Using link analysis to identify aspects in faceted web search. In Proceedings of the International ACM SIGIR Conference on Research and Development Information Retrieval (SIGIR'06).
40. Weize Kong and James Allan. 2013. Extracting query facets from search results. In Proceedings of the 36th International ACMSIGIR Conference on Research and Development in Information Retrieval (SIGIR'13). ACM, New York, NY, 93-102. DOI: http://dx.doi.org/10.1145/2484028.2484097
41. Alexander Kotov, Paul N. Bennett, Ryen W. White, Susan T. Dumais, and Jaime Teevan. 2011. Modeling and analysis of cross-session search tasks. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'11). ACM, New York, NY, 5-14. DOI: http://dx.doi.org/10.1145/2009916.2009922
42. Zhen Liao, Yang Song, Li-Wei He, and Yalou Huang. 2012. Evaluating the effectiveness of search task trails. In Proceedings of the 21st International Conference on World Wide Web (WWW'12). ACM, New York, NY, 489-498. DOI: http://dx.doi.org/10.1145/2187836.2187903
43. Daniel J. Liebling, Paul N. Bennett, and Ryen W. White. 2012. Anticipatory search: Using context to initiate search. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'12). ACM, New York, NY, 1035-1036. DOI: http://dx.doi.org/10.1145/2348283.2348456
44. Chang Liu, Si Sun, Michael J. Cole, and Nicholas J. Belkin. 2011. Rutgers at the TREC 2011 session track. In Proceedings of the 20th Text Retrieval Conference (TREC).
45. Jingjing Liu and Nicholas J. Belkin. 2010. Personalizing information retrieval for multi-session tasks: The roles of task stage and task type. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'10). ACM, New York, NY, 26-33. DOI: http://dx.doi.org/10.1145/1835449.1835457
46. Jingjing Liu, Michael J. Cole, Chang Liu, Ralf Bierig, Jacek Gwizdka, Nicholas J. Belkin, Jun Zhang, and Xiangmin Zhang. 2010. Search behaviors in different task types. In Proceedings of the 10th Annual Joint Conference on Digital Libraries (JCDL'10). ACM, New York, NY, 69-78. DOI: http://dx.doi.org/10.1145/1816123.1816134
47. Claudio Lucchese, Salvatore Orlando, Raffaele Perego, Fabrizio Silvestri, and Gabriele Tolomei. 2011. Identifying task-based sessions in search engine query logs. In Proceedings of the 4th ACM International Conference on Web Search and Data Mining (WSDM'11). ACM, New York, NY, 277-286. DOI: http://dx.doi.org/10.1145/1935826.1935875
48. Hao Ma, Michael R. Lyu, and Irwin King. 2010. Diversifying query suggestion results. In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI'10). AAAI Press.
49. Gary Marchionini. 2006. Exploratory search: From finding to understanding. Commun. ACM 49, 4 (April 2006), 41-46. DOI: http://dx.doi.org/10.1145/1121949.1121979
50. Dan Morris, Meredith Ringel Morris, and Gina Venolia. 2008. SearchBar: A search-centric web history for task resumption and information re-finding. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'08). ACM, New York, NY, 1207-1216. DOI: http://dx.doi.org/10.1145/1357054.1357242
51. Jeffrey Pound, Stelios Paparizos, and Panayiotis Tsaparas. 2011. Facet discovery for structured web search: A query-log mining approach. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'11). ACM, New York, NY, 169-180. DOI: http://dx.doi.org/10.1145/1989323.1989342
52. Pernilla Qvarfordt, Gene Golovchinsky, Tony Dunnigan, and Elena Agapie. 2013. Looking ahead: Query preview in exploratory search. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'13). ACM, New York, NY, 243-252. DOI: http://dx.doi.org/10.1145/2484028.2484084
53. Filip Radlinski, Paul N. Bennett, Ben Carterette, and Thorsten Joachims. 2009. Redundancy, diversity and interdependent document relevance. SIGIR Forum 43, 2 (2009), 46-52. DOI: http://dx.doi.org/10.1145/1670564.1670572
54. Filip Radlinski and Susan Dumais. 2006. Improving personalized web search using result diversification. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'06). ACM, New York, NY, 691-692. DOI: http://dx.doi.org/10.1145/1148170.1148320
55. Filip Radlinski and Thorsten Joachims. 2005. Query chains: Learning to rank from implicit feedback. In Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining (KDD'05). ACM, New York, NY, 239-248. DOI: http://dx.doi.org/10.1145/1081870.1081899
56. Karthik Raman, Paul N. Bennett, and Kevyn Collins-Thompson. 2013. Toward whole-session relevance: Exploring intrinsic diversity in web search. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'13). ACM, New York, NY, 463-472. DOI: http://dx.doi.org/10.1145/2484028.2484089
57. Karthik Raman and Thorsten Joachims. 2013. Learning socially optimal information systems from egoistic users. In Machine Learning and Knowledge Discovery in Databases, Hendrik Blockeel, Kristian Kersting, Siegfried Nijssen, and Filip elezn (Eds.), Lecture Notes in Computer Science, Vol. 8189, Springer, Berlin Heidelberg, 128-144. DOI: http://dx.doi.org/10.1007/978-3-642-40991-2-9
58. Karthik Raman, Thorsten Joachims, and Pannaga Shivaswamy. 2011. Structured learning of two-level dynamic rankings. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management (CIKM'11). ACM, New York, NY, 291-296. DOI: http://dx.doi.org/10.1145/2063576.2063623
59. Karthik Raman, Pannaga Shivaswamy, and Thorsten Joachims. 2012. Online learning to diversify from implicit feedback. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'12). ACM, New York, NY, 705-713. DOI: http://dx.doi.org/10.1145/2339530.2339642
60. Eldar Sadikov, Jayant Madhavan, Lu Wang, and Alon Halevy. 2010. Clustering query refinements by user intent. In Proceedings of the 19th International Conference on World Wide Web (WWW'10). ACM, New York, NY, 10. DOI: http://dx.doi.org/10.1145/1772690.1772776
61. Tetsuya Sakai and Zhicheng Dou. 2013. Summaries, ranked retrieval and sessions: A unified framework for information access evaluation. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'13). ACM, New York, NY, 473-482. DOI: http://dx.doi.org/10.1145/2484028.2484031
62. Rodrygo L. T. Santos, Craig Macdonald, and Iadh Ounis. 2010a. Exploiting query reformulations for web search result diversification. In Proceedings of the World Wide Web Conference (WWW'10). ACM, New York, NY, 881-890. DOI: http://dx.doi.org/10.1145/1772690.1772780
63. Rodrygo L. T. Santos, Craig Macdonald, and Iadh Ounis. 2010b. Selectively diversifying web search results. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management (CIKM'10). ACM, New York, NY, 1179-1188. DOI: http://dx.doi.org/10.1145/1871437.1871586
64. Adish Singla, Ryen White, and Jeff Huang. 2010. Studying trailfinding algorithms for enhanced web search. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'10). ACM, New York, NY, 443-450. DOI: http://dx.doi.org/10.1145/1835449.1835524
65. Aleksandrs Slivkins, Filip Radlinski, and Sreenivas Gollapudi. 2010. Learning optimally diverse rankings over large document collections. In Proceedings of the 27th International Conference on Machine Learning (ICML'10). http://www.icml2010.org/papers/446.pdf.
66. Mark D. Smucker and Charles L. A. Clarke. 2012. Time-based calibration of effectiveness measures. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'12). ACM, New York, NY, 95-104. DOI: http://dx.doi.org/10.1145/2348283.2348300
67. Yang Song, Dengyong Zhou, and Li-Wei He. 2011. Post-ranking query suggestion by diversifying search results. In Proceedings of the ACM Conference on Research and Development in Information Retrieval (SIGIR'11). ACM, New York, NY, 815-824. DOI: http://dx.doi.org/10.1145/2009916.2010025
68. Ashwin Swaminathan, Cherian V. Mathew, and Darko Kirovski. 2009. Essential pages. In Proceedings of the IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT'09), Vol. 1, IEEE Computer Society, 173-182. DOI: http://dx.doi.org/10.1109/WI-IAT.2009.33
69. Ryen W. White, Paul N. Bennett, and Susan T. Dumais. 2010. Predicting short-term interests using activitybased search context. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management (CIKM'10). ACM, New York, NY, 1009-1018. DOI: http://dx.doi.org/10.1145/1871437.1871565
70. Ryen W. White and Steven M. Drucker. 2007. Investigating behavioral variability in web search. In Proceedings of the 16th International Conference on World Wide Web (WWW'07). ACM, New York, NY, 21-30. DOI: http://dx.doi.org/10.1145/1242572.1242576
71. Ryen W. White, Bill Kules, Steven M. Drucker, and M. C. Schraefel. 2006. Supporting exploratory search, introduction. Commun. ACM 49, 4 (April 2006), 36-39. DOI: http://eprints.soton.ac.uk/263649/.
72. Ryen W. White, Gary Marchionini, and Gheorghe Muresan. 2008. Editorial: Evaluating exploratory search systems. Inf. Process. Manag. 44, 2 (2008), 433-436. DOI: http://dx.doi.org/10.1016/j.ipm.2007.09.011
73. Xiaojun Yuan and Ryen White. 2012. Building the trail best traveled: Effects of domain knowledge on web search trailblazing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'12). ACM, New York, NY, USA, 1795-1804. DOI: http://dx.doi.org/10.1145/2207676.2208312
74. Yisong Yue and Thorsten Joachims. 2008. Predicting diverse subsets using structural SVMs. In Proceedings of the 25th International Conference on Machine Learning (ICML'08). ACM, New York, NY, 8. DOI: http://dx.doi.org/10.1145/1390156.1390310
75. Cheng Xiang Zhai, William W. Cohen, and John Lafferty. 2003. Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'03). ACM, New York, NY, 10-17. DOI: http://dx.doi.org/10.1145/860435.860440
76. Lanbo Zhang and Yi Zhang. 2010. Interactive retrieval based on faceted feedback. In Proceedings of the 33rd International ACMSIGIR Conference on Research and Development in Information Retrieval (SIGIR'10). ACM, New York, NY, 363-370. DOI: http://dx.doi.org/10.1145/1835449.1835511
77. Qiankun Zhao, Steven C. H. Hoi, Tie-Yan Liu, Sourav S. Bhowmick, Michael R. Lyu, and Wei-Ying Ma. 2006. Time-dependent semantic similarity measure of queries using historical click-through data. In Proceedings of the 15th International Conference on World Wide Web (WWW'06). ACM, New York, NY, 543-552. DOI: http://dx.doi.org/10.1145/1135777.1135858