Introduction: Search result diversification

Foundations and Trends in Information Retrieval

Volumn 9, Issue 1, 2015, Pages 1-90

  Santos, Rodrygo L T ^a   MacDonald, Craig ^b   Ounis, Iadh ^b  

^a FEDERAL UNIVERSITY OF MINAS GERAIS   (Brazil)

^b UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SYSTEMS;

COMPLEX INFORMATION; FORMAL DEFINITION; MULTIPLE SEARCH;

COMPUTER SCIENCE;

EID: 84925010903     PISSN: 15540669     EISSN: 15540677     Source Type: Journal    
DOI: 10.1561/1500000040     Document Type: Article

References (214)

1. R. Agrawal, S. Gollapudi, A. Halverson, and S. Ieong. Diversifying search results. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining, pages 5-14, Barcelona, Spain, 2009. ACM.
2. G. Amati. Frequentist and Bayesian approach to information retrieval. In Proceedings of the 28th European Conference on IR Research on Advances in Information Retrieval, pages 13-24, London, UK, 2006. Springer.
3. G. Amati. Probability models for information retrieval based on Divergence From Randomness. PhD thesis, University of Glasgow, 2003.
4. A. Arasu, J. Cho, H. Garcia-Molina, A. Paepcke, and S. Raghavan. Searching the Web. ACM Transactions on Internet Technology, 1(1):2-43, 2001. ISSN 1533-5399.
5. A. Ashkan and C. L. A. Clarke. On the informativeness of cascade and intent-aware effectiveness measures. In Proceedings of the 20th Interna- Tional Conference on World Wide Web, pages 407-416, Hyderabad, India, 2011. ACM.
6. J. A. Aslam, E. Yilmaz, and V. Pavlu. The maximum entropy method for analyzing retrieval measures. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 27-34, Salvador, Brazil, 2005. ACM.
7. R. Baeza-Yates, C. Hurtado, and M. Mendoza. Query recommendation using query logs in search engines. In Proceedings of the 9th International Conference on Current Trends in Database Technology, pages 588-596, Heraklion, Greece, 2004. Springer-Verlag.
8. R. A. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval. Pearson Education Ltd., Harlow, UK, 2 edition, 2011.
9. D. Beeferman and A. Berger. Agglomerative clustering of a search engine query log. In Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 407-416, Boston, MA, USA, 2000. ACM.
10. F. Belém, R. L. T. Santos, J. Almeida, and M. A. Gonçalves. Topic diversity in tag recommendation. In Proceedings of the 7th ACM Conference on Recommender Systems, pages 141-148, Hong Kong, China, 2013. ACM.
11. Y. Bernstein and J. Zobel. Redundant documents and search effectiveness. In Proceedings of the 14th ACM International Conference on Information and Knowledge Management, pages 736-743, Bremen, Germany, 2005. ACM.
12. D. Berry and B. Fristedt. Bandit problems: Sequential allocation of experiments. Chapman and Hall, 1985.
13. S. Bhatia, D. Majumdar, and P. Mitra. Query suggestions in the absence of query logs. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 795-804, Beijing, China, 2011. ACM.
14. D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. The Journal of Machine Learning Research, 3:993-1022, 2003. ISSN 1532-4435.
15. V. D. Blondel, J.-L. Guillaume, R. Lambiotte, and E. Lefebvre. Fast unfolding of communities in large networks. Journal of Statistical Mechanics, 2008 (10):P10008+, 2008. ISSN 1742-5468.
16. P. Boldi, F. Bonchi, C. Castillo, D. Donato, A. Gionis, and S. Vigna. The query-flow graph: Model and applications. In Proceedings of the 17th ACM Conference on Information and Knowledge Management, pages 609-618, Napa Valley, CA, USA, 2008. ACM.
17. P. Boldi, F. Bonchi, C. Castillo, D. Donato, and S. Vigna. Query suggestions using query-flow graphs. In Proceedings of the 2009 Workshop on Web Search Click Data, pages 56-63. ACM, 2009a.
18. P. Boldi, F. Bonchi, C. Castillo, and S. Vigna. From "Dango" to "Japanese cakes": Query reformulation models and patterns. In Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, pages 183-190, Milan, Italy, 2009b. IEEE Computer Society.
19. P. Borlund. The concept of relevance in IR. Journal of the American Society for Information Science and Technology, 54(10):913-925, 2003. ISSN 1532- 2882.
20. A. Bouchoucha, J. He, and J.-Y. Nie. Diversified query expansion using Concept Net. In Proceedings of the 22nd ACM International Conference on Information and Knowledge Management, pages 1861-1864, San Francisco, CA, USA, 2013. ACM.
21. D. Broccolo, L.Marcon, F. M. Nardini, R. Perego, and F. Silvestri. Generating suggestions for queries in the long tail with an inverted index. Information Processing and Management, 48(2):326-339, 2012. ISSN 0306-4573.
22. A. Broder. A taxonomy of web search. SIGIR Forum, 36(2):3-10, 2002. ISSN 0163-5840.
23. C. Buckley. Why current IR engines fail. In Proceedings of the 27th Annual International Conference on Research and Development in Information Retrieval, pages 584-585, Sheffield, UK, 2004. ACM Press.
24. G. Capannini, F. M. Nardini, R. Perego, and F. Silvestri. Efficient diversification of web search results. Proceedings of the VLDB Endowment, 4(7): 451-459, 2011. ISSN 2150-8097.
25. J. Carbonell and J. Goldstein. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 335-336, Melbourne, Australia, 1998. ACM.
26. B. Carterette. Robust test collections for retrieval evaluation. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 55-62, Amsterdam, The Netherlands, 2007. ACM.
27. B. Carterette. An analysis of NP-completeness in novelty and diversity ranking. In Proceedings of the 2nd International Conference on Theory of Information Retrieval, pages 200-211, Cambridge, UK, 2009. Springer-Verlag.
28. B. Carterette and P. Chandar. Probabilistic models of ranking novel documents for faceted topic retrieval. In Proceedings of the 18th ACM Conference on Information and Knowledge Management, pages 1287-1296, Hong Kong, China, 2009. ACM.
29. P. Chandar and B. Carterette. Preference based evaluation measures for novelty and diversity. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 413-422, Dublin, Ireland, 2013. ACM.
30. O. Chapelle, D. Metlzer, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded relevance. In Proceedings of the 18th ACM Conference on Information and Knowledge Management, pages 621-630, Hong Kong, China, 2009. ACM.
31. O. Chapelle, Y. Chang, and T.-Y. Liu. Future directions in learning to rank. Journal of Machine Learning Research, Proceedings Track, pages 91-100, 2011a.
32. O. Chapelle, S. Ji, C. Liao, E. Velipasaoglu, L. Lai, and S.-L.Wu. Intent-based diversification of web search results: Metrics and algorithms. Information Retrieval, 14(6):572-592, 2011b.
33. H. Chen and D. R. Karger. Less is more: Probabilistic models for retrieving fewer relevant documents. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 429-436, Seattle, WA, USA, 2006. ACM.
34. C. L. A. Clarke, M. Kolla, G. V. Cormack, O. Vechtomova, A. Ashkan, S. Büttcher, and I. MacKinnon. Novelty and diversity in information retrieval evaluation. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 659-666, Singapore, Singapore, 2008. ACM.
35. C. L. A. Clarke, N. Craswell, and I. Soboroff. Overview of the TREC 2009Web track. In Proceedings of the 18th Text REtrieval Conference, Gaithersburg, MD, USA, 2009a.
36. C. L. A. Clarke, M. Kolla, and O. Vechtomova. An effectiveness measure for ambiguous and underspecified queries. In Proceedings of the 2nd International Conference on Theory of Information Retrieval, pages 188-199, Cambridge, UK, 2009b. Springer-Verlag.
37. C. L. A. Clarke, N. Craswell, I. Soboroff, and G. V. Cormack. Overview of the TREC 2010 Web track. In Proceedings of the 19th Text REtrieval Conference, Gaithersburg, MD, USA, 2010.
38. C. L. A. Clarke, N. Craswell, I. Soboroff, and A. Ashkan. A comparative analysis of cascade measures for novelty and diversity. In Proceedings of the 4th ACM International Conference on Web Search and Data Mining, pages 75-84, Hong Kong, China, 2011a. ACM.
39. C. L. A. Clarke, N. Craswell, I. Soboroff, and E. M. Voorhees. Overview of the TREC 2011 Web track. In Proceedings of the 20th Text REtrieval Conference, Gaithersburg, MD, USA, 2011b.
40. C. L. A. Clarke, N. Craswell, and E. M. Voorhees. Overview of the TREC 2012 Web track. In Proceedings of the 21st Text REtrieval Conference, Gaithersburg, MD, USA, 2012.
41. C. Cleverdon. The Cranfield tests on index language devices. Aslib Proceedings, 19(6):173-194, 1967.
42. C. W. Cleverdon. The significance of the Cranfield tests on index languages. In Proceedings of the 14th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3-12, Chicago, IL, USA, 1991. ACM.
43. P. Clough, M. Sanderson, M. Abouammoh, S. Navarro, and M. Paramita. Multiple approaches to analysing query diversity. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 734-735, Boston, MA, USA, 2009. ACM.
44. K. Collins-Thompson, P. Bennett, F. Diaz, C. L. A. Clarke, and E. M. Voorhees. TREC 2013 Web track overview. In Proceedings of the 22nd Text REtrieval Conference, Gaithersburg, MD, USA, 2013.
45. W. S. Cooper. The inadequacy of probability of usefulness as a ranking criterion for retrieval system output. Technical report, University of California, Berkeley, Berkeley, CA, USA, 1971.
46. W. S. Cooper. Some inconsistencies and misidentified modeling assumptions in probabilistic information retrieval. ACM Transactions on Information Systems, 13(1):100-111, 1995.
47. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein. Introduction to Algorithms. The MIT Press, 2nd edition, 2001.
48. N. Craswell and M. Szummer. Random walks on the click graph. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 239-246, Amsterdam, The Netherlands, 2007. ACM.
49. N. Craswell, O. Zoeter, M. Taylor, and B. Ramsey. An experimental comparison of click position-bias models. In Proceedings of the 1st International Conference on Web Search and Data Mining, pages 87-94. ACM, 2008.
50. S. Cronen-Townsend and W. B. Croft. Quantifying query ambiguity. In Proceedings of the 2nd International Conference on Human Language Technology Research, pages 104-109, San Diego, CA, USA, 2002. Morgan Kaufmann Publishers Inc.
51. M. Cutts. Spotlight keynote. In Proceedings of Search Engine Strategies, San Francisco, CA, USA, 2012.
52. V. Dang and B. W. Croft. Term level search result diversification. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 603-612, Dublin, Ireland, 2013. ACM.
53. V. Dang and W. B. Croft. Query reformulation using anchor text. In Proceedings of the 3rd ACM International Conference on Web Search and Data Mining, pages 41-50, New York, NY, USA, 2010. ACM.
54. V. Dang and W. B. Croft. Diversity by proportionality: An election-based approach to search result diversification. In Proceedings of the 35th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval, pages 65-74, Portland, OR, USA, 2012. ACM.
55. V. Dang, X. Xue, and W. B. Croft. Inferring query aspects from reformulations using clustering. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management, pages 2117-2120, Glasgow, UK, 2011. ACM.
56. G. Demartini. ARES: A retrieval engine based on sentiments sentiment-based search result annotation and diversification. In Proceedings of the 33rd European Conference on IR Research on Advances in Information Retrieval, pages 772-775, Dublin, Ireland, 2011. Springer-Verlag.
57. E. Demidova, P. Fankhauser, X. Zhou, and W. Nejdl. Divq: Diversification for keyword search over structured databases. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 331-338, Geneva, Switzerland, 2010. ACM.
58. T. Deselaers, T. Gass, P. Dreuw, and H. Ney. Jointly optimising relevance and diversity in image retrieval. In Proceedings of the ACM International Conference on Image and Video Retrieval, pages 1-8, Santorini, Greece, 2009. ACM.
59. F. Diaz, M. Lalmas, and M. Shokouhi. From federated to aggregated search. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, page 910, 2010.
60. Z. Dou, S. Hu, K. Chen, R. Song, and J.-R. Wen. Multi-dimensional search result diversification. In Proceedings of the fourth ACM international Conference on Web Search and Data Mining, pages 475-484, Hong Kong, China, 2011. ACM.
61. D. Downey, S. Dumais, and E. Horvitz. Heads and tails: Studies of web search with common and rare queries. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 847-848, Amsterdam, The Netherlands, 2007. ACM.
62. M. Dundar, B. Krishnapuram, J. Bi, and R. B. Rao. Learning classifiers when the training data is not IID. In Proceedings of the 20th International Joint Conference on Artifical Intelligence, pages 756-761, Hyderabad, India, 2007. Morgan Kaufmann Publishers Inc.
63. N. Eiron and K. S. McCurley. Analysis of anchor text for web search. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 459-460, Toronto, Canada, 2003. ACM.
64. U. Feige. A threshold of ln(n) for approximating set cover. Journal of the ACM, 45:634-652, 1998. ISSN 0004-5411.
65. B. M. Fonseca, P. B. Golgher, E. S. De Moura, B. Pôssas, and N. Ziviani. Discovering search engine related queries using association rules. Journal of Web Engineering, 2(4):215-227, October 2003. ISSN 1540-9589.
66. E. A. Fox and J. A. Shaw. Combination of multiple searches. In Proceedings of the 2nd Text REtrieval Conference, pages 243-252, Gaithersburg, MD, USA, 1993.
67. J. H. Friedman. Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5):1189-1232, 2001.
68. X. Geng, T.-Y. Liu, T. Qin, A. Arnold, H. Li, and H.-Y. Shum. Query dependent ranking using k-nearest neighbor. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 115-122, Singapore, Singapore, 2008. ACM.
69. V. Gil-Costa, R. L. T. Santos, C. Macdonald, and I. Ounis. Sparse spatial selection for novelty-based search result diversification. In Proceedings of the 18th International Symposium on String Processing and Information Retrieval, pages 344-355, Pisa, Italy, 2011. Springer.
70. V. Gil-Costa, R. L. T. Santos, C. Macdonald, and I. Ounis. Modelling efficient novelty-based search result diversification in metric spaces. Journal of Discrete Algorithms, 18:75-88, 2013. ISSN 1570-8667.
71. W. Goffman. On relevance as a measure. Information Storage and Retrieval, 2(3):201-203, 1964.
72. P. B. Golbus, J. A. Aslam, and C. L. Clarke. Increasing evaluation sensitivity to diversity. Information Retrieval, 16(4):530-555, 2013. ISSN 1386-4564.
73. S. Gollapudi and A. Sharma. An axiomatic approach for result diversification. In Proceedings of the 18th International Conference on World Wide Web, pages 381-390, Madrid, Spain, 2009. ACM.
74. M. D. Gordon and P. Lenk. A utility theoretic examination of the probability ranking principle in information retrieval. Journal of the American Society for Information Science and Technology, 42(10):703-714, 1991.
75. M. D. Gordon and P. Lenk. When is the probability ranking principle suboptimal? Journal of the American Society for Information Science and Technology, 43(1):1-14, 1992.
76. D. Harman. Overview of the second Text REtrieval Conference (TREC-2). In Proceedings of the 2nd Text REtrieval Conference, Gaithersburg, MD, USA, 1993.
77. S. P. Harter. A probabilistic approach to automatic keyword indexing. Part I: On the distribution of specialty words in a technical literature. Journal of the American Society for Information Science, 26(4):197-206, 1975a.
78. S. P. Harter. A probabilistic approach to automatic keyword indexing. Part II: An algorithm for pro babilistc indexing. Journal of the American Society for Information Science, 26(4):280-289, 1975b.
79. J. He, E. Meij, and M. de Rijke. Result diversification based on query-specific cluster ranking. Journal of the American Society for Information Science and Technology, 62(3):550-571, 2011. ISSN 1532-2882.
80. J. He, V. Hollink, and A. de Vries. Combining implicit and explicit topic representations for result diversification. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 851-860, Portland, OR, USA, 2012. ACM.
81. W. R. Hersh and P. Over. TREC-8 Interactive track report. In Proceedings of the 8th Text REtrieval Conference, Gaithersburg, MD, USA, 1999.
82. D. Hiemstra. A linguistically motivated probabilistic model of information retrieval. In Proceedings of the 2nd European Conference on Research and Advanced Technology for Digital Libraries, pages 569-584, Heraklion, Greece, 1998. Springer.
83. D. S. Hochbaum, editor. Approximation algorithms for NP-hard problems. PWS Publishing Co., Boston, MA, USA, 1997.
84. D. Jannach, M. Zanker, A. Felfernig, and G. Friedrich. Recommender Systems: An Introduction. Cambridge University Press, New York, NY, USA, 1st edition, 2010.
85. B. J. Jansen, A. Spink, J. Bateman, and T. Saracevic. Real life information retrieval: A study of user queries on the Web. SIGIR Forum, 32(1):5-17, 1998. ISSN 0163-5840.
86. B. J. Jansen, A. Spink, and T. Saracevic. Real life, real users, and real needs: A study and analysis of user queries on the Web. Information Processing and Management, 36(2):207-227, 2000. ISSN 0306-4573.
87. K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4):422-446, 2002. ISSN 1046-8188.
88. R. Jones, B. Rey, O. Madani, and W. Greiner. Generating query substitutions. In Proceedings of the 15th international conference on World Wide Web, pages 387-396, Edinburgh, UK, 2006. ACM.
89. I.-H. Kang and G. Kim. Query type classification for web document retrieval. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 64-71, Toronto, Canada, 2003. ACM.
90. J. G. Kemeny and J. L. Snell. Finite Markov Chains. Springer, 1960.
91. S. Kharazmi, M. Sanderson, F. Scholer, and D. Vallet. Using score differences for search result diversification. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1143-1146. ACM, 2014.
92. E. Kharitonov, C. Macdonald, P. Serdyukov, and I. Ounis. Intent models for contextualising and diversifying query suggestions. In Proceedings of the 22nd ACM International Conference on Conference on information and Knowledge Management, pages 2303-2308, San Francisco, CA, USA, 2013. ACM.
93. S. Khuller, A. Moss, and J. S. Naor. The budgeted maximum coverage problem. Information Processing Letters, 70:39-45, 1999. ISSN 0020-0190.
94. Y. Kim and W. B. Croft. Diversifying query suggestions based on query documents. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 891-894, Gold Coast, QLD, Australia, 2014. ACM.
95. R. Kraft and J. Zien. Mining anchor text for query refinement. In Proceedings of the 13th International Conference on World Wide Web, pages 666-674, New York, NY, USA, 2004. ACM.
96. R. Krestel and N. Dokoohaki. Diversifying product review rankings: Getting the full picture. In Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, pages 138-145, Washington, DC, USA, 2011. IEEE Computer Society.
97. U. Kruschwitz, D. Lungley, M.-D. Albakour, and D. Song. Deriving query suggestions for site search. Journal of the American Society for Information Science and Technology, 64(10):1975-1994, 2013. ISSN 1532-2890.
98. E. Lagergren and P. Over. Comparing interactive information retrieval systems across sites: The TREC-6 Interactive track matrix experiment. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 164-172, Melbourne, Australia, 1998. ACM.
99. N. Lathia, S. Hailes, L. Capra, and X. Amatriain. Temporal diversity in recommender systems. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 210-217, Geneva, Switzerland, 2010. ACM.
100. V. Lavrenko and W. B. Croft. Relevance based language models. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 120-127, New Orleans, LA, USA, 2001. ACM.
101. T. Leelanupab, G. Zuccon, and J. M. Jose. A comprehensive analysis of parameter settings for novelty-biased cumulative gain. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, pages 1950-1954, Maui, HI, USA, 2012. ACM.
102. S. Liang, Z. Ren, and M. de Rijke. Fusion helps diversification. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 303-312, Gold Coast, QLD, Australia, 2014. ACM.
103. N. Limsopatham, C. Macdonald, and I. Ounis. Modelling relevance towards multiple inclusion criteria when ranking patients. In Proceedings of the 23rd ACM International Conference on Information and Knowledge Management, pages 1639-1648, Shanghai, China, 2014. ACM.
104. H. Liu and P. Singh. Concept Net-a practical commonsense reasoning toolkit. BT Technology Journal, 22(4):211-226, 2004. ISSN 1358-3948.
105. H. Ma, M. R. Lyu, and I. King. Diversifying query suggestion results. In Proceedings of the 24th AAAI Conference on Artificial Intelligence, Atlanta, GA, USA, 2010. AAAI Press.
106. J. I. Marden. Analyzing and modeling rank data. Taylor & Francis, 1996.
107. H. Markowitz. Portfolio selection. The Journal of Finance, 7(1):77-91, 1952. ISSN 00221082.
108. M. E. Maron and J. L. Kuhns. On relevance, probabilistic indexing and information retrieval. Journal of the ACM, 7(3):216-244, 1960. ISSN 0004- 5411.
109. Q. Mei, D. Zhou, and K. Church. Query suggestion using hitting time. In Proceedings of the 17th ACM Conference on Information and Knowledge Management, pages 469-478, Napa Valley, CA, USA, 2008. ACM.
110. M. Melucci. Contextual search: A computational framework. Foundations and Trends in Information Retrieval, 6(4-5):257-405, 2012.
111. S. Mizzaro. Relevance: The whole history. Journal of the American Society for Information Science, 48(9):810-832, 1997. ISSN 0002-8231.
112. A. Moffat and J. Zobel. Rank-biased precision for measurement of retrieval effectiveness. ACM Transactions on Information Systems, 27(1):1-27, 2008. ISSN 1046-8188.
113. V. Murdock and M. Lalmas. Workshop on aggregated search. SIGIR Forum, 42:80-83, 2008. ISSN 0163-5840.
114. G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher. An analysis of approximations for maximizing sub modular set functions-I. Mathematical Programming, 14:265-294, 1978. ISSN 0025-5610.
115. T. N. Nguyen and N. Kanhabua. Leveraging dynamic query subtopics for time-aware search result diversification. In Proceedings of the 36th European Conference on Information Retrieval, pages 222-234, Amsterdam, The Netherlands, 2014. Springer.
116. P. Over. TREC-6 Interactive report. In Proceedings of the 6th Text Retrieval Conference, pages 73-81, Gaithersburg, MD, USA, 1997.
117. P. Over. TREC-7 Interactive track report. In Proceedings of the 7th Text Retrieval Conference, pages 33-39, Gaithersburg, MD, USA, 1998.
118. A. M. Ozdemiray and I. S. Altingovde. Score and rank aggregation methods for explicit search result diversification. Technical Report METU-CENG- 2013-01, Middle East Technical University, Ankara, Turkey, 2013.
119. M. L. Paramita, J. Tang, and M. Sanderson. Generic and spatial approaches to image search results diversification. In Proceedings of the 31st European Conference on IR Research on Advances in Information Retrieval, pages 603-610, Toulouse, France, 2009. Springer.
120. J. Peng, C. Macdonald, and I. Ounis. Learning to select a ranking function. In Proceedings of the 31st European Conference on IR Research on Advances in Information Retrieval, pages 114-126, Milton Keynes, UK, 2010. Springer.
121. V. Plachouras. Diversity in expert search. In Proceedings of the 1st International Workshop on Diversity in Document Retrieval, pages 63-67, Dublin, Ireland, 2011.
122. A. Plakhov. Entity-oriented search result diversification. In Proceedings of the 1st International Workshop on Entity-Oriented Search, Beijing, China, 2011.
123. J. M. Ponte and W. B. Croft. A language modeling approach to information retrieval. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 275-281, Melbourne, Australia, 1998. ACM.
124. F. Radlinski and S. Dumais. Improving personalized web search using result diversification. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 691-692, Seattle, WA, USA, 2006. ACM.
125. F. Radlinski, R. Kleinberg, and T. Joachims. Learning diverse rankings with multi-armed bandits. In Proceedings of the 25th International Conference on Machine Learning, pages 784-791, Helsinki, Finland, 2008. ACM.
126. F. Radlinski, P. N. Bennett, B. Carterette, and T. Joachims. Redundancy, diversity and interdependent document relevance. SIGIR Forum, 43(2): 46-52, 2009. ISSN 0163-5840.
127. F. Radlinski, M. Szummer, and N. Craswell. Metrics for assessing sets of subtopics. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 853-854, Geneva, Switzerland, 2010a. ACM.
128. F. Radlinski, M. Szummer, and N. Craswell. Inferring query intent from reformulations and clicks. In Proceedings of the 19th International Conference on World Wide Web, pages 1171-1172, Raleigh, NC, USA, 2010b.
129. D. Rafiei, K. Bharat, and A. Shukla. Diversifying web search results. In Proceedings of the 19th International Conference on World Wide Web, pages 781-790, Raleigh, NC, USA, 2010.
130. K. Raman, P. Shivaswamy, and T. Joachims. Online learning to diversify from implicit feedback. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Beijing, China, 2012. ACM.
131. S. E. Robertson and K. Spärck Jones. Relevance weighting of search terms. Journal of the American Society for Information Science, 27:129-146, 1976.
132. S. E. Robertson, C. J. van Rijsbergen, and M. F. Porter. Probabilistic models of indexing and searching. In Proceedings of the 3rd Annual ACM Conference on Research and Development in Information Retrieval, pages 35-56. Butterworth & Co., 1981.
133. S. Robertson and H. Zaragoza. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333- 389, 2009. ISSN 1554-0669.
134. S. Robertson, H. Zaragoza, and M. Taylor. Simple bm25 extension to multiple weighted fields. In Proceedings of the 13th ACM International Conference on Information and Knowledge Management, pages 42-49, Washington, DC, USA, 2004. ACM.
135. S. E. Robertson. The probability ranking principle in IR. Journal of Documentation, 33(4):294-304, 1977.
136. S. E. Robertson, S. Walker, S. Jones, M. Hancock-Beaulieu, and M. Gatford. Okapi at TREC-3. In Proceedings of the 3rd Text Retrieval Conference, Gaithersburg, MD, USA, 1994.
137. D. E. Rose and D. Levinson. Understanding user goals in web search. In Proceedings of the 13th International Conference on World Wide Web, pages 13-19, New York, NY, USA, 2004. ACM.
138. B. R. Rowe, D. W. Wood, A. N. Link, and D. A. Simoni. Economic impact assessment of NIST's Text Retrieval Conference (TREC) program. Technical Report 0211875, RTI International, 2010.
139. T. Sakai. Evaluating evaluation metrics based on the bootstrap. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '06, pages 525-532, Seattle, WA, USA, 2006. ACM.
140. T. Sakai. Alternatives to bpref. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 71-78, Amsterdam, The Netherlands, 2007. ACM.
141. T. Sakai. Evaluation with informational and navigational intents. In Proceedings of the 21st International Conference on World Wide Web, pages 499-508, Lyon, France, 2012. ACM.
142. T. Sakai. The unreusability of diversified search test collections. In Proceedings of the 5th International Workshop on Evaluating Information Access, pages 1-8, Tokyo, Japan, 2013.
143. T. Sakai and R. Song. Diversified search evaluation: Lessons from the NTCIR- 9 Intent task. Information Retrieval, 2012. ISSN 1386-4564.
144. T. Sakai, N. Craswell, R. Song, S. Robertson, Z. Dou, and C.-Y. Lin. Simple evaluation metrics for diversified search results. In Proceedings of the 3rd International Workshop on Evaluating Information Access, pages 42-50, Tokyo, Japan, 2010. NII.
145. T. Sakai, Z. Dou, and C. L. Clarke. The impact of intent selection on diversified search evaluation. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 921-924, Dublin, Ireland, 2013a. ACM.
146. T. Sakai, Z. Dou, T. Yamamoto, Y. Liu, M. Zhang, and R. Song. Overview of the NTCIR-10 Intent-2 task. In Proceedings of the 10th NTCIR Workshop Meeting on Evaluation of Information Access Technologies, Tokyo, Japan, 2013b.
147. P. A. Samuelson and W. D. Nordhaus. Micro economics. McGraw-Hill, 2001.
148. M. Sanderson. Ambiguous queries: Test collections need more sense. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 499-506, Singapore, Singapore, 2008. ACM.
149. M. Sanderson. Test collection based evaluation of information retrieval systems. Foundations and Trends in Information Retrieval, 4(4):247-375, 2010.
150. M. Sanderson, M. L. Paramita, P. Clough, and E. Kanoulas. Do user preferences and evaluation measures line up? In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 555-562, Geneva, Switzerland, 2010. ACM.
151. R. L. T. Santos. Explicit web search result diversification. PhD thesis, School of Computing Science, University of Glasgow, Glasgow, UK, 2013.
152. R. L. T. Santos and I. Ounis. Diversifying for multiple information needs. In Proceedings of the 1st International Workshop on Diversity in Document Retrieval, pages 37-41, Dublin, Ireland, 2011.
153. R. L. T. Santos, C. Macdonald, and I. Ounis. Selectively diversifying web search results. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management, pages 1179-1188, Toronto, Canada, 2010a. ACM.
154. R. L. T. Santos, C. Macdonald, and I. Ounis. Exploiting query reformulations for web search result diversification. In Proceedings of the 19th International Conference on World Wide Web, pages 881-890, Raleigh, NC, USA, 2010b. ACM.
155. R. L. T. Santos, J. Peng, C. Macdonald, and I. Ounis. Explicit search result diversification through sub-queries. In Proceedings of the 31st European Conference on IR Research on Advances in Information Retrieval, pages 87-99, Milton Keynes, UK, 2010c. Springer.
156. R. L. T. Santos, C. Macdonald, and I. Ounis. Aggregated search result diversification. In Proceedings of the 3rd International Conference on the Theory of Information Retrieval, pages 250-261, Bertinoro, Italy, 2011a. Springer.
157. R. L. T. Santos, C. Macdonald, and I. Ounis. Intent-aware search result diversification. In Proceedings of the 34th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 595-604, Beijing, China, 2011b. ACM.
158. R. L. T. Santos, C. Macdonald, R. McCreadie, I. Ounis, and I. Soboroff. Information retrieval on the blogosphere. Foundations and Trends in Information Retrieval, 6(1):1-125, 2012a.
159. R. L. T. Santos, C. Macdonald, and I. Ounis. On the role of novelty for search result diversification. Information Retrieval, 15(5):478-502, 2012b.
160. R. L. T. Santos, C. Macdonald, and I. Ounis. Learning to rank query suggestions for adhoc and diversity search. Information Retrieval, 16(4):429-451, 2013. ISSN 1386-4564.
161. M. Searcóid. Metric Spaces. Springer Undergraduate Mathematics Series. Springer, 2006.
162. C. Silverstein, H. Marais, M. Henzinger, and M. Moricz. Analysis of a very large web search engine query log. SIGIR Forum, 33(1):6-12, 1999. ISSN 0163-5840.
163. F. Silvestri. Mining query logs: Turning search usage data into knowledge. Foundations and Trends in Information Retrieval, 4(1-2):1-174, 2010.
164. A. Slivkins, F. Radlinski, and S. Gollapudi. Learning optimally diverse rankings over large document collections. In Proceedings of the 27th Annual International Conference on Machine Learning, pages 983-990, Haifa, Israel, 2010. Omnipress.
165. R. Song, Z. Luo, J.-Y. Nie, Y. Yu, and H.-W. Hon. Identification of ambiguous queries in web search. Information Processing and Management, 45(2):216- 229, 2009. ISSN 0306-4573.
166. R. Song, M. Zhang, T. Sakai, M. P. Kato, Y. Liu, M. Sugimoto, Q. Wang, and N. Orii. Overview of the NTCIR-9 Intent task. In Proceedings of the 9th NTCIR Workshop Meeting on Evaluation of Information Access Technologies, Tokyo, Japan, 2011a.
167. Y. Song, D. Zhou, and L. wei He. Post-ranking query suggestion by diversifying search results. In Proceedings of the 34th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 815-824, Beijing, China, 2011b. ACM.
168. K. Spärck-Jones, S. E. Robertson, and M. Sanderson. Ambiguous requests: Implications for retrieval tests, systems and theories. SIGIR Forum, 41(2): 8-17, 2007. ISSN 0163-5840.
169. I. Szpektor, A. Gionis, and Y. Maarek. Improving recommendation for long tail queries via templates. In Proceedings of the 20th international conference on World wide web, pages 47-56, Hyderabad, India, 2011. ACM.
170. J. Teevan, S. T. Dumais, and E. Horvitz. Characterizing the value of personalizing search. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 757-758, Amsterdam, The Netherlands, 2007. ACM.
171. I. Tsochantaridis, T. Joachims, T. Hofmann, and Y. Altun. Large margin methods for structured and interdependent output variables. Journal of Machine Learning Research, 6:1453-1484, 2005. ISSN 1532-4435.
172. H. R. Turtle andW. B. Croft. Uncertainty in information retrieval systems. In Uncertainty Management in Information Systems, pages 189-224. Kluwer Academic Publishers, Norwell, MA, USA, 1996.
173. D. Vallet. Crowd sourced evaluation of personalization and diversification techniques in web search. In Proceedings of the ACM SIGIR Workshop on Crowd sourcing for Information Retrieval, Beijing, China, 2011. ACM.
174. D. Vallet and P. Castells. Personalized diversification of search results. In Proceedings of the 35th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 841-850, Portland, OR, USA, 2012. ACM.
175. R. H. van Leuken, L. Garcia, X. Olivares, and R. van Zwol. Visual diversification of image search results. In Proceedings of the 18th International Conference on World Wide Web, pages 341-350, Madrid, Spain, 2009. ACM.
176. C. J. van Rijsbergen. The Geometry of Information Retrieval. Cambridge University Press, New York, NY, USA, 2004.
177. S. Vargas and P. Castells. Rank and relevance in novelty and diversity metrics for recommender systems. In Proceedings of the 5th ACM Conference on Recommender Systems, pages 109-116, Chicago, IL, USA, 2011. ACM.
178. S. Vargas, P. Castells, and D. Vallet. Intent-oriented diversity in recommender systems. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1211-1212, Beijing, China, 2011. ACM.
179. S. Vargas, R. L. T. Santos, C. Macdonald, and I. Ounis. Selecting effective expansion terms for diversity. In Proceedings of the 10th Conference on Open Research Areas in Information Retrieval, pages 69-76, Lisbon, Portugal, 2013. CID.
180. E. Vee, U. Srivastava, J. Shanmugasundaram, P. Bhat, and S. A. Yahia. Efficient computation of diverse query results. In Proceedings of the 24th International Conference on Data Engineering, pages 228-236, Cancún, Mexico, 2008. IEEE Computer Society.
181. R. V. Vohra and N. G. Hall. A probabilistic analysis of the maximal covering location problem. Discrete Applied Mathematics, 43(2):175-183, 1993. ISSN 0166-218X.
182. J. von Neumann and O. Morgenstern. Theory of Games and Economic Behavior. Princeton University Press, 1944.
183. E. M. Voorhees. TREC: Continuing information retrieval's tradition of experimentation. Communications of the ACM, 50(11):51-54, 2007. ISSN 0001-0782.
184. E. M. Voorhees and D. Harman. Overview of the 6th Text Retrieval Conference. In Proceedings of the 6th Text Retrieval Conference, Gaithersburg, MD, USA, 1997.
185. E. M. Voorhees and D. Harman. Overview of the 7th Text Retrieval Conference. In Proceedings of the 7th Text Retrieval Conference, Gaithersburg, MD, USA, 1998.
186. E. M. Voorhees and D. Harman. Overview of the 8th Text Retrieval Conference. In Proceedings of the 8th Text Retrieval Conference, Gaithersburg, MD, USA, 1999.
187. E. M. Voorhees and D. K. Harman. TREC: Experiment and Evaluation in Information Retrieval. Digital Libraries and Electronic Publishing. MIT Press, 2005.
188. J. Wang and J. Zhu. Portfolio theory of information retrieval. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 115-122, Boston, MA, USA, 2009. ACM.
189. Q.Wang, Y. Qian, R. Song, Z. Dou, F. Zhang, T. Sakai, and Q. Zheng. Mining subtopics from text fragments for a web query. Information Retrieval, 16 (4):484-503, 2013. ISSN 1386-4564.
190. X. Wang and C. Zhai. Mining term association patterns from search logs for effective query reformulation. In Proceedings of the 17th ACM Conference on Information and Knowledge Management, pages 479-488, Napa Valley, CA, USA, 2008. ACM.
191. X. Wang, H. Fang, and C. Zhai. A study of methods for negative relevance feedback. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 219-226, Singapore, Singapore, 2008. ACM.
192. M. J. Welch, J. Cho, and C. Olston. Search result diversity for informational queries. In Proceedings of the 20th International Conference on World Wide Web, pages 237-246, Hyderabad, India, 2011. ACM.
193. G. J. Woeginger. Exact algorithms for NP-hard problems: A survey. In Combinatorial Optimization-Eureka, You Shrink!, pages 185-207. Springer, 2003.
194. M. A. Woodbury. Inverting modified matrices. Technical Report MR38136, Statistical Research Group, Princeton University, Princeton, NJ, USA, 1950.
195. F. Wu, J. Madhavan, and A. Halevy. Identifying aspects for web-search queries. Journal of Artificial Intelligence Research, 40(1):677-700, 2011. ISSN 1076-9757.
196. X. Yin, J. X. Huang, X. Zhou, and Z. Li. A survival modeling approach to biomedical search result diversification using Wikipedia. In Proceedings of the 33rd international ACM SIGIR Conference on Research and Development in Information Retrieval, pages 901-902, Geneva, Switzerland, 2010. ACM.
197. C. Yu, L. Lakshmanan, and S. Amer-Yahia. It takes variety to make a world: Diversification in recommender systems. In Proceedings of the 12th International Conference on Extending Database Technology, pages 368-378, Saint Petersburg, Russia, 2009. ACM.
198. Y. Yue and T. Joachims. Predicting diverse subsets using structural svms. In Proceedings of the 25th International Conference on Machine Learning, pages 1224-1231, Helsinki, Finland, 2008. ACM.
199. H. Zaragoza, N. Craswell, M. J. Taylor, S. Saria, and S. E. Robertson. Microsoft Cambridge at TREC 13: Web and Hard tracks. In Proceedings of the 13th Text Retrieval Conference, Gaithersburg, MD, USA, 2004.
200. C. Zhai. Statistical language models for information retrieval: A critical review. Foundations and Trends in Information Retrieval, 2(3):137-213, 2008. ISSN 1554-0669.
201. C. Zhai and J. Lafferty. A risk minimization framework for information retrieval. Information Processing and Management, 42(1):31-55, 2006. ISSN 0306-4573.
202. C. Zhai, W. W. Cohen, and J. Lafferty. Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 10-17, Toronto, Canada, 2003. ACM.
203. Z. Zhang and O. Nasraoui. Mining search engine query logs for query recommendation. In Proceedings of the 15th international conference on World Wide Web, pages 1039-1040, Edinburgh, UK, 2006. ACM.
204. W. Zheng, H. Fang, C. Yao, and M. Wang. Search result diversification for enterprise data. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management, pages 1901-1904, Glasgow, UK, 2011a. ACM.
205. W. Zheng, X. Wang, H. Fang, and H. Cheng. An exploration of pattern based subtopic modeling for search result diversification. In Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries, pages 387-388, Ottawa, ON, Canada, 2011b. ACM.
206. W. Zheng, H. Fang, and C. Yao. Exploiting concept hierarchy for result diversification. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, pages 1844-1848, Maui, HI, USA, 2012. ACM.
207. W. Zheng, H. Fang, C. Yao, and M. Wang. Leveraging integrated information to extract query subtopics for search result diversification. Information Retrieval, 17(1):52-73, 2014. ISSN 1386-4564.
208. T. Zhou, Z. Kuscsik, J. Liu, M. Medo, J. Wakeling, and Y. Zhang. Solving the apparent diversity-accuracy dilemma of recommender systems. Proceedings of the National Academy of Sciences, 107(10):4511-4515, 2010.
209. X. Zhu, A. B. Goldberg, J. V. Gael, and D. Andrzejewski. Improving diversity in ranking using absorbing random walks. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics-Human Language Technologies, pages 97-104, Rochester, NY, USA, 2007. ACL.
210. Y. Zhu, Y. Lan, J. Guo, X. Cheng, and S. Niu. Learning for search result diversification. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 293-302, Gold Coast, QLD, Australia, 2014. ACM.
211. C.-N. Ziegler, S. M. McNee, J. A. Konstan, and G. Lausen. Improving recommendation lists through topic diversification. In Proceedings of the 14th International Conference on World Wide Web, pages 22-32, Chiba, Japan, 2005. ACM.
212. J. Zobel. How reliable are the results of large-scale information retrieval experiments? In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 307-314, Melbourne, Australia, 1998. ACM.
213. G. Zuccon and L. Azzopardi. Using the quantum probability ranking principle to rank interdependent documents. In Proceedings of the 32nd European Conference on IR Research on Advances in Information Retrieval, pages 357-369, Milton Keynes, UK, 2010. Springer.
214. G. Zuccon, L. Azzopardi, D. Zhang, and J. Wang. Top-k retrieval using facility location analysis. In Proceedings of the 34th European Conference on Advances in Information Retrieval, pages 305-316, Barcelona, Spain, 2012. Springer-Verlag.