Efficient search engine measurements

ACM Transactions on the Web

Volumn 5, Issue 4, 2011, Pages

  Bar Yossef, Ziv ^a,c   Gurevich, Maxim ^b  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b YAHOO RESEARCH   (United States)

^c Google Inc   (Israel)

Author keywords

Corpus size estimation; Evaluation; Sampling; Search engines

Indexed keywords

AGGREGATE FUNCTION; CORPUS SIZE; EMPIRICAL ANALYSIS; EVALUATION; GENERIC METHOD; IMPORTANCE SAMPLING; INDEX FRESHNESS; PERFORMANCE IMPROVEMENTS; RAO-BLACKWELLIZATION; STATE OF THE ART;

INFORMATION RETRIEVAL; SEARCH ENGINES;

ESTIMATION;

EID: 80455127332     PISSN: 15591131     EISSN: 1559114X     Source Type: Journal    
DOI: 10.1145/2019643.2019645     Document Type: Article

References (33)

1. ANAGNOSTOPOULOS, A., BRODER, A. Z., AND CARMEL, D. 2006. Sampling search-engine results. World Wide Web 9, 4, 397-429.
2. BAR-YOSSEF, Z., BERG, A., CHIEN, S., FAKCHAROENPHOL, J., AND WEITZ, D. 2000. Approximating aggregate queries about Web pages via random walks. In Proceedings of the 26th Annual Conference on Very Large Databases. 535-544.
3. BAR-YOSSEF, Z. AND GUREVICH, M. 2007. Efficient search engine measurements. In Proceedings of the 16th International World Wide Web Conference (WWW). ACM, New York, 401-410. (Pubitemid 47582269)
4. BAR-YOSSEF, Z. AND GUREVICH,M. 2008a. Mining search engine query logs via suggestion sampling. Proc. VLDB 1, 1, 54-65.
5. BAR-YOSSEF, Z. AND GUREVICH, M. 2008b. Random sampling from a search engine's index. J. ACM 55, 5, 1-74.
6. BAR-YOSSEF, Z. AND GUREVICH, M. 2009. Estimating the impressionrank of web pages. In Proceedings of the 18th International Conference on World Wide Web (WWW'09). ACM, New York, 41-50.
7. BAYKAN, E., HENZINGER, M. R., KELLER, S. F., CASTELBERG, S. D., AND KINZLER, M. 2009. A comparison of techniques for sampling web pages. In Proceedings of the Symposium on Theoretical Aspects of Computer Science (STACS). 13-30.
8. BHARAT, K. AND BRODER, A. 1998. A technique for measuring the relative size and overlap of public Web search engines. In Proceedings of the 7th International Conference on World Wide Web. ACM, New York, 379-388.
9. BRADLOW, E. T. AND SCHMITTLEIN, D. C. 2000. The little engines that could: Modeling the performance of World Wide Web search engines. Market. Sci. 19, 43-62.
10. BRODER, A., FONTOURA, M., JOSIFOVSKI, V., KUMAR, R., MOTWANI, R., NABAR, S., PANIGRAHY, R., TOMKINS, A., AND XU, Y. 2006. Estimating corpus size via queries. In Proceedings of the 15th International Conference on Information and Knowledge Management (CIKM). ACM, New York, 594-603. (Pubitemid 47203605)
11. CASELLA, G. AND ROBERT, C. P. 1996. Rao-Blackwellisation of sampling schemes. Biometrika 83, 1, 81-94.
12. CHENEY, M. AND PERRY, M. 2005. A comparison of the size of the Yahoo! and Google indices. http://www.mostpopularsites.net/MPS News/Search Engine Facts/ A Comparison of the Size of the Yahoo! and Google Indices.
13. DASGUPTA, A., DAS, G., AND MANNILA, H. 2007. A random walk approach to sampling hidden databases. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'07). ACM, New York, 629-640. (Pubitemid 47630840)
14. DMOZ. The open directory project. http://dmoz.org.
15. DOBRA, A. AND FIENBERG, S. E. 2004. How large is the worldwide web? In Web Dynamics, 23-44.
16. GOLDREICH, O. 1997. A sample of samplers - A computational perspective on sampling (survey). Electron. Colloq. Computat. Complex. (ECCC) 4, 20.
17. GULLI, A. AND SIGNORINI, A. 2005. The indexable Web is more than 11.5 billion pages. In Proceedings of the 14th International Conference on the World Wide Web (WWW). 902-903.
18. HASTINGS, W. K. 1970. Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 1, 97-109.
19. HENZINGER, M. R., HEYDON, A., MITZENMACHER, M., AND NAJORK, M. 1999. Measuring index quality using random walks on the Web. In Proceedings of the 8th International Conference on the World Wide Web (WWW). 213-225.
20. HENZINGER, M. R., HEYDON, A., MITZENMACHER, M., AND NAJORK, M. 2000. On near-uniform URL sampling. In Proceedings of the 9th International Conference on the World Wide Web (WWW). 295-308.
21. HESTERBERG, T. C. 1988. Advances in importance sampling. Ph.D. dissertation, Stanford University.
22. LAWRENCE, S. AND GILES, C. L. 1998. Searching the World Wide Web. Science 5360, 280, 98.
23. LAWRENCE, S. AND GILES, C. L. 1999. Accessibility of information on the Web. Nature 400, 107-109. (Pubitemid 29327520)
24. LEE, S.-M. AND CHAO, A. 1994. Estimating population size via sample coverage for closed capturerecapture models. Biometrics 50, 1, 88-97. (Pubitemid 24138576)
25. LIU, J. S. 1996. Metropolized independent sampling with comparisons to rejection sampling and importance sampling. Stat. Comput. 6, 113-119. (Pubitemid 126724248)
26. LIU, J. S. 2001. Monte Carlo Strategies in Scientific Computing. Springer.
27. MARSHALL, A. W. 1956. The use of multi-stage sampling schemes in Monte Carlo computations. In Proceedings of the Symposium on Monte Carlo Methods. 123-140.
28. MCCURLEY, K. 2007. Income inequality in the attention economy. http://mccurley.org/papers/effective.
29. METROPOLIS, N., ROSENBLUTH, A., ROSENBLUTH, M., TELLER, A., AND TELLER, E. 1953. Equations of state calculations by fast computing machines. J. Chem. Phys. 21, 1087-1091.
30. RUSMEVICHIENTONG, P., PENNOCK, D., LAWRENCE, S., AND GILES, C. L. 2001. Methods for sampling pages uniformly from theWorldWideWeb. In Proceedings of the AAAI Symposium on Using Uncertainty within Computation.
31. SIEGMUND, D. 1985. Sequential Analysis - Tests and Confidence Intervals. Springer-Verlag.
32. STUART, A. AND ORD, J. K. 1994. Kendall's Advanced Theory of Statistics. Vol.1: Distribution Theory. Hodder Arnold, London.
33. VON NEUMANN, J. 1963. Various techniques used in connection with random digits. In John von Neumann, Collected Works. Vol. V. Oxford.