Size estimation of non-cooperative data collections

ACM International Conference Proceeding Series

Volumn , Issue , 2012, Pages 239-246

  Khelghati, Mohammadreza ^a   Hiemstra, Djoerd ^a   Van Keulen, Maurice ^a  

^a UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

deep web; estimation bias; pool based size estimation; query based sampling; regression equations; size estimation; stochastic simulation

Indexed keywords

DEEP WEB; ESTIMATION BIAS; REGRESSION EQUATION; SIZE ESTIMATION; STOCHASTIC SIMULATIONS;

ALGORITHMS; COMPETITION; INFORMATION RETRIEVAL; SEARCH ENGINES; STOCHASTIC MODELS; WEB SERVICES;

ESTIMATION;

EID: 84873378043     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2428736.2428774     Document Type: Conference Paper

References (19)

1. Amstrup, S., McDonald, T., and Manly, B. F. Handbook of Capture-Recapture Analysis. Princeton University Press, Princeton, NJ, Oct. 2005.
2. Anagnostopoulos, A., Broder, A. Z., and Carmel, D. Sampling search-engine results. In WWW '05: Proceedings of the 14th international conference on World Wide Web (New York, NY, USA, 2005), ACM Press, pp. 245-256.
3. Bar-Yossef, Z., and Gurevich, M. Random sampling from a search engine's index. In Proceedings of the 15th international conference on World Wide Web (New York, NY, USA, 2006), WWW '06, ACM, pp. 367-376.
4. Bar-Yossef, Z., and Gurevich, M. Efficient search engine measurements. Proceedings of the 16th international conference on World Wide Web (2007), 401-410.
5. Bar-Yossef, Z., and Gurevich, M. Efficient search engine measurements. ACM Trans. Web 5, 4 (Oct. 2011), 18:1-18:48.
6. Bharat, K., and Broder, A. A technique for measuring the relative size and overlap of public web search engines. Comput. Netw. ISDN Syst. 30 (April 1998), 379-388.
7. Broder, A. Z., Fontoura, M., Josifovski, V., Kumar, R., Motwani, R., Nabar, S. U., Panigrahy, R., Tomkins, A., and Xu, Y. Estimating corpus size via queries. In CIKM (2006), pp. 594-603.
8. Callan, J. P., and Connell, M. E. Query-based sampling of text databases. ACM Trans. Inf. Syst. 19, 2 (2001), 97-130.
9. Dasgupta, A., Jin, X., Jewell, B., Zhang, N., and Das, G. Unbiased estimation of size and other aggregates over hidden web databases. In Proceedings of the 2010 international conference on Management of data (New York, NY, USA, 2010), SIGMOD '10, ACM, pp. 855-866.
10. dmoz. http://dmoz.org, Title=The open directory project.
11. Geiger, D., Heckerman, D., and Meek, C. Introduction to monte carlo methods. In Learning in graphical models, M. Jordan, Ed. Kluwer, 1998, pp. 175-289.
12. Gulli, A., and Signorini, A. The indexable web is more than 11.5 billion pages. In Special interest tracks and posters of the 14th international conference on World Wide Web (New York, NY, USA, 2005), WWW '05, ACM, pp. 902-903.
13. Kern, J. C. An introduction to regression analysis. American Statistician 61, 1 (2007), 101-101.
14. Lu, J. Ranking bias in deep web size estimation using capture recapture method. Data Knowl. Eng. 69, 8 (Aug. 2010), 866-879.
15. Lu, J., and Li, D. Estimating deep web data source size by capture-recapture method. Inf. Retr. 13, 1 (Feb. 2010), 70-95.
16. lup Liu, K., Yu, C., and Meng, W. Discovering the representative of a search engine. In In Proc. CIKM (2001).
17. Shokouhi, M., Zobel, J., Scholer, F., and Tahaghoghi, S. M. M. Capturing collection size for distributed non-cooperative retrieval. In SIGIR (2006), pp. 316-323.
18. Thomas, P. Generalising multiple capture-recapture to non-uniform sample sizes. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval (New York, NY, USA, 2008), SIGIR '08, ACM, pp. 839-840.
19. Xu, J., Wu, S., and Li, X. Estimating collection size with logistic regression. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval (New York, NY, USA, 2007), SIGIR '07, ACM, pp. 789-790.