Towards unbiased BFS sampling

IEEE Journal on Selected Areas in Communications

Volumn 29, Issue 9, 2011, Pages 1799-1809

  Kurant, MacIej ^a   Markopoulou, Athina ^a   Thiran, Patrick ^b  

^a University of California Irvine   (United States)

^b EPFL   (Switzerland)

Author keywords

bias; Breadth First Search (BFS); estimation; network topology; online social networks; sampling methods

Indexed keywords

BIAS; BREADTH-FIRST-SEARCH; NETWORK TOPOLOGY; ONLINE SOCIAL NETWORKS; SAMPLING METHODS;

DEFORESTATION; ELECTRIC NETWORK TOPOLOGY; ESTIMATION; INTERNET; TOPOLOGY;

SOCIAL NETWORKING (ONLINE);

ESTIMATION; INTERNET; NETWORKS;

EID: 80053653453     PISSN: 07338716     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSAC.2011.111005     Document Type: Article

References (58)

1. M. Kurant, A. Markopoulou, and P. Thiran, "On the bias of BFS (Breadth First Search)", in Proc. 22nd Int. Teletraffic Congr., also in arXiv:1004.1729, 2010.
2. M. Gjoka, M. Kurant, C. T. Butts, and A. Markopoulou, "Practical Recommendations on Sampling OSN Users by Crawling the Social Graph", To appear in IEEE J. Sel. Areas Commun. on Measurement of Internet Topologies, 2011.
3. L. Lovász, "Random walks on graphs: A survey", Combinatorics, Paul Erdos is Eighty, vol. 2, no. 1, pp. 1-46, 1993.
4. M. R. Henzinger, A. Heydon, M. Mitzenmacher, and M. Najork, "On near-uniform URL sampling", in Proc. 9th Int. Conf. on World Wide Web, Amsterdam, Netherlands, 2000.
5. C. Gkantsidis, M. Mihail, and A. Saberi, "Random walks in peer-to-peer networks", in Proc. IEEE INFOCOM, Hong Kong, China, 2004.
6. D. Stutzbach, R. Rejaie, N. Duffield, S. Sen, and W. Willinger, "On unbiased sampling for unstructured peer-to-peer networks", in Proc. 6th ACM SIGCOMM Conf. on Internet measurement, Rio de Janeiro, Brazil, 2006.
7. A. Rasti, M. Torkjazi, R. Rejaie, N. Duffield, W. Willinger, and D. Stutzbach, "Respondent-driven sampling for characterizing unstructured overlays", in Proc. IEEE INFOCOM Mini-conference, Rio de Janeiro, Brazil, 2009, pp. 2701-2705.
8. B. Ribeiro and D. Towsley, "Estimating and sampling graphs with multidimensional random walks", in Proc. 10th ACM SIGCOMM Conf. on Internet measurement, Melbourne, Australia, 2010.
9. K. Avrachenkov, B. Ribeiro, and D. Towsley, "Improving Random Walk Estimation Accuracy with Uniform Restarts", in I7th Workshop on Algorithms and Models for the Web Graph, 2010.
10. M. Kurant, M. Gjoka, C. T. Butts, and A. Markopoulou, "Walking on a Graph with a Magnifying Glass: Stratified Sampling via Weighted Random Walks", in Proc. ACM SIGMETRICS, 2011.
11. M. Gjoka, M. Kurant, C. T. Butts, and A. Markopoulou, "Walking in Facebook: A Case Study of Unbiased Sampling of OSNs", in Proc. IEEE INFOCOM, San Diego, CA, 2010.
12. B. Krishnamurthy, P. Gill, and M. Arlitt, "A few chirps about twitter", in Proc. 1st workshop on Online social networks, Seattle, WA, 2008, pp. 19-24.
13. J. Leskovec and C. Faloutsos, "Sampling from large graphs", in Proc. 12th ACM SIGKDD Int. Conf. on Knowledge discovery and data mining, Philadelphia, PA, 2006, pp. 631-636. (Pubitemid 44535566)
14. S. L. Feld, "Why Your Friends Have More Friends Than You Do", American Journal of Sociology, vol. 96, no. 6, p. 1464, May 1991.
15. M. Newman, "Ego-centered networks and the ripple effect", Social Networks, vol. 25, pp. 83-95, 2003.
16. M. Salganik and D. D. Heckathorn, "Sampling and estimation in hidden populations using respondent-driven sampling", Sociological Methodology, vol. 34, no. 1, pp. 193-240, 2004.
17. E. Volz and D. D. Heckathorn, "Probability based estimation theory for respondent driven sampling", J. Official Statistics, vol. 24, no. 1, pp. 79-97, 2008.
18. L. A. Goodman, "Snowball sampling", Annals of Mathematical Statistics, vol. 32, pp. 148-170, 1961.
19. D. D. Heckathorn, "Respondent-Driven Sampling: A New Approach to the Study of Hidden Populations", Social Problems, vol. 44, pp. 174-199, 1997.
20. M. Najork and J. L. Wiener, "Breadth-first search crawling yields highquality pages", in WWW, 2001.
21. Y. Ahn, S. Han, H. Kwak, S. Moon, and H. Jeong, "Analysis of topological characteristics of huge online social networking services", in Proc. 16th Int. Conf. on World Wide Web, Banff, Alberta, Canada, 2007, pp. 835-844. (Pubitemid 47582313)
22. A. Mislove, M. Marcon, K. Gummadi, P. Druschel, and B. Bhattacharjee, "Measurement and analysis of online social networks", in Proc. 7th ACM SIGCOMM Conf. on Internet measurement, San Diego, CA, 2007, pp. 29-42.
23. C. Wilson, B. Boe, A. Sala, K. Puttaswamy, and B. Zhao, "User interactions in social networks and their implications", in Proc. 4th ACM European Conf. on Computer systems, Nuremberg, Germany, 2009, pp. 205-218.
24. S. H. Lee, P.-J. Kim, and H. Jeong, "Statistical properties of sampled networks", Physical Review E, vol. 73, p. 16102, 2006.
25. L. Becchetti, C. Castillo, D. Donato, A. Fazzone, and I. Rome, "A comparison of sampling techniques for web graph characterization", in Proc. Workshop on Link Analysis, Philadelphia, PA, 2006.
26. S. Ye, J. Lang, and F. Wu, "Crawling online social graphs", in Proc. 12th Asia-Pacific Web Conference, Busan, Korea, 2010, pp. 236-242.
27. D. Achlioptas, A. Clauset, D. Kempe, and C. Moore, "On the bias of traceroute sampling: or, power-law degree distributions in regular graphs", in Proc. 37th Annu. ACM Symp. on Theory of computing, 2005, pp. 694-703. (Pubitemid 43098642)
28. M. Kurant, "Python scripts for BFS sampling and bias correction: http://mkurant.com/maciej/publications/papers/traversals.zip"
29. A. Mislove, H. Koppula, K. Gummadi, P. Druschel, and B. Bhattacharjee, "Growth of the flickr social network", in Proc. 1st workshop on Online social networks, Seattle, WA, 2008, pp. 25-30.
30. J. H. Kim, "Poisson cloning model for random graphs", in International Congress of Mathematicians (ICM), 2006.
31. M. Molloy and B. Reed, "A critical point for random graphs with a given degree sequence", Random structures and algorithms, vol. 6, no. 2-3, pp. 161-180, 1995.
32. K. Gile and M. Handcock, "Respondent-driven sampling: An assessment of current methodology", To appear in Sociological Methodology, 2011.
33. K. Gile, "Improved Inference for Respondent-Driven Sampling Data with Application to HIV Prevalence Estimation", arXiv:1006.4837, 2010.
34. J. Illenberger, G. Flötteröd, and N. Kai, "An approach to correct bias induced by snowball sampling", in Sunbelt Social Networks Conference, 2009.
35. F. Yates and P. Grundy, "Selection without replacement from within strata with probability proportional to size", J. Royal Statistical Society. Series B (Methodological), vol. 15, no. 2, pp. 253-261, 1953.
36. D. Raj, "Some estimators in sampling with varying probabilities without replacement", J. American Statistical Association, pp. 269-284, 1956.
37. M. Murthy, "Ordered and unordered estimators in sampling without replacement", Sankhyà: The Indian Journal of Statistics, vol. 18, no. 3, pp. 379-390, 1957.
38. H. Hartley and J. Rao, "Sampling with unequal probabilities and without replacement", The Annals of Mathematical Statistics, 1962.
39. G. Andreatta and G. Kaufman, "Estimation of finite population properties when sampling is without replacement and proportional to magnitude", J. American Statistical Association, vol. 81, no. 395, pp. 657-666, 1986.
40. T. J. Rao, S. Sengupta, and B. K. Sinha, "Some Order Relations Between Selection and Inclusion Probabilities for PPSWOR Sampling Scheme", Metrika, vol. 38, no. 1, pp. 335-343, Dec. 1991.
41. S. Kochar and R. Korwar, "On random sampling without replacement from a finite population", Annals of the Institute of Statistical Mathematics, vol. 53, no. 3, pp. 631-646, 2001. (Pubitemid 33617907)
42. L. Fattorini, "Applying the Horvitz-Thompson criterion in complex designs: A computer-intensive perspective for estimating inclusion probabilities", Biometrika, vol. 93, no. 2, pp. 269-278, Jun. 2006. (Pubitemid 43992190)
43. J. Leskovec, J. Kleinberg, and C. Faloutsos, "Graphs over time: densification laws, shrinking diameters and possible explanations", in KDD, 2005.
44. F. Chung and L. Lu, "Connected components in random graphs with given degree sequences", Annals of Combinatorics, vol. 6, p. 125145, 2002.
45. R. Motwani and P. Raghavan, Randomized Algorithms. Cambridge University Press, 1990.
46. F. Viger and M. Latapy, "Efficient and simple generation of random simple connected graphs with prescribed degree sequence", LNCS Computing and Combinatorics, vol. 3595, pp. 440-449, 2005. (Pubitemid 41450222)
47. M. Hansen and W. Hurwitz, "On the Theory of Sampling from Finite Populations", Annals of Mathematical Statistics, vol. 14, no. 3, 1943.
48. S. Maslov and K. Sneppen, "Specificity and stability in topology of protein networks", Science, vol. 296, no. 5569, p. 910, 2002.
49. D. Horvitz and D. Thompson, "A generalization of sampling without replacement from a finite universe", J. American Statistical Association, vol. 47, no. 260, pp. 663-685, 1952.
50. J. Leskovec, J. Kleinberg, and C. Faloutsos, "Graph evolution: Densification and shrinking diameters", ACM Trans. Knowledge Discovery from Data, vol. 1, no. 1, p. 2, Mar. 2007.
51. B. Viswanath, A. Mislove, M. Cha, and K. Gummadi, "On the evolution of user interaction in facebook", in Proc. 2nd workshop on Online social networks, Barcelona, Spain, 2009, pp. 37-42.
52. J. Leskovec, D. Huttenlocher, and J. Kleinberg, "Predicting positive and negative links in online social networks", in WWW, New York, New York, USA, 2010, p. 641.
53. M. Richardson, R. Agrawal, and P. Domingos, "Trust management for the semantic web", The SemanticWeb-ISWC 2003, pp. 351-368, 2003.
54. J. Leskovec, K. Lang, A. Dasgupta, and M. Mahoney, "Community structure in large networks: Natural cluster sizes and the absence of large well-defined clusters", Internet Mathematics, vol. 6, no. 1, pp. 29-123, 2009.
55. M. Newman, "Assortative mixing in networks", Physical Review Letters, vol. 89, no. 20, p. 208701, 2002.
56. "SNAP Graph Library." [Online]. Available: http://snap. stanford. edu/data
57. Z. Anwar, W. Yurcik, V. Pandey, A. Shankar, I. Gupta, and R. Campbell, "Leveraging Social-Network Infrastructure to Improve Peer-to-Peer Overlay Performance: Results from Orkut", Arxiv preprint cs/0509095, 2005.
58. R. Albert, H. Jeong, and A. Barabási, "Diameter of the world-wide web", Nature, vol. 401, no. 6749, pp. 130-131, 1999. (Pubitemid 29436307)