Real-time community detection in full social networks on a laptop

PLoS ONE

Volumn 13, Issue 1, 2018, Pages

  Chamberlain, Benjamin Paul ^a   Levy Kramer, Josh ^b   Humby, Clive ^b   Deisenroth, Marc Peter ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b Starcount Insights   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COMPRESSION; HUMAN; MACHINE; NEIGHBORHOOD; NOISE; REACTION TIME; SCIENTIST; SOCIAL NETWORK; SOFTWARE; COMPUTER; SOCIAL SUPPORT;

COMPUTERS; HUMANS; SOCIAL SUPPORT; SOFTWARE;

EID: 85040773826     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0188702     Document Type: Article

References (61)

1. Hewett T, Baecker R, Card S, Carey T. ACM SIGCHI curricula for human-computer interaction. ACM Press; 1992.
2. Gleich DF, Seshadhri C. Vertex neighborhoods, low conductance cuts, and good seeds for local community methods. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining; 2012. p. 597–605.
3. Liben-Nowell D, Kleinberg J. The link-prediction problem for social networks. Journal of the American Society for Information Science and Technology. 2007; 58(7):1019–1031. https://doi.org/10.1002/asi. 20591
4. McPherson M, Smith-Lovin L, Cook JM. Birds of a feather: Homophily in social networks. Annual Review of Sociology. 2001; 27(1):415–444. https://doi.org/10.1146/annurev.soc.27.1.415
5. Shang Y. Deffuant model of opinion formation in one-dimensional multiplex networks. Journal of Physics A: Mathematical and Theoretical. 2015; 48(39):395101. https://doi.org/10.1088/1751-8113/48/39/ 395101
6. Yang J, Leskovec J. Overlapping community detection at scale: a nonnegative matrix factorization approach. In: Proceedings of the 6th international conference on Web search and data mining. ACM; 2013. p. 587–596.
7. Clauset A. Finding local community structure in networks. Physical Review E. 2005; 72.2:026132.https://doi.org/10.1103/PhysRevE.72.026132
8. Bahmani B, Chakrabarti K, Xin D. Fast personalized pagerank on mapreduce. Proceedings of the International Conference on Management of Data. 2011; p. 973–984.
9. Pons P, Latapy M. Computing communities in large networks using random walks. Computer and Information Sciences. 2005; p. 284–293.
10. Gibson D, Tomkins A, Jose S. Discovering Large Dense Subgraphs in Massive Graphs. In: Proceedings of the International Conference on Very Large Data Bases. VLDB Endowment; 2005. p. 721–732.
11. Becchetti L, Boldi P, Castillo C, Gionis A. Efficient algorithms for large-scale local triangle counting. ACM Transactions on Knowledge Discovery from Data. 2010; 4.3:13. https://doi.org/10.1145/1839490. 1839494
12. Newman M. Fast algorithm for detecting community structure in networks. Physical review E. 2004; 69 (6)(066133). https://doi.org/10.1103/PhysRevE.69.066133
13. Rosvall M, Bergstrom C. Maps of random walks on complex networks reveal community structure. Proceedings of the National Academy of Sciences. 2008; 105.4:1118–1123. https://doi.org/10.1073/pnas. 0706851105
14. Blondel VD, Guillaume JL, Lambiotte R, Lefebvre E. Fast unfolding of community hierarchies in large networks. Journal of Statistical Mechanics: Theory and Experiment. 2008; p. 10008. https://doi.org/10.1088/1742-5468/2008/10/P10008
15. Low Y, Gonzalez J, Kyrola A. Graphlab: A new framework for parallel machine learning; 2014. Available from: http://arxiv.org/abs/1408.2041.
16. Malewicz G, Austern MH, Bik AJ, Dehnert JC, Horn I, Leiser N, et al. Pregel: A system for large-scale graph processing. In: Proceedings of the International Conference on Management of Data; 2010. p. 135–146.
17. Chen R, Weng X, He B, Yang M. Large graph processing in the cloud. In: Proceedings of the International Conference on Management of Data; 2010. p. 1123–1126.
18. Kyrola A, Blelloch G, Guestrin C. Graphchi: Large-scale graph computation on just a PC. Symposium on Operating Systems Design and Implementation. 2012; p. 31–46.
19. Gupta P, Goel A, Lin J, Sharma A, Wang D, Zadeh R. WTF: The who to follow service at Twitter. Proceedings of the International Conference on World Wide Web. 2013; p. 505–514.
20. Li Y, He K, Bindel D, Hopcroft JE. Uncovering the small community structure in large networks: A local spectral approach. In: Proceedings of the International Conference on World Wide Web; 2015. p. 658–668.
21. Bullmore E, Sporns O. Complex brain networks: graph theoretical analysis of structural and functional systems. Nature Reviews Neuroscience. 2009; p. 186–198. https://doi.org/10.1038/nrn2575 PMID: 19190637
22. Reichardt J, Bornholdt S. Statistical mechanics of community detection. Physical Review E. 2006; 74. https://doi.org/10.1103/PhysRevE.74.016110
23. Fortunato S. Community detection in graphs. Physics Reports. 2010; 486(3):75–174. https://doi.org/10.1016/j.physrep.2009.11.002
24. Newman M. The structure and function of complex networks. Society for Industrial and Applied Mathematics Review. 2003;.
25. Flake GW, Lawrence S, Giles CL. Efficient identification of web communities. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining; 2000. p. 150–160.
26. Page L, Brin S, Motwani R, Winograd T. The PageRank citation ranking: Bringing order to the web.; 1998. Available from: http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf.
27. Kloumann I, Kleinberg J. Community membership identification from small seed sets. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining; 2014. p. 1366–1375.
28. Haveliwala TH. Topic-sensitive PageRank. In: Proceedings of the International Conference on World Wide Web; 2002. p. 517–526.
29. Sampson SF. Crisis in a cloister. Cornell University, Ithaca; 1969.
30. Zachary W. An information flow model for conflict and fission in small groups. Journal of Anthropological Research. 1977; p. 452–473. https://doi.org/10.1086/jar.33.4.3629752
31. Lusseau D. The emergent properties of a dolphin social network. In: Proceedings of the Royal Society of London B: Biological Sciences. vol. 270(Suppl2); 2003. p. S186–S188. https://doi.org/10.1098/rsbl.2003.0057
32. Yang J, Leskovec J. Defining and evaluating network communities based on ground-truth. Knowledge and Information Systems. 2015; 42.1:181–213. https://doi.org/10.1007/s10115-013-0693-z
33. Pace MF. BSP vs MapReduce. Procedia Computer Science. 2012; 9:246–255. https://doi.org/10.1016/j.procs.2012.04.026
34. Adler M, Mitzenmacher M. Towards compressing web graphs; 2001. p. 203–212. Available from: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=917151.
35. Boldi P, Vigna S. The webgraph framework I: compression techniques. In: Proceedings of the International Conference on World Wide Web; 2004. p. 595–602.
36. Chierichetti F, Kumar R, Lattanzi S. On compressing social networks. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining; 2009. p. 219–228.
37. Broder AZ. On the resemblance and containment of documents. In: Proceedings of the International Conference on Compression and Complexity of Sequences; 1997. p. 21–29.
38. Charikar MS. Similarity estimation techniques from rounding algorithms. In: Proceedings of the Annual Symposium on Theory of Computing; 2002. p. 380–388.
39. Philbin J. Near Duplicate Image Detection: min-Hash and tf-idf weighting. In: Proceedings of the British Machine Vision Conference. vol. 3; 2008. p. 4.
40. Manasse M, Mcsherry F, Talwar K. Consistent weighted sampling. In: Proceedings of the International Conference on World Wide Web; 2007.
41. Haveliwala T, Gionis A, Indyk P. Scalable techniques for clustering the Web. International Workshop on the Web and Databases. 2000;.
42. Li P, König C. b-Bit minwise hashing. Proceedings of the International Conference on World Wide Web. 2009;.
43. Mitzenmacher M, Pagh R, Pham N. Efficient estimation for high similarities using odd sketches. In: Proceedings of the International Conference on World Wide Web; 2014. p. 109–118.
44. Indyk P, Motwani R. Approximate nearest neighbors: Towards removing the curse of dimensionality. In: Proceedings of the Annual Symposium on Theory of Computing. vol. 126; 1998. p. 604–613.
45. Motwani R, Naor A, Panigrahy R. Lower bounds on locality sensitive hashing. SIAM Journal on Discrete Mathematics. 2005; p. 930–935.
46. O’Donnell R, Wu Y, Zhou Y. Optimal lower bounds for locality sensitive hashing (except when q is tiny). ACM Transactions on Computation Theory. 2009; 6(1):9.
47. Andoni A, Indyk P, Nguyen HL, Razenshteyn I. Beyond locality-sensitive hashing. In: Proceedings of the Annual Symposium on Discrete Algorithms; 2014. p. 1018–1028.
48. Leskovec J, Kleinberg J, Faloutsos C. Graph Evolution: Densification and Shrinking Diameters. ACM Transactions on Knowledge Discovery from Data. 2007; 1(1). https://doi.org/10.1145/1217299. 1217301
49. Karoński M, Scheinerman ER, Singer-Cohen KB. On random intersection graphs: The subgraph problem. Combinatorics, Probability and Computing. 1999; 8(1999):131–159.
50. Shang Y. Degree distributions in general random intersection graphs. Electronic Journal of Combinatorics. 2010; 17(1):R23.
51. Shang Y. Joint probability generating function for degrees of active/passive random intersection graphs. Frontiers of Mathematics in China. 2012; 7(1):117–124. https://doi.org/10.1007/s11464-011-0165-2
52. Broder AZ, Charikar M, Frieze AM, Mitzenmacher M. Min-wise independent permutations. Journal of Computer and System Sciences. 2000; 60(3):630–659. https://doi.org/10.1006/jcss.1999.1690
53. Lancichinetti A, Fortunato S, Kertész J. Detecting the overlapping and hierarchical community structure in complex networks. New Journal of Physics. 2009; 11.3(033015). https://doi.org/10.1088/1367-2630/11/3/033015
54. Leskovec J, Lang KJ, Mahoney MW. Empirical comparison of algorithms for network community detection. In: Proceedings of the International Conference on World Wide Web; 2010. p. 631–640.
55. Raghavan UN, Albert R, Kumara S. Near linear time algorithm to detect community structures in large-scale networks. Physical Review E. 2007; 76(3):036106. https://doi.org/10.1103/PhysRevE.76.036106
56. Newman MEJ. Finding community structure in networks using the eigenvectors of matrices. Physical Review E. 2006; 74(3):1–19. https://doi.org/10.1103/PhysRevE.74.036104
57. Newman M. Detecting community structure in networks. The European Physical Journal B. 2004; 38.2:321–330. https://doi.org/10.1140/epjb/e2004-00124-y
58. Reichardt J, Bornholdt S. Statistical Mechanics of Community Detection. Physics Review E. 2006; 74(1):016110. https://doi.org/10.1103/PhysRevE.74.016110
59. Leskovec J, Lang KJ, Dasgupta A, Mahoney MW. Statistical properties of community structure in large social and information networks. Proceedings of the 17th international conference on World Wide Web. 2008; p. 695–704.
60. Fortunato S, Barthelemy M. Resolution limit in community detection. In: Proceedings of the National Academy of Sciences. vol. 104.1; 2007. p. 36–41. https://doi.org/10.1073/pnas.0605965104
61. Bastian M, Heymann S, Jacomy M. Gephi: An open source software for exploring and manipulating networks. In: Proceedings of the International Conference on Weblogs and Social Media; 2009. p. 361–362.