Entropy of stochastic blockmodel ensembles

Physical Review E - Statistical, Nonlinear, and Soft Matter Physics

Volumn 85, Issue 5, 2012, Pages

  Peixoto, Tiago P ^a  

^a UNIVERSITY OF BREMEN   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BLOCK STRUCTURES; BLOCKMODEL; DEGREE CORRELATION; DEGREE SEQUENCE; GENERAL NATURE; GROUP MEMBERSHIPS; LOG-LIKELIHOOD FUNCTIONS; MULTIGRAPHS; NETWORK MODELS; OBSERVED DATA;

DIRECTED GRAPHS; ENTROPY;

STOCHASTIC SYSTEMS;

EID: 84862006668     PISSN: 15393755     EISSN: 15502376     Source Type: Journal    
DOI: 10.1103/PhysRevE.85.056122     Document Type: Article

References (77)

1. P. W. Holland, K. B. Laskey, and S. Leinhardt, Social Networks 0378-8733 10.1016/0378-8733(83)90021-7 5, 109 (1983).
2. S. E. Fienberg, M. M. Meyer, and S. S. Wasserman, J. Am. Stat. Assoc. 80, 51 (1985).
3. K. Faust and S. Wasserman, Social Networks 0378-8733 10.1016/0378- 8733(92)90013-W 14, 5 (1992).
4. C. J. Anderson, S. Wasserman, and K. Faust, Social Networks 0378-8733 10.1016/0378-8733(92)90017-2 14, 137 (1992).
5. P. Doreian, V. Batagelj, and A. Ferligoj, Generalized Blockmodeling (Cambridge University Press, Cambridge, 2004).
6. U. Brandes, J. Lerner, and U. Nagel, Advances in Data Analysis and Classification 1862-5347 10.1007/s11634-010-0074-3 5, 81 (2010).
7. S. Fortunato, Phys. Rep. PRPLCM 0370-1573 10.1016/j.physrep.2009.11.002 486, 75 (2010).
8. M. E. J. Newman and E. A. Leicht, Proceedings of the National Academy of Sciences PNASA6 0027-8424 10.1073/pnas.0610537104 104, 9564 (2007). (Pubitemid 47175306)
9. J. Reichardt and D. R. White, Eur. Phys. J. B EPJBFY 1434-6028 10.1140/epjb/e2007-00340-y 60, 217 (2007). (Pubitemid 350277664)
10. P. J. Bickel and A. Chen, Proceedings of the National Academy of Sciences PNASA6 0027-8424 10.1073/pnas.0907096106 106, 21068 (2009).
11. R. Guimerà and M. Sales-Pardo, Proceedings of the National Academy of Sciences PNASA6 0027-8424 10.1073/pnas.0908366106 106, 22073 (2009).
12. B. Karrer and M. E. J. Newman, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.83.016107 83, 016107 (2011).
13. B. Ball, B. Karrer, and M. E. J. Newman, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.84.036103 84, 036103 (2011).
14. J. Reichardt, R. Alamino, and D. Saad, PLoS ONE 1932-6203 10.1371/journal.pone.0021282 6, e21282 (2011).
15. A. Decelle, F. Krzakala, C. Moore, and L. Zdeborová, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.84.066106 84, 066106 (2011).
16. T. P. Peixoto, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.85.041908 85, 041908 (2012).
17. T. P. Peixoto, J. Stat. Mech.: Theory and Experiment 1742-5468 10.1088/1742-5468/2012/01/P01006 (2012) P01006.
18. T. Gross and B. Blasius, J. R. Soc. Interface 1742-5689 10.1098/rsif.2007.1229 5, 259 (2008).
19. M. Newman, Networks: An Introduction (Oxford University Press, Oxford, 2010).
20. M. E. J. Newman, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.67.026126 67, 026126 (2003).
21. M. Girvan and M. E. J. Newman, Proceedings of the National Academy of Sciences PNASA6 0027-8424 10.1073/pnas.122653799 99, 7821 (2002). (Pubitemid 34650963)
22. G. Bianconi, EPL (Europhysics Letters) EULEEJ 0295-5075 10.1209/0295-5075/81/28005 81, 28005 (2008).
23. K. Anand and G. Bianconi, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.80.045102 80, 045102 (2009).
24. G. Bianconi, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.79.036114 79, 036114 (2009).
25. K. Anand and G. Bianconi, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.82.011116 82, 011116 (2010).
26. C. Castellano, S. Fortunato, and V. Loreto, Reviews of Modern Physics RMPHAT 0034-6861 10.1103/RevModPhys.81.591 81, 591 (2009).
27. D. Strauss, SIAM Review SIREAD 0036-1445 10.1137/1028156 28, 513 (1986).
28. Z. Burda, J. D. Correia, and A. Krzywicki, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.64.046118 64, 046118 (2001).
29. J. Park and M. E. J. Newman, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.70.066117 70, 066117 (2004). (Pubitemid 40429804)
30. J. Park and M. E. J. Newman, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.70.066146 70, 066146 (2004). (Pubitemid 40429826)
31. G. Palla, I. Derényi, I. Farkas, and T. Vicsek, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.69.046117 69, 046117 (2004).
32. Z. Burda, J. Jurkiewicz, and A. Krzywicki, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.69.026106 69, 026106 (2004).
33. C. Biely and S. Thurner, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.74.066116 74, 066116 (2006). (Pubitemid 46130662)
34. A. Fronczak, P. Fronczak, and J. A. Hołyst, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.73.016108 73, 016108 (2006). (Pubitemid 43254395)
35. P. Fronczak, A. Fronczak, and J. A. Hołyst, Eur. Phys. J. B EPJBFY 1434-6028 10.1140/epjb/e2007-00270-8 59, 133 (2007). (Pubitemid 47536706)
36. D. Jeong, M. Y. Choi, and H. Park, J. Phys. A: Math. Theor. 1751-8113 10.1088/1751-8113/40/32/001 40, 9723 (2007).
37. D. Foster, J. Foster, M. Paczuski, and P. Grassberger, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.81.046115 81, 046115 (2010).
38. K. Nowicki and T. A. B. Snijders, J. Am. Stat. Assoc. JSTNAL 0162-1459 10.1198/016214501753208735 96, 1077 (2001). (Pubitemid 33333511)
39. E. A. Bender, Discrete Mathematics DSMHA4 0012-365X 10.1016/0012-365X(74) 90118-6 10, 217 (1974).
40. E. Bender and J. Butler, Computers, IEEE Transactions on ITCOB4 0018-9340 10.1109/TC.1978.1675021 C-27, 1180 (1978).
41. N. Wormald, in Surveys in Combinatorics, London Mathematical Society Lecture Note Series, edited by, J. D. Lamb, and, D. A. Preece, Vol. 276 (Cambridge University Press, Cambridge, 1999), pp. 239-298.
42. F. Chung and L. Lu, Proceedings of the National Academy of Sciences PNASA6 0027-8424 10.1073/pnas.252631999 99, 15879 (2002). (Pubitemid 35462716)
43. F. Chung and L. Lu, Annals of Combinatorics 0218-0006 10.1007/PL00012580 6, 125 (2002).
44. T. M. Cover and J. A. Thomas, Elements of Information Theory, 99th ed. (Wiley-Interscience, New York, 1991).
45. M. Boguñá, R. Pastor-Satorras, and A. Vespignani, Eur. Phys. J. B EPJBFY 1434-6028 10.1140/epjb/e2004-00038-8 38, 205 (2004).
46. J. Park and M. E. J. Newman, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.68.026112 68, 026112 (2003).
47. S. Johnson, J. J. Torres, J. Marro, and M. A. Muñoz, Phys. Rev. Lett. PRLTAO 0031-9007 10.1103/PhysRevLett.104.108702 104, 108702 (2010).
48. B. McKay and N. Wormald, European J. Combin 11, 565 (1990).
49. Note that not all degree sequences are allowed in the first place, since they must be graphical. Imposing a block structure further complicates things, since the graphical condition needs to be generalized to blockmodels. We will not pursue this here, as we consider only the sufficiently sparse situation, where this issue can be neglected.
50. B. Bollobás, Eur. J. Comb. 1, 311 (1980).
51. M. E. J. Newman, S. H. Strogatz, and D. J. Watts, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.64.026118 64, 026118 (2001).
52. M. E. J. Newman, SIAM Rev. SIREAD 0036-1445 10.1137/S003614450342480 45, 167 (2003).
53. O. D. King, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.70.058101 70, 058101 (2004). (Pubitemid 40086459)
54. The difference between ensembles with "hard" and "soft" degree constraints is analyzed in detail in Ref. for the case without block structures.
55. S. Janson, Combinatorics, Probability and Computing 0963-5483 10.1017/S0963548308009644 18, 205 (2009).
56. N. Wormald, J. Comb. Theory, Ser. B JCBTB8 0095-8956 10.1016/S0095- 8956(81)80022-6 31, 168 (1981).
57. P. W. Holland and S. Leinhardt, J. Am. Stat. Assoc., 76, 33 (1981).
58. S. Wasserman and P. Pattison, Psychometrika 0033-3123 10.1007/BF02294547 61, 401 (1996).
59. Y. Zhao, E. Levina, and J. Zhu, arXiv: 1110.3854.
60. We could easily use any of the other entropy expressions derived previously, to accommodate the diverse variants of the ensemble, which could be directed, multigraphs, etc. However, the use of the expressions derived for the "hard" degree constraints have a more limited validity, since it assumes stronger sparsity conditions. We focus therefore on ensembles with soft degree constraints, since they are more generally applicable.
61. This simple method can be very inefficient in certain cases, especially if the network is very large, since one may always finish in local maxima which are far away from the global optimum. We have also used the better variant know as the Kernighan-Lin algorithm, adapted to the blockmodel problem in Ref., which can escape such local solutions. However, for the simple examples considered here, we found almost no difference in the results.
62. N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, J. Chem. Phys. JCPSA6 0021-9606 10.1063/1.1699114 21, 1087 (1953).
63. W. K. Hastings, Biometrika BIOKAX 0006-3444 10.1093/biomet/57.1.97 57, 97 (1970).
64. R. Taylor, in Combinatorial Mathematics VIII, edited by, K. L. McAvaney, Vol. 884 (Springer, Berlin/Heidelberg, 1981), pp. 314-336.
65. R. B. Eggleton and D. A. Holton, in Combinatorial Mathematics VIII, edited by, K. L. McAvaney, Vol. 884 (Springer, Berlin/Heidelberg, 1981), pp. 155-172.
66. A. C. C. Coolen, A. Martino, and A. Annibale, J. Stat. Phys. JSTPBS 0022-4715 10.1007/s10955-009-9821-2 136, 1035 (2009).
67. r s. However, as mentioned in Sec., this ensemble is equivalent to the microcanonical version for sufficiently large samples.
68. An alternative which circumvents this problem is the so-called maximum a posteriori (MAP) approach, which uses parameter distributions, instead of a single set of parameters when maximizing the log-likelihood. Instead of the log-likelihood increasing monotonically with B, the parameter distributions become broader. This approach has been applied to the degree-corrected stochastic blockmodel in Ref., using belief propagation. This method, however, has the disadvantage of being numerically less efficient for large networks.
69. A. Treves and S. Panzeri, Neural Computation NEUCEB 0899-7667 10.1162/neco.1995.7.2.399 7, 399 (1995).
70. We note that Eq. should be understood only as a rule of thumb which gives a lower bound on the bias of L, since it is obtained only from the first term of Eq., and assumes that the number of blocks in each partition fluctuates independently, which is not likely to hold in general since the block partition is a result of an optimization algorithm.
71. G. Bianconi, P. Pin, and M. Marsili, Proceedings of the National Academy of Sciences PNASA6 0027-8424 10.1073/pnas.0811511106 106, 11433 (2009).
72. A. Lancichinetti, F. Radicchi, and J. J. Ramasco, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.81.046110 81, 046110 (2010).
73. F. Radicchi, A. Lancichinetti, and J. J. Ramasco, Phys. Rev. E PLEEE8 1539-3755 10.1103/PhysRevE.82.026102 82, 026102 (2010).
74. A. Lancichinetti, F. Radicchi, J. J. Ramasco, and S. Fortunato, PLoS ONE 1932-6203 10.1371/journal.pone.0018961 6, e18961 (2011).
75. P. Erdos and T. Gallai, Mat. Lapok. 11, 264 (1960).
76. C. I. Del Genio, T. Gross, and K. E. Bassler, Phys. Rev. Lett. PRLTAO 0031-9007 10.1103/PhysRevLett.107.178701 107, 178701 (2011).
77. B. Kernighan and S. Lin, Bell Syst. Tech. J. 49, 291 (1970).