On the Universality of the Stochastic Block Model

Presented at the SYNS Warm-up event of NetSci 2019 (https://www.networkscienceinstitute.org/syns).

Paper: https://journals.aps.org/pre/abstract/10.1103/PhysRevE.98.032309
arXiv: https://arxiv.org/abs/1806.04214
Code: https://github.com/jg-you/sbm_canonical_mcmc

Abstract
========
Mesoscopic pattern extraction (MPE) is the problem of finding a partition of the nodes of a complex network that maximizes some objective function. Many well-known network inference problems fall in this category, including, for instance, community detection, core-periphery identification, and imperfect graph coloring. In this paper, we show that the most popular algorithms designed to solve MPE problems can in fact be understood as special cases of the maximum likelihood formulation of the stochastic block model (SBM) or one of its direct generalizations. These equivalence relations show that the SBM is nearly universal with respect to MPE problems.

Transcript

1. O U S B SYNS : I’ ... Jean-Gabriel Young Center for the Study of Complex Systems, University of Michigan, Ann Arbor, MI, USA jgyoung.ca @_jgyou May th, Based on joint work with Guillaume St-Onge, Patrick Desrosiers, Louis J. Dubé
2. “I made this...” projects
3. “I made this...” projects Marie Kondo projects
4. Political Books co-purchase network V. Krebs, unpublished, http://www.orgnet.com/
5. Political Books co-purchase network V. Krebs, unpublished, http://www.orgnet.com/
6. Political Books co-purchase network V. Krebs, unpublished, http://www.orgnet.com/
7. Political Books co-purchase network V. Krebs, unpublished, http://www.orgnet.com/
8. Political Books co-purchase network V. Krebs, unpublished, http://www.orgnet.com/
9. Political Books co-purchase network V. Krebs, unpublished, http://www.orgnet.com/
10. Political Books co-purchase network V. Krebs, unpublished, http://www.orgnet.com/
11. Political Books co-purchase network V. Krebs, unpublished, http://www.orgnet.com/
12. Political Books co-purchase network V. Krebs, unpublished, http://www.orgnet.com/
13. Our goal : Organize MPE algorithms.
14. Mesoscopic pattern extraction (MPE) with objective functions MPE Algorithm
15. Mesoscopic pattern extraction (MPE) with objective functions What truly matters : the objective function. Explore Partition Space Objective Function E : Propose partitions σ1 , σ2 , ... O : Order partitions H(σo ) ≥ H(σi ) ∀i
16. A large class of MPE objectives G The class of objective functions that can be decomposed as H(σ, A; f ) (ij) f (aij , , σi , σj ), with f (aij , σi , σj ) a score function for node pair (i, j). f (1 , , ) f (0, , ) 2 1 3 W.W. Zachary, J. Anthropol. Res. ( ), http://www-persona .umich.edu/∼mejn/netdata/
17. Two-part algorithms in practice Core-periphery Greedy (Kernighan-Lin type)
18. Two-part algorithms in practice Core-periphery Greedy (Kernighan-Lin type) Modularity Greedy (Kernighan-Lin type)
19. Two-part algorithms in practice Core-periphery Greedy (Kernighan-Lin type) Modularity Greedy (Kernighan-Lin type) Spectral + Gaussian Mixture
20. Our goal : Organize MPE algorithms
21. Our goal : Organize MPE algorithms using their objective functions
22. Organization tools relationships to organize objective functions
23. Organization tools relationships to organize objective functions E H1 ∼ H2 Order partitions σA , σB the same way
24. Organization tools relationships to organize objective functions E H1 ∼ H2 Order partitions σA , σB the same way S H1 ⇓ H2 Restrict the “expressiveness” of H1 to get H2.
25. We’ll use equivalence and specialization to construct a hierarchy.
26. Stochastic Block Model Near the top of the hierarchy sits...
27. Stochastic Block Model Near the top of the hierarchy sits... T A Generative model that creates network based on a hidden partition σ, a density matrix ω P(A|σ, ω) ij P(Aij |σi , σj , ω) ij (ωσi σj )Aij Aij e−ωσi σj 2 1 3 ω
28. Stochastic Block Model To find communities we : maximize the likelihood w.r.t. to σ. L SBM P(A|σ, ω) ij (ωσi σj )Aij Aij e−ωσi σj
29. Stochastic Block Model To find communities we : maximize the likelihood w.r.t. to σ. L SBM P(A|σ, ω) ij (ωσi σj )Aij Aij e−ωσi σj Since argmax f (x) argmax log f (x), we can also use log P(σ|A, ω) ∼ ij [Aij log ωσi σj − ωσi σj ]
30. Stochastic Block Model To find communities we : maximize the likelihood w.r.t. to σ. L SBM P(A|σ, ω) ij (ωσi σj )Aij Aij e−ωσi σj Since argmax f (x) argmax log f (x), we can also use log P(σ|A, ω) ∼ ij [Aij log ωσi σj − ωσi σj ] But! That’s suspiciously close to : H(σ, A; f ) (ij) f (aij , σi , σj )
31. By restricting ω : we get a whole hierarchy!
32. By restricting ω : we get a whole hierarchy! Max. likelihood of the SISBM aij log ωσiσj − ωσiσj λij ωσiσj ≥ 0, λij ≥ 0 Max. likelihood of the SBM aij log ωσiσj − ωσiσj ωσiσj ≥ 0 Max. likelihood of the SIGMGM xσiσj aij + γλij λij ≥ 0, xσiσj ∈ {0, 1}, γ < 0 General modularities xσiσj aij + γλij λij ≥ 0, γ < 0 Modularities δσiσj aij + γλij λij ≥ 0, γ < 0 Max. likelihood of the GMGM xσiσj aij + γ xσiσj ∈ {0, 1}, γ < 0 Edge counts with quadratic size constraints xσiσj aij + γ xσiσj ∈ {0, 1}, γ < 0 Balanced cut δσiσj aij + γ γ < 0 Balanced coloring (1 − δσiσj ) aij + γ γ < 0 Core-periphery e.g.: δσi1 δσj 1 aij + γ γ < 0 *we actually need to use a more general SBM
33. The Stochastic Block Model is in that it is at the top of the hierarchy
34. Why does this matter? . S ! There is good code that will fit the SBM.
35. Why does this matter? . S ! There is good code that will fit the SBM. . S MPE ! You can see them as fitting a model -> statistically principled.
36. Why does this matter? . S ! There is good code that will fit the SBM. . S MPE ! You can see them as fitting a model -> statistically principled. . D MPE ? They are fitting ill-defined models.
37. Why does this matter? . S ! There is good code that will fit the SBM. . S MPE ! You can see them as fitting a model -> statistically principled. . D MPE ? They are fitting ill-defined models. . D ’ ! If algorithm A and B find the same partition, not necessarily a sign of consensus.
38. Why does this matter? . S ! There is good code that will fit the SBM. . S MPE ! You can see them as fitting a model -> statistically principled. . D MPE ? They are fitting ill-defined models. . D ’ ! If algorithm A and B find the same partition, not necessarily a sign of consensus. . T ! Trivial reduction for NP-hardness. Detectability is a general phenomenon. And more?
39. Further reading O ( ) J.-G. Young, . St-Onge and P. Desrosiers, Louis J. Dubé Phys. Rev. E, ( ) Inspiration / related work ( ) J. Reichardt and S. Bornholdt, Phys. Rev. E ( ) ( ) M.E.J Newman, Phys. Rev. E ( ) ( ) S. C. Olhede and P. J. Wolfe, PNAS , ( )
40. Take-home message Objective function = important part of MPE algorithms Organization : via equivalence and specialization Max. likelihood SBM is almost universal as MPE goes Reference : Phys. Rev. E, ( ) Software : github.com/jg-you/sbm_canonica _mcmc
41. Reference : Phys. Rev. E, ( ) arXiv : 1806.04214 jgyou@umich.edu jgyoung.ca @_jgyou