On the Universality of the Stochastic Block Model

Presented at the SYNS Warm-up event of NetSci 2019 (https://www.networkscienceinstitute.org/syns).

Paper: https://journals.aps.org/pre/abstract/10.1103/PhysRevE.98.032309
arXiv: https://arxiv.org/abs/1806.04214
Code: https://github.com/jg-you/sbm_canonical_mcmc

Abstract
========
Mesoscopic pattern extraction (MPE) is the problem of finding a partition of the nodes of a complex network that maximizes some objective function. Many well-known network inference problems fall in this category, including, for instance, community detection, core-periphery identification, and imperfect graph coloring. In this paper, we show that the most popular algorithms designed to solve MPE problems can in fact be understood as special cases of the maximum likelihood formulation of the stochastic block model (SBM) or one of its direct generalizations. These equivalence relations show that the SBM is nearly universal with respect to MPE problems.

Transcript

1. O U
   S B
   SYNS : I’ ...
   Jean-Gabriel Young
   Center for the Study of Complex Systems, University of Michigan, Ann Arbor, MI, USA
   jgyoung.ca @_jgyou
   May th,
   Based on joint work with Guillaume St-Onge, Patrick Desrosiers, Louis J. Dubé
   

   View Slide

2. “I made this...” projects
   

   View Slide

3. “I made this...” projects Marie Kondo projects
   

   View Slide

4. Political Books co-purchase network
   V. Krebs, unpublished, http://www.orgnet.com/
   

   View Slide

5. Political Books co-purchase network
   V. Krebs, unpublished, http://www.orgnet.com/
   

   View Slide

6. Political Books co-purchase network
   V. Krebs, unpublished, http://www.orgnet.com/
   

   View Slide

7. Political Books co-purchase network
   V. Krebs, unpublished, http://www.orgnet.com/
   

   View Slide

8. Political Books co-purchase network
   V. Krebs, unpublished, http://www.orgnet.com/
   

   View Slide

9. Political Books co-purchase network
   V. Krebs, unpublished, http://www.orgnet.com/
   

   View Slide

10. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/
    

    View Slide

11. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/
    

    View Slide

12. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/
    

    View Slide

13. Our goal :
    Organize MPE algorithms.
    

    View Slide

14. Mesoscopic pattern extraction (MPE) with objective functions
    MPE
    Algorithm
    

    View Slide

15. Mesoscopic pattern extraction (MPE) with objective functions
    What truly matters : the objective function.
    Explore
    Partition Space
    Objective
    Function
    E : Propose partitions σ1
    , σ2
    , ...
    O : Order partitions H(σo
    ) ≥ H(σi
    ) ∀i
    

    View Slide

16. A large class of MPE objectives
    G
    The class of objective functions that can be decomposed as
    H(σ, A; f )
    (ij)
    f (aij
    , , σi
    , σj
    ),
    with f (aij
    , σi
    , σj
    ) a score function for node pair (i, j).
    f (1 , , )
    f (0, , )
    2
    1
    3
    W.W. Zachary, J. Anthropol. Res. ( ), http://www-persona .umich.edu/∼mejn/netdata/
    

    View Slide

17. Two-part algorithms in practice
    Core-periphery
    Greedy (Kernighan-Lin type)
    

    View Slide

18. Two-part algorithms in practice
    Core-periphery
    Greedy (Kernighan-Lin type)
    Modularity
    Greedy (Kernighan-Lin type)
    

    View Slide

19. Two-part algorithms in practice
    Core-periphery
    Greedy (Kernighan-Lin type)
    Modularity
    Greedy (Kernighan-Lin type)
    Spectral + Gaussian Mixture
    

    View Slide

20. Our goal :
    Organize MPE algorithms
    

    View Slide

21. Our goal :
    Organize MPE algorithms
    using their objective functions
    

    View Slide

22. Organization tools
    relationships to organize objective functions
    

    View Slide

23. Organization tools
    relationships to organize objective functions
    E
    H1
    ∼ H2
    Order partitions σA
    , σB the
    same way
    

    View Slide

24. Organization tools
    relationships to organize objective functions
    E
    H1
    ∼ H2
    Order partitions σA
    , σB the
    same way
    S
    H1
    ⇓
    H2
    Restrict the “expressiveness”
    of H1 to get H2.
    

    View Slide

25. We’ll use equivalence and specialization to construct a hierarchy.
    

    View Slide

26. Stochastic Block Model
    Near the top of the hierarchy sits...
    

    View Slide

27. Stochastic Block Model
    Near the top of the hierarchy sits...
    T A
    Generative model that creates network based on a hidden
    partition σ, a density matrix ω
    P(A|σ, ω)
    ij
    P(Aij
    |σi
    , σj
    , ω)
    ij
    (ωσi
    σj
    )Aij
    Aij
    e−ωσi σj
    2
    1
    3
    ω
    

    View Slide

28. Stochastic Block Model
    To find communities we : maximize the likelihood w.r.t. to σ.
    L SBM
    P(A|σ, ω)
    ij
    (ωσi
    σj
    )Aij
    Aij
    e−ωσi σj
    

    View Slide

29. Stochastic Block Model
    To find communities we : maximize the likelihood w.r.t. to σ.
    L SBM
    P(A|σ, ω)
    ij
    (ωσi
    σj
    )Aij
    Aij
    e−ωσi σj
    Since argmax f (x) argmax log f (x), we can also use
    log P(σ|A, ω) ∼
    ij
    [Aij log ωσi
    σj
    − ωσi
    σj
    ]
    

    View Slide

30. Stochastic Block Model
    To find communities we : maximize the likelihood w.r.t. to σ.
    L SBM
    P(A|σ, ω)
    ij
    (ωσi
    σj
    )Aij
    Aij
    e−ωσi σj
    Since argmax f (x) argmax log f (x), we can also use
    log P(σ|A, ω) ∼
    ij
    [Aij log ωσi
    σj
    − ωσi
    σj
    ]
    But! That’s suspiciously close to :
    H(σ, A; f )
    (ij)
    f (aij
    , σi
    , σj
    )
    

    View Slide

31. By restricting ω : we get a whole hierarchy!
    

    View Slide

32. By restricting ω : we get a whole hierarchy!
    Max. likelihood of the SISBM
    aij
    log ωσiσj
    − ωσiσj
    λij
    ωσiσj
    ≥ 0, λij ≥ 0
    Max. likelihood of the SBM
    aij
    log ωσiσj
    − ωσiσj
    ωσiσj
    ≥ 0
    Max. likelihood of the SIGMGM
    xσiσj
    aij
    + γλij
    λij ≥ 0, xσiσj
    ∈ {0, 1}, γ < 0
    General modularities
    xσiσj
    aij
    + γλij
    λij ≥ 0, γ < 0
    Modularities
    δσiσj
    aij
    + γλij
    λij ≥ 0, γ < 0
    Max. likelihood of the GMGM
    xσiσj
    aij
    + γ
    xσiσj
    ∈ {0, 1}, γ < 0
    Edge counts with
    quadratic size constraints
    xσiσj
    aij
    + γ
    xσiσj
    ∈ {0, 1}, γ < 0
    Balanced cut
    δσiσj
    aij
    + γ
    γ < 0
    Balanced coloring
    (1 − δσiσj
    ) aij
    + γ
    γ < 0
    Core-periphery
    e.g.: δσi1
    δσj 1
    aij
    + γ
    γ < 0
    *we actually need to use a more general SBM
    

    View Slide

33. The Stochastic Block Model is
    in that it is at the top of the hierarchy
    

    View Slide

34. Why does this matter?
    . S ! There is good code that will fit the SBM.
    

    View Slide

35. Why does this matter?
    . S ! There is good code that will fit the SBM.
    . S MPE ! You can see them as fitting a
    model -> statistically principled.
    

    View Slide

36. Why does this matter?
    . S ! There is good code that will fit the SBM.
    . S MPE ! You can see them as fitting a
    model -> statistically principled.
    . D MPE ? They are fitting ill-defined
    models.
    

    View Slide

37. Why does this matter?
    . S ! There is good code that will fit the SBM.
    . S MPE ! You can see them as fitting a
    model -> statistically principled.
    . D MPE ? They are fitting ill-defined
    models.
    . D ’ ! If algorithm A and B find the same
    partition, not necessarily a sign of consensus.
    

    View Slide

38. Why does this matter?
    . S ! There is good code that will fit the SBM.
    . S MPE ! You can see them as fitting a
    model -> statistically principled.
    . D MPE ? They are fitting ill-defined
    models.
    . D ’ ! If algorithm A and B find the same
    partition, not necessarily a sign of consensus.
    . T ! Trivial reduction for NP-hardness.
    Detectability is a general phenomenon. And more?
    

    View Slide

39. Further reading
    O
    ( ) J.-G. Young, . St-Onge and P. Desrosiers, Louis J. Dubé
    Phys. Rev. E, ( )
    Inspiration / related work
    ( ) J. Reichardt and S. Bornholdt, Phys. Rev. E ( )
    ( ) M.E.J Newman, Phys. Rev. E ( )
    ( ) S. C. Olhede and P. J. Wolfe, PNAS , ( )
    

    View Slide

40. Take-home message
    Objective function = important part of MPE algorithms
    Organization : via equivalence and specialization
    Max. likelihood SBM is almost universal as MPE goes
    Reference : Phys. Rev. E, ( )
    Software : github.com/jg-you/sbm_canonica _mcmc
    

    View Slide

41. Reference : Phys. Rev. E, ( )
    arXiv : 1806.04214
    jgyou@umich.edu jgyoung.ca @_jgyou
    

    View Slide