Upgrade to Pro — share decks privately, control downloads, hide ads and more …

On the Universality of the Stochastic Block Model

On the Universality of the Stochastic Block Model

Presented at the SYNS Warm-up event of NetSci 2019 (https://www.networkscienceinstitute.org/syns).

Paper: https://journals.aps.org/pre/abstract/10.1103/PhysRevE.98.032309
arXiv: https://arxiv.org/abs/1806.04214
Code: https://github.com/jg-you/sbm_canonical_mcmc

Abstract
========
Mesoscopic pattern extraction (MPE) is the problem of finding a partition of the nodes of a complex network that maximizes some objective function. Many well-known network inference problems fall in this category, including, for instance, community detection, core-periphery identification, and imperfect graph coloring. In this paper, we show that the most popular algorithms designed to solve MPE problems can in fact be understood as special cases of the maximum likelihood formulation of the stochastic block model (SBM) or one of its direct generalizations. These equivalence relations show that the SBM is nearly universal with respect to MPE problems.

Jean-Gabriel Young

May 26, 2019
Tweet

More Decks by Jean-Gabriel Young

Other Decks in Science

Transcript

  1. O U
    S B
    SYNS : I’ ...
    Jean-Gabriel Young
    Center for the Study of Complex Systems, University of Michigan, Ann Arbor, MI, USA
    jgyoung.ca @_jgyou
    May th,
    Based on joint work with Guillaume St-Onge, Patrick Desrosiers, Louis J. Dubé

    View full-size slide

  2. “I made this...” projects

    View full-size slide

  3. “I made this...” projects Marie Kondo projects

    View full-size slide

  4. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/

    View full-size slide

  5. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/

    View full-size slide

  6. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/

    View full-size slide

  7. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/

    View full-size slide

  8. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/

    View full-size slide

  9. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/

    View full-size slide

  10. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/

    View full-size slide

  11. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/

    View full-size slide

  12. Political Books co-purchase network
    V. Krebs, unpublished, http://www.orgnet.com/

    View full-size slide

  13. Our goal :
    Organize MPE algorithms.

    View full-size slide

  14. Mesoscopic pattern extraction (MPE) with objective functions
    MPE
    Algorithm

    View full-size slide

  15. Mesoscopic pattern extraction (MPE) with objective functions
    What truly matters : the objective function.
    Explore
    Partition Space
    Objective
    Function
    E : Propose partitions σ1
    , σ2
    , ...
    O : Order partitions H(σo
    ) ≥ H(σi
    ) ∀i

    View full-size slide

  16. A large class of MPE objectives
    G
    The class of objective functions that can be decomposed as
    H(σ, A; f )
    (ij)
    f (aij
    , , σi
    , σj
    ),
    with f (aij
    , σi
    , σj
    ) a score function for node pair (i, j).
    f (1 , , )
    f (0, , )
    2
    1
    3
    W.W. Zachary, J. Anthropol. Res. ( ), http://www-persona .umich.edu/∼mejn/netdata/

    View full-size slide

  17. Two-part algorithms in practice
    Core-periphery
    Greedy (Kernighan-Lin type)

    View full-size slide

  18. Two-part algorithms in practice
    Core-periphery
    Greedy (Kernighan-Lin type)
    Modularity
    Greedy (Kernighan-Lin type)

    View full-size slide

  19. Two-part algorithms in practice
    Core-periphery
    Greedy (Kernighan-Lin type)
    Modularity
    Greedy (Kernighan-Lin type)
    Spectral + Gaussian Mixture

    View full-size slide

  20. Our goal :
    Organize MPE algorithms

    View full-size slide

  21. Our goal :
    Organize MPE algorithms
    using their objective functions

    View full-size slide

  22. Organization tools
    relationships to organize objective functions

    View full-size slide

  23. Organization tools
    relationships to organize objective functions
    E
    H1
    ∼ H2
    Order partitions σA
    , σB the
    same way

    View full-size slide

  24. Organization tools
    relationships to organize objective functions
    E
    H1
    ∼ H2
    Order partitions σA
    , σB the
    same way
    S
    H1

    H2
    Restrict the “expressiveness”
    of H1 to get H2.

    View full-size slide

  25. We’ll use equivalence and specialization to construct a hierarchy.

    View full-size slide

  26. Stochastic Block Model
    Near the top of the hierarchy sits...

    View full-size slide

  27. Stochastic Block Model
    Near the top of the hierarchy sits...
    T A
    Generative model that creates network based on a hidden
    partition σ, a density matrix ω
    P(A|σ, ω)
    ij
    P(Aij
    |σi
    , σj
    , ω)
    ij
    (ωσi
    σj
    )Aij
    Aij
    e−ωσi σj
    2
    1
    3
    ω

    View full-size slide

  28. Stochastic Block Model
    To find communities we : maximize the likelihood w.r.t. to σ.
    L SBM
    P(A|σ, ω)
    ij
    (ωσi
    σj
    )Aij
    Aij
    e−ωσi σj

    View full-size slide

  29. Stochastic Block Model
    To find communities we : maximize the likelihood w.r.t. to σ.
    L SBM
    P(A|σ, ω)
    ij
    (ωσi
    σj
    )Aij
    Aij
    e−ωσi σj
    Since argmax f (x) argmax log f (x), we can also use
    log P(σ|A, ω) ∼
    ij
    [Aij log ωσi
    σj
    − ωσi
    σj
    ]

    View full-size slide

  30. Stochastic Block Model
    To find communities we : maximize the likelihood w.r.t. to σ.
    L SBM
    P(A|σ, ω)
    ij
    (ωσi
    σj
    )Aij
    Aij
    e−ωσi σj
    Since argmax f (x) argmax log f (x), we can also use
    log P(σ|A, ω) ∼
    ij
    [Aij log ωσi
    σj
    − ωσi
    σj
    ]
    But! That’s suspiciously close to :
    H(σ, A; f )
    (ij)
    f (aij
    , σi
    , σj
    )

    View full-size slide

  31. By restricting ω : we get a whole hierarchy!

    View full-size slide

  32. By restricting ω : we get a whole hierarchy!
    Max. likelihood of the SISBM
    aij
    log ωσiσj
    − ωσiσj
    λij
    ωσiσj
    ≥ 0, λij ≥ 0
    Max. likelihood of the SBM
    aij
    log ωσiσj
    − ωσiσj
    ωσiσj
    ≥ 0
    Max. likelihood of the SIGMGM
    xσiσj
    aij
    + γλij
    λij ≥ 0, xσiσj
    ∈ {0, 1}, γ < 0
    General modularities
    xσiσj
    aij
    + γλij
    λij ≥ 0, γ < 0
    Modularities
    δσiσj
    aij
    + γλij
    λij ≥ 0, γ < 0
    Max. likelihood of the GMGM
    xσiσj
    aij
    + γ
    xσiσj
    ∈ {0, 1}, γ < 0
    Edge counts with
    quadratic size constraints
    xσiσj
    aij
    + γ
    xσiσj
    ∈ {0, 1}, γ < 0
    Balanced cut
    δσiσj
    aij
    + γ
    γ < 0
    Balanced coloring
    (1 − δσiσj
    ) aij
    + γ
    γ < 0
    Core-periphery
    e.g.: δσi1
    δσj 1
    aij
    + γ
    γ < 0
    *we actually need to use a more general SBM

    View full-size slide

  33. The Stochastic Block Model is
    in that it is at the top of the hierarchy

    View full-size slide

  34. Why does this matter?
    . S ! There is good code that will fit the SBM.

    View full-size slide

  35. Why does this matter?
    . S ! There is good code that will fit the SBM.
    . S MPE ! You can see them as fitting a
    model -> statistically principled.

    View full-size slide

  36. Why does this matter?
    . S ! There is good code that will fit the SBM.
    . S MPE ! You can see them as fitting a
    model -> statistically principled.
    . D MPE ? They are fitting ill-defined
    models.

    View full-size slide

  37. Why does this matter?
    . S ! There is good code that will fit the SBM.
    . S MPE ! You can see them as fitting a
    model -> statistically principled.
    . D MPE ? They are fitting ill-defined
    models.
    . D ’ ! If algorithm A and B find the same
    partition, not necessarily a sign of consensus.

    View full-size slide

  38. Why does this matter?
    . S ! There is good code that will fit the SBM.
    . S MPE ! You can see them as fitting a
    model -> statistically principled.
    . D MPE ? They are fitting ill-defined
    models.
    . D ’ ! If algorithm A and B find the same
    partition, not necessarily a sign of consensus.
    . T ! Trivial reduction for NP-hardness.
    Detectability is a general phenomenon. And more?

    View full-size slide

  39. Further reading
    O
    ( ) J.-G. Young, . St-Onge and P. Desrosiers, Louis J. Dubé
    Phys. Rev. E, ( )
    Inspiration / related work
    ( ) J. Reichardt and S. Bornholdt, Phys. Rev. E ( )
    ( ) M.E.J Newman, Phys. Rev. E ( )
    ( ) S. C. Olhede and P. J. Wolfe, PNAS , ( )

    View full-size slide

  40. Take-home message
    Objective function = important part of MPE algorithms
    Organization : via equivalence and specialization
    Max. likelihood SBM is almost universal as MPE goes
    Reference : Phys. Rev. E, ( )
    Software : github.com/jg-you/sbm_canonica _mcmc

    View full-size slide

  41. Reference : Phys. Rev. E, ( )
    arXiv : 1806.04214
    [email protected] jgyoung.ca @_jgyou

    View full-size slide