Upgrade to Pro — share decks privately, control downloads, hide ads and more …

LASAGNE Multilayer Network Translation Framework

Matt J Williams
November 06, 2014
93

LASAGNE Multilayer Network Translation Framework

Presentation on the dataset translation tool built for the EU LASAGNE research project. Presented at Queen Mary University of London on 6 Nov 2014.

Matt J Williams

November 06, 2014
Tweet

Transcript

  1. A webapp and repository for multilayer networks
    LASAGNE Project Meeting
    6-7 Nov 2014, QMUL
    The LASAGNE Multilayer Network
    Translation Framework
    Matt Williams, Mirco Musolesi
    University of Birmingham

    View Slide

  2. LASAGNE Multilayer Data Tools
    • Common format for multilayer network
    data within LASAGNE
    • Common format for algorithms
    • Dissemination to broader multi* research
    community

    View Slide

  3. Outline
    • LASAGNE data formats recap
    • Dataset connectors
    • Translation webapp
    • Multilayer network repository

    View Slide

  4. LMF Formats
    Source Dataset
    Connector
    Metaformat Files
    Node List
    Layer List
    Intra-Layer Edge List
    Inter-Layer Edge List
    Aggregate
    Multiplex File
    LASAGNE Multilayer Network Framework
    (LMF)

    View Slide

  5. Metaformat Files
    • Node list
    • Layer list
    • Intra-layer edge list
    • Inter-layer edge list

    View Slide

  6. Metaformat Files

    L1
    L2

    View Slide

  7. Metaformat Files

    node
    id
    time
    stamp
    weight
    ... ... ...
    ... ... ...
    ... ... ...
    ... ... ...
    ... ... ...
    node list
    L1
    L2

    View Slide

  8. Metaformat Files

    layer
    id
    label/
    weight
    time
    stamp
    ... ... ...
    ... ... ...
    layer list
    L1
    L2

    View Slide

  9. Metaformat Files
    ↵ i j α t w
    intra-layer edge list
    ... ... ... ... ...
    ... ... ... ... ...
    ... ... ... ... ...
    ... ... ... ... ...
    ... ... ... ... ...
    ... ... ... ... ...
    one file per layer
    L1
    L2

    View Slide

  10. Metaformat Files

    inter-layer edge list
    i j α β t w
    ... ... ... ... ... ...
    ... ... ... ... ... ...
    ... ... ... ... ... ...
    ... ... ... ... ... ...
    ... ... ... ... ... ...
    L1
    L2

    View Slide

  11. LMF Aggregate File
    nodeID_i nodeID_j layerID_alpha layerID_beta timestamp weight
    w
    vi vj ↵ t
    ... ... ... ... ... ...
    • Single file representing whole multilayer
    network

    View Slide

  12. Dataset Connectors
    • Translates source data to LMF metaformat and
    aggregate
    • Output as either CSV files or MySQL database

    View Slide

  13. Implemented Connectors
    Noordin Top
    Terrorist Network
    Human Brain
    Connectivity
    Flight
    Routes
    Online
    Multiplayer
    Game
    45 subjects, obtained via CNRS
    Subset obtained via MUW

    View Slide

  14. Data Format Public Layers Nodes Weighted
    Time-
    varying
    Explicit
    Inter-Edges
    Terrorist Pajek ✔ 4 78 ✘ ✘ ✘
    Flights
    Open
    Flights
    ✔ 10 100 ✘ ✘ ✘
    Pardus Bespoke ✘ 4 2,058 ✘ ✘ ✘
    Brain Bespoke ✘ 2 90 ✔ ✘ ✘
    Implemented Connectors

    View Slide

  15. Command-Line Tools
    • 4x connectors
    • Export to MySQL and CSV file
    • Open source
    https://github.com/nsds/LMTF-Tools

    View Slide

  16. Translation Framework
    • Easy-to-use web tool for translating data sets
    • 4x connectors available through webapp

    View Slide

  17. LMTF Demo!

    View Slide

  18. Translation Framework
    Web Browser
    Web Server
    (Apache)
    Application
    Server
    (LMTF)
    Translation
    Working Storage
    Dataset
    Repository
    Job Queue &
    Dispatcher
    (beanstalk)
    http
    e-mail (smtp)
    • LMF data packaged as ZIP archive
    • Delivery via e-mail (URL to ZIP archive) or in-browser

    View Slide

  19. Translation Framework

    View Slide

  20. LMF Data Repository
    • Public repository of many multilayer
    network datasets, stored in LASAGNE data
    formats
    • Available as direct download
    • Current datasets:
    OpenFlights Routes
    Noordin Top Terrorist Networks

    View Slide

  21. Data Format Public Layers Nodes Weighted
    Time-
    varying
    Explicit
    Inter-Edges
    Terrorist Pajek ✔ 4 78 ✘ ✘ ✘
    Flights
    Open
    Flights
    ✔ 10 100 ✘ ✘ ✘
    Pardus Bespoke ✘ 4 2,058 ✘ ✘ ✘
    Brain Bespoke ✘ 2 90 ✔ ✘ ✘
    LMF Data Repository

    View Slide

  22. Data Format Public Layers Nodes Weighted
    Time-
    varying
    Explicit
    Inter-Edges
    Terrorist Pajek ✔ 4 78 ✘ ✘ ✘
    Flights
    Open
    Flights
    ✔ 10 100 ✘ ✘ ✘
    Pardus Bespoke ✘ 4 2,058 ✘ ✘ ✘
    Brain Bespoke ✘ 2 90 ✔ ✘ ✘
    LMF Data Repository

    View Slide

  23. Deployment
    • High-performance machine at UoB
    • 128GB RAM and 2TB working storage
    • High bandwidth
    http://pizza.cs.bham.ac.uk/lmtf

    View Slide

  24. LMTF Translation Demo!

    View Slide

  25. Outlook
    • LMF: LASAGNE Multilayer Network Format
    • LMTF: LASAGNE Multilayer Network Translation Framework
    ‣ Command line tools for translation
    ‣ Webapp for translation
    • Connectors so far – testing the framework
    • Open-source – encourage third party connectors
    • Ready for dissemination to community

    View Slide

  26. Opportunities
    • Call for more data sources!
    ‣ New connectors, for variety of types of multilayer dataset
    ‣ Especially: non-aligned, non-diagonal, time-varying, weighted
    ‣ Connectors for standard data formats:
    Gephi GEXF, ...
    • Expand multilayer network dataset repository
    • Suggestions very welcome! Propose a dataset, we’ll write a
    connector!
    • Collaborate on tools and algorithms for this format

    View Slide

  27. Translation framework:
    Command-line tools and connectors:
    http://pizza.cs.bham.ac.uk/lmtf
    https://github.com/nsds/LMTF-Tools
    https://github.com/nsds/LMTF
    Source code to be available

    View Slide

  28. A webapp and repository for multilayer networks
    The LASAGNE Multilayer Network
    Translation Framework
    Matt Williams, Mirco Musolesi
    University of Birmingham
    With thanks to...
    LMTF contributors: Victor Darvariu, Kevin Pomorski, Demet Turan
    Data providers: CNRS, MUW
    M.J.Williams
    @cs.bham.ac.uk

    View Slide

  29. Acknowledgements
    Brain connectivity diagram
    Yasser Iturria-Medina, et al. Brain Hemispheric Structural Efficiency and Interconnectivity
    Rightward Asymmetry in Human and Nonhuman Primates. 2011.
    [http://cercor.oxfordjournals.org/content/21/1/56]
    Flight paths visualisation
    Michael Markieta [http://www.arupconnect.com/2012/12/14/global-flight-paths/]
    Pizza slice
    [http://icons.mysitemyway.com/magic-marker-icons-food-beverage/]
    WWW click icon
    Freepik [http://www.flaticon.com/free-icon/world-wide-web_1198]

    View Slide