LASAGNE Multilayer Network Translation Framework

Presentation on the dataset translation tool built for the EU LASAGNE research project. Presented at Queen Mary University of London on 6 Nov 2014.

Transcript

1. A webapp and repository for multilayer networks
   LASAGNE Project Meeting
   6-7 Nov 2014, QMUL
   The LASAGNE Multilayer Network
   Translation Framework
   Matt Williams, Mirco Musolesi
   University of Birmingham
   

   View Slide

2. LASAGNE Multilayer Data Tools
   • Common format for multilayer network
   data within LASAGNE
   • Common format for algorithms
   • Dissemination to broader multi* research
   community
   

   View Slide

3. Outline
   • LASAGNE data formats recap
   • Dataset connectors
   • Translation webapp
   • Multilayer network repository
   

   View Slide

4. LMF Formats
   Source Dataset
   Connector
   Metaformat Files
   Node List
   Layer List
   Intra-Layer Edge List
   Inter-Layer Edge List
   Aggregate
   Multiplex File
   LASAGNE Multilayer Network Framework
   (LMF)
   

   View Slide

5. Metaformat Files
   • Node list
   • Layer list
   • Intra-layer edge list
   • Inter-layer edge list
   

   View Slide

6. Metaformat Files
   ↵
   L1
   L2
   

   View Slide

7. Metaformat Files
   ↵
   node
   id
   time
   stamp
   weight
   ... ... ...
   ... ... ...
   ... ... ...
   ... ... ...
   ... ... ...
   node list
   L1
   L2
   

   View Slide

8. Metaformat Files
   ↵
   layer
   id
   label/
   weight
   time
   stamp
   ... ... ...
   ... ... ...
   layer list
   L1
   L2
   

   View Slide

9. Metaformat Files
   ↵ i j α t w
   intra-layer edge list
   ... ... ... ... ...
   ... ... ... ... ...
   ... ... ... ... ...
   ... ... ... ... ...
   ... ... ... ... ...
   ... ... ... ... ...
   one file per layer
   L1
   L2
   

   View Slide

10. Metaformat Files
    ↵
    inter-layer edge list
    i j α β t w
    ... ... ... ... ... ...
    ... ... ... ... ... ...
    ... ... ... ... ... ...
    ... ... ... ... ... ...
    ... ... ... ... ... ...
    L1
    L2
    

    View Slide

11. LMF Aggregate File
    nodeID_i nodeID_j layerID_alpha layerID_beta timestamp weight
    w
    vi vj ↵ t
    ... ... ... ... ... ...
    • Single file representing whole multilayer
    network
    

    View Slide

12. Dataset Connectors
    • Translates source data to LMF metaformat and
    aggregate
    • Output as either CSV files or MySQL database
    

    View Slide

13. Implemented Connectors
    Noordin Top
    Terrorist Network
    Human Brain
    Connectivity
    Flight
    Routes
    Online
    Multiplayer
    Game
    45 subjects, obtained via CNRS
    Subset obtained via MUW
    

    View Slide

14. Data Format Public Layers Nodes Weighted
    Time-
    varying
    Explicit
    Inter-Edges
    Terrorist Pajek ✔ 4 78 ✘ ✘ ✘
    Flights
    Open
    Flights
    ✔ 10 100 ✘ ✘ ✘
    Pardus Bespoke ✘ 4 2,058 ✘ ✘ ✘
    Brain Bespoke ✘ 2 90 ✔ ✘ ✘
    Implemented Connectors
    

    View Slide

15. Command-Line Tools
    • 4x connectors
    • Export to MySQL and CSV file
    • Open source
    https://github.com/nsds/LMTF-Tools
    

    View Slide

16. Translation Framework
    • Easy-to-use web tool for translating data sets
    • 4x connectors available through webapp
    

    View Slide

17. LMTF Demo!
    

    View Slide

18. Translation Framework
    Web Browser
    Web Server
    (Apache)
    Application
    Server
    (LMTF)
    Translation
    Working Storage
    Dataset
    Repository
    Job Queue &
    Dispatcher
    (beanstalk)
    http
    e-mail (smtp)
    • LMF data packaged as ZIP archive
    • Delivery via e-mail (URL to ZIP archive) or in-browser
    

    View Slide

19. Translation Framework
    

    View Slide

20. LMF Data Repository
    • Public repository of many multilayer
    network datasets, stored in LASAGNE data
    formats
    • Available as direct download
    • Current datasets:
    OpenFlights Routes
    Noordin Top Terrorist Networks
    

    View Slide

21. Data Format Public Layers Nodes Weighted
    Time-
    varying
    Explicit
    Inter-Edges
    Terrorist Pajek ✔ 4 78 ✘ ✘ ✘
    Flights
    Open
    Flights
    ✔ 10 100 ✘ ✘ ✘
    Pardus Bespoke ✘ 4 2,058 ✘ ✘ ✘
    Brain Bespoke ✘ 2 90 ✔ ✘ ✘
    LMF Data Repository
    

    View Slide

22. Data Format Public Layers Nodes Weighted
    Time-
    varying
    Explicit
    Inter-Edges
    Terrorist Pajek ✔ 4 78 ✘ ✘ ✘
    Flights
    Open
    Flights
    ✔ 10 100 ✘ ✘ ✘
    Pardus Bespoke ✘ 4 2,058 ✘ ✘ ✘
    Brain Bespoke ✘ 2 90 ✔ ✘ ✘
    LMF Data Repository
    

    View Slide

23. Deployment
    • High-performance machine at UoB
    • 128GB RAM and 2TB working storage
    • High bandwidth
    http://pizza.cs.bham.ac.uk/lmtf
    

    View Slide

24. LMTF Translation Demo!
    

    View Slide

25. Outlook
    • LMF: LASAGNE Multilayer Network Format
    • LMTF: LASAGNE Multilayer Network Translation Framework
    ‣ Command line tools for translation
    ‣ Webapp for translation
    • Connectors so far – testing the framework
    • Open-source – encourage third party connectors
    • Ready for dissemination to community
    

    View Slide

26. Opportunities
    • Call for more data sources!
    ‣ New connectors, for variety of types of multilayer dataset
    ‣ Especially: non-aligned, non-diagonal, time-varying, weighted
    ‣ Connectors for standard data formats:
    Gephi GEXF, ...
    • Expand multilayer network dataset repository
    • Suggestions very welcome! Propose a dataset, we’ll write a
    connector!
    • Collaborate on tools and algorithms for this format
    

    View Slide

27. Translation framework:
    Command-line tools and connectors:
    http://pizza.cs.bham.ac.uk/lmtf
    https://github.com/nsds/LMTF-Tools
    https://github.com/nsds/LMTF
    Source code to be available
    

    View Slide

28. A webapp and repository for multilayer networks
    The LASAGNE Multilayer Network
    Translation Framework
    Matt Williams, Mirco Musolesi
    University of Birmingham
    With thanks to...
    LMTF contributors: Victor Darvariu, Kevin Pomorski, Demet Turan
    Data providers: CNRS, MUW
    M.J.Williams
    @cs.bham.ac.uk
    

    View Slide

29. Acknowledgements
    Brain connectivity diagram
    Yasser Iturria-Medina, et al. Brain Hemispheric Structural Efficiency and Interconnectivity
    Rightward Asymmetry in Human and Nonhuman Primates. 2011.
    [http://cercor.oxfordjournals.org/content/21/1/56]
    Flight paths visualisation
    Michael Markieta [http://www.arupconnect.com/2012/12/14/global-flight-paths/]
    Pizza slice
    [http://icons.mysitemyway.com/magic-marker-icons-food-beverage/]
    WWW click icon
    Freepik [http://www.flaticon.com/free-icon/world-wide-web_1198]
    

    View Slide