LASAGNE Multilayer Network Translation Framework

Presentation on the dataset translation tool built for the EU LASAGNE research project. Presented at Queen Mary University of London on 6 Nov 2014.

Transcript

1. A webapp and repository for multilayer networks LASAGNE Project Meeting 6-7 Nov 2014, QMUL The LASAGNE Multilayer Network Translation Framework Matt Williams, Mirco Musolesi University of Birmingham
2. LASAGNE Multilayer Data Tools • Common format for multilayer network data within LASAGNE • Common format for algorithms • Dissemination to broader multi* research community
3. Outline • LASAGNE data formats recap • Dataset connectors • Translation webapp • Multilayer network repository
4. LMF Formats Source Dataset Connector Metaformat Files Node List Layer List Intra-Layer Edge List Inter-Layer Edge List Aggregate Multiplex File LASAGNE Multilayer Network Framework (LMF)
5. Metaformat Files • Node list • Layer list • Intra-layer edge list • Inter-layer edge list
6. Metaformat Files ↵ L1 L2
7. Metaformat Files ↵ node id time stamp weight ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... node list L1 L2
8. Metaformat Files ↵ layer id label/ weight time stamp ... ... ... ... ... ... layer list L1 L2
9. Metaformat Files ↵ i j α t w intra-layer edge list ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... one file per layer L1 L2
10. Metaformat Files ↵ inter-layer edge list i j α β t w ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... L1 L2
11. LMF Aggregate File nodeID_i nodeID_j layerID_alpha layerID_beta timestamp weight w vi vj ↵ t ... ... ... ... ... ... • Single file representing whole multilayer network
12. Dataset Connectors • Translates source data to LMF metaformat and aggregate • Output as either CSV files or MySQL database
13. Implemented Connectors Noordin Top Terrorist Network Human Brain Connectivity Flight Routes Online Multiplayer Game 45 subjects, obtained via CNRS Subset obtained via MUW
14. Data Format Public Layers Nodes Weighted Time- varying Explicit Inter-Edges Terrorist Pajek ✔ 4 78 ✘ ✘ ✘ Flights Open Flights ✔ 10 100 ✘ ✘ ✘ Pardus Bespoke ✘ 4 2,058 ✘ ✘ ✘ Brain Bespoke ✘ 2 90 ✔ ✘ ✘ Implemented Connectors
15. Command-Line Tools • 4x connectors • Export to MySQL and CSV file • Open source https://github.com/nsds/LMTF-Tools
16. Translation Framework • Easy-to-use web tool for translating data sets • 4x connectors available through webapp
17. LMTF Demo!
18. Translation Framework Web Browser Web Server (Apache) Application Server (LMTF) Translation Working Storage Dataset Repository Job Queue & Dispatcher (beanstalk) http e-mail (smtp) • LMF data packaged as ZIP archive • Delivery via e-mail (URL to ZIP archive) or in-browser
19. Translation Framework
20. LMF Data Repository • Public repository of many multilayer network datasets, stored in LASAGNE data formats • Available as direct download • Current datasets: OpenFlights Routes Noordin Top Terrorist Networks
21. Data Format Public Layers Nodes Weighted Time- varying Explicit Inter-Edges Terrorist Pajek ✔ 4 78 ✘ ✘ ✘ Flights Open Flights ✔ 10 100 ✘ ✘ ✘ Pardus Bespoke ✘ 4 2,058 ✘ ✘ ✘ Brain Bespoke ✘ 2 90 ✔ ✘ ✘ LMF Data Repository
22. Data Format Public Layers Nodes Weighted Time- varying Explicit Inter-Edges Terrorist Pajek ✔ 4 78 ✘ ✘ ✘ Flights Open Flights ✔ 10 100 ✘ ✘ ✘ Pardus Bespoke ✘ 4 2,058 ✘ ✘ ✘ Brain Bespoke ✘ 2 90 ✔ ✘ ✘ LMF Data Repository
23. Deployment • High-performance machine at UoB • 128GB RAM and 2TB working storage • High bandwidth http://pizza.cs.bham.ac.uk/lmtf
24. LMTF Translation Demo!
25. Outlook • LMF: LASAGNE Multilayer Network Format • LMTF: LASAGNE Multilayer Network Translation Framework ‣ Command line tools for translation ‣ Webapp for translation • Connectors so far – testing the framework • Open-source – encourage third party connectors • Ready for dissemination to community
26. Opportunities • Call for more data sources! ‣ New connectors, for variety of types of multilayer dataset ‣ Especially: non-aligned, non-diagonal, time-varying, weighted ‣ Connectors for standard data formats: Gephi GEXF, ... • Expand multilayer network dataset repository • Suggestions very welcome! Propose a dataset, we’ll write a connector! • Collaborate on tools and algorithms for this format
27. Translation framework: Command-line tools and connectors: http://pizza.cs.bham.ac.uk/lmtf https://github.com/nsds/LMTF-Tools https://github.com/nsds/LMTF Source code to be available
28. A webapp and repository for multilayer networks The LASAGNE Multilayer Network Translation Framework Matt Williams, Mirco Musolesi University of Birmingham With thanks to... LMTF contributors: Victor Darvariu, Kevin Pomorski, Demet Turan Data providers: CNRS, MUW M.J.Williams @cs.bham.ac.uk
29. Acknowledgements Brain connectivity diagram Yasser Iturria-Medina, et al. Brain Hemispheric Structural Efficiency and Interconnectivity Rightward Asymmetry in Human and Nonhuman Primates. 2011. [http://cercor.oxfordjournals.org/content/21/1/56] Flight paths visualisation Michael Markieta [http://www.arupconnect.com/2012/12/14/global-flight-paths/] Pizza slice [http://icons.mysitemyway.com/magic-marker-icons-food-beverage/] WWW click icon Freepik [http://www.flaticon.com/free-icon/world-wide-web_1198]