2021-09-30_UNVT presentation at FOSS4G 2021

Slide for the UNVT presentation at FOSS4G2021

Transcript

1. Deployment of Open Source Vector Tile Technology with UN Vector Tile Toolkit Taro Ubukawa (United Nations Geospatial Information Section), Hidenori Fujimura (Geospatial Information Authority of Japan), Diego Gonzalez Ferreiro (United Nations Global Service Centre), Paolo Frizzera (United Nations Global Service Centre), Oliva Martin Sanchez (United Nations Global Service Centre) 30 Sept 2021 on Aconcagua room
2. Geospatial Strategy for the United Nations The effective, efficient and universal use of geospatial information in support of all mandates and operations of the United Nations for a better world The UN Open GIS Initiative aims to identify and develop an Open Source GIS bundle that meets the requirements of UN operations for both peace-building and peace-keeping. Introduction: Geospatial Information for a Better World. With open source GIS bundle! About Me: Taro Ubukawa (Mr.) taro.ubukawa@un.org Senior Geospatial Expert (since December 2019), UN Geospatial Information Section (Dispatched from Japan) Our strategy and Initiative
3. • UNVT is a collection of Open Source Software (OSS) to produce, host, style and optimize vector tiles for web mapping. It was initiated by Mr. Hidenori Fujimura in 2018. • Goal: To facilitate the production of Vector Tiles by public organizations for further use as basemaps. UN Vector Tile Toolkit (UNVT) The toolkit works even in a single-board computer
4. • One of UNVT projects is the deployment of UNVT in UN Global Service Center (UNGSC) in Italy. • I will introduce our effort and share our experience. Develop a vector tile map of covering the world from both OSM and the UN internal data (from our PostGIS database). Update it everyday/week. UNVT Deployment in UN (One of the key UNVT projects) UNGSC supports various UN operations, missions, agencies, etc., with ICT and geospatial information services.
5. • How can we efficiently combine OSM data and UN data? • UN Mappers update OSM everyday. How can we update the vector map frequently? → Wise use of nodejs and open source tools. Key targets OSM source UN source
6. • Production: Vector Tiles of the whole globe • Styling: We made a Web Map Style with efficient method • Hosting: Vector tile delivery and web map server (Simple hosting server) What we made
7. With the nodejs scripts, open source tools are combined efficiently. • Conversion Sever: RHEL • Tools used: nodejs (pg to read PostGIS), Tippecanoe, tile-join • Repository: https://github.com/un-vector-tile-toolkit/produce-gsc-4-pro • Other tips: • Scripts are executed as scheduled task to update daily/weekly. • Only updated OSM sourced data everyday/week while UN sourced will be updated when needs arise Our Experience: Production Source (PostGIS) Vector Tiles (mbtiles) GeoJSON seq GeoJSON sequence is directly imported in Tippecanoe Our work flow nodejs Tippica noe
8. • 841 spatial modules • Total size: 164 GB (ZL0-15) • 42 layers covering these themes: • Landcover • Hydrography • Admin boundary • POI • Transportation • Place • Label, etc Area Number of spatial modules Data size Conversion Time Priority 69 12 GB 2h48m 1 114 25 GB 4h24m 2 116 20 GB 4h19m 3 31 27 GB 4h27m 4 118 24 GB 5h49m 5 126 20 GB 4h03m 6 137 16 GB 3h47m 7 130 20 GB 5h03m Daily update for the priority area. The other areas are updated weekly to reduce the burden to the source server. (If needed, about 35 straight hours for the whole globe.)
9. Style description • Style.json based on the mapbox style spec. • Starting from Maputnik, now we are using HOCON to efficiently edit the map style. (HOCON: Human-Optimized Config Object Notation) Our Experience: Styling See https://github.com/un-vector-tile-toolkit/style-gsc-prod Assemblage config files to make a single style.json!!
10. • We have prepared several style files for different Map libraries/tools, and we found some differences. • We made styles for: • Mapbox gl js • QGIS • ArcGIS API for Javascript, ArcGIS online • Findings • Not all libraries/tools support advanced styling expression. • Some tools only read only a single source per style file. • Some tool uses system fonts (not “glyghs”) • RGB, RGBA, RGB color code • Authentication for vector tiles delivery may have not been well established in many tools. Our Experience: Styling -Tips
11. • We made a simple hosting server using nodejs/express. PBF tiles are derived from mbtiles. Our Experience: Hosting Vector Tile Data (mbtiles) * They will be delivered as pbf tiles Map Library Map Symbol Icons (sprite) Text font (Glyphs) HTML files for webmap Hosting server(s) Our vector tile server (nodejs/express) https://github.com/un-vector-tile-toolkit/onyx Extended server (Azure AD authentication.) https://github.com/un-vector-tile-toolkit/coesite Components: • Server itself • Data • Map Symbol • Font • Library • Style info Style info
12. This year, we had several online workshops/exercises: • UNVT workshop (5 May 2021) • UNVT workshop in Japanese (12 May 2021) • A Series of UNVT exercise (June and July 2021) Knowledge-sharing efforts: Workshop(s) Picture by Prof. Furuhashi and Ms. Yasuda. https://github.com/unvt/512 Materials are available! • UNVT Workshop in Japanese: https://github.com/unvt/512 • UNVT exercise materials: https://speakerdeck.com/ubukawa • UNVT naru: https://github.com/unvt/naru We have a UNVT storytelling workshop on 22 October 2021. Please contact me if you want to join us!
13. My message: • I have shared our experience of deploying UNVT for vector tile development project. • I feel that making/styling/hosting vector tiles is not that difficult thanks to the open source tools. • If you are interested in our efforts, join us in UNVT activity!! Thank you and I will invite Mr. Fujimura to show the latest UNVT examples in other projects.
14. Expansion of UNVT in Japan UNVT optgeo FTS EV 130 repos. Adopt Geodata
15. Topography & Landform classifications https://optgeo.github.io/one-styled
16. Voxel Tiles from point cloud (LAS) data https://x.optgeo.org/kid-a
17. Expansion of UNVT in Japan UNVT optgeo FTS EV Earth Observation plus Vector Tiles
18. JAXA HRLULC & Topography & Elevation https://optgeo.github.io/hrlulc3d512
19. Expansion of UNVT in Japan UNVT optgeo FTS EV Tile Service
20. FTS FTS is a tile service Tiling and hosting, free, as in freedom * Ghostscript Tiger is AGPL-ed.
21. x.optgeo.org UNVT on Raspberry Pi Flock of Raspberry Pi in GSI 500GB HDD Produce Host Style Optimize
22. FTS Tile Service SSH DNS Let’s Encrypt NFS HTTP/2 Tiles & Apps HTTP/2 HTTP Project partners in: Japanese Antarctic Research Expedition (JARE) 63rd Wintering Party Furuhashi Laboratory, Aoyama Gakuin University
23. Meet us at https://github.com/unvt https://github.com/optgeo Thanks.