2021-06-14_UNVT exercise day01

Material for a casual vector tile exercise (Day 01)

Transcript

1. UNVT Exercise Day 1 14 June 2021 By Taro Ubukawa (Senior Geospatial Expert at UN Geospatial Information Section)
2. Whole schedule
3. Today’s contents • Excercise Part 1: Open the vector tile • Excercise Part 2: Understand and edit style.json • Excercise Part 3: Simple hosting of vector tile map
4. Revisit: Vector Tile • I will not go into details of vector tile explanation. Please visit the materials of UNVT workshop on 5 May 2021. https://speakerdeck.com/hfu/unvt-workshop- introduction-and-application?slide=11 From https://maps.gsi.go.jp/development/siyou.html Each tile is represented by the combination of Z, X, Y.
5. Exercise 1: Let’s read VT by yourself As the first step, let’s read the vector tile from the Internet. • What: • Existing Vector Tiles • How: • With QGIS (ver 3.16 or later) • Or, with Maputnik (https://maputnik.github.io/ )
6. Prep: What do you need to know to use VT? • URL of vector tile • https://xxxxxx/{z}/{x}/{y}.ext • XYZ specify the tile location. The order of zxy in the file path could change. • Extension could be “pbf” or “mvt”. Recently, “pbf” is widely used. • Structure of the vector tile • Layer name, zoom level (max/min), attribution These information is needed to create you own vector tile map (to design details of your web map). • Style info of vector tile map (if any) • You can open the styled vector tile map. • A single vector tile source can have multiple styles.
7. Practice 1-1: Explore the sample data Sample data for testing purpose • Source: NaturalEarth, public domain data set • Repository: https://github.com/ubukawa/vt-test • Vector tile, simple style.json, and simple webmap using mapbox-gl.js (ver 1.x) and ArcGIS API for JavaScript are there. • VT URL: https://ubukawa.github.io/vt-test/ne-test/{z}/{x}/{y}.pbf • VT Structure • Zoom Level: from 0 to 5 • Layers: • landmass (from ne_110m_land.shp), • bndl (from ne_110m_admin_0_boundary_lines_land.shp), • popp (from ne_110m_populated_places.shp), • coastl (from ne_110m_coastline.shp) * Attributions values come from original shape files. • Style URL: https://ubukawa.github.io/vt-test/maps/test-simple.json The boundaries in the map is just for practice. They do not imply official endorsement/acceptance by the UN or any agencies.
8. (With QGIS – step 1) Vector Tiles connection in Browser Panel How to add vector tile to QGIS? • Start QGIS. • At the Browser Panel, right click at “Vector Tile,” then select “new generic Connection.”
9. (With QGIS – step 2) Add vector tile without any style.json (1) • Name Vector Tile Connection. • Specify tile URL Try the following if necessary https://ubukawa.github.io/vt-test/ne-test/{z}/{x}/{y}.pbf • Confirm Max/Min Zoom of the tile max is 5 if you use above tile. • Then, click “OK” This time, we will open vector tile without any style. So, please leave “Style URL” as brank. (We will practice this later.)
10. (With QGIS – step 2) Add vector tile without any style.json (2) • Choose your tile in the Browser Panel. • Then, your vector tile will be added to the layers panel. • You will see vector tile without any customized style. Displayed tile is from NaturalEarth (free data). The boundaries in the map is just for practice. They do not imply official endorsement/acceptance by the UN or any agencies.
11. (With QGIS – step 3) Add vector tile with style.json (1) • Name Vector Tile Connection. • Specify tile URL Try the following if necessary https://ubukawa.github.io/vt-test/ne-test/{z}/{x}/{y}.pbf • Confirm Max/Min Zoom of the tile max is 5 if you use the above tile. • Specify the style URL use the following if you use the above tile. https://ubukawa.github.io/vt-test/maps/test-simple.json • Then, click “OK”
12. (With QGIS – step 3) Add vector tile with style.json (2) Displayed tile is from NaturalEarth (free data). The boundaries in the map is just for practice. They do not imply official endorsement/acceptance by the UN or any agencies. • Choose your tile in the Browser Panel. • Then, your vector tile will be added to the layers panel. • You will see vector tile with customized style. • QGIS does not work exactly same with mapbox-gl js library. So, the style for QGIS should be simple. (This case, the point layer uses icon and has no paint-info, so it was skipped.)
13. Practice 1-2 (If you have time): Please practice by yourself • GSI Vector by GSI Japan (Japan area only): • Tile URL: https://cyberjapandata.gsi.go.jp/xyz/{t}/{z}/{x}/{y}.pbf • Style URL: https://gsi-cyberjapan.github.io/gsivectortile-mapbox-gl-js/std.json • UN internal network only (small scale map, for testing purpose): • Tile URL: https://dev-unvt.dfs.un.org/clearmap-vector/{z}/{x}/{y}.pbf • Style URL: https://dev-unvt.dfs.un.org/styles/clearmapVector/clearmap-adm.json * This is accessible only from UN internal network. • Sample data from OSM (Around New York only, test repository for personal test): • Repository: https://github.com/ubukawa/osm-neighbor • Sample map: https://ubukawa.github.io/osm-neighbor/index.html • Tile URL: https://ubukawa.github.io/osm-neighbor/osm-ny/{z}/{x}/{y}.pbf • Style URL: https://ubukawa.github.io/osm-neighbor/style.json You will find that this style cannot be open with QGIS nor maputnik. Because there are some advanced style description. For these case, 1) you can see this map as web map app, or 2) you can prepare a simple style by yourself.
14. Tips: (with maputnik) • If you cannot use QGIS, you can use maputnik to see vector tiles. • This tool is mainly for editing style. So, please see the part 2 of today’s lesson.
15. Tips: QGIS may not fully display VT map Differences • Background • Labels of the point • Width of the line. • Etc… If you see the same with mapbox-gl js, you will find some difference. https://ubukawa.github.io/vt-test/maps/test-simple-mapbox.html
16. Tips: Exploring ESRI vector tiles • How are vector tiles treated in ArcGIS online? • In the item page, scroll down the page and click View to view the vector tile layer metadata and its service URL. To access the vector tile data in an application, use the URL+tile/{z}/{y}/{x}.pbf. Example "sources":{"esri":{"type":"vector","url":"https://base maps.arcgis.com/arcgis/rest/services/World_Bas emap_GCS_v2/VectorTileServer"}} For detail, https://developers.arcgis.com/documentation/mapping-apis-and-services/data-hosting/tutorials/tools/convert-a- feature-layer-vector-tile/ IF the data is not open to the public, it may not be displayed in QGIS. Tile URL: https://basemaps.arcgis.com/arcgis/rest/services/World_B asemap_GCS_v2/VectorTileServer/tile/{z}/{y}/{x}.pbf Note: Pay attention to the order (not ZXY)
17. Tips: Exploring ESRI vector tiles (cont.) • How about seeing ArcGIS vector tile? • https://basemaps.arcgis.com/arcgis/rest/services/World_Basemap_v2/Vector TileServer/tile/{z}/{y}/{x}.pbf → OK • https://basemaps.arcgis.com/arcgis/rest/services/World_Basemap_GCS_v2/Vec torTileServer/tile/{z}/{y}/{x}.pbf → no good (projection?)
18. Summary of Part 1 You have learned: 1. Vector tiles are accessible by specifying URL of pbf tiles. 2. If you have style information, the vector tiles can be displayed as a vector tile map.
19. Part 2: Styling
20. Styling – what? / why? For example, boundary lines are classified using their attribute values. • Why is styling needed? • Vector tiles do not have any styling information. They have geometries and attributions. • We need a way to specify the style of each layer. If you edit the style file, you can freely change the design of your web map.
21. Styling - A format of styling information How cat we add style information? mapbox-gl.js takes map styles that conform to the Mapbox Style Specification. For detail, go to: https://docs.mapbox.com/mapbox-gl-js/style-spec/ (Other map libraries also follows main part of that specification.) • A “style.json” • is a JSON object • with specific root level and nested properties (You can name it as you like.) • How we can make it? • By manual writing using text editor (it is tough work though…) • Mapbox Studio • Maputnik (an open source visual editor for mapbox style spec.) • UN Vector Tile Toolkit (e.g. naru with HOCON) ArcGIS Pro or ArcGIS Vector Tile Style editor may be able to edit style, but I felt that they were not so flexible for external source data.
22. Here are the main properties at the root level • version: Style specification number. Must be 8. (required) • source: Data source. (required) • glyphs: A URL for glyphs (text font, etc). (optional) • sprite: A base URL for the sprite image (map symbols). (optional) • layers: Layers drawn in the order of this array. (required) • center: Default map center (optional) • zoom: default zoom level (optional) Source and layers are important parts. From https://docs.mapbox.com/mapbox-gl-js/style-spec/root/ Styling - Main components of a style.json
23. Practice: Styling with maputunik • There is a good tool, called Maptnik. For beginners, it is really useful! • Maputnik A free and open visual editor for the Mapbox GL styles targeted at developers and map designers. • https://github.com/maputnik/editor • https://maputnik.github.io/editor/ I used this at the initial stage so that I can confirm the converted vector tile. However, this tool does not support the advanced expression like match.
24. Let’s try!! • Open any style.json with maputnik • You can import style.json (online) by specifying its path as below: https://maputnik.github.io/editor?style=(xxxx.json) • Let’s practice together with the following example. • https://maputnik.github.io/editor?style=https://ubukawa.github.io/vt- test/maps/test-simple.json
25. Maputnik Practice (1) • “Style Setting” to check the sprite and glyphs location. (map symbols and text fonts) • “Data Sources” to confirm the vector tile source. • Click “export” to export the current style. • Edit layers • Fill (Polygon) • Line • Symbol Displayed tile is from NaturalEarth (free data). The boundaries in the map is just for practice. They do not imply official endorsement/acceptance by the UN or any agencies.  Top Menu Layer Property editor Map View
26. Maputnik Practice (2): Property editor (1) Layer • Source, Min zoom, Max zoom Filter • You can apply filter using attribution. Layer Property editor
27. Paint properties • Opacity • Color • Width (line) • Etc.. Maputnik Practice (2): Property editor (2)
28. You can see the JSON object of the layer in the bottom of the layer. “layers”: [ { “id”: “A”, …. }, { “id”: “B”, …. } ] Maputnik Practice (2): Property editor (3)
29. Maputnik Practice (3): Export You can download json file from the “export” in the top menu. Once you become familiar with style.json, you may want to directly edit your style. (Because you can use advanced style expression by referring the mapbox’s style specifications.)
30. Summary of the part 2 • You understood what style file is and why we need it. • Now, you can edit style.json by using maputnik or by using text editor.
31. Part 3: Simple Web map APP hosting
32. Hosting – overview • Necessary elements for Vector Tile Hosting Server Vector Tile Data Map Library Map Symbol Icons (sprite) Text font (Glyphs) HTML Hosting server(s) • Server itself → GitHub page • Data → Your own / External tile • Map Symbol → Your own / External source • Font → Your own / External source • Library → External source / or upload • Style info → Your own / external source • Html → your own. Style info
33. Demo: Create a GitHub Repository In order to establish a web page, here, we will use a GitHub repository.
34. Demo: Start GitHub page Setting > GitHub Page Confirm the root URL of your repository.
35. Then, upload the necessary files • Copy the file from the shared directory in MS Teams channel. • Make sure that you have adjust your file path in style.json and html.
36. Tips: Some more info about our files • Text fonts With this practice, we are using the open sans font (location name is “sans”). You can find/use several fonts. • Sprite Mapbox provides map icons (CC0 licensed) https://labs.mapbox.com/maki- icons/ • Map library Here, we are using mapbox-gl-js ver. 1.10. If you want to version 2, please check its pricing policy.
37. Result • Then, you can establish your own page. (Tips) • You can easily make such map if you have any web server. • You can also use other libraries.
38. Summary of the part 3 • You understood how we can host a simple web map using mapbox-gl-js or other libraries.
39. End of the exercise today. • Thank you for your kind attention. Any question? • Next session, we will use Docker Desktop for windows (Docker toolkit would be also okay). Please make sure that you can use it in PowerShell, etc. (Please also make sure that you can use “git” if possible.)
40. References: • Test data: https://github.com/ubukawa/vt-test • NaturalEarth: https://www.naturalearthdata.com/ • ESRI: https://developers.arcgis.com/documentation/mapping-apis-and-services/data- hosting/tutorials/tools/convert-a-feature-layer-vector-tile/ • Maputnik: https://maputnik.github.io/ • Fujimura, H. (2021): UNVT Workshop: Introduction and Application https://speakerdeck.com/hfu/unvt-workshop-introduction-and-application • Matsuzawa, T. (2019): Create/distribute vector tile, https://speakerdeck.com/smellman/distrubute-vector-tile