Share
Embed
Transcript
70 slides
-
OSGeo.JP Workshop for the UN Vector Tile Toolkit – Let’s
make sustainable web maps Supported by OSGeo.JP 2018-12-02T13:30/17:00 @Moratuwa University -
Today’s schedule 2 1 Introduction 13:30 2 Fully guided hands-on
14:00 3 More individual challenges 15:30 4 Comments and wrap-up 16:40 -
4 By the end of the workshop, You create a
web map as good as this. Or you can make it even better. -
Questionnaire 5 Q1: What Operating System do you mainly use
today? ü Windows 10? ü Windows 7? ü OS X? ü Linux? ü Others? Q2: What Browser do you use today? (choose all you want to use) ü Google Chrome? ü Firefox? ü Safari? ü Microsoft Edge? ü Others? -
Step 0: network check 6 open http://vectortiles.xyz:8765/hands-on You will see
this map; otherwise plese let us know. -
7 Natural Earth Natural Earth Global Map Survey Department, Sri
Lanka OpenStreetMap OpenStreetMap contributors Integrated into a single vector tileset using the UN Vector Tile Toolkit -
8 And this is: 100% vector tiles. 100% open source.
No access token required. -
11 Reference: https://tmcw.github.io/presentations/jsgeo 1 Untiled WMS Selection by BBOX 2
Image tiles Predictable & cached 3 Tiles and overlays Vector retrofit 4 Vector tiles Everything in vector Evolution of web maps for performance -
12 Reference: https://tmcw.github.io/presentations/jsgeo/#33 Image tiles are facets of a gigantic
image Vector tiles are facets into datastructure, a straightforward way to vector geospatial data. -
14 9.C Significantly increase access to information and communications technology
and strive to provide universal and affordable access to the Internet in least developed countries by 2020 + Vector Tiles even saves the bandwidth use. Vector tiles are gradually getting ready for everyone. -
UN Vector Tile Toolkit Objective To facilitate the production of
vector tiles for basemaps by public organizations. Structure Import Produce Host Style Optimize Existing datasource e.g. PostGIS, Shapefile, GeoJSON Tippecanoe Maputnik Interoperable with both Open Source and other frameworks. Tailor-made Node.js scripts Openly maintained on GitHub. Maximum use of existing OSS while adding tailor-made scripts shared openly. vt-optimizer 15 You have accessed this in the step 0. -
16 Technical details Import data and assign layer names and
zoom levels. Custom open source scripts. Create vector tile packages in mbtiles format. Tippecanoe Host modular vector tile packages in a simple and scalable manner. Custom open source scripts. Style hosted vector tiles. Maputnik Enhance the optimization of the size of individual vector tiles by visualizing them. vt-optimizer and other custom open source scripts. Import Produce Host Style Optimize -
17 The UN Vector Tile Toolkit is a collection of
tools, rather than a monolithic DB/Server suite. Because: ü Inputs are diverse. ü Depolyment environments are diverse. Flexibility is the core value. And sometimes the server shall be isolated from the master database (for security and scarability). -
The stages 19 1. Get source data 2. Add design
properties 3. Build vector tilesets 4. Host the vector tilesets 5. Style the hosted vector tiles -
The stages 20 1. Get source data 2. Add design
properties 3. Build vector tilesets 4. Host the vector tilesets 5. Style the hosted vector tiles -
22 [Please check your workspace] You probably need about 2GB
for your work area. This material refers the path to your work area as ${your-workspace} -
Step 1: setup scripts to produce vector tileset 23 $
cd ${your-workspace} $ git clone https://github.com/un-vector-tile- toolkit/spinel-produce $ cd spinel-produce Download and install the scripts for producing vector tiles. If this does not work… ü Install git. ü Setup git (next slide). [spinel-produce]$ npm install If this does not work… ü Setup Node.js (next to the next slide) -
Setup Node.js 25 ü Recommendation: sudo apt install nodejs Ø
In Ubuntu, you may want to run the following command, too. curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash - For other Linux-based platforms: https://github.com/nodesource/distributions -
Or you still have a problem with “git clone”… 26
You can download a ZIP file from https://github.com/{user}/{repo} In this case https://github.com/un-vector-tile-toolkit/spinel-produce -
Step 2: get the source data 27 [spinel-produce]$ node download.js
# If you have time… [spinel-produce]$ gzcat src/6-46-30.ndjson.gz [spinel-produce]$ gzcat src/gmlk20.ndjson.gz This may take time depending on the Internet bandwidth. These are Newline Delimited GeoJSON features. i.e. each line is a GeoJSON Feature. This format is also called GeoJSON Text Sequences (RFC 8142) with Record Separator (RS) omitted. -
The stages 29 1. Get source data 2. Add design
properties 3. Build vector tilesets 4. Host the vector tilesets 5. Style the hosted vector tiles -
Step 3: Add design properties 31 # If you have
time… [spinel-produce]$ less 6-46-30.ndjson [spinel-produce]$ node import.js > 6-46- 30.ndjson -
Break Point A: definition of what3numbers, import.js, and modify.js 32
what3numbers import.js modify.js The number assigned to a tile (in slippy map tilenames) used as the extent of the vector tile package. e.g. 4-46-30 overwraps Sri Lanka. The code to read each features, apply “modify function” to add design properties (layer name, minimum zoom and maximum zoom), and store the resulting features to NDJSON. The code that contains a function to modify a feature by adding design properties and edit other properties of the feature. You can edit this to change the “vector tile schema”. -
The stages 35 1. Get source data 2. Add design
properties 3. Build vector tilesets 4. Host the vector tilesets 5. Style the hosted vector tiles -
Step 4: build vector tilesets 36 [spinel-produce]$ tippecanoe -f --read-parallel
--drop-densest-as-needed --simplification=2 -- minimum-zoom=6 --maximum-zoom=15 --base- zoom=15 --output=6-46-30.mbtiles 6-46- 30.ndjson This may take time. You’re rendering areas as big as Sri Lanka. If this does not work… ü Install Tippecanoe (next slide) -
Troubleshooting: install Tippecanoe 37 $ cd ${your-workspace} $ git clone
https://github.com/mapbox/tippecanoe $ cd tippecanoe [tippecanoe]$ make –j [tippecanoe]$ sudo make install # Or you can just use ./tippecanoe without make install [tippecanoe]$ tippecanoe -v If this still does not work, see the next slide. -
Troubleshooting in installing Tippecanoe • If you see errors at
“make -j”, posslble solutions are: • Install g++ • Install libsqlite3 38 $ sudo apt install build-essential $ sudo apt install libsqlite3-dev zlib1g-dev For details, see https://github.com/mapbox/tippecanoe#development -
39 This is a vector tile package that contains the
module 6-46-30. Format: mbtiles-spec (CC-BY) 6-46-30.mbtiles -
Step 5: Quick-check the content of vector tileset 40 $
sudo npm install –g tileserver-gl-light $ tileserver-gl-light 6-46-30.mbtiles $ open http://localhost:8080 Please see a workaround if you see errors (e.g. in Ubuntu) Don’t worry if tileserver-gl-light doesn’t work. You can simply skip this step. -
tileserver-gl-light install error workaround 41 $ sudo npm install -g
tileserver-gl-light --unsafe-perm=true --allow-root Source: https://github.com/npm/npm/issues/17268 -
On the console 43 Ctrl-C to stop. The vector tiles
you created were sent to the browser. -
The stages 44 1. Get source data 2. Add design
properties 3. Build vector tilesets 4. Host the vector tilesets 5. Style the hosted vector tiles -
Step 6: install the server script 45 $ cd {your-workspace}
# probably “cd ..” $ git clone https://github.com/un-vector-tile- toolkit/spinel $ cd spinel [spinel]$ npm install [spinel]$ mv ../spinel-produce/6-46-30.mbtiles mbtiles/hands-on [spinel]$ node index.js spinel vector tile server starts on 8765 # After you see “spinel vector tile server starts on 8765” on the original console; $ open http://localhost:8765 $ open http://localhost:8765/hands-on Ctrl-C to stop. -
Step 7: a little bit more professional way to run
the server 48 $ npm install -g pm2 $ pm2 start index.js -name spinel $ open http://localhost:8765/hands-on # when you want to monitor the server $ pm2 monit # when you want to stop and delete the server $ pm2 stop all $ pm2 delete all You don’t have to do it now. You will use the server in the next stage. -
The stages 49 1. Get source data 2. Add design
properties 3. Build vector tilesets 4. Host the vector tilesets 5. Style the hosted vector tiles -
Step 8: install Maputnik 50 $ cd {your-workspace} # probably
“cd ..” $ git clone https://github.com/maputnik/editor $ cd editor [editor]$ npm install $ open http://localhost:8888 $ open http://localhost:8888?style=http://localhost:8 765/hands-on/style.json [editor]$ npm start -
Additional tutorials 55 6. Evaluate the size of vector tiles
7. Convert Shapefile to NDJSON 8. Extract OpenStreetMap data -
Additional hands-on exercise 56 6. Evaluate the size of vector
tiles 7. Convert Shapefile to NDJSON 8. Extract OpenStreetMap data -
Step 9: evaluate the size of vector tiles 57 $
cd {your-workspace} # probably “cd ..” $ git clone https://github.com/ibesora/vt- optimizer $ cd vt-optimizer $ npm install $ node index.js –m ../spinel/mbtiles/hands-on/6- 46-30.mbtiles -
Additional technical introduction 59 6. Evaluate the size of vector
tiles 7. Convert Shapefile to NDJSON 8. Extract OpenStreetMap data -
Step 10: Convert Shapefile to NDJSON 60 Recommendation A: Use
ver. 2.4.0dev of GDAL Challenge: RFC 8142 is still a new format for most software. Recommendation B: Use node-shapefile and script in Node.js Example: https://github.com/hfu/natural-coast Application code example: https://github.com/hfu/shapefile-pass-through https://github.com/mbostock/shapefile -
Additional hands-on exercise 61 6. Evaluate the size of vector
tiles 7. Convert Shapefile to NDJSON 8. Extract OpenStreetMap data -
Step 10: Extract OpenStreetMap data 62 Source: https://github.com/hfu/extract-all/blob/master/index.js e.g. Use
Osmium Extract Then use Osmium Export to convert OSM PBF to NDJSON -
I have the following extracts for your testing purpose 63
Today, you can get these modules from http://first.vectortiles.xyz:8765/modules/{z}-{x}-{y}.pbf e.g. http://first.vectortiles.xyz:8765/modules/6-29-30.pbf -
Example for individual challenges 65 # challenge difficulty 1 Change
the style of the map EASIER 2 Change vector tile schema MODERATE 3 Add your own data HARD 4 Add HTTPS or SPDY for better interoperability and performance HARD -
Setup procedure for the “lifeboat” 1. Set-up the workshop LAN.
2. Start a UNIX server inside the LAN. 3. [op] Check the IP address of the server and provide a domain name using a public DNS for the ease of access. 4. Prepare tickets that contains account information, e.g. [user1, some_fancy_password] and setup such temporal accounts on the server. 5. If a participant want a ”lifeboat”, provide the ticket and allow the participant to access the server. ü If the participant does not have an ssh client, the participant may want to install “Chrome Secure Shell Extension” that provide an ssh client on top of Chrome. 69 -
Providing a ssh server Set up a server ifconfig |
less to find the local IP address, probably 192.168.0.{some} sudo service ssh start sudo adduser user1 Test a connection from the client ssh –l user1 192.168.0.{some} After the session sudo deluser user1 sudo rm –rf /home/user1 70 Similar on mac, but use Settings to start the server.