Packaged Exercise of the UN Vector Tile Toolkit for the GGIM

For the "Planning and Management of National Mapping and Surveying" course
2019-12-03T13:10/16:30 @ GSI

Transcript

1. Packaged Exercise of the UN Vector Tile Toolkit for the GGIM Hidenori Executive Officer for Geospatial Information Policy, GSI Lead, the United Nations Vector Tile Toolkit Lead, Task Group B, UN-GGIM WG-Disasters 2019-12-03T13:10/16:30
2. words • UN Vector Tile Toolkit (UNVT): an open source software toolkit to support anyone to produce, host, style and optimize basemap-grade vector tiles. • Vector Tile: a de-facto standard vector geospatial data for web maps. • GGIM: the United Nations Committee of Experts for Global Geospatial Information Management. This session tries to demonstrate the UN Vector Tile Toolkit in the context of supporting the UN Member States.
3. This presentation will be on Speakerdeck https://speakderdeck.com/hfu
4. related session Taro’s session for Web Mapping This session uses the same technology. Today we (in four teams) try the UNVT on Raspberry Pi.
5. Please let me know if you mind being on the photos.
6. Agenda 1. Introduction to UNVT 2. Introduction to Raspberry Pi 3 Model B+ (P3) 3. Hands-on exercise of UNVT on P3 4. Invitation to the UNVT
7. Hidenori: self introduction 1 Background: information technology Robot vision, parallel processing 2 Global Mapping project Capacity building of digital mapping 3 Information Access Division Digital Japan Web System 4 Leibniz University of Hannover Road verification using ALOS-1 5 Technology Development Office Address, geographic identifier 6 Overseas Project Division Mapping / sewage projects 7 Director, Information Access Division GSI Maps and GSI Tiles 8 Director, International Affairs Division Conclusion of the Global Mapping project 9 United Nations Geospatial Information Section (UNGIS) Senior Geospatial Expert UN Vector Tile Toolkit 10 Executive Officer for Geospatial Information Policy Technology policy coordination 11 Involved in ISO/TC 211 for a long time e.g. Head of Delegation 7
8. GSIʼs experience regarding the UNVT 2003 GSI has started a web map using the vector tile technology. →Technologically right, and so natural decision. 2012 GSI adopted Open Source Software for the operation of web maps. FOSS4G of that days was based on the image tile technology. →A compromise in technology; improved transparency and accountability. 2014 Vector Tile Technology with Open Source License became available. GSI Vector Tile Engineering Experiment was started. →Recovered a good technology direction. 2015 UN Open GIS Initiative was established. →Open Source GIS for more efficient UN Operations. 2017 Japan dispatched Senior Geospatial Expert to UN Geospatial Information Section. →On request by the UN Secretariat for implementation-oriented web map technology with capacity building component based on the experience of the Global Mapping Project.． 2018 The UNVT was established under the UN Open GIS Initiative →For automatic and continuous update of the UN internal basemap vector tiles. The UNVT is an FOSS4G for more stable sustainment of the technology and to facilitate the diversity of web maps especially by Member State organizations. 2019 GSI released GSI Maps Vector as another product of the UNVT. 8
9. Composition of the UNVT A package of best existing Open Source Software that enable everyone to produce, host, style and optimize basemap-grade vector tiles. Produce Tippecanoe Host Node.js program (budo) Style Custom Node.js program Optimize vt-optimizer 9
10. 100% Open Source transparency, accountability, and scalability Not dependent on any cloud service so that we can choose the best cloud service
11. What is Vector Tiles? 11
12. 12 Tiles
13. 13 Image Tiles Vector Tiles
14. 14 Image Tiles Vector Tiles Rendered Images Machine- readable data
15. UN implementation #1 The original implementation based on image tiles 15 UN data used Not responsive No global coverage in the large scale
16. UN implementation #2 Vector Tile implementation before my arrival 16 16 UN data used Less responsive Global coverage in the large scale
17. UN implementation #4 Implementation using the UNVT 17 UN data used Global coverage of building details Responsive as in video games Continuous zoom and responsiveness matters for good user experience.
18. GSI Maps Vector uses the UNVT, too. 18 UNVT aims for a sustainable project by combining efforts of the partners by solving common problems.
19. The UNVT Team 19 Interdisciplinary Cross-sector We expand our community in the future UNGIS GeoThings Geospatial Information Authority of Japan National Astronomical Observatory of Japan National Institute for Agro-Environmental Sciences Mapbox Mapple On OSGeo Japan Chapter UN Geospatial Information Section UN Global Service Centre
20. The UNVT Vision Interdisciplinary & cross-sector partnership Diverse application fields Free and Open Source Software 20 enable expand sustain
21. Recent Activities 21
22. 2018-12 FOSS4G Asia 2018 in Moratuwa, Sri Lanka OSGeo.JP Workshop for the UN Vector Tile Toolkit
26. 2019-08 FOSS4G 2019 Bucharest
31. The UNVT Core: features ① “Around the World in 80 hours” ② Server-side image tile rendering for interoperability ③ Single board PC implementation for demos and capacity building 31
32. “Around the World in 80 hours” Successfully implemented a production tools as fast as 80 hours per a global dataset which contains the whole OpenStreetMap data and other 10GBs of data. ü By carefully designing a data flow without temporary data storage. Source data Vector tiles Before After temp data Step 1 Step 2 Step 3 Source data Vector tiles Step 1 Step 2 Step 3 temp data 32
33. Server-side image tile rendering Only vector tiles are inside the continuous production loop. Image tiles are rendered on request at the server side. ü Take heavy process off the production loop. ü A proven method from private sector. ü Reduced 80% of data production. Before After Source data Vector tiles Image tiles Source data Vector tiles Image tiles Legacy Apps Legacy Apps Push Push Pull Pull Pull 33 ～80 hours ～400 hours
34. Interoperability guaranteed Server-side image-tile rendering function of the UNVT guaranteed the interoperability with proprietary solutions and legacy solutions, without compromising on the production speed. 34
35. 2019-11 UN-GGIM-AP Plenary
36. Integrating geospatial information and statistics 36
37. Population density visualized in 3D Interdisciplinary 37
38. Without adopting vector tile, we might simply disappear from the Web
39. Plenary presentation 39
40. 2019-11 ESCAP Workshop in Central Asia for Land Management Tashkent, Uzbekistan
41. 41 UNVT for Land Information Management?
42. The UNVT Team 42 UNGIS GeoThings Geospatial Information Authority of Japan National Astronomical Observatory of Japan National Institute for Agro-Environmental Sciences Mapbox Mapple On OSGeo Japan Chapter UN Geospatial Information Section UN Global Service Centre
43. Application of UNVT on land information management Agricultural land parcel vector tiles on the latest topographic map 43
44. Agricultural land parcel vector tiles on the latest orthophotos Application of UNVT on land information management 44
45. Agricultural land parcel vector tiles on the topographic map of 1880s Application of UNVT on land information management 45
46. The UNVT Vision Interdisciplinary & cross-sector partnership Diverse application fields Free and Open Source Software 46 enable expand sustain
47. Next step: Contribute to UN Agenda by Tangible Running Code Planning a scenario-based exercise using the UNVT. 47
49. Single board PC implementation • UNVT Core is available as Open Source Software, but it requires enterprise server environment and expertise. • Single board PC implementation (P3 UNVT) was developed for community building by easier demo and capacity building. UNVT Core P3 UNVT
50. The UNVT Team 50 UNGIS GeoThings Geospatial Information Authority of Japan National Astronomical Observatory of Japan National Institute for Agro-Environmental Sciences Mapbox Mapple On OSGeo Japan Chapter UN Geospatial Information Section UN Global Service Centre
52. Agenda 1. Introduction to UNVT 2. Introduction to Raspberry Pi 3 Model B+ (P3) 3. Hands-on exercise of UNVT on P3 4. Invitation to the UNVT
53. Raspberry Pi: a single board computer
56. P3 is globally available
57. P3 is globally available
58. P4 is also globally available
60. Raspberry Pi prepared for you by: Yoichi David A. Hastings Award Winner (FOSS4G Asia 2018) Nobusuke OSGeo Japan Chapter Representative Hidenori me
61. Yoichi Yoichi Nobusuke Hidenori Hidenori
62. Agenda 1. Introduction to UNVT 2. Introduction to Raspberry Pi 3 Model B+ (P3) 3. Hands-on exercise of P3 UNVT 4. Invitation to the UNVT
63. Purpose of P3 UNVT Aim: To streamline the Demo and Capacity Building of UNVT. By providing: UNVT on real Linux (industry standard server OS) All the data and the web contents are also included. Without requiring: Network connection, specific client OS.
64. Learn about the hardware • The board (the computer) • The ethernet cable (for communication) • The USB cable (for power) • The microSD card (the storage) → the software part
65. The software part Inside the 32GB microSD card… Raspbian (Linux) Docker unvt/kawagoe (Docker image) UNVT software such as node, Tippecanoe, vt- optimizer, ogr2ogr Payload data
66. Connect P3 to a PC 1. Connect P3 and a PC by an Ethernet cable. 2. Connect P3 and the PC by a USB cable.
67. login to P3 ssh pi@raspberrypi.local Raspbian (Linux) Docker unvt/kawagoe (Docker image) UNVT software such as node, Tippecanoe, vt- optimizer, ogr2ogr Payload data Start “Command Prompt” and Password: raspberrypi (This is the default of Raspberry Pi.) If you have time… PC df -h docker -v uname -a
68. inside P3, log in to Docker docker run -ti --rm –p 9966:9966 unvt/kawagoe Raspbian (Linux) Docker unvt/kawagoe (Docker image) UNVT software such as node, Tippecanoe, vt- optimizer, ogr2ogr Payload data In the “Command Prompt” If you have time… PC tippecanoe –v ogr2ogr –v ls nano -p 9966:9966 Map the port 9966 of the container to the port 9966 of P3. unvt/kawagoe The name of
69. unvt/kawagoe; a P3 UNVT
70. Kawagoe: Open Source on the GitHub
71. Kawagoe: Product also available on the Web
72. Kawagoe: suburban city near Tokyo
73. Kawagoe: hit by Typhoon Hagibis (#19)
74. P3 UNVT exercise cont’d 1.Produce 2.Host 3.Style 4.Optimize
75. Edit kawagoe/hocon/style.conf cd kawagoe nano hocon/style.conf In unvt/kawagoe instance on Command Prompt Update location to http://raspberrypi.local:9966 and save (Ctrl+s then Ctrl+x) BEFORE AFTER
76. Create the style file rake build In unvt/kawagoe instance on Command Prompt
77. Start the server budo -d docs In unvt/kawagoe instance on Command Prompt Open http://raspberrypi.local:9966 from the browser on your PC.
79. Power off In unvt/kawagoe instance on Command Prompt Wait for a few minutes and disconnect wires.
80. What to do to create vector tiles from your data 1. Learn more details about unvt/kawagoe. 2. Download your own data. 3. Convert your data into GeoJSON Text Sequence Ø ogr2ogr will help. 4. Convert your GeoJSON Text Sequence into vector tiles Ø Tippecanoe will help. 5. Develop the style.json Ø nano hocon/*.conf & rake build 6. rake host & copy the content of the docs to the server
81. What to do to create your own P3 UNVT 1. Create a copy of (‘fork’) un-vector-tile-toolkit/kawagoe 2. Edit the content of the directory 3. rake docker_build 4. docker login 5. docker push your/container_image
82. Agenda 1. Introduction to UNVT 2. Introduction to Raspberry Pi 3 Model B+ (P3) 3. Hands-on exercise of P3 UNVT 4. Invitation to the UNVT
83. UNVT is open to everyone Community participation is not necessary to use the UNVT. https://github.com/un-vector-tile-toolkit
84. We have a Slack For internal communications of contributors. https://unvtt.slack.com We welcome you anytime. Let me know for an invite.
85. We hope to welcome you in the team You don’t have to agree to show the logo of your organization, to join the community. UNGIS GeoThings Geospatial Information Authority of Japan National Astronomical Observatory of Japan National Institute for Agro-Environmental Sciences Mapbox Mapple On OSGeo Japan Chapter UN Geospatial Information Section UN Global Service Centre 85