Mobile app development with routing and voice navigation Taro Matsuzawa (@smellman) Georepublic Open Street Map Foundation Japan OSGeo.JP Japan Unix Society (jus) 1

$ whoami • Georepublic Senior Engineer • Programmer(Ruby/Python/Javascript) • System / Network designer of Linux system • Tile guru. • OSM Tile server • Mapbox Vector Tile • Gamemap (L.CRS.Simple) 2

SOTM2017 Local Team 3

Programming / Works / OSS Community 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 Highshool Tokyo Denki Univ Part time job Part time job SI Company EC engineer Georepublic Linux/Mozilla Japan Unix Society OSMFJ OSGeo.JP 4

Programming / Works / OSS Community 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 Highshool Tokyo Denki Univ Part time job Part time job SI Company EC engineer Georepublic Linux/Mozilla Japan Unix Society OSMFJ OSGeo.JP I buy computer and start learning programming and Linux. 4

Programming / Works / OSS Community 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 Highshool Tokyo Denki Univ Part time job Part time job SI Company EC engineer Georepublic Linux/Mozilla Japan Unix Society OSMFJ OSGeo.JP I started my first GIS programming at part time job. 4

Programming / Works / OSS Community 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 Highshool Tokyo Denki Univ Part time job Part time job SI Company EC engineer Georepublic Linux/Mozilla Japan Unix Society OSMFJ OSGeo.JP But too many years, I didn’t any GIS programming. 4

Programming / Works / OSS Community 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 Highshool Tokyo Denki Univ Part time job Part time job SI Company EC engineer Georepublic Linux/Mozilla Japan Unix Society OSMFJ OSGeo.JP I became GIS programmer again. 4

Programming / Works / OSS Community 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 Highshool Tokyo Denki Univ Part time job Part time job SI Company EC engineer Georepublic Linux/Mozilla Japan Unix Society OSMFJ OSGeo.JP I join in OSS community over 17 years. 4

Programming / Works / OSS Community 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 Highshool Tokyo Denki Univ Part time job Part time job SI Company EC engineer Georepublic Linux/Mozilla Japan Unix Society OSMFJ OSGeo.JP 4

Two Books Too old… 5

Topics • Overview of “Daredemo Navi” app • Routing function • Voice Navigation • Tiles • Issue of Webview 6

ͩΕͰ΋ Navi • Smartphone application (iOS and Android) dedicated to “͋͠Θͤͷଜ(Village of Happiness)” in Kobe City. • ͩΕͰ΋ means “Everyone” • Application name in English is “UDNavi” • UD = Universal Design 7

4IJBXBTFOPNVSB 4IJBXBTF OP NVSB ޾ͤ ଜ )BQQZ 7JMMBHF 7JMMBHFPG)BQQJOFTT 8

Where? 9

Where? 9

Where? 9

Where? 9

Where? 9

Where? 9

Functions • Navigation • Supports both wheelchair user and blind people • Supports both Outdoor and Indoor (Game map). • Voice navigation enabled for blind users • The map is designed by Kobe city’s Universal design guideline. 10

Technology overview API Server & Map View Tile Server Administrator 11

Technology overview API Server & Map View Tile Server Varnish Cache Administrator 11

Technology overview API Server & Map View Tile Server Varnish Cache Administrator 11

Technology overview API Server & Map View Tile Server Smartphone Application React Native Varnish Cache Administrator 11

Demo 12

https://youtu.be/ EuteOqqCVKo 13

Routing 14

pgRouting • We use pgRouting to make routing API. • We use OpenStreetMap data to support outdoor routing. • We don’t use OpenStreetMap data for indoor routing. 15

osm2pgrouting • We use osm2pgrouting develop branch. • https://github.com/pgRouting/osm2pgrouting/tree/ develop • Develop branch supports “hstore”. • The mean is “We can write queries with tag”. 16

Prepare database createdb xxx_production psql --dbname xxx_production -c 'CREATE EXTENSION postgis' psql --dbname xxx_production -c 'CREATE EXTENSION pgRouting' psql --dbname xxx_production -c 'CREATE EXTENSION hstore' In this application, we do it with Ruby on Rails “db:migrate” task. class AddExtensions < ActiveRecord::Migration[5.0] def up execute "CREATE EXTENSION hstore" execute "CREATE EXTENSION pgrouting" end def down execute "DROP EXTENSION pgrouting" execute "DROP EXTENSION hstore" end end 17

build osm2pgrouting # install package sudo apt install -y \ build-essential expat libexpat1-dev \ libboost-dev libboost-program-options-dev \ libpq-dev libpqxx-dev \ cmake # clone develop branch git clone --branch develop \ https://github.com/pgRouting/osm2pgrouting.git # build osm2pgrouting binary cd osm2pgrouting cmake -H. -Bbuild cd build make # test it ./osm2pgrouting 18

Import task #/bin/sh # download file wget --progress=dot:meta -O shiawasenomura.osm \ ”http://www.overpass-api.de/api/xapi? *[bbox=135.101,34.699,135.131,34.718][@meta]” # import it osm2pgrouting/build/osm2pgrouting -f shiawasenomura.osm \ --conf osm2pgrouting/mapconfig_for_pedestrian.xml \ --dbname xxx_production \ --username xxx \ --schema routing \ --addnodes --hstore --attributes --tags \ --chunk 5000 --postgis --clean \ --password xxxx # remove old file rm shiawasenomura.osm 19

How to make API • API get start position, end position and query type (pedestrian, wheelchair and blind_low_vision). 1. Make a virtual model with position information and query type. 2. Virtual model calculates route. • pedestrian and wheelchair: Return Well-Known Text. • blind_low_vision: pgRouting query get each “ways” and make array of building text. 20

API Client Rails API Ourdoor Model pgRouting 21

API Client Rails API Ourdoor Model Request pgRouting 21

API Client Rails API Ourdoor Model get_routing function Initialize instance Request pgRouting 21

API Client Rails API Ourdoor Model get_routing function Initialize instance Request pgRouting Call function Search route 21

API Client Rails API Ourdoor Model get_routing function Initialize instance Request pgRouting Call function Search route Return JSON 21

Builder Making Query Function result Function get_ routing function WKT Array pedestri an wheel chair blind_ low_ vision pgRouting Query result One Way Way Way Way Way Way Ways Text
 Builder 22 API

Model class OutdoorRouting include ActiveModel::Model attr_reader :result, :source_optional_id, :target_optional_id ……… def initialize(source_lon, source_lat, target_lon, target_lat, search_type, source_name, target_name, source_optional_id = nil, target_optional_id = nil) @source_lon = source_lon @source_lat = source_lat @target_lon = target_lon @target_lat = target_lat @search_type = search_type @source_name = source_name @target_name = target_name @source_optional_id = source_optional_id @target_optional_id = target_optional_id end 23

get_routing def get_routing(language, text) if text return get_routing_text(language) else return get_routing_map end end get_routing_text: blindness get_routing_map: pedestrian and wheelchair 24

get_routing_map def get_routing_map text = false calculate(text) way = get_routing_wkt first_position = get_first_position last_position = get_last_position return { type: "outdoor", source_name: @source_name, target_name: @target_name, first_position: first_position, last_position: last_position, way: way, ……… } end 25

get_routing_text def get_routing_text(language) text = true calculate(text) texts = get_routing_texts(language) return { type: "outdoor", source_name: @source_name, target_name: @target_name, texts: texts, ……… } end 26

calculate def calculate(text) return if @result if @search_type == :pedestrain @result = ActiveRecord::Base.connection.execute(routing_for_pedestrain(text)) elsif @search_type == :wheelchair @result = ActiveRecord::Base.connection.execute(routing_for_wheelchair(text)) elsif @search_type == :blind_low_vision @result = ActiveRecord::Base.connection.execute(routing_for_blind_low_vision(text)) end end misspelling @result is PostgreSQL result (Array). routing_for functions are query builder. 27

Query for pedestrian 28 def routing_for_pedestrain(text) dijkstra_condition = <<-EOS SELECT id, source, target, length_m as cost FROM routing.ways EOS routing_sql(dijkstra_condition, text) end

Query for wheelchair 29 def routing_for_wheelchair(text) dijkstra_condition = <<-EOS SELECT routing.ways.id, routing.ways.source, routing.ways.target, routing.ways.length_m * CASE WHEN routing.osm_ways.tags ? 'incline' THEN 2 ELSE 1 END as cost FROM routing.ways JOIN routing.osm_ways ON (routing.ways.osm_id = routing.osm_ways.osm_id AND NOT routing.osm_ways.tags @> 'highway=>steps' AND NOT routing.osm_ways.tags @> 'highway=>path' AND NOT routing.osm_ways.tags @> 'surface=>wood' AND NOT routing.osm_ways.tags @> 'surface=>unpaved') EOS routing_sql(dijkstra_condition, text) end Define the cost per ways If way has incline tag, the cost will be double. Define ignore ways. • highway=steps • highway=path • surface=wood • surface=unpaved

Query for blind low vision 30 def routing_for_blind_low_vision(text) dijkstra_condition = <<-EOS SELECT routing.ways.id, routing.ways.source, routing.ways.target, routing.ways.length_m * CASE WHEN routing.osm_ways.tags @> 'highway=>steps' THEN 40 WHEN routing.osm_ways.tags @> 'tactile_paving=>yes' THEN 0.5 ELSE 2 END as cost FROM routing.ways JOIN routing.osm_ways ON ( routing.ways.osm_id = routing.osm_ways.osm_id AND ( routing.osm_ways.tags @> 'tactile_paving=>yes' OR ( routing.osm_ways.tags @> 'handrail:left=>yes' OR routing.osm_ways.tags @> 'handrail:right=>yes' OR routing.osm_ways.tags @> 'handrail=>left' OR routing.osm_ways.tags @> 'handrail=>right' OR routing.osm_ways.tags @> 'handrail=>yes' OR routing.osm_ways.tags @> 'handrail=>both' ) ) ) EOS routing_sql(dijkstra_condition, text) end Define the cost per ways If way has highway=step, the cost will be 40 times.
 But way has tactile_paving=yes, the cost will be half. Define to enable to walk
 for blind low vision. • tactile_paving=yes or • the way has handrail

routing_sql 31 def routing_sql(dijkstra_condition, text) grouping_text = "" text_query = "" if text text_query = <<-EOS with_geom.osm_id as osm_id, degrees(ST_azimuth( ST_StartPoint(ST_MakeLine(route_geom)), ST_EndPoint(ST_MakeLine(route_geom)) )) as heading, routing.osm_ways.tags as tags, max(with_geom.seq) as max_seq, EOS grouping_text = <<-EOS JOIN routing.osm_ways ON (with_geom.osm_id = routing.osm_ways.osm_id) group by with_geom.osm_id, routing.osm_ways.tags order by max_seq; EOS end <<-EOS WITH dijkstra AS ( SELECT * FROM pgr_dijkstra( $$#{dijkstra_condition}$$, (SELECT id FROM routing.ways_vertices_pgr ORDER BY the_geom <-> ST_SetSRID(ST_Point(#{source_lon},#{source_lat}),4326) LIMIT 1), (SELECT id FROM routing.ways_vertices_pgr ORDER BY the_geom <-> ST_SetSRID(ST_Point(#{target_lon},#{target_lat}),4326) LIMIT 1), false) ), with_geom AS ( SELECT dijkstra.seq, dijkstra.cost, routing.ways.name, routing.ways.osm_id, CASE WHEN dijkstra.node = routing.ways.source THEN the_geom ELSE ST_Reverse(the_geom) END AS route_geom FROM dijkstra JOIN routing.ways ON (edge = ways.id) ORDER BY seq ) SELECT sum(with_geom.cost) as cost, ST_Length(ST_MakeLine(route_geom)::geography) as length_m, ST_AsText(ST_MakeLine(route_geom)) as geom_text, #{text_query} ST_AsText(ST_StartPoint(ST_MakeLine(route_geom))) as start_position, ST_AsText(ST_EndPoint(ST_MakeLine(route_geom))) as end_position FROM with_geom #{grouping_text} EOS end Making additional query and grouping for text builder. select pgr_dijkstra with custom query with source location and target location Making result

Full of model code 32 https://gist.github.com/smellman/b6198065fd89fa9ac653b9ad48a40379 https://gist.github.com/smellman/ Uploaded at my gist.

How text builder works 33 wheelchair user can look map, the result will be one line.

How text builder works 34 blind low vision need each lines, the result will be 4 lines.

How text builder works 35 Blind low vision need to turn along the way.

generate_builder 36 def generate_builder return if @builder @builder = RoutingTextBuilder.new @result.each_with_index do |r, index| @builder.put(index, r["heading"], r["length_m"]) end @builder.finish end

checking heading 37 Go ahead. Turn right. Turn left. 45 degree 135 degree 315 degree 225 degree If next way is “Go ahead”, set current heading and add way’s length. If next way is “Turn right” or “Turn left”, close current way and start check with new way.

Result 38 curl "http://localhost:3000/api/ routing/text? departure_type=indoor_poi&dep arture_value=57&arrival_type=ind oor_poi&arrival_value=187&route _type=blind_low_vision&languag e=en" { "results": [ … { "type": "outdoor", "source_name": "Tanpoponoie entrance", "target_name": "Main Bldg. , Lodge entrance", "texts": [ "Face towards the east", "Go forward 27m", "Turn left, Go forward 16m", "Turn right, Go forward 114m", "Turn left, Go forward 89m", "Turn right, Go forward 36m", "Turn right, Go forward 5m" ], … }, … ] }

Tile 39

Making custom tile. • Customer want to create map follow a Kobe city’s guideline. • At just the right time, OpenMapTiles was released. • I followed it as a hobby before when it was named OSM2VectorTile. • Also, I know about Maputnik. • Alternative of Mapbox Studio (But I didn’t access current Mapbox Studio…) • For this reason why we use openmaptiles :-) 40

How to host MVT 1. Making Mapbox Vector Tile with openmaptiles. 2. Upload mbtiles to server. 3. Setup tileserver-gl and nginx. 4. Making mapbox-gl-style json file. 5. Add new design to tileserver-gl. 41

How to make MVT. 1. Get machine. 2. Setup docker environments. 3. Run git clone 4. Run docker-compose pull 5. Run quickstart.sh script. 6. Check created tiles. 42

Get machine • Buy Thinkpad X220 (Junk). • 15,000 JPY. • Buy 16GB memory and 240GB SSD. • 18,000 JPY. • Total cost: 33,000 JPY. 43

Setup docker environments. 44 sudo apt-get install docker.io docker-compose sudo service docker start sudo usermod -aG docker $USER sudo reboot If you use Ubuntu < 17.04, don’t install docker-compose with apt. Use following command. curl -L https://github.com/docker/compose/releases/download/1.14.0/docker- compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

git clone to check tile 45 mkdir -p ~/develop/osm cd develop/osm git clone https://github.com/openmaptiles/openmaptiles.git cd ./openmaptiles docker-compose pull ./quickstart.sh japan make start-tileserver Open http://localhost:8080/ in your browser.

Enable to upper levels • openmaptiles makes MVT between level 0 to level 7 by default. • Enable to upper levels, you need following. 1. edit .env file’s QUICKSTART_MAX_ZOOM 2. edit data/docker-compose-config.yml 3. run quickstart.sh again. • NOTE: make download-geofabrik task doesn’t make new data/docker-compose-config.yml. 46

Upload mbtiles to server • Only scp data/tiles.mbtiles to server. 47 scp data/tiles.mbtiles your_account@your_domain:~/

Setup tileserver-gl and nginx 48 ssh your_account@your_domain # setup docker sudo apt-get install nginx mkdir tileserver cd tileserver cp ~/tiles.mbtiles . git clone -b gh-pages \ https://github.com/smellman/fonts.git git clone -b gh-pages \ https://github.com/smellman/osm-bright-gl-style.git sprites mkdir styles vim docker-compose.yml

system layout 49 UJMFTFSWFSHM UJMFTFSWFSHM UJMFTFSWFSHM UJMFTFSWFSHM WBSOJTIDBDIF 1PSU 1PSU 1PSU 1PSU 1PSU %PDLFS$PNQPTF IUUQ IUUQT

docker-compose.yml 50 version: '2' services: varnish: image: eeacms/varnish ports: - "6081:6081" depends_on: - raster-tileserver environment: BACKENDS: "raster-tileserver" BACKENDS_PORT: "80" BACKENDS_PROBE_INTERVAL: "10s" BACKENDS_PROBE_TIMEOUT: "2s" DNS_ENABLED: "true" restart: always vector-tileserver: image: klokantech/tileserver-gl ports: - "8080:80" volumes: - .:/data restart: always raster-tileserver: image: klokantech/tileserver-gl volumes: - .:/data restart: always

nginx 51 server { listen 443 ssl http2; listen [::]:443 ssl http2; # ssl configuration insert here server_name your_domain; location / { proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_pass http://localhost:8080; } location ~ ^/.*\.png { proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_pass http://localhost:6081; } } png resource only forward varnish

start docker 52 docker-compose pull docker-compose up -d docker-compose scale raster-tileserver=3

Making mapbox-gl-style json file • Go to https://editor.openmaptiles.org 53

editor 54 Select style

editor 55 1. Move to your MVT support area. 2. Select “Source” and “Sytle Settings” tag.

Source 56 1. Copy TileJSON URL in your domain. 2. Change “#openmaptiles” TileJSON URL.

Style Settings 57 Change “Sprite URL” to https://smellman.github.io/osm-bright-gl-style/sprite Change “Glyphs URL” to https://smellman.github.io/fonts/{fontstack}/{range}.pbf

Edit style 58

Export style 59 Select “Export” tab and download style. The style is json file.

Edit style in editor 60 "sources": { "openmaptiles": { "type": "vector", "url": "mbtiles://{japan-vector}" } }, "sprite": "sprite", "glyphs": "{fontstack}/{range}.pbf", Change sources’s url, sprite, glyphs.

Add new design into tileserver-gl 61 scp yourstyle.json \ youraccount@yourdomain:~/tileserver/styles ssh youraccount@yourdomain cd tileserver vim config.json

config.json 62 { "options": { "paths": { "root": "", "fonts": "fonts", "sprites": "sprites", "styles": "styles", "mbtiles": "" }, "formatQuality": { "jpeg": 80, "webp": 90, "pngQuantization": false, "png": 90 }, "maxSize": 2048, "pbfAlias": "pbf", "serveAllFonts": false }, "styles": { "kobe": { "style": "kobe.json", "tilejson": { } } }, "data": { "japan-vector": { "mbtiles": "japan.mbtiles" } } } And restart docker-compose

Support iOS and Android in Webview • ReactNative supports native Webview. • iOS’s Webview can use mapbox-gl-js. • Android’s Webview has many problem to use mapbox- gl-js. • Use Leaflet.js instead of mapbox-gl-js. 63

Platform.OS 64 const OUTDOOR_MAP_PATH = 'map' const OUTDOOR_ANDROID_MAP_PATH = 'android_map' export const getOutdoorMapURL = () => { if (Platform.OS === 'ios') { return get_service_url(OUTDOOR_MAP_PATH) } return get_service_url(OUTDOOR_ANDROID_MAP_PATH) }

Client code use 65 import { getOutdoorMapURL, getIndoorMapURL, I18n } from '../../constants' export default class ShowRoute extends Component { … return ( { this.outdoor_webview = webview }} source={{uri: getOutdoorMapURL()}} style={{flex: 1}} startInLoadingState={true} domStorageEnabled={true} onMessage={(e) => {}} onLoadEnd={() => {this.outdoor_map_loaded(current_map)}} /> {paging_component} ) … }

outdoor_map_loaded 66 outdoor_map_loaded = (current_map) => { if (this.outdoor_webview) { const value = current_map this.outdoor_webview.postMessage(JSON.stringify(value)) console.log(JSON.stringify(value)) } } Call same javascript in WebView.

Server side for iOS 67 $(document).on "ready", -> if $("#map").is('div') and $("#map").data("map-type") == "normal" map = new exports.Map "map" map.initMap() document.addEventListener 'message', (e) -> data = JSON.parse(e.data) if (data.geojson) showData(data) if (data.heading) updateHeading(data) return showData = (data) -> if map.map.loaded() map.showData(data) else map.map.on 'load', -> map.showData(data)

Server side for Android 68 $(document).on "ready", -> if $("#android_map").is('div') and $("#android_map").data("map-type") == "normal" map = new exports.AndroidMap "android_map" map.initMap() document.addEventListener 'message', (e) -> data = JSON.parse(e.data) if (data.geojson) showData(data) return showData = (data) -> setTimeout -> map.showData(data) , 1000 return

Two classes 69 class Map constructor: (map_id) -> @map = new mapboxgl.Map container: map_id, style: ‘…’ … showData: (data) -> @map.addSource 'routing', type: 'geojson', data: data.geojson … class AndroidMap constructor: (map_id) -> @map = L.map map_id L.tileLayer(“…”, { … }).addTo(@map) … showData: (data) -> map = @map obj = L.geoJson data.geojson, pointToLayer: (geoJsonPoint, latlng) -> … … for iOS and Admin View for Android

Summary 70