$30 off During Our Annual Pro Sale. View Details »

Day1-1030-Mobile app development with routing and voice navigation

sotm2017
September 01, 2017

Day1-1030-Mobile app development with routing and voice navigation

sotm2017

September 01, 2017
Tweet

More Decks by sotm2017

Other Decks in Research

Transcript

  1. Mobile app development
    with routing and voice
    navigation
    Taro Matsuzawa (@smellman)

    Georepublic

    Open Street Map Foundation Japan

    OSGeo.JP

    Japan Unix Society (jus)
    1

    View Slide

  2. $ whoami
    • Georepublic Senior Engineer

    • Programmer(Ruby/Python/Javascript)

    • System / Network designer of Linux system

    • Tile guru.

    • OSM Tile server

    • Mapbox Vector Tile

    • Gamemap (L.CRS.Simple)
    2

    View Slide

  3. SOTM2017 Local Team
    3

    View Slide

  4. Programming / Works / OSS
    Community
    97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17
    Highshool
    Tokyo Denki Univ
    Part time job
    Part time job
    SI Company
    EC engineer
    Georepublic
    Linux/Mozilla
    Japan Unix Society
    OSMFJ
    OSGeo.JP
    I started my first GIS programming at part time job.
    But too many years, I didn’t any GIS programming.
    I buy computer and start learning programming and Linux.
    I became GIS programmer again.
    I join in OSS community over 17 years.
    4

    View Slide

  5. Two Books
    Too old…
    5

    View Slide

  6. Topics
    • Overview of “Daredemo Navi” app

    • Routing function

    • Voice Navigation

    • Tiles

    • Issue of Webview
    6

    View Slide

  7. ͩΕͰ΋ Navi
    • Smartphone application (iOS and Android) dedicated to
    “͋͠Θͤͷଜ(Village of Happiness)” in Kobe City.

    • ͩΕͰ΋ means “Everyone”

    • Application name in English is “UDNavi”

    • UD = Universal Design
    7

    View Slide

  8. 4IJBXBTFOPNVSB
    4IJBXBTF OP NVSB
    ޾ͤ ଜ
    )BQQZ 7JMMBHF
    7JMMBHFPG)BQQJOFTT
    8

    View Slide

  9. Where?
    9

    View Slide

  10. Functions
    • Navigation

    • Supports both wheelchair user and blind people

    • Supports both Outdoor and Indoor (Game map).

    • Voice navigation enabled for blind users

    • The map is designed by Kobe city’s Universal design
    guideline.
    10

    View Slide

  11. Technology overview
    API Server & Map View Tile Server
    Smartphone Application
    React Native
    Varnish
    Cache
    Administrator
    11

    View Slide

  12. Demo
    12

    View Slide

  13. 13

    View Slide

  14. Routing
    14

    View Slide

  15. pgRouting
    • We use pgRouting to make routing API.

    • We use OpenStreetMap data to support outdoor
    routing.

    • We don’t use OpenStreetMap data for indoor routing.
    15

    View Slide

  16. osm2pgrouting
    • We use osm2pgrouting develop branch.

    • https://github.com/pgRouting/osm2pgrouting/tree/
    develop

    • Develop branch supports “hstore”.

    • The mean is “We can write queries with tag”.
    16

    View Slide

  17. Prepare database
    createdb xxx_production
    psql --dbname xxx_production -c 'CREATE EXTENSION postgis'
    psql --dbname xxx_production -c 'CREATE EXTENSION pgRouting'
    psql --dbname xxx_production -c 'CREATE EXTENSION hstore'
    In this application, we do it with Ruby on Rails “db:migrate” task.
    class AddExtensions < ActiveRecord::Migration[5.0]
    def up
    execute "CREATE EXTENSION hstore"
    execute "CREATE EXTENSION pgrouting"
    end
    def down
    execute "DROP EXTENSION pgrouting"
    execute "DROP EXTENSION hstore"
    end
    end
    17

    View Slide

  18. build osm2pgrouting
    # install package
    sudo apt install -y \
    build-essential expat libexpat1-dev \
    libboost-dev libboost-program-options-dev \
    libpq-dev libpqxx-dev \
    cmake
    # clone develop branch
    git clone --branch develop \
    https://github.com/pgRouting/osm2pgrouting.git
    # build osm2pgrouting binary
    cd osm2pgrouting
    cmake -H. -Bbuild
    cd build
    make
    # test it
    ./osm2pgrouting
    18

    View Slide

  19. Import task
    #/bin/sh
    # download file
    wget --progress=dot:meta -O shiawasenomura.osm \
    ”http://www.overpass-api.de/api/xapi?
    *[bbox=135.101,34.699,135.131,34.718][@meta]”
    # import it
    osm2pgrouting/build/osm2pgrouting -f shiawasenomura.osm \
    --conf osm2pgrouting/mapconfig_for_pedestrian.xml \
    --dbname xxx_production \
    --username xxx \
    --schema routing \
    --addnodes --hstore --attributes --tags \
    --chunk 5000 --postgis --clean \
    --password xxxx
    # remove old file
    rm shiawasenomura.osm
    19

    View Slide

  20. How to make API
    • API get start position, end position and query type
    (pedestrian, wheelchair and blind_low_vision).

    1. Make a virtual model with position information and
    query type.

    2. Virtual model calculates route.

    • pedestrian and wheelchair: Return Well-Known Text.

    • blind_low_vision: pgRouting query get each “ways”
    and make array of building text.
    20

    View Slide

  21. API
    Client
    Rails
    API
    Ourdoor Model
    get_routing
    function
    Initialize instance
    Request
    pgRouting
    Call function
    Search
    route
    Return JSON
    21

    View Slide

  22. Builder Making
    Query
    Function
    result
    Function
    get_
    routing
    function
    WKT
    Array
    pedestri
    an
    wheel
    chair
    blind_
    low_
    vision
    pgRouting
    Query
    result
    One Way
    Way
    Way
    Way
    Way
    Way
    Ways
    Text

    Builder
    22
    API

    View Slide

  23. Model
    class OutdoorRouting
    include ActiveModel::Model
    attr_reader :result, :source_optional_id, :target_optional_id
    ………
    def initialize(source_lon, source_lat, target_lon, target_lat,
    search_type, source_name, target_name, source_optional_id = nil,
    target_optional_id = nil)
    @source_lon = source_lon
    @source_lat = source_lat
    @target_lon = target_lon
    @target_lat = target_lat
    @search_type = search_type
    @source_name = source_name
    @target_name = target_name
    @source_optional_id = source_optional_id
    @target_optional_id = target_optional_id
    end
    23

    View Slide

  24. get_routing
    def get_routing(language, text)
    if text
    return get_routing_text(language)
    else
    return get_routing_map
    end
    end
    get_routing_text: blindness
    get_routing_map: pedestrian and wheelchair
    24

    View Slide

  25. get_routing_map
    def get_routing_map
    text = false
    calculate(text)
    way = get_routing_wkt
    first_position = get_first_position
    last_position = get_last_position
    return {
    type: "outdoor",
    source_name: @source_name,
    target_name: @target_name,
    first_position: first_position,
    last_position: last_position,
    way: way,
    ………
    }
    end
    25

    View Slide

  26. get_routing_text
    def get_routing_text(language)
    text = true
    calculate(text)
    texts = get_routing_texts(language)
    return {
    type: "outdoor",
    source_name: @source_name,
    target_name: @target_name,
    texts: texts,
    ………
    }
    end
    26

    View Slide

  27. calculate
    def calculate(text)
    return if @result
    if @search_type == :pedestrain
    @result =
    ActiveRecord::Base.connection.execute(routing_for_pedestrain(text))
    elsif @search_type == :wheelchair
    @result =
    ActiveRecord::Base.connection.execute(routing_for_wheelchair(text))
    elsif @search_type == :blind_low_vision
    @result =
    ActiveRecord::Base.connection.execute(routing_for_blind_low_vision(text))
    end
    end
    misspelling
    @result is PostgreSQL result (Array).
    routing_for functions are query builder.
    27

    View Slide

  28. Query for pedestrian
    28
    def routing_for_pedestrain(text)
    dijkstra_condition = <<-EOS
    SELECT id, source, target, length_m as cost
    FROM routing.ways
    EOS
    routing_sql(dijkstra_condition, text)
    end

    View Slide

  29. Query for wheelchair
    29
    def routing_for_wheelchair(text)
    dijkstra_condition = <<-EOS
    SELECT
    routing.ways.id, routing.ways.source,
    routing.ways.target,
    routing.ways.length_m *
    CASE
    WHEN routing.osm_ways.tags ? 'incline' THEN 2
    ELSE 1
    END
    as cost
    FROM routing.ways
    JOIN routing.osm_ways
    ON (routing.ways.osm_id = routing.osm_ways.osm_id
    AND
    NOT routing.osm_ways.tags @> 'highway=>steps'
    AND
    NOT routing.osm_ways.tags @> 'highway=>path'
    AND
    NOT routing.osm_ways.tags @> 'surface=>wood'
    AND
    NOT routing.osm_ways.tags @> 'surface=>unpaved')
    EOS
    routing_sql(dijkstra_condition, text)
    end
    Define the cost per ways
    If way has incline tag,
    the cost will be double.
    Define ignore ways.
    • highway=steps
    • highway=path
    • surface=wood
    • surface=unpaved

    View Slide

  30. Query for blind low vision
    30
    def routing_for_blind_low_vision(text)
    dijkstra_condition = <<-EOS
    SELECT
    routing.ways.id, routing.ways.source, routing.ways.target,
    routing.ways.length_m *
    CASE
    WHEN routing.osm_ways.tags @> 'highway=>steps' THEN 40
    WHEN routing.osm_ways.tags @> 'tactile_paving=>yes' THEN 0.5
    ELSE 2
    END
    as cost
    FROM routing.ways
    JOIN routing.osm_ways
    ON (
    routing.ways.osm_id = routing.osm_ways.osm_id
    AND
    (
    routing.osm_ways.tags @> 'tactile_paving=>yes'
    OR
    (
    routing.osm_ways.tags @> 'handrail:left=>yes'
    OR
    routing.osm_ways.tags @> 'handrail:right=>yes'
    OR
    routing.osm_ways.tags @> 'handrail=>left'
    OR
    routing.osm_ways.tags @> 'handrail=>right'
    OR
    routing.osm_ways.tags @> 'handrail=>yes'
    OR
    routing.osm_ways.tags @> 'handrail=>both'
    )
    )
    )
    EOS
    routing_sql(dijkstra_condition, text)
    end
    Define the cost per ways
    If way has highway=step,
    the cost will be 40 times.

    But way has tactile_paving=yes,
    the cost will be half.
    Define to enable to walk

    for blind low vision.
    • tactile_paving=yes
    or
    • the way has handrail

    View Slide

  31. routing_sql
    31
    def routing_sql(dijkstra_condition, text)
    grouping_text = ""
    text_query = ""
    if text
    text_query = <<-EOS
    with_geom.osm_id as osm_id,
    degrees(ST_azimuth(
    ST_StartPoint(ST_MakeLine(route_geom)),
    ST_EndPoint(ST_MakeLine(route_geom))
    )) as heading,
    routing.osm_ways.tags as tags,
    max(with_geom.seq) as max_seq,
    EOS
    grouping_text = <<-EOS
    JOIN routing.osm_ways
    ON (with_geom.osm_id = routing.osm_ways.osm_id)
    group by with_geom.osm_id, routing.osm_ways.tags
    order by max_seq;
    EOS
    end
    <<-EOS
    WITH
    dijkstra AS (
    SELECT * FROM pgr_dijkstra(
    $$#{dijkstra_condition}$$,
    (SELECT id FROM routing.ways_vertices_pgr
    ORDER BY the_geom <-> ST_SetSRID(ST_Point(#{source_lon},#{source_lat}),4326) LIMIT 1),
    (SELECT id FROM routing.ways_vertices_pgr
    ORDER BY the_geom <-> ST_SetSRID(ST_Point(#{target_lon},#{target_lat}),4326) LIMIT 1),
    false)
    ),
    with_geom AS (
    SELECT dijkstra.seq, dijkstra.cost, routing.ways.name, routing.ways.osm_id,
    CASE
    WHEN dijkstra.node = routing.ways.source THEN the_geom
    ELSE ST_Reverse(the_geom)
    END AS route_geom
    FROM dijkstra JOIN routing.ways
    ON (edge = ways.id) ORDER BY seq
    )
    SELECT
    sum(with_geom.cost) as cost,
    ST_Length(ST_MakeLine(route_geom)::geography) as length_m,
    ST_AsText(ST_MakeLine(route_geom)) as geom_text,
    #{text_query}
    ST_AsText(ST_StartPoint(ST_MakeLine(route_geom))) as start_position,
    ST_AsText(ST_EndPoint(ST_MakeLine(route_geom))) as end_position
    FROM with_geom
    #{grouping_text}
    EOS
    end
    Making additional query
    and grouping
    for text builder.
    select pgr_dijkstra
    with custom query
    with source location and
    target location
    Making result

    View Slide

  32. Full of model code
    32
    https://gist.github.com/smellman/b6198065fd89fa9ac653b9ad48a40379
    https://gist.github.com/smellman/
    Uploaded at my gist.

    View Slide

  33. How text builder works
    33
    wheelchair user can look map,
    the result will be one line.

    View Slide

  34. How text builder works
    34
    blind low vision need each lines,
    the result will be 4 lines.

    View Slide

  35. How text builder works
    35
    Blind low vision need
    to turn along the way.

    View Slide

  36. generate_builder
    36
    def generate_builder
    return if @builder
    @builder = RoutingTextBuilder.new
    @result.each_with_index do |r, index|
    @builder.put(index, r["heading"], r["length_m"])
    end
    @builder.finish
    end

    View Slide

  37. checking heading
    37
    Go ahead.
    Turn right.
    Turn left.
    45 degree
    135 degree
    315 degree
    225 degree
    If next way is “Go ahead”,
    set current heading
    and add way’s length.
    If next way is “Turn right”
    or “Turn left”,
    close current way
    and start check with
    new way.

    View Slide

  38. Result
    38
    curl "http://localhost:3000/api/
    routing/text?
    departure_type=indoor_poi&dep
    arture_value=57&arrival_type=ind
    oor_poi&arrival_value=187&route
    _type=blind_low_vision&languag
    e=en"
    {
    "results": [

    {
    "type": "outdoor",
    "source_name": "Tanpoponoie entrance",
    "target_name": "Main Bldg. , Lodge entrance",
    "texts": [
    "Face towards the east",
    "Go forward 27m",
    "Turn left, Go forward 16m",
    "Turn right, Go forward 114m",
    "Turn left, Go forward 89m",
    "Turn right, Go forward 36m",
    "Turn right, Go forward 5m"
    ],

    },

    ]
    }

    View Slide

  39. Tile
    39

    View Slide

  40. Making custom tile.
    • Customer want to create map follow a Kobe city’s guideline.

    • At just the right time, OpenMapTiles was released.

    • I followed it as a hobby before when it was named
    OSM2VectorTile.

    • Also, I know about Maputnik.

    • Alternative of Mapbox Studio (But I didn’t access
    current Mapbox Studio…)

    • For this reason why we use openmaptiles :-)
    40

    View Slide

  41. How to host MVT
    1. Making Mapbox Vector Tile with openmaptiles.

    2. Upload mbtiles to server.

    3. Setup tileserver-gl and nginx.

    4. Making mapbox-gl-style json file.

    5. Add new design to tileserver-gl.
    41

    View Slide

  42. How to make MVT.
    1. Get machine.

    2. Setup docker environments.

    3. Run git clone

    4. Run docker-compose pull

    5. Run quickstart.sh script.

    6. Check created tiles.
    42

    View Slide

  43. Get machine
    • Buy Thinkpad X220 (Junk).

    • 15,000 JPY.

    • Buy 16GB memory and
    240GB SSD.

    • 18,000 JPY.

    • Total cost: 33,000 JPY.
    43

    View Slide

  44. Setup docker environments.
    44
    sudo apt-get install docker.io docker-compose
    sudo service docker start
    sudo usermod -aG docker $USER
    sudo reboot
    If you use Ubuntu < 17.04,
    don’t install docker-compose with apt.
    Use following command.
    curl -L https://github.com/docker/compose/releases/download/1.14.0/docker-
    compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

    View Slide

  45. git clone to check tile
    45
    mkdir -p ~/develop/osm
    cd develop/osm
    git clone https://github.com/openmaptiles/openmaptiles.git
    cd ./openmaptiles
    docker-compose pull
    ./quickstart.sh japan
    make start-tileserver
    Open http://localhost:8080/ in your browser.

    View Slide

  46. Enable to upper levels
    • openmaptiles makes MVT between level 0 to level 7 by
    default.

    • Enable to upper levels, you need following.

    1. edit .env file’s QUICKSTART_MAX_ZOOM

    2. edit data/docker-compose-config.yml

    3. run quickstart.sh again.

    • NOTE: make download-geofabrik task doesn’t make new
    data/docker-compose-config.yml.
    46

    View Slide

  47. Upload mbtiles to server
    • Only scp data/tiles.mbtiles to server.
    47
    scp data/tiles.mbtiles your_account@your_domain:~/

    View Slide

  48. Setup tileserver-gl and
    nginx
    48
    ssh your_account@your_domain
    # setup docker
    sudo apt-get install nginx
    mkdir tileserver
    cd tileserver
    cp ~/tiles.mbtiles .
    git clone -b gh-pages \
    https://github.com/smellman/fonts.git
    git clone -b gh-pages \
    https://github.com/smellman/osm-bright-gl-style.git sprites
    mkdir styles
    vim docker-compose.yml

    View Slide

  49. system layout
    49
    UJMFTFSWFSHM
    UJMFTFSWFSHM
    UJMFTFSWFSHM
    UJMFTFSWFSHM
    WBSOJTIDBDIF
    1PSU
    1PSU
    1PSU
    1PSU
    1PSU
    %PDLFS$PNQPTF
    IUUQ
    IUUQT

    View Slide

  50. docker-compose.yml
    50
    version: '2'
    services:
    varnish:
    image: eeacms/varnish
    ports:
    - "6081:6081"
    depends_on:
    - raster-tileserver
    environment:
    BACKENDS: "raster-tileserver"
    BACKENDS_PORT: "80"
    BACKENDS_PROBE_INTERVAL: "10s"
    BACKENDS_PROBE_TIMEOUT: "2s"
    DNS_ENABLED: "true"
    restart: always
    vector-tileserver:
    image: klokantech/tileserver-gl
    ports:
    - "8080:80"
    volumes:
    - .:/data
    restart: always
    raster-tileserver:
    image: klokantech/tileserver-gl
    volumes:
    - .:/data
    restart: always

    View Slide

  51. nginx
    51
    server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    # ssl configuration insert here
    server_name your_domain;
    location / {
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_pass http://localhost:8080;
    }
    location ~ ^/.*\.png {
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_pass http://localhost:6081;
    }
    } png resource only forward varnish

    View Slide

  52. start docker
    52
    docker-compose pull
    docker-compose up -d
    docker-compose scale raster-tileserver=3

    View Slide

  53. Making mapbox-gl-style
    json file
    • Go to https://editor.openmaptiles.org
    53

    View Slide

  54. editor
    54
    Select style

    View Slide

  55. editor
    55
    1. Move to your MVT support area.
    2. Select “Source” and “Sytle Settings” tag.

    View Slide

  56. Source
    56
    1. Copy TileJSON URL in your domain.
    2. Change “#openmaptiles” TileJSON URL.

    View Slide

  57. Style Settings
    57
    Change “Sprite URL” to
    https://smellman.github.io/osm-bright-gl-style/sprite
    Change “Glyphs URL” to
    https://smellman.github.io/fonts/{fontstack}/{range}.pbf

    View Slide

  58. Edit style
    58

    View Slide

  59. Export style
    59
    Select “Export” tab and
    download style.
    The style is json file.

    View Slide

  60. Edit style in editor
    60
    "sources": {
    "openmaptiles": {
    "type": "vector",
    "url": "mbtiles://{japan-vector}"
    }
    },
    "sprite": "sprite",
    "glyphs": "{fontstack}/{range}.pbf",
    Change sources’s url, sprite, glyphs.

    View Slide

  61. Add new design into
    tileserver-gl
    61
    scp yourstyle.json \
    youraccount@yourdomain:~/tileserver/styles
    ssh youraccount@yourdomain
    cd tileserver
    vim config.json

    View Slide

  62. config.json
    62
    {
    "options": {
    "paths": {
    "root": "",
    "fonts": "fonts",
    "sprites": "sprites",
    "styles": "styles",
    "mbtiles": ""
    },
    "formatQuality": {
    "jpeg": 80,
    "webp": 90,
    "pngQuantization": false,
    "png": 90
    },
    "maxSize": 2048,
    "pbfAlias": "pbf",
    "serveAllFonts": false
    },
    "styles": {
    "kobe": {
    "style": "kobe.json",
    "tilejson": {
    }
    }
    },
    "data": {
    "japan-vector": {
    "mbtiles": "japan.mbtiles"
    }
    }
    }
    And restart docker-compose

    View Slide

  63. Support iOS and Android in
    Webview
    • ReactNative supports native Webview.

    • iOS’s Webview can use mapbox-gl-js.

    • Android’s Webview has many problem to use mapbox-
    gl-js.

    • Use Leaflet.js instead of mapbox-gl-js.
    63

    View Slide

  64. Platform.OS
    64
    const OUTDOOR_MAP_PATH = 'map'
    const OUTDOOR_ANDROID_MAP_PATH = 'android_map'
    export const getOutdoorMapURL = () => {
    if (Platform.OS === 'ios') {
    return get_service_url(OUTDOOR_MAP_PATH)
    }
    return get_service_url(OUTDOOR_ANDROID_MAP_PATH)
    }

    View Slide

  65. Client code use
    65
    import {
    getOutdoorMapURL,
    getIndoorMapURL,
    I18n
    } from '../../constants'
    export default class ShowRoute extends Component {

    return (

    ref={webview => { this.outdoor_webview = webview }}
    source={{uri: getOutdoorMapURL()}}
    style={{flex: 1}}
    startInLoadingState={true}
    domStorageEnabled={true}
    onMessage={(e) => {}}
    onLoadEnd={() => {this.outdoor_map_loaded(current_map)}}
    />
    {paging_component}

    )

    }

    View Slide

  66. outdoor_map_loaded
    66
    outdoor_map_loaded = (current_map) => {
    if (this.outdoor_webview) {
    const value = current_map
    this.outdoor_webview.postMessage(JSON.stringify(value))
    console.log(JSON.stringify(value))
    }
    }
    Call same javascript in WebView.

    View Slide

  67. Server side for iOS
    67
    $(document).on "ready", ->
    if $("#map").is('div') and $("#map").data("map-type") == "normal"
    map = new exports.Map "map"
    map.initMap()
    document.addEventListener 'message', (e) ->
    data = JSON.parse(e.data)
    if (data.geojson)
    showData(data)
    if (data.heading)
    updateHeading(data)
    return
    showData = (data) ->
    if map.map.loaded()
    map.showData(data)
    else
    map.map.on 'load', ->
    map.showData(data)

    View Slide

  68. Server side for Android
    68
    $(document).on "ready", ->
    if $("#android_map").is('div') and $("#android_map").data("map-type") ==
    "normal"
    map = new exports.AndroidMap "android_map"
    map.initMap()
    document.addEventListener 'message', (e) ->
    data = JSON.parse(e.data)
    if (data.geojson)
    showData(data)
    return
    showData = (data) ->
    setTimeout ->
    map.showData(data)
    , 1000
    return

    View Slide

  69. Two classes
    69
    class Map
    constructor: (map_id) ->
    @map = new mapboxgl.Map
    container: map_id,
    style: ‘…’

    showData: (data) ->
    @map.addSource 'routing',
    type: 'geojson',
    data: data.geojson

    class AndroidMap
    constructor: (map_id) ->
    @map = L.map map_id
    L.tileLayer(“…”, {

    }).addTo(@map)

    showData: (data) ->
    map = @map
    obj = L.geoJson data.geojson,
    pointToLayer: (geoJsonPoint, latlng) ->


    for iOS and Admin View for Android

    View Slide

  70. Summary
    70

    View Slide