Upgrade to Pro — share decks privately, control downloads, hide ads and more …

讓地圖在手機上活起來 — 離線公車導覽

0e543123a2cc239ba3b6c45068663a5a?s=47 Timothy
November 09, 2013

讓地圖在手機上活起來 — 離線公車導覽

0e543123a2cc239ba3b6c45068663a5a?s=128

Timothy

November 09, 2013
Tweet

Transcript

  1. 讓地圖在⼿手機上活起來 — 離線公⾞車導覽
 Makes OSM maps alive and kicking on

    iOS — offline transit routing 
 
 iOS 離線地圖實作的⼯工具介紹跟觀念 Timothy Wu 2013-11-09
  2. •Not affiliated with Academic Sinica. •Was involved in Bioinformatics •Entrepreneur

    wanna-be •I think I have a little Prosopagnosia (Face-blindness) (臉盲症) •Wanna see what’s like to be one? •http://tinyurl.com/mlesk4n •OSM newbie (and probably will be for a long time) A little about myself
  3. What do we mean by alive and kicking? I want

    to do that all inside an iOS cell phone, all without the internet. Nada.
  4. Why OpenStreetMap? •Google License restrictions •Can’t save map tiles for

    offline viewing (at least not on iOS) •Can’t store requested Google map Point of Interests (POIs). •Can’t save routes and directions for query •No real time navigation allowed
  5. Thinking backward — what it involves in the grand picture

    offline map & routing Map images OSM data Stamen’s toner map graph structure
  6. 忠孝東路 中⼭山北路 南京東路 市⺠民⼤大道 松江路 重慶北路 What graph? Treat all

    road data as nodes and edges Then find the “best” way around it
 (may mean the shortest) node edge
  7. 忠孝東路 中⼭山北路 南京東路 市⺠民⼤大道 松江路 重慶北路 How to find the

    best path? • The easy answer is: try it all and then get the best (or shortest) one. • There is Dijkstra’s algorithm. Really, it tries everything from origin to destination, just efficiently. We’ll leave it as that. ?
  8. How about bus? offline map & routing Map images OSM

    data graph structure Bus data Calculate in a similar fashion
  9. OSM data The OSM data Editor: id, or josm, or

    others generate XML data
  10. OSM data The OSM data — nodes

  11. OSM data The OSM data — ways

  12. The grand picture offline map & routing Map images OSM

    data graph structure Bus data
  13. Getting the images Map images OSM data •One way is

    we DON’T generate it ourself •Just grab the images from the servers
  14. Getting the images Map images OSM data • http://tile.stamen.com/toner/<zoom>/<x>/<y>.png •

    http://tile.stamen.com/terrain/<zoom>/<x>/<y>.jpg • http://tile.stamen.com/watercolor/<zoom>/<x>/<y>.jpg
  15. Getting the images Map images OSM data • http://tile.stamen.com/watercolor/<zoom>/<x>/<y>.jpg Zoom

    level: 0 x, y = 0, 0 0, 1 1, 1 1, 0 Zoom level: 1 Zoom level Max: 20, min: 0 ! Zoom levels 20, 19, 18 corresponds to scale 1, 1/2, 1/4
  16. Landez to get images Map images OSM data https://github.com/makinacorpus/landez Landez

    Landez is a code library written in Python Tiles MBTiles MBTiles is a simply a SQLite3 database with a specific schema defined by MapBox, inc
  17. Landez to get images Map images OSM data Three things

    we tell Landez with A, B, C A: Bounding box ! B: Zoom level
 
 C: URL
  18. Map images OSM data A: Bounding box: 1. Left 2.

    bottom 3. right 4. top In another word:
 “min_Longitude , min_Latitude , max_Longitude , max_Latitude” Landez to get images
  19. Landez to get images Map images OSM data B: zoom

    level Say, 12 to 15. ! You need to be careful with tiles. 
 With each increase in level, you get more tiles then you all prior lower zoom levels combined! ! At the maximum zoom level (20), the whole world is a square of 262,144 screens wide and 262,144 screens tall (assuming a screen is 1024 resolution in width and height)
  20. Landez to get images Map images OSM data C: URL

    http://tile.stamen.com/watercolor/{z}/{x}/{y}.jpg
  21. Landez to get images from landez import MBTilesBuilder ! mb

    = MBTilesBuilder(filepath='output.mbtiles', tiles_url='http://tile.stamen.com/watercolor/{z}/{x}/ {y}.jpg')
 mb.add_coverage(bbox=(-122.1099,37.4531,-121.8768,37.6107), zoomlevels=range(12, 16))
 mb.run() If no URL given, it’ll just grab from the official OpenStreetMaps server. 
 Guarantee ugly tiles (sorry, OSM), but easy peasy. Python script to get images from server
  22. Mapnik to generate image Map images OSM data •The second

    way is to generate the tiles ourself •The most well known is Mapnik, a C++ & Python program.
  23. Mapnik flow chart Map images OSM data OSM XML Tiles

    Mapnik osm2pqsql, or Imposm PostGIS A pain
  24. Setting up PostGIS •Here is what I do on PostGIS

    8.4 (granted, an older vsion) ! •Install Postgresql and PostGIS ! •Create cluster (just DB) ! ! ! •Start postgres initdb -D /var/lib/postgresql/data postgres -D /var/lib/postgresql/data
  25. Setting up PostGIS •Set it so I bind to the

    IP and port and expose it ! ! ! ! ! •Trust the connection to the database to my computer In /var/lib/postgresql/data/postgresql.conf In /var/lib/postgresql/data/pg_hba.conf
  26. Setting up PostGIS •Create database • As “postgres" user, create

    postgresql tables using Imposm’s create-db.sh script (and modified for the right db name) • (I was using Imposm 2.0) create-db.sh
  27. Importing into PostGIS •Finally, import the data osm2pgsql -c -G

    -U postgres -d gis -S /usr/share/ osm2pgsql/default.style your_file.osm.pbf (or .osm)
  28. Next, Mapnik Map images OSM data Tiles Mapnik osm2pqsql, or

    Imposm PostGIS OSM XML Stylesheet
  29. Mapnik Stylesheet Map images OSM data •It ONLY took 14,613

    lines of stylesheet to make this! •I can’t find any editors for it. •Documents are also difficult to come by •Someone wrote a pdf explaining version 0.7 (it’s 2.2.0 now?) and seems difficult but understandable
  30. Mapnik Stylesheet — 14,613 LINES! Yikes!

  31. I don’t know the gibberish it says

  32. 這不是⼈人幹的
 Not for the 
 faint of heart

  33. The real pain is Mapnik, especially if you want to

    deal with stylesheet Map images OSM data Tiles Mapnik osm2pqsql, or Imposm PostGIS Landez Lots of pain OSM XML
  34. Landez as proxy to Mapnik Map images OSM data from

    landez import MBTilesBuilder ! mb = MBTilesBuilder(stylefile="yourstyle.xml") mb.add_coverage(bbox=(-122.1099,37.4531,-121.8768,37.6107), zoomlevels=range(12, 16))
 mb.run() Get XML from server using the bounding box and zoom range Watch out for bugs, though
  35. If you must make your own Map images OSM data

    •Much easier Cascadenik
 Stylesheet Mapnik
 Stylesheet CSS-like XML Cascadenik
 translation
  36. TileMill Map images OSM data Tiles Mapnik osm2pqsql, or Imposm

    PostGIS Landez OSM XML TileMill
  37. TileMill, the even better option from MapBox Map images OSM

    data •Uses Mapnik to generate tiles internally. •Generate MBTiles directly •Takes a CSS-like stylesheet called Carto (derived from Cascadenik) WYSIWYG editor
  38. TileMill, with OSM-Bright stylesheet show and hide layers export Just

    altogether 1203 lines
  39. TileMill, the alternative from MapBox MBTiles

  40. Just an estimate, and not very good

  41. OSM-Bright under OS X quick-look MBTiles

  42. MBTiles SELECT tile_data from tiles WHERE zoom_level = z and

    tile_column = x and tile_row = y Gives you the tiles from SQL command, except that the coordinate is flipped from that of Stamen’s. MBTiles
  43. So far… Map images OSM data •Covered how to get

    OSM data into map images. ! •One way is to just grab tiles from others using Landez. ! •The other way is to generate ourselves. •You can either use Mapnik directly, or Landez, or Cascadenik •Or uses TileMill which renders using… Mapnik.
  44. The grand picture offline map & routing Map images OSM

    data graph structure Bus data
  45. Putting it on the phone as offline map MBTiles Objective-C


    SDK Map images offline map & routing
  46. iOS SDK Route-me Open source Route-me 
 iOS sdk Alpstein


    fork MapBox iOS 
 sdk Objective-C
 SDK Map images offline map & routing Best choice in my opinion
  47. Putting it on the phone as offline map Objective-C
 SDK

    Map images offline map & routing •Starting with iOS 7,
 MapKit itself supports tiles
 with MKTileOverLay and can 
 completely tell it not to load
 Apple’s map.
  48. Putting it on the phone as offline map Map images

    offline map & routing
  49. I made an tiny library open source library on top

    of MapKit Map images offline map & routing •OK…just a couple of classes. •Read MBTiles •Constraint map range and zoom level base on MBTiles bounding box
  50. The grand picture offline map & routing Map images OSM

    data graph structure Bus data
  51. This complicated thing… offline map & routing OSM data graph

    structure is already done on the server side
  52. Using the Graphserver OSM XML Bus data (GTFS) After executing

    a couple of scripts… Graphserver Web service
  53. Using the Graphserver Graphserver Web service http://blahblah/…
 from: osm-53040714 


    to: osm-1366392295
 time: 1260839444 json with narratives telling you directions
  54. Code structure Python wrappers C routing core High-level representation, edge

    including street, bus route, each with a different “cost” for shortest path computation Data parsing, database creation, graph creation, human-readable narratives
  55. Porting it to iPhone Python wrappers C routing core •I

    ported almost EVERYTHING •How stupid
  56. Porting it to iPhone Objective-C C routing core

  57. Reminds you GraphServer’s role offline map & routing OSM data

    graph structure
  58. Parsing and generate graph OSM data graph structure •Generate from

    OSM XML (or PBF) to graph <node id="52978567" lat="37.5534643" lon=“-122.0136585" />
 <node id="52978568" lat="37.5524912" lon=“-122.0145082" /> ...
 ! <way id="6318398"> <nd ref="52978567"/> <nd ref="52978568"/> <nd ref="52978569"/> <nd ref="52978570"/>
 </way> .osm (simplified)
  59. Parsing OSM data OSM data graph structure •From the Gaia-SINS

    federated projects •C code •Both XML and PBF readOSM •Objective-C library •XML only •Generate SQLitedb (but I have to generate my own’s)
  60. Database schema? OSM data graph structure •I’ll just make whatever

    GraphServer uses… •Well, almost. Except I am too smart for that, I uses SpatiaLite •Compiling SpatiaLite to iOS, and use FMDB to interface it •Sometimes work, sometimes not (may be my own fault) GraphServer calls this “OSMDB”, stores the node and way information OSMDB
  61. Generate graph and store in DB OSM data graph structure

    OSMDB GDB Generate graph (nodes and edges), store in another SQLite3 called GDB I store graph structure, nodes and edges I store OSM nodes and ways
  62. GDB details: Only keep interesting nodes OSM data graph structure

    GDB Routing relevant Not at intersection, irrelevant
  63. GDB details: one way street OSM data graph structure GDB

    One-way street Two-way street For simplicity, I’ll ignore that detail in the following slides.
  64. offline map & routing OSM data graph structure

  65. The shortest path graph structure offline map & routing GDB

    Generate on server/app starts (or deserialize from disk)
  66. The shortest path graph structure offline map & routing The

    narratives? 52978580 52978515 32817060 32816488 52978591 “Start at Herschel Ave. Turn right on Torrey Pines Rd. Turn left on Girard Ave. Slight right onto Forward St. Straight to Market St. Arrive at destination.”
  67. The shortest path graph structure offline map & routing 52978580

    52978515 32817060 32816488 52978591 GDB GDB is only stores ID numbers of nodes and edges. And it doesn’t have any human understandable information. OSMDB Hi! Me!
  68. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? No previous edge, a start.
 
 Where are we starting? OSMDB
  69. Walking through the shortest path to generate the narratives How

    far are we walking? OSMDB
  70. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? Has previous and next edges, a “turn”.
 Direction? Hard turn? Slight turn? Straight? OSMDB
  71. Walking through the shortest path to generate the narratives Keep

    track of distance walked so far. OSMDB How far are we walking?
  72. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? OSMDB Has previous and next edges, a “turn”.
 Direction?
  73. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? OSMDB No next edge. It’s an end.
  74. I have been ignoring bus routing, it’s actually part of

    GraphServer as well. offline map & routing graph structure Map images OSM data Bus data
  75. GTFS Bus data •Google Transit Feed Specification •Bus stop locations

    •Bus schedule (weekly) •Service dates •Bus route shape file (optional)
  76. GTFS Bus data •Six required tables •Route — Bus lines,

    Example: 247 •Trips — A journey from each bus with stop times •Stop_times: All stop times https://developers.google.com/transit/gtfs/reference
  77. GTFS Bus data https://support.google.com/transitpartners/answer/1106431?hl=en

  78. Source of GTFS

  79. GTFS data exchange

  80. Make into yet another SQLite3 Database GTFSDB •Store all GTFS

    files into GTFSDB
  81. How can we make it route? •Essentially, each bus stop

    is a node, each trip is another edge 10:00 AM 10:30 AM 11:00 AM 10:10 AM 10:40 AM 11:10 AM GTFSDB GDB
  82. 10:00 AM 10:30 AM 11:00 AM 10:10 AM 10:40 AM

    11:10 AM Links the 
 bus stop to the closest node When calculating the shortest path, we actually calculate “cost”
  83. Cost Something like… ! Regular road average walking speed x

    walking_reluctance + rise x hill_reluctance
 
 Bus current_time + waiting_panelty + transfer_panelty ! ! It’s a a complicated scheme
  84. Complications •You don’t start nor end on intersections. You are

    here. •Generate temporary edges at route request
  85. Problems •You need to draw the shape of road and

    bus route •Road we just pull out from the database •If shape file is not in the GTFS, I’m in trouble
  86. Some city does not have usable OSM •Taipei, for one.

    ! •Giving directions require a lot of road names
  87. Some transit services do not have GTFS •Not Taipei •Hong

    kong’s has a web server, no publicly data.
  88. Talking to the guy makes me realize the data is

    very precious and well guarded
  89. How about Android? •OpenTripPlanner (Java, LGPL) •Originates from a rewrite

    of the GraphServer •Actively maintained (unlike GraphServer). It has support for real-time bus information as well •Multiple routing algorithms.
  90. Roll your own handling of everything •You could also starts

    with simple shortest path (or other path finding) libraries and starts from there •But I have no experience for those.
  91. pgRouting (C++, Boost licence) •There are support for multiple algorithms

    •A*, bi-direction A*, bi-directional Dijkstra
  92. PESGraph (Objective-C, MIT(?)) •Dijkstra's algorithm

  93. Summary •Making or getting tile images •Landez •PostGIS & TileMill

    •Routing •Graphserver •Bus data with GTFS
  94. Future directions •Making better-looking tiles •What if hand-drawn? ! •Reporting

    more than one routes ! •Contributing to Landez?
  95. FAQ •Image attributions: •https://secure.flickr.com/photos/captainmcdan/ 233091932/ •https://secure.flickr.com/photos/nnova/2964382517/ •https://farm9.staticflickr.com/8392/ •https://secure.flickr.com/photos/kobakou/8459284083/ •https://secure.flickr.com/photos/bruceberrien/ 4262228892/

    Timothy Wu 2013-11-09