Upgrade to Pro — share decks privately, control downloads, hide ads and more …

讓地圖在手機上活起來 — 離線公車導覽

Timothy
November 09, 2013

讓地圖在手機上活起來 — 離線公車導覽

Timothy

November 09, 2013
Tweet

Other Decks in Programming

Transcript

  1. 讓地圖在⼿手機上活起來 — 離線公⾞車導覽
 Makes OSM maps alive and kicking on

    iOS — offline transit routing 
 
 iOS 離線地圖實作的⼯工具介紹跟觀念 Timothy Wu 2013-11-09
  2. •Not affiliated with Academic Sinica. •Was involved in Bioinformatics •Entrepreneur

    wanna-be •I think I have a little Prosopagnosia (Face-blindness) (臉盲症) •Wanna see what’s like to be one? •http://tinyurl.com/mlesk4n •OSM newbie (and probably will be for a long time) A little about myself
  3. What do we mean by alive and kicking? I want

    to do that all inside an iOS cell phone, all without the internet. Nada.
  4. Why OpenStreetMap? •Google License restrictions •Can’t save map tiles for

    offline viewing (at least not on iOS) •Can’t store requested Google map Point of Interests (POIs). •Can’t save routes and directions for query •No real time navigation allowed
  5. Thinking backward — what it involves in the grand picture

    offline map & routing Map images OSM data Stamen’s toner map graph structure
  6. 忠孝東路 中⼭山北路 南京東路 市⺠民⼤大道 松江路 重慶北路 What graph? Treat all

    road data as nodes and edges Then find the “best” way around it
 (may mean the shortest) node edge
  7. 忠孝東路 中⼭山北路 南京東路 市⺠民⼤大道 松江路 重慶北路 How to find the

    best path? • The easy answer is: try it all and then get the best (or shortest) one. • There is Dijkstra’s algorithm. Really, it tries everything from origin to destination, just efficiently. We’ll leave it as that. ?
  8. How about bus? offline map & routing Map images OSM

    data graph structure Bus data Calculate in a similar fashion
  9. Getting the images Map images OSM data •One way is

    we DON’T generate it ourself •Just grab the images from the servers
  10. Getting the images Map images OSM data • http://tile.stamen.com/toner/<zoom>/<x>/<y>.png •

    http://tile.stamen.com/terrain/<zoom>/<x>/<y>.jpg • http://tile.stamen.com/watercolor/<zoom>/<x>/<y>.jpg
  11. Getting the images Map images OSM data • http://tile.stamen.com/watercolor/<zoom>/<x>/<y>.jpg Zoom

    level: 0 x, y = 0, 0 0, 1 1, 1 1, 0 Zoom level: 1 Zoom level Max: 20, min: 0 ! Zoom levels 20, 19, 18 corresponds to scale 1, 1/2, 1/4
  12. Landez to get images Map images OSM data https://github.com/makinacorpus/landez Landez

    Landez is a code library written in Python Tiles MBTiles MBTiles is a simply a SQLite3 database with a specific schema defined by MapBox, inc
  13. Landez to get images Map images OSM data Three things

    we tell Landez with A, B, C A: Bounding box ! B: Zoom level
 
 C: URL
  14. Map images OSM data A: Bounding box: 1. Left 2.

    bottom 3. right 4. top In another word:
 “min_Longitude , min_Latitude , max_Longitude , max_Latitude” Landez to get images
  15. Landez to get images Map images OSM data B: zoom

    level Say, 12 to 15. ! You need to be careful with tiles. 
 With each increase in level, you get more tiles then you all prior lower zoom levels combined! ! At the maximum zoom level (20), the whole world is a square of 262,144 screens wide and 262,144 screens tall (assuming a screen is 1024 resolution in width and height)
  16. Landez to get images Map images OSM data C: URL

    http://tile.stamen.com/watercolor/{z}/{x}/{y}.jpg
  17. Landez to get images from landez import MBTilesBuilder ! mb

    = MBTilesBuilder(filepath='output.mbtiles', tiles_url='http://tile.stamen.com/watercolor/{z}/{x}/ {y}.jpg')
 mb.add_coverage(bbox=(-122.1099,37.4531,-121.8768,37.6107), zoomlevels=range(12, 16))
 mb.run() If no URL given, it’ll just grab from the official OpenStreetMaps server. 
 Guarantee ugly tiles (sorry, OSM), but easy peasy. Python script to get images from server
  18. Mapnik to generate image Map images OSM data •The second

    way is to generate the tiles ourself •The most well known is Mapnik, a C++ & Python program.
  19. Mapnik flow chart Map images OSM data OSM XML Tiles

    Mapnik osm2pqsql, or Imposm PostGIS A pain
  20. Setting up PostGIS •Here is what I do on PostGIS

    8.4 (granted, an older vsion) ! •Install Postgresql and PostGIS ! •Create cluster (just DB) ! ! ! •Start postgres initdb -D /var/lib/postgresql/data postgres -D /var/lib/postgresql/data
  21. Setting up PostGIS •Set it so I bind to the

    IP and port and expose it ! ! ! ! ! •Trust the connection to the database to my computer In /var/lib/postgresql/data/postgresql.conf In /var/lib/postgresql/data/pg_hba.conf
  22. Setting up PostGIS •Create database • As “postgres" user, create

    postgresql tables using Imposm’s create-db.sh script (and modified for the right db name) • (I was using Imposm 2.0) create-db.sh
  23. Importing into PostGIS •Finally, import the data osm2pgsql -c -G

    -U postgres -d gis -S /usr/share/ osm2pgsql/default.style your_file.osm.pbf (or .osm)
  24. Mapnik Stylesheet Map images OSM data •It ONLY took 14,613

    lines of stylesheet to make this! •I can’t find any editors for it. •Documents are also difficult to come by •Someone wrote a pdf explaining version 0.7 (it’s 2.2.0 now?) and seems difficult but understandable
  25. The real pain is Mapnik, especially if you want to

    deal with stylesheet Map images OSM data Tiles Mapnik osm2pqsql, or Imposm PostGIS Landez Lots of pain OSM XML
  26. Landez as proxy to Mapnik Map images OSM data from

    landez import MBTilesBuilder ! mb = MBTilesBuilder(stylefile="yourstyle.xml") mb.add_coverage(bbox=(-122.1099,37.4531,-121.8768,37.6107), zoomlevels=range(12, 16))
 mb.run() Get XML from server using the bounding box and zoom range Watch out for bugs, though
  27. If you must make your own Map images OSM data

    •Much easier Cascadenik
 Stylesheet Mapnik
 Stylesheet CSS-like XML Cascadenik
 translation
  28. TileMill, the even better option from MapBox Map images OSM

    data •Uses Mapnik to generate tiles internally. •Generate MBTiles directly •Takes a CSS-like stylesheet called Carto (derived from Cascadenik) WYSIWYG editor
  29. MBTiles SELECT tile_data from tiles WHERE zoom_level = z and

    tile_column = x and tile_row = y Gives you the tiles from SQL command, except that the coordinate is flipped from that of Stamen’s. MBTiles
  30. So far… Map images OSM data •Covered how to get

    OSM data into map images. ! •One way is to just grab tiles from others using Landez. ! •The other way is to generate ourselves. •You can either use Mapnik directly, or Landez, or Cascadenik •Or uses TileMill which renders using… Mapnik.
  31. iOS SDK Route-me Open source Route-me 
 iOS sdk Alpstein


    fork MapBox iOS 
 sdk Objective-C
 SDK Map images offline map & routing Best choice in my opinion
  32. Putting it on the phone as offline map Objective-C
 SDK

    Map images offline map & routing •Starting with iOS 7,
 MapKit itself supports tiles
 with MKTileOverLay and can 
 completely tell it not to load
 Apple’s map.
  33. I made an tiny library open source library on top

    of MapKit Map images offline map & routing •OK…just a couple of classes. •Read MBTiles •Constraint map range and zoom level base on MBTiles bounding box
  34. This complicated thing… offline map & routing OSM data graph

    structure is already done on the server side
  35. Using the Graphserver OSM XML Bus data (GTFS) After executing

    a couple of scripts… Graphserver Web service
  36. Using the Graphserver Graphserver Web service http://blahblah/…
 from: osm-53040714 


    to: osm-1366392295
 time: 1260839444 json with narratives telling you directions
  37. Code structure Python wrappers C routing core High-level representation, edge

    including street, bus route, each with a different “cost” for shortest path computation Data parsing, database creation, graph creation, human-readable narratives
  38. Porting it to iPhone Python wrappers C routing core •I

    ported almost EVERYTHING •How stupid
  39. Parsing and generate graph OSM data graph structure •Generate from

    OSM XML (or PBF) to graph <node id="52978567" lat="37.5534643" lon=“-122.0136585" />
 <node id="52978568" lat="37.5524912" lon=“-122.0145082" /> ...
 ! <way id="6318398"> <nd ref="52978567"/> <nd ref="52978568"/> <nd ref="52978569"/> <nd ref="52978570"/>
 </way> .osm (simplified)
  40. Parsing OSM data OSM data graph structure •From the Gaia-SINS

    federated projects •C code •Both XML and PBF readOSM •Objective-C library •XML only •Generate SQLitedb (but I have to generate my own’s)
  41. Database schema? OSM data graph structure •I’ll just make whatever

    GraphServer uses… •Well, almost. Except I am too smart for that, I uses SpatiaLite •Compiling SpatiaLite to iOS, and use FMDB to interface it •Sometimes work, sometimes not (may be my own fault) GraphServer calls this “OSMDB”, stores the node and way information OSMDB
  42. Generate graph and store in DB OSM data graph structure

    OSMDB GDB Generate graph (nodes and edges), store in another SQLite3 called GDB I store graph structure, nodes and edges I store OSM nodes and ways
  43. GDB details: Only keep interesting nodes OSM data graph structure

    GDB Routing relevant Not at intersection, irrelevant
  44. GDB details: one way street OSM data graph structure GDB

    One-way street Two-way street For simplicity, I’ll ignore that detail in the following slides.
  45. The shortest path graph structure offline map & routing GDB

    Generate on server/app starts (or deserialize from disk)
  46. The shortest path graph structure offline map & routing The

    narratives? 52978580 52978515 32817060 32816488 52978591 “Start at Herschel Ave. Turn right on Torrey Pines Rd. Turn left on Girard Ave. Slight right onto Forward St. Straight to Market St. Arrive at destination.”
  47. The shortest path graph structure offline map & routing 52978580

    52978515 32817060 32816488 52978591 GDB GDB is only stores ID numbers of nodes and edges. And it doesn’t have any human understandable information. OSMDB Hi! Me!
  48. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? No previous edge, a start.
 
 Where are we starting? OSMDB
  49. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? Has previous and next edges, a “turn”.
 Direction? Hard turn? Slight turn? Straight? OSMDB
  50. Walking through the shortest path to generate the narratives Keep

    track of distance walked so far. OSMDB How far are we walking?
  51. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? OSMDB Has previous and next edges, a “turn”.
 Direction?
  52. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? OSMDB No next edge. It’s an end.
  53. I have been ignoring bus routing, it’s actually part of

    GraphServer as well. offline map & routing graph structure Map images OSM data Bus data
  54. GTFS Bus data •Google Transit Feed Specification •Bus stop locations

    •Bus schedule (weekly) •Service dates •Bus route shape file (optional)
  55. GTFS Bus data •Six required tables •Route — Bus lines,

    Example: 247 •Trips — A journey from each bus with stop times •Stop_times: All stop times https://developers.google.com/transit/gtfs/reference
  56. How can we make it route? •Essentially, each bus stop

    is a node, each trip is another edge 10:00 AM 10:30 AM 11:00 AM 10:10 AM 10:40 AM 11:10 AM GTFSDB GDB
  57. 10:00 AM 10:30 AM 11:00 AM 10:10 AM 10:40 AM

    11:10 AM Links the 
 bus stop to the closest node When calculating the shortest path, we actually calculate “cost”
  58. Cost Something like… ! Regular road average walking speed x

    walking_reluctance + rise x hill_reluctance
 
 Bus current_time + waiting_panelty + transfer_panelty ! ! It’s a a complicated scheme
  59. Complications •You don’t start nor end on intersections. You are

    here. •Generate temporary edges at route request
  60. Problems •You need to draw the shape of road and

    bus route •Road we just pull out from the database •If shape file is not in the GTFS, I’m in trouble
  61. Some city does not have usable OSM •Taipei, for one.

    ! •Giving directions require a lot of road names
  62. Some transit services do not have GTFS •Not Taipei •Hong

    kong’s has a web server, no publicly data.
  63. Talking to the guy makes me realize the data is

    very precious and well guarded
  64. How about Android? •OpenTripPlanner (Java, LGPL) •Originates from a rewrite

    of the GraphServer •Actively maintained (unlike GraphServer). It has support for real-time bus information as well •Multiple routing algorithms.
  65. Roll your own handling of everything •You could also starts

    with simple shortest path (or other path finding) libraries and starts from there •But I have no experience for those.
  66. pgRouting (C++, Boost licence) •There are support for multiple algorithms

    •A*, bi-direction A*, bi-directional Dijkstra
  67. Summary •Making or getting tile images •Landez •PostGIS & TileMill

    •Routing •Graphserver •Bus data with GTFS