讓地圖在手機上活起來 — 離線公車導覽

Transcript

1. 讓地圖在⼿手機上活起來 — 離線公⾞車導覽
 Makes OSM maps alive and kicking on iOS — offline transit routing 
 
 iOS 離線地圖實作的⼯工具介紹跟觀念 Timothy Wu 2013-11-09
2. •Not affiliated with Academic Sinica. •Was involved in Bioinformatics •Entrepreneur wanna-be •I think I have a little Prosopagnosia (Face-blindness) (臉盲症) •Wanna see what’s like to be one? •http://tinyurl.com/mlesk4n •OSM newbie (and probably will be for a long time) A little about myself
3. What do we mean by alive and kicking? I want to do that all inside an iOS cell phone, all without the internet. Nada.
4. Why OpenStreetMap? •Google License restrictions •Can’t save map tiles for offline viewing (at least not on iOS) •Can’t store requested Google map Point of Interests (POIs). •Can’t save routes and directions for query •No real time navigation allowed
5. Thinking backward — what it involves in the grand picture offline map & routing Map images OSM data Stamen’s toner map graph structure
6. 忠孝東路 中⼭山北路 南京東路 市⺠民⼤大道 松江路 重慶北路 What graph? Treat all road data as nodes and edges Then find the “best” way around it
 (may mean the shortest) node edge
7. 忠孝東路 中⼭山北路 南京東路 市⺠民⼤大道 松江路 重慶北路 How to find the best path? • The easy answer is: try it all and then get the best (or shortest) one. • There is Dijkstra’s algorithm. Really, it tries everything from origin to destination, just efficiently. We’ll leave it as that. ?
8. How about bus? offline map & routing Map images OSM data graph structure Bus data Calculate in a similar fashion
9. OSM data The OSM data Editor: id, or josm, or others generate XML data
10. OSM data The OSM data — nodes
11. OSM data The OSM data — ways
12. The grand picture offline map & routing Map images OSM data graph structure Bus data
13. Getting the images Map images OSM data •One way is we DON’T generate it ourself •Just grab the images from the servers
14. Getting the images Map images OSM data • http://tile.stamen.com/toner/<zoom>/<x>/<y>.png • http://tile.stamen.com/terrain/<zoom>/<x>/<y>.jpg • http://tile.stamen.com/watercolor/<zoom>/<x>/<y>.jpg
15. Getting the images Map images OSM data • http://tile.stamen.com/watercolor/<zoom>/<x>/<y>.jpg Zoom level: 0 x, y = 0, 0 0, 1 1, 1 1, 0 Zoom level: 1 Zoom level Max: 20, min: 0 ! Zoom levels 20, 19, 18 corresponds to scale 1, 1/2, 1/4
16. Landez to get images Map images OSM data https://github.com/makinacorpus/landez Landez Landez is a code library written in Python Tiles MBTiles MBTiles is a simply a SQLite3 database with a specific schema defined by MapBox, inc
17. Landez to get images Map images OSM data Three things we tell Landez with A, B, C A: Bounding box ! B: Zoom level
 
 C: URL
18. Map images OSM data A: Bounding box: 1. Left 2. bottom 3. right 4. top In another word:
 “min_Longitude , min_Latitude , max_Longitude , max_Latitude” Landez to get images
19. Landez to get images Map images OSM data B: zoom level Say, 12 to 15. ! You need to be careful with tiles. 
 With each increase in level, you get more tiles then you all prior lower zoom levels combined! ! At the maximum zoom level (20), the whole world is a square of 262,144 screens wide and 262,144 screens tall (assuming a screen is 1024 resolution in width and height)
20. Landez to get images Map images OSM data C: URL http://tile.stamen.com/watercolor/{z}/{x}/{y}.jpg
21. Landez to get images from landez import MBTilesBuilder ! mb = MBTilesBuilder(filepath='output.mbtiles', tiles_url='http://tile.stamen.com/watercolor/{z}/{x}/ {y}.jpg')
 mb.add_coverage(bbox=(-122.1099,37.4531,-121.8768,37.6107), zoomlevels=range(12, 16))
 mb.run() If no URL given, it’ll just grab from the official OpenStreetMaps server. 
 Guarantee ugly tiles (sorry, OSM), but easy peasy. Python script to get images from server
22. Mapnik to generate image Map images OSM data •The second way is to generate the tiles ourself •The most well known is Mapnik, a C++ & Python program.
23. Mapnik flow chart Map images OSM data OSM XML Tiles Mapnik osm2pqsql, or Imposm PostGIS A pain
24. Setting up PostGIS •Here is what I do on PostGIS 8.4 (granted, an older vsion) ! •Install Postgresql and PostGIS ! •Create cluster (just DB) ! ! ! •Start postgres initdb -D /var/lib/postgresql/data postgres -D /var/lib/postgresql/data
25. Setting up PostGIS •Set it so I bind to the IP and port and expose it ! ! ! ! ! •Trust the connection to the database to my computer In /var/lib/postgresql/data/postgresql.conf In /var/lib/postgresql/data/pg_hba.conf
26. Setting up PostGIS •Create database • As “postgres" user, create postgresql tables using Imposm’s create-db.sh script (and modified for the right db name) • (I was using Imposm 2.0) create-db.sh
27. Importing into PostGIS •Finally, import the data osm2pgsql -c -G -U postgres -d gis -S /usr/share/ osm2pgsql/default.style your_file.osm.pbf (or .osm)
28. Next, Mapnik Map images OSM data Tiles Mapnik osm2pqsql, or Imposm PostGIS OSM XML Stylesheet
29. Mapnik Stylesheet Map images OSM data •It ONLY took 14,613 lines of stylesheet to make this! •I can’t find any editors for it. •Documents are also difficult to come by •Someone wrote a pdf explaining version 0.7 (it’s 2.2.0 now?) and seems difficult but understandable
30. Mapnik Stylesheet — 14,613 LINES! Yikes!
31. I don’t know the gibberish it says
32. 這不是⼈人幹的
 Not for the 
 faint of heart
33. The real pain is Mapnik, especially if you want to deal with stylesheet Map images OSM data Tiles Mapnik osm2pqsql, or Imposm PostGIS Landez Lots of pain OSM XML
34. Landez as proxy to Mapnik Map images OSM data from landez import MBTilesBuilder ! mb = MBTilesBuilder(stylefile="yourstyle.xml") mb.add_coverage(bbox=(-122.1099,37.4531,-121.8768,37.6107), zoomlevels=range(12, 16))
 mb.run() Get XML from server using the bounding box and zoom range Watch out for bugs, though
35. If you must make your own Map images OSM data •Much easier Cascadenik
 Stylesheet Mapnik
 Stylesheet CSS-like XML Cascadenik
 translation
36. TileMill Map images OSM data Tiles Mapnik osm2pqsql, or Imposm PostGIS Landez OSM XML TileMill
37. TileMill, the even better option from MapBox Map images OSM data •Uses Mapnik to generate tiles internally. •Generate MBTiles directly •Takes a CSS-like stylesheet called Carto (derived from Cascadenik) WYSIWYG editor
38. TileMill, with OSM-Bright stylesheet show and hide layers export Just altogether 1203 lines
39. TileMill, the alternative from MapBox MBTiles
40. Just an estimate, and not very good
41. OSM-Bright under OS X quick-look MBTiles
42. MBTiles SELECT tile_data from tiles WHERE zoom_level = z and tile_column = x and tile_row = y Gives you the tiles from SQL command, except that the coordinate is flipped from that of Stamen’s. MBTiles
43. So far… Map images OSM data •Covered how to get OSM data into map images. ! •One way is to just grab tiles from others using Landez. ! •The other way is to generate ourselves. •You can either use Mapnik directly, or Landez, or Cascadenik •Or uses TileMill which renders using… Mapnik.
44. The grand picture offline map & routing Map images OSM data graph structure Bus data
45. Putting it on the phone as offline map MBTiles Objective-C
 SDK Map images offline map & routing
46. iOS SDK Route-me Open source Route-me 
 iOS sdk Alpstein
 fork MapBox iOS 
 sdk Objective-C
 SDK Map images offline map & routing Best choice in my opinion
47. Putting it on the phone as offline map Objective-C
 SDK Map images offline map & routing •Starting with iOS 7,
 MapKit itself supports tiles
 with MKTileOverLay and can 
 completely tell it not to load
 Apple’s map.
48. Putting it on the phone as offline map Map images offline map & routing
49. I made an tiny library open source library on top of MapKit Map images offline map & routing •OK…just a couple of classes. •Read MBTiles •Constraint map range and zoom level base on MBTiles bounding box
50. The grand picture offline map & routing Map images OSM data graph structure Bus data
51. This complicated thing… offline map & routing OSM data graph structure is already done on the server side
52. Using the Graphserver OSM XML Bus data (GTFS) After executing a couple of scripts… Graphserver Web service
53. Using the Graphserver Graphserver Web service http://blahblah/…
 from: osm-53040714 
 to: osm-1366392295
 time: 1260839444 json with narratives telling you directions
54. Code structure Python wrappers C routing core High-level representation, edge including street, bus route, each with a different “cost” for shortest path computation Data parsing, database creation, graph creation, human-readable narratives
55. Porting it to iPhone Python wrappers C routing core •I ported almost EVERYTHING •How stupid
56. Porting it to iPhone Objective-C C routing core
57. Reminds you GraphServer’s role offline map & routing OSM data graph structure
58. Parsing and generate graph OSM data graph structure •Generate from OSM XML (or PBF) to graph <node id="52978567" lat="37.5534643" lon=“-122.0136585" />
 <node id="52978568" lat="37.5524912" lon=“-122.0145082" /> ...
 ! <way id="6318398"> <nd ref="52978567"/> <nd ref="52978568"/> <nd ref="52978569"/> <nd ref="52978570"/>
 </way> .osm (simplified)
59. Parsing OSM data OSM data graph structure •From the Gaia-SINS federated projects •C code •Both XML and PBF readOSM •Objective-C library •XML only •Generate SQLitedb (but I have to generate my own’s)
60. Database schema? OSM data graph structure •I’ll just make whatever GraphServer uses… •Well, almost. Except I am too smart for that, I uses SpatiaLite •Compiling SpatiaLite to iOS, and use FMDB to interface it •Sometimes work, sometimes not (may be my own fault) GraphServer calls this “OSMDB”, stores the node and way information OSMDB
61. Generate graph and store in DB OSM data graph structure OSMDB GDB Generate graph (nodes and edges), store in another SQLite3 called GDB I store graph structure, nodes and edges I store OSM nodes and ways
62. GDB details: Only keep interesting nodes OSM data graph structure GDB Routing relevant Not at intersection, irrelevant
63. GDB details: one way street OSM data graph structure GDB One-way street Two-way street For simplicity, I’ll ignore that detail in the following slides.
64. offline map & routing OSM data graph structure
65. The shortest path graph structure offline map & routing GDB Generate on server/app starts (or deserialize from disk)
66. The shortest path graph structure offline map & routing The narratives? 52978580 52978515 32817060 32816488 52978591 “Start at Herschel Ave. Turn right on Torrey Pines Rd. Turn left on Girard Ave. Slight right onto Forward St. Straight to Market St. Arrive at destination.”
67. The shortest path graph structure offline map & routing 52978580 52978515 32817060 32816488 52978591 GDB GDB is only stores ID numbers of nodes and edges. And it doesn’t have any human understandable information. OSMDB Hi! Me!
68. Walking through the shortest path to generate the narratives Is it a start?
 Is it a turn? Is it an end? No previous edge, a start.
 
 Where are we starting? OSMDB
69. Walking through the shortest path to generate the narratives How far are we walking? OSMDB
70. Walking through the shortest path to generate the narratives Is it a start?
 Is it a turn? Is it an end? Has previous and next edges, a “turn”.
 Direction? Hard turn? Slight turn? Straight? OSMDB
71. Walking through the shortest path to generate the narratives Keep track of distance walked so far. OSMDB How far are we walking?
72. Walking through the shortest path to generate the narratives Is it a start?
 Is it a turn? Is it an end? OSMDB Has previous and next edges, a “turn”.
 Direction?
73. Walking through the shortest path to generate the narratives Is it a start?
 Is it a turn? Is it an end? OSMDB No next edge. It’s an end.
74. I have been ignoring bus routing, it’s actually part of GraphServer as well. offline map & routing graph structure Map images OSM data Bus data
75. GTFS Bus data •Google Transit Feed Specification •Bus stop locations •Bus schedule (weekly) •Service dates •Bus route shape file (optional)
76. GTFS Bus data •Six required tables •Route — Bus lines, Example: 247 •Trips — A journey from each bus with stop times •Stop_times: All stop times https://developers.google.com/transit/gtfs/reference
77. GTFS Bus data https://support.google.com/transitpartners/answer/1106431?hl=en
78. Source of GTFS
79. GTFS data exchange
80. Make into yet another SQLite3 Database GTFSDB •Store all GTFS files into GTFSDB
81. How can we make it route? •Essentially, each bus stop is a node, each trip is another edge 10:00 AM 10:30 AM 11:00 AM 10:10 AM 10:40 AM 11:10 AM GTFSDB GDB
82. 10:00 AM 10:30 AM 11:00 AM 10:10 AM 10:40 AM 11:10 AM Links the 
 bus stop to the closest node When calculating the shortest path, we actually calculate “cost”
83. Cost Something like… ! Regular road average walking speed x walking_reluctance + rise x hill_reluctance
 
 Bus current_time + waiting_panelty + transfer_panelty ! ! It’s a a complicated scheme
84. Complications •You don’t start nor end on intersections. You are here. •Generate temporary edges at route request
85. Problems •You need to draw the shape of road and bus route •Road we just pull out from the database •If shape file is not in the GTFS, I’m in trouble
86. Some city does not have usable OSM •Taipei, for one. ! •Giving directions require a lot of road names
87. Some transit services do not have GTFS •Not Taipei •Hong kong’s has a web server, no publicly data.
88. Talking to the guy makes me realize the data is very precious and well guarded
89. How about Android? •OpenTripPlanner (Java, LGPL) •Originates from a rewrite of the GraphServer •Actively maintained (unlike GraphServer). It has support for real-time bus information as well •Multiple routing algorithms.
90. Roll your own handling of everything •You could also starts with simple shortest path (or other path finding) libraries and starts from there •But I have no experience for those.
91. pgRouting (C++, Boost licence) •There are support for multiple algorithms •A*, bi-direction A*, bi-directional Dijkstra
92. PESGraph (Objective-C, MIT(?)) •Dijkstra's algorithm
93. Summary •Making or getting tile images •Landez •PostGIS & TileMill •Routing •Graphserver •Bus data with GTFS
94. Future directions •Making better-looking tiles •What if hand-drawn? ! •Reporting more than one routes ! •Contributing to Landez?
95. FAQ •Image attributions: •https://secure.flickr.com/photos/captainmcdan/ 233091932/ •https://secure.flickr.com/photos/nnova/2964382517/ •https://farm9.staticflickr.com/8392/ •https://secure.flickr.com/photos/kobakou/8459284083/ •https://secure.flickr.com/photos/bruceberrien/ 4262228892/ Timothy Wu 2013-11-09