讓地圖在手機上活起來 — 離線公車導覽

Transcript

1. 讓地圖在⼿手機上活起來 — 離線公⾞車導覽
 Makes OSM maps alive and kicking on

   iOS — offline transit routing 
 
 iOS 離線地圖實作的⼯工具介紹跟觀念 Timothy Wu 2013-11-09

2. •Not affiliated with Academic Sinica. •Was involved in Bioinformatics •Entrepreneur

   wanna-be •I think I have a little Prosopagnosia (Face-blindness) (臉盲症) •Wanna see what’s like to be one? •http://tinyurl.com/mlesk4n •OSM newbie (and probably will be for a long time) A little about myself

3. What do we mean by alive and kicking? I want

   to do that all inside an iOS cell phone, all without the internet. Nada.

4. Why OpenStreetMap? •Google License restrictions •Can’t save map tiles for

   offline viewing (at least not on iOS) •Can’t store requested Google map Point of Interests (POIs). •Can’t save routes and directions for query •No real time navigation allowed

5. Thinking backward — what it involves in the grand picture

   offline map & routing Map images OSM data Stamen’s toner map graph structure

6. 忠孝東路 中⼭山北路 南京東路 市⺠民⼤大道 松江路 重慶北路 What graph? Treat all

   road data as nodes and edges Then find the “best” way around it
 (may mean the shortest) node edge

7. 忠孝東路 中⼭山北路 南京東路 市⺠民⼤大道 松江路 重慶北路 How to find the

   best path? • The easy answer is: try it all and then get the best (or shortest) one. • There is Dijkstra’s algorithm. Really, it tries everything from origin to destination, just efficiently. We’ll leave it as that. ?

8. How about bus? offline map & routing Map images OSM

   data graph structure Bus data Calculate in a similar fashion

9. OSM data The OSM data Editor: id, or josm, or

   others generate XML data

10. OSM data The OSM data — nodes

11. OSM data The OSM data — ways

12. The grand picture offline map & routing Map images OSM

    data graph structure Bus data

13. Getting the images Map images OSM data •One way is

    we DON’T generate it ourself •Just grab the images from the servers

14. Getting the images Map images OSM data • http://tile.stamen.com/toner/<zoom>/<x>/<y>.png •

    http://tile.stamen.com/terrain/<zoom>/<x>/<y>.jpg • http://tile.stamen.com/watercolor/<zoom>/<x>/<y>.jpg

15. Getting the images Map images OSM data • http://tile.stamen.com/watercolor/<zoom>/<x>/<y>.jpg Zoom

    level: 0 x, y = 0, 0 0, 1 1, 1 1, 0 Zoom level: 1 Zoom level Max: 20, min: 0 ! Zoom levels 20, 19, 18 corresponds to scale 1, 1/2, 1/4

16. Landez to get images Map images OSM data https://github.com/makinacorpus/landez Landez

    Landez is a code library written in Python Tiles MBTiles MBTiles is a simply a SQLite3 database with a specific schema defined by MapBox, inc

17. Landez to get images Map images OSM data Three things

    we tell Landez with A, B, C A: Bounding box ! B: Zoom level
 
 C: URL

18. Map images OSM data A: Bounding box: 1. Left 2.

    bottom 3. right 4. top In another word:
 “min_Longitude , min_Latitude , max_Longitude , max_Latitude” Landez to get images

19. Landez to get images Map images OSM data B: zoom

    level Say, 12 to 15. ! You need to be careful with tiles. 
 With each increase in level, you get more tiles then you all prior lower zoom levels combined! ! At the maximum zoom level (20), the whole world is a square of 262,144 screens wide and 262,144 screens tall (assuming a screen is 1024 resolution in width and height)

20. Landez to get images Map images OSM data C: URL

    http://tile.stamen.com/watercolor/{z}/{x}/{y}.jpg

21. Landez to get images from landez import MBTilesBuilder ! mb

    = MBTilesBuilder(filepath='output.mbtiles', tiles_url='http://tile.stamen.com/watercolor/{z}/{x}/ {y}.jpg')
 mb.add_coverage(bbox=(-122.1099,37.4531,-121.8768,37.6107), zoomlevels=range(12, 16))
 mb.run() If no URL given, it’ll just grab from the official OpenStreetMaps server. 
 Guarantee ugly tiles (sorry, OSM), but easy peasy. Python script to get images from server

22. Mapnik to generate image Map images OSM data •The second

    way is to generate the tiles ourself •The most well known is Mapnik, a C++ & Python program.

23. Mapnik flow chart Map images OSM data OSM XML Tiles

    Mapnik osm2pqsql, or Imposm PostGIS A pain

24. Setting up PostGIS •Here is what I do on PostGIS

    8.4 (granted, an older vsion) ! •Install Postgresql and PostGIS ! •Create cluster (just DB) ! ! ! •Start postgres initdb -D /var/lib/postgresql/data postgres -D /var/lib/postgresql/data

25. Setting up PostGIS •Set it so I bind to the

    IP and port and expose it ! ! ! ! ! •Trust the connection to the database to my computer In /var/lib/postgresql/data/postgresql.conf In /var/lib/postgresql/data/pg_hba.conf

26. Setting up PostGIS •Create database • As “postgres" user, create

    postgresql tables using Imposm’s create-db.sh script (and modified for the right db name) • (I was using Imposm 2.0) create-db.sh

27. Importing into PostGIS •Finally, import the data osm2pgsql -c -G

    -U postgres -d gis -S /usr/share/ osm2pgsql/default.style your_file.osm.pbf (or .osm)

28. Next, Mapnik Map images OSM data Tiles Mapnik osm2pqsql, or

    Imposm PostGIS OSM XML Stylesheet

29. Mapnik Stylesheet Map images OSM data •It ONLY took 14,613

    lines of stylesheet to make this! •I can’t find any editors for it. •Documents are also difficult to come by •Someone wrote a pdf explaining version 0.7 (it’s 2.2.0 now?) and seems difficult but understandable

30. Mapnik Stylesheet — 14,613 LINES! Yikes!

31. I don’t know the gibberish it says

32. 這不是⼈人幹的
 Not for the 
 faint of heart

33. The real pain is Mapnik, especially if you want to

    deal with stylesheet Map images OSM data Tiles Mapnik osm2pqsql, or Imposm PostGIS Landez Lots of pain OSM XML

34. Landez as proxy to Mapnik Map images OSM data from

    landez import MBTilesBuilder ! mb = MBTilesBuilder(stylefile="yourstyle.xml") mb.add_coverage(bbox=(-122.1099,37.4531,-121.8768,37.6107), zoomlevels=range(12, 16))
 mb.run() Get XML from server using the bounding box and zoom range Watch out for bugs, though

35. If you must make your own Map images OSM data

    •Much easier Cascadenik
 Stylesheet Mapnik
 Stylesheet CSS-like XML Cascadenik
 translation

36. TileMill Map images OSM data Tiles Mapnik osm2pqsql, or Imposm

    PostGIS Landez OSM XML TileMill

37. TileMill, the even better option from MapBox Map images OSM

    data •Uses Mapnik to generate tiles internally. •Generate MBTiles directly •Takes a CSS-like stylesheet called Carto (derived from Cascadenik) WYSIWYG editor

38. TileMill, with OSM-Bright stylesheet show and hide layers export Just

    altogether 1203 lines

39. TileMill, the alternative from MapBox MBTiles

40. Just an estimate, and not very good

41. OSM-Bright under OS X quick-look MBTiles

42. MBTiles SELECT tile_data from tiles WHERE zoom_level = z and

    tile_column = x and tile_row = y Gives you the tiles from SQL command, except that the coordinate is flipped from that of Stamen’s. MBTiles

43. So far… Map images OSM data •Covered how to get

    OSM data into map images. ! •One way is to just grab tiles from others using Landez. ! •The other way is to generate ourselves. •You can either use Mapnik directly, or Landez, or Cascadenik •Or uses TileMill which renders using… Mapnik.

44. The grand picture offline map & routing Map images OSM

    data graph structure Bus data

45. Putting it on the phone as offline map MBTiles Objective-C


    SDK Map images offline map & routing

46. iOS SDK Route-me Open source Route-me 
 iOS sdk Alpstein


    fork MapBox iOS 
 sdk Objective-C
 SDK Map images offline map & routing Best choice in my opinion

47. Putting it on the phone as offline map Objective-C
 SDK

    Map images offline map & routing •Starting with iOS 7,
 MapKit itself supports tiles
 with MKTileOverLay and can 
 completely tell it not to load
 Apple’s map.

48. Putting it on the phone as offline map Map images

    offline map & routing

49. I made an tiny library open source library on top

    of MapKit Map images offline map & routing •OK…just a couple of classes. •Read MBTiles •Constraint map range and zoom level base on MBTiles bounding box

50. The grand picture offline map & routing Map images OSM

    data graph structure Bus data

51. This complicated thing… offline map & routing OSM data graph

    structure is already done on the server side

52. Using the Graphserver OSM XML Bus data (GTFS) After executing

    a couple of scripts… Graphserver Web service

53. Using the Graphserver Graphserver Web service http://blahblah/…
 from: osm-53040714 


    to: osm-1366392295
 time: 1260839444 json with narratives telling you directions

54. Code structure Python wrappers C routing core High-level representation, edge

    including street, bus route, each with a different “cost” for shortest path computation Data parsing, database creation, graph creation, human-readable narratives

55. Porting it to iPhone Python wrappers C routing core •I

    ported almost EVERYTHING •How stupid

56. Porting it to iPhone Objective-C C routing core

57. Reminds you GraphServer’s role offline map & routing OSM data

    graph structure

58. Parsing and generate graph OSM data graph structure •Generate from

    OSM XML (or PBF) to graph <node id="52978567" lat="37.5534643" lon=“-122.0136585" />
 <node id="52978568" lat="37.5524912" lon=“-122.0145082" /> ...
 ! <way id="6318398"> <nd ref="52978567"/> <nd ref="52978568"/> <nd ref="52978569"/> <nd ref="52978570"/>
 </way> .osm (simplified)

59. Parsing OSM data OSM data graph structure •From the Gaia-SINS

    federated projects •C code •Both XML and PBF readOSM •Objective-C library •XML only •Generate SQLitedb (but I have to generate my own’s)

60. Database schema? OSM data graph structure •I’ll just make whatever

    GraphServer uses… •Well, almost. Except I am too smart for that, I uses SpatiaLite •Compiling SpatiaLite to iOS, and use FMDB to interface it •Sometimes work, sometimes not (may be my own fault) GraphServer calls this “OSMDB”, stores the node and way information OSMDB

61. Generate graph and store in DB OSM data graph structure

    OSMDB GDB Generate graph (nodes and edges), store in another SQLite3 called GDB I store graph structure, nodes and edges I store OSM nodes and ways

62. GDB details: Only keep interesting nodes OSM data graph structure

    GDB Routing relevant Not at intersection, irrelevant

63. GDB details: one way street OSM data graph structure GDB

    One-way street Two-way street For simplicity, I’ll ignore that detail in the following slides.

64. offline map & routing OSM data graph structure

65. The shortest path graph structure offline map & routing GDB

    Generate on server/app starts (or deserialize from disk)

66. The shortest path graph structure offline map & routing The

    narratives? 52978580 52978515 32817060 32816488 52978591 “Start at Herschel Ave. Turn right on Torrey Pines Rd. Turn left on Girard Ave. Slight right onto Forward St. Straight to Market St. Arrive at destination.”

67. The shortest path graph structure offline map & routing 52978580

    52978515 32817060 32816488 52978591 GDB GDB is only stores ID numbers of nodes and edges. And it doesn’t have any human understandable information. OSMDB Hi! Me!

68. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? No previous edge, a start.
 
 Where are we starting? OSMDB

69. Walking through the shortest path to generate the narratives How

    far are we walking? OSMDB

70. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? Has previous and next edges, a “turn”.
 Direction? Hard turn? Slight turn? Straight? OSMDB

71. Walking through the shortest path to generate the narratives Keep

    track of distance walked so far. OSMDB How far are we walking?

72. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? OSMDB Has previous and next edges, a “turn”.
 Direction?

73. Walking through the shortest path to generate the narratives Is

    it a start?
 Is it a turn? Is it an end? OSMDB No next edge. It’s an end.

74. I have been ignoring bus routing, it’s actually part of

    GraphServer as well. offline map & routing graph structure Map images OSM data Bus data

75. GTFS Bus data •Google Transit Feed Specification •Bus stop locations

    •Bus schedule (weekly) •Service dates •Bus route shape file (optional)

76. GTFS Bus data •Six required tables •Route — Bus lines,

    Example: 247 •Trips — A journey from each bus with stop times •Stop_times: All stop times https://developers.google.com/transit/gtfs/reference

77. GTFS Bus data https://support.google.com/transitpartners/answer/1106431?hl=en

78. Source of GTFS

79. GTFS data exchange

80. Make into yet another SQLite3 Database GTFSDB •Store all GTFS

    files into GTFSDB

81. How can we make it route? •Essentially, each bus stop

    is a node, each trip is another edge 10:00 AM 10:30 AM 11:00 AM 10:10 AM 10:40 AM 11:10 AM GTFSDB GDB

82. 10:00 AM 10:30 AM 11:00 AM 10:10 AM 10:40 AM

    11:10 AM Links the 
 bus stop to the closest node When calculating the shortest path, we actually calculate “cost”

83. Cost Something like… ! Regular road average walking speed x

    walking_reluctance + rise x hill_reluctance
 
 Bus current_time + waiting_panelty + transfer_panelty ! ! It’s a a complicated scheme

84. Complications •You don’t start nor end on intersections. You are

    here. •Generate temporary edges at route request

85. Problems •You need to draw the shape of road and

    bus route •Road we just pull out from the database •If shape file is not in the GTFS, I’m in trouble

86. Some city does not have usable OSM •Taipei, for one.

    ! •Giving directions require a lot of road names

87. Some transit services do not have GTFS •Not Taipei •Hong

    kong’s has a web server, no publicly data.

88. Talking to the guy makes me realize the data is

    very precious and well guarded

89. How about Android? •OpenTripPlanner (Java, LGPL) •Originates from a rewrite

    of the GraphServer •Actively maintained (unlike GraphServer). It has support for real-time bus information as well •Multiple routing algorithms.

90. Roll your own handling of everything •You could also starts

    with simple shortest path (or other path finding) libraries and starts from there •But I have no experience for those.

91. pgRouting (C++, Boost licence) •There are support for multiple algorithms

    •A*, bi-direction A*, bi-directional Dijkstra

92. PESGraph (Objective-C, MIT(?)) •Dijkstra's algorithm

93. Summary •Making or getting tile images •Landez •PostGIS & TileMill

    •Routing •Graphserver •Bus data with GTFS

94. Future directions •Making better-looking tiles •What if hand-drawn? ! •Reporting

    more than one routes ! •Contributing to Landez?

95. FAQ •Image attributions: •https://secure.flickr.com/photos/captainmcdan/ 233091932/ •https://secure.flickr.com/photos/nnova/2964382517/ •https://farm9.staticflickr.com/8392/ •https://secure.flickr.com/photos/kobakou/8459284083/ •https://secure.flickr.com/photos/bruceberrien/ 4262228892/

    Timothy Wu 2013-11-09