Route Planning with FOSS: OpenStreetMap, PostGIS, OSRM and OpenLayers

ch/open Workshop, Zurich.

Transcript

1. Route Planning with FOSS: OpenStreetMap, PostGIS, OSRM and OpenLayers Michael Rüegg Institute For Software University of Applied Sciences Rapperswil /ch/open Workshop, Zürich November, 2012
2. Outline 1 GIS 101 2 PostGIS 3 OpenStreetMap 4 Open Source Routing Machine 5 OpenLayers 6 Showcase: Tourpl Michael Rüegg (IFS) Route Planning with FOSS November, 2012 2 / 40
3. Outline 1 GIS 101 2 PostGIS 3 OpenStreetMap 4 Open Source Routing Machine 5 OpenLayers 6 Showcase: Tourpl Michael Rüegg (IFS) Route Planning with FOSS November, 2012 3 / 40
4. What is GIS? GIS stands for Geographic Information System Used to answer what is where on the Earth’s surface GIS data is typically stored in a Spatial Database A spatial database defines special data types for geometric objects (e. g., polygons) allows to save geometric data in regular (relational) databases provides special functions and indexes for querying and manupulating spatial data Michael Rüegg (IFS) Route Planning with FOSS November, 2012 4 / 40
5. Map projections & spatial reference systems Basic problem: How to get from a round world to a flat map? ⇒ Map projections: portray Earth’s curved to a flat surface Spatial Reference System (SRS) European Petrolueum Survey Group Geodesy (EPSG) Geodesy is the study of the shape of the Earth WGS84: The World Geodetic System is a standard for use in cartography, geodesy and navigation CHLV03: The coordinate system for Switzerland Michael Rüegg (IFS) Route Planning with FOSS November, 2012 5 / 40
6. Layer concept GIS is closely associated with the layer concept A GIS allows to work in various levels and their proper representations ⇒ Goal: Combination of data to get new information Figure: http://www.digital-geography.com/concept-of-layers-in-a-gis Michael Rüegg (IFS) Route Planning with FOSS November, 2012 6 / 40
7. GIS file formats Standards for encoding geographical information into a file Raster vs. vector formats Common vector formats: Drawing Interchange Format (DXF): CAD data file format developed by Autodesk ESRI Shapefile: popular geospatial vector data format from Environmental Systems Research Institute (ESRI) Keyhole Markup Language (KML): XML dialect for expressing geographic visualization within two-dimensional maps and three-dimensional Earth browsers Spatialite: free GIS that enables spatial queries and objects for SQLite Michael Rüegg (IFS) Route Planning with FOSS November, 2012 7 / 40
8. Geometric data types A feature represents the basic data unit in a GIS How can we store and represent geographic features in a database? ⇒ Simple Feature for SQL (SFS) Model Storage model for geographical data using well-known text (WKT) Basic geometric data types: points, linestrings, polygons Figure: Source Wikipedia Michael Rüegg (IFS) Route Planning with FOSS November, 2012 8 / 40
9. Overview SFS data types Figure: Source Wikipedia Michael Rüegg (IFS) Route Planning with FOSS November, 2012 9 / 40
10. Well-known text (WKT) Text markup language for representing vector geometry objects on a map Examples (Source Wikipedia): Type WKT Figure Point POINT (30 10) LineString LINESTRING (30 10, 10 30, 40 40) Polygon POLYGON ((30 10, 10 20, 20 40, 40 40, 30 10)) Polygon with hole POLYGON ((35 10, 10 20, 15 40, 45 45, 35 10), (20 30, 35 35, 30 20, 20 30)) Michael Rüegg (IFS) Route Planning with FOSS November, 2012 10 / 40
11. Spatial Relationships An important goal when using a GIS is determining the spatial relationships between geometric objects Figure: Source Wikipedia Michael Rüegg (IFS) Route Planning with FOSS November, 2012 11 / 40
12. Spatial Indices A spatial index is used to optimize spatial queries Spatial order structure: What is spatially close together should also be close in memory Figure: R-Tree example Michael Rüegg (IFS) Route Planning with FOSS November, 2012 12 / 40
13. Outline 1 GIS 101 2 PostGIS 3 OpenStreetMap 4 Open Source Routing Machine 5 OpenLayers 6 Showcase: Tourpl Michael Rüegg (IFS) Route Planning with FOSS November, 2012 13 / 40
14. What is PostGIS? Extension of PostgreSQL, “spatially enables” the PostgreSQL server Implements the SFS model Open Source (GNU General Public License) Data loaders (e. g., ESRI shape files) >330 geometric functions: distance, area, union, intersection, etc. Latest release is 2.0.1 (June 2012) Michael Rüegg (IFS) Route Planning with FOSS November, 2012 14 / 40
15. How to create PostGIS enabled databases Create PostGIS database template: $ createdb template_postgis $ psql template_postgis -c "create extension postgis" $ psql template_postgis -f <installdir>/postgresql/share/contrib/ postgis-2.0/legacy.sql Now we can create a DB from our template: $ createdb -T template_postgis gis_db To verify the installed PostGIS version, use this: $ psql gis_db -c "SELECT PostGIS_Full_Version()" Michael Rüegg (IFS) Route Planning with FOSS November, 2012 15 / 40
16. PostGIS Metadata tables There are two OpenGIS meta-data tables SPATIAL_REF_SYS has >3’000 spatial reference systems: CREATE TABLE spatial_ref_sys ( srid INTEGER NOT NULL PRIMARY KEY, auth_name VARCHAR(256), auth_srid INTEGER, srtext VARCHAR(2048), proj4text VARCHAR(2048)); GEOMETRY_COLUMNS contains the spatial columns: CREATE TABLE geometry_columns ( f_geometry_column VARCHAR(256) NOT NULL, coord_dimension INTEGER NOT NULL, srid INTEGER NOT NULL, type VARCHAR(30) NOT NULL); Michael Rüegg (IFS) Route Planning with FOSS November, 2012 16 / 40
17. Creating basic geometries with PostGIS We create a basic point object (x,y): SELECT ST_Point(1, 3) AS ExamplePoint; No spatial reference system used (default is Cartesian grid). Let’s use WGS84: SELECT ST_SetSRID(ST_Point(8.515342,47.389058), 4326); The same can be done by using WKT: SELECT ST_GeomFromText(’POINT(8.515342,47.389058)’, 4326); The same for LineStrings and Polygons: SELECT ST_GeomFromText(’LINESTRING(1 1,2 2,3 3)’) AS MyLine; SELECT ST_GeomFromText(’POLYGON((0 1,1 -1,-1 -1,0 1))’) As MyTriangle; Michael Rüegg (IFS) Route Planning with FOSS November, 2012 17 / 40
18. Creating a spatial table 1 Create a “normal” non-spatial table: CREATE TABLE beers ( id NOT NULL PRIMARY KEY, name VARCHAR, price FLOAT4); 2 Add a spatial column: SELECT AddGeometryColumn(’public’, ’beers’, ’geom’, 4326, ’POINT’, 2); Syntax of AddGeometryColumn: AddGeometryColumn(<schema_name>, <table_name>, <column_name>, <srid>, <type>, <dimension>) Michael Rüegg (IFS) Route Planning with FOSS November, 2012 18 / 40
19. Creating a spatial index Create a spatial index: CREATE INDEX in_beers ON beers USING gist(geom); New dataset for PostGIS functions: INSERT INTO beers(name, price, geom) VALUES ( ’Manor’, 1.15, GeometryFromText(’POINT(8.81915 47.22640)’,4326)); SELECT * from beers order by price; 21;"Coop Sunnehof"; 1.05;"01010000..." 18;"Manor"; 1.15;"01010000..." 20;"Dorflade"; 2.4; "01010000..." 23;"Rappi Bier Factory";2.9; "01010000..." 22;"Lido Markt"; 3.1; "01010000..." 19;"Biergarten"; 3.5; "01010000..." 17;"Nelson Pub"; 4.5; "01010000..." Michael Rüegg (IFS) Route Planning with FOSS November, 2012 19 / 40
20. PostGIS Functions SELECT name, price, round( st_distance_sphere(geom, GeometryFromText(’POINT(8.816511 47.223064)’, 4326) )::numeric, 0) AS "distance" FROM beers ORDER BY distance; "Biergarten"; 3.5; 235 "Dorflade"; 2.4; 300 "Nelson Pub"; 4.5; 381 "Manor"; 1.15;422 "Coop Sunnehof";1.05;706 Note that we must convert the point to EPSG 4326! ST_Distance_Sphere: Returns minimum distance in meters between two lon/lat geometries float ST_Distance_Sphere(geometry lonlatA,geometry lonlatB) Michael Rüegg (IFS) Route Planning with FOSS November, 2012 20 / 40
21. Outline 1 GIS 101 2 PostGIS 3 OpenStreetMap 4 Open Source Routing Machine 5 OpenLayers 6 Showcase: Tourpl Michael Rüegg (IFS) Route Planning with FOSS November, 2012 21 / 40
22. OpenStreetMap Data OpenStreetMap (OSM) export format is a XML format Topological data structure with four data types: Nodes are points with a geographic position, stored as WGS84 coordinates (e. g. POIs) Ways are ordered node lists representing a polyline (e. g. streets, rivers, parks) Tags are used to store metadata about the map objects Relations are used for representing the relationship of existing nodes and ways (e. g. turn restrictions on roads) OpenStreetMap data: www.geofabrik.de OSM data for Switzerland in PBF Format (Protocolbuffer Binary Format): download.geofabrik.de/osm/ europe/switzerland.osm.pbf Michael Rüegg (IFS) Route Planning with FOSS November, 2012 22 / 40
23. Load OSM data into PostgreSQL osm2pgsql is a utility that converts OSM data to PostgreSQL databases osm2pgsql is a lossy conversion utility Two main modes of running: normal and slim mode Normal mode uses memory for intermedia storage Slim mode uses 3 on-disk tracking tables: planet_osm_nodes, planet_osm_ways, planet_osm_rels Michael Rüegg (IFS) Route Planning with FOSS November, 2012 23 / 40
24. Load OSM data into PostgreSQL $ osm2pgsql --create --slim --extra-attributes \ --database <DB_NAME> --prefix osm --style \ /usr/local/share/osm2pgsql/xyz.style \ --username <DB_USER> --port <DB_PORT> \ --hstore-all --input-reader pbf \ --extra-attributes switzerland.osm.pbf -hstore: Generate a hstore (key/value) column By using hstore we can use any tag in SQL queries: gis_db=> SELECT COUNT(*) FROM planet_osm_point WHERE ((tags->’man_made’) = ’tower’); osm2pgsql overwrites the tables and create them fresh by default (use -append instead if necessary) Michael Rüegg (IFS) Route Planning with FOSS November, 2012 24 / 40
25. Outline 1 GIS 101 2 PostGIS 3 OpenStreetMap 4 Open Source Routing Machine 5 OpenLayers 6 Showcase: Tourpl Michael Rüegg (IFS) Route Planning with FOSS November, 2012 25 / 40
26. Introduction High performance routing machine Does not use an A* variant to compute shortest path, but Contraction Hierarchies (CH) Written in C++, available under the GNU license Michael Rüegg (IFS) Route Planning with FOSS November, 2012 26 / 40
27. Installation For Ubuntu 12.04: $ sudo apt-get install build-essential git scons \ pkg-config libprotoc-dev libprotobuf7 \ protobuf-compiler libprotobuf-dev libosmpbf-dev \ libpng-dev libbz2-dev libstxxl-dev libstxxl-doc \ libstxxl1 libxml2-dev libzip-dev \ libboost-thread-dev libboost-system-dev \ libboost-regex-dev libboost-filesystem-dev \ lua5.1 liblua5.1-0-dev libluabind-dev $ git clone https://github.com/DennisOSRM/Project-OSRM.git $ scons -j4 More information: https://github.com/DennisOSRM/ Project-OSRM/wiki/Building%20OSRM Michael Rüegg (IFS) Route Planning with FOSS November, 2012 27 / 40
28. Configuration server.ini server.ini contains properties for the routing engine: Threads = 8 IP = 0.0.0.0 Port = 5000 hsgrData=switzerland.osrm.hsgr nodesData=switzerland.osrm.nodes edgesData=switzerland.osrm.edges ramIndex=switzerland.osrm.ramIndex fileIndex=switzerland.osrm.fileIndex namesData=switzerland.osrm.names Michael Rüegg (IFS) Route Planning with FOSS November, 2012 28 / 40
29. Configuration Routing profiles Routing profiles are specified in Lua files Example car.lua: peed_profile = { ["motorway"] = 90, ["motorway_link"] = 75, ["trunk"] = 85, ["trunk_link"] = 70, ["primary"] = 65, ["primary_link"] = 60, ["secondary"] = 55, ["tertiary"] = 40, ["residential"] = 25, ["living_street"] = 10, ["service"] = 15, ["ferry"] = 5, ["default"] = 50 } Michael Rüegg (IFS) Route Planning with FOSS November, 2012 29 / 40
30. Running OSRM Extracting the Road Network 1/2 OSM data contains information irrelevant to routing OSM data needs to be normalized before being processed This is done by the OSRM tool named extractor Parses the content of the OSM file and writes: A .osrm file containing the routing data A .osrm.restrictions file containing the restrictions to make certain turns during navigation A .osrm.names file which contains the road names Let’s execute it: $ ./osrm-extract switzerland.pbf Michael Rüegg (IFS) Route Planning with FOSS November, 2012 30 / 40
31. Running OSRM Extracting the Road Network 2/2 The extractor is able to handle bzip2 compressed files as well as PBF files External memory accesses are handles by the stxxl library stxxl must be configured in a file .stxxl File format: disk=full_disk_filename,capacity,access_method Example: disk=/tmp/stxxl,25000,syscall Michael Rüegg (IFS) Route Planning with FOSS November, 2012 31 / 40
32. Running OSRM Creating the hierachy and run OSRM To create hierarchy (the precomputed data that enables OSRM to find the shortest path extermly fast), type: $ ./osrm-prepare switzerland.osrm \ switzerland.osrm.restrictions Nearest-neighbor data structure and node map are created Afterwards, 4 files should exist: map.osrm.hsgr (the hierarchy) map.osrm.nodes (the nodemap) map.osrm.ramIndex (stage 1 index) map.osrm.fileIndex (stage 2 index) To run the engine, just type: ./osrm-routed Michael Rüegg (IFS) Route Planning with FOSS November, 2012 32 / 40
33. OSRM Services locate: Location of nearest node to a given coordinate nearest: Location of nearest point on any street segment for a given coordinate viaroute: Computation of shortest path between two coordinates given an ordered list of via points Query format: http://server:5000/_service_?param1=value&...&paramX=value Example (from here to Zürich Mainstation): $ curl "server:5000/viaroute?loc=47.389058,\ > 8.515342&loc=47.378188,8.539204" | python -mjson.tool { "version": 0.3,"status":0,"status_message": "Found route between points","route_geometry": ["4","Duttweilerstrasse",239,4,17,"239m","NE",38], ["2","Foerrlibuckstrasse",363,7,27,"363m","E",84], "route_summary":{"total_distance":3220,"total_time":237,... Michael Rüegg (IFS) Route Planning with FOSS November, 2012 33 / 40
34. Outline 1 GIS 101 2 PostGIS 3 OpenStreetMap 4 Open Source Routing Machine 5 OpenLayers 6 Showcase: Tourpl Michael Rüegg (IFS) Route Planning with FOSS November, 2012 34 / 40
35. Introduction OpenLayers is a JavaScript library for displaying map data in web browsers (no server-side dependencies) Most popular open source software web-mapping client Ability to overlay proprietary non-OGC-compliant mapping layers with OGC WMS, WFS and WFS-T layers Various controls to build custom toolbars, menus and widgets for editing the map Michael Rüegg (IFS) Route Planning with FOSS November, 2012 35 / 40
36. Example <script src="OpenLayers.js"></script> <script type="text/javascript"> var map; function init() { map = new OpenLayers.Map(’map’, { controls: [ new OpenLayers.Control.Navigation(), new OpenLayers.Control.PanZoomBar(), new OpenLayers.Control.LayerSwitcher({’ascending’:false}), new OpenLayers.Control.Permalink(), new OpenLayers.Control.ScaleLine(), new OpenLayers.Control.Permalink(’permalink’), new OpenLayers.Control.MousePosition(), new OpenLayers.Control.OverviewMap(), new OpenLayers.Control.KeyboardDefaults()], numZoomLevels: 6 }); map.addLayer(new OpenLayers.Layer.Google("Google Hybrid", { type : google.maps.MapTypeId.HYBRID, numZoomLevels : 20 })); var position = new OpenLayers.LonLat(lon, lat).transform(fromProjection, toProjection); var markers = new OpenLayers.Layer.Markers("Markers"); map.addLayer(markers); markers.addMarker(new OpenLayers.Marker(position)); } </script> Michael Rüegg (IFS) Route Planning with FOSS November, 2012 36 / 40
37. Outline 1 GIS 101 2 PostGIS 3 OpenStreetMap 4 Open Source Routing Machine 5 OpenLayers 6 Showcase: Tourpl Michael Rüegg (IFS) Route Planning with FOSS November, 2012 37 / 40
38. Tourpl Tourpl — “Der Tourenplaner” A route planning web application developed at University of Applied Sciences Rapperswil Travelling Salesman Problem (TSP) Uses PostGIS, OSM data, osm2pgsql, OSRM and OpenLayers Available at www.tourpl.ch ⇒ Demonstration Michael Rüegg (IFS) Route Planning with FOSS November, 2012 38 / 40
39. Thanks for your attention! INSTITUTE FOR SOFTWARE IFS Michael Rüegg (IFS) Route Planning with FOSS November, 2012 39 / 40
40. Bibliography I Regina O. Obe and Leo S. Hsu PostGIS in Action. Manning, 2011 Stefan Keller Unterrichtsmaterialien “Geodatenbanksysteme”. University of Applied Sciences Rapperswil, 2012 Dennis Luxen OSRM Wiki. https: //github.com/DennisOSRM/Project-OSRM/wiki Michael Rüegg (IFS) Route Planning with FOSS November, 2012 40 / 40