Share
Embed
Open geo data in The Netherlands (and beyond)
A brief introduction to the Dutch open geo data scene.
Provides answers to:
- why it is perceived as being crap
- why it is not crap
- where's the data
- how to get it
- how to make a map using TileMill / CartoDB
Go to Google Drive for working links: https://docs.google.com/presentation/d/1lgoJ4R6CGEcBKrfzs2dn5eGswsjoSpbqgntU8TuywI0/edit#slide=id.p
Simeon Nedkov
April 02, 2013
Transcript
76 slides
-
Bio and outlook @simeonnedkov Past: MSc Geomatics at TU Delft
Current: PhD research at Vrije Universiteit on geo tech use in the city/people/government triad - redefine the smart city Today - Dutch/European open geo data and stores - View and get data - Display (in fancy ways) and host your own data - No D3 and OpenGeo Suite -
... everything you ever wanted to know about geographical information,
standards and services? -
1. Why it sucks. 2. When it will get better.
3. What's in it for me. -
1. Out of sight, out of mind.. well, almost. 2.
Gradually... 3. Easy access to pan-European data -
It sucks. Verbose Unpolished and obscure docs SHP? EPSG: 28992??
GIS? WMS? WFS? Reference servers and implementations are mostly Java (and C++) Geoportals.. ugh! Lack of decent geo data sources and services Politics -
It needs to get better And it is! Better mapping
tech: OpenLayers 3, MapBox, CartoDB, Shapely, Fiona More tutorials by non-geo developers Favourable legislation: Wet Basisregistraties Publieke Dienstverlening op de Kaart (PDOK) Nationaal Georegister 2... INSPIRE -
Why bother? Well established and powerful standards Extensive infrastructure Access
to European open geo data Interoperability and ease of use Use it to make it better Encourage cities / open data initiatives to adopt standards You have no choice... :) -
Client Data Discover View Download Spatial Data Infrastructures? w e
b s i t e c a t a l o g machine readable geo services -
Client Data Discover View Download Spatial Data Infrastructures? w e
b s i t e c a t a l o g Geoportal -
Client Data Discover View Download Spatial Data Infrastructures? w e
b s i t e c a t a l o g Geoportal SDI -
Geo data in theory Client Metadata - location URL -
format - extent - accuracy - temporal info - coordinate system - owner - access conditions Data store I Data store III Data store II Data store IV Catalog Discover data Location of data URL Query for data Download Data - georeferenced - vector (shp) - raster (geotiff) View -
Geo data in theory Client (Machine) searchable catalog NLR Universities
Schools Governmental bodies Provinces Cities Research institutes Aerial images? NLR has them at this URL Geo extent json, svg .png, jpg -
Geo data in practice Client NLR Universities Schools Provinces Research
institutes Data store V Provinciaal GeoRegister Governmental bodies Cities Most stores CKAN flat files -
Geo data in practice Client NLR Universities Schools Provinces Research
institutes Data store V Provinciaal GeoRegister Governmental bodies Cities Most stores CKAN flat files Politics and organisational issues -
Geo data in practice Client NLR Universities Schools Provinces Research
institutes Data store V Provinciaal GeoRegister Governmental bodies Cities Most stores CKAN flat files Politics and organisational issues The tech works! -
nationaalgeoregister.nl Needs to grow... Back to reality pdok.nl national data
sets robust up-to-date great use it! provinciaalgeoregister.nl provinces, no services KNMI / NLR -
More data Local announcements (live!) Topographic maps Rail/Roads/Canals/Parks CBS data
Aerial and satellite images -
Even more data INSPIRE "... an EU initiative to establish
an infrastructure for spatial information in Europe ..." 1. Data from all EU member states 2. More Dutch data -
Geo services View Web Map Service (WMS) Raster: images Web
Map Tile Service (WMTS) Tiles Download Web Feature Service (WFS) Vector: geometry and attributes Web Coverage Service (WCS) Raw raster data -
http://geo.serv.er/wms Server View vector and raster Download vector From data
to services Tile Cache geometry sttyling http://geo.serv.er/wfs - (Spatial) filters - (Spatial) operations - Coordinate transformations vector raster values -
http://geodata.nationaalgeoregister.nl/ahn25m/wms? request=GetCapabilities& service=WMS& version=1.1.3 Interacting with services Request: HTTP GET
/ POST Response: XML, .png, .jpg, .GML, .JSON, .svg -
Client Server - Description of service - Layers - Requests
- GetMap - GetFeatureInfo - GetLegendGraphic Client GetCapabilities Capabilities document (XML) Server Subsequent requests .png, .jpg and text Parse -
GetFeatureInfo - request http://geodata.nationaalgeoregister.nl/ahn25m/wms? request=GetFeatureInfo& service=WMS& version=1.1.1& query_layers=ahn25m& info_format=text/html& x=353&
y=145& layers=ahn25m& srs=EPSG:28892& bbox=197896, 429027, 205562, 433818& format=image/png& width=780& height=330& -
Viewing data: WMS PDOK kaart wizard kaart.pdok.nl QGIS - open
source GIS OpenLayers 2 >> WMS made easy -
QGIS add WMS layer QGIS add WMS layer Add WMS
layer button -
OpenLayers - WMS var map = new OpenLayers.Map('map', { maxExtent:
new OpenLayers.Bounds(10000.0, 305000.0, 280000.0, 619000.0), projection: new OpenLayers.Projection("EPSG:28992") }); var wms_url = "http://geodata.nationaalgeoregister.nl/ahn25m/wms" var layer = new OpenLayers.Layer.WMS( "Actueel Hoogtebestand Nederland 25 m", wms_url, { layers: "ahn25m", format: "image/png", srs: "EPSG:28992", styles: "" }, { transitionEffect: 'resize' } ); map.addLayer(layer); map.setCenter(new OpenLayers.LonLat(445463, 156271), 1); http://bl.ocks.org/ndkv/ -
OpenLayers - Feature info var getFeatureControl = new OpenLayers.Control.WMSGetFeatureInfo({ url:
proxy + wms_url, layers: [layer], queryVisible: true, eventListeners: { getfeatureinfo: function (event) { var pop = new OpenLayers.Popup.FramedCloud( "ahn25m", map.getLonLatFromPixel(event.xy), null, event.text, null, true) pop.panMapIfOutOfView = false; map.addPopup(pop); } } }); -
WMS - a note on tiling WMS: single images, no
tiling, no caching WMS-C: tiling, caching - same endpoint as "pure" WMS - invoke with parameter: tiled=true - OSM and Leaflet do this automagically TMS / WMTS - used for the Basic Registry Topography service - TMS docs: http://wiki.osgeo.org/wiki/Tile_Map_Service_Specification - WMTS docs: ... -
Anatomy of WFS Delivers geometry Geographical Markup Language GeoJSON Anatomy
of WFS BAG http://geodata.nationaalgeoregister.nl/bagviewer/wfs? -
Client Server - Description of service - Filters - Functions
- Feature types i.e. layers - Requests - GetFeature [ filter ] - DescribeFeatureType - ... Client GetCapabilities Capabilities document (XML) Server Subsequent requests .gml, .json, .shp Parse -
{ "type": "Feature", "id": "pand.fid--2e3a32e5_13db0af9ceb_65e", "geometry": { "type": "Polygon", "coordinates":
[ [ [ 70418.262, 429235.086 ], [ ... ] ] ] } "geometry_name": "geometrie", "properties": { "gid": 6571756, "identificatie": 530100002242340, "bouwjaar": 2002, "status": "Pand in gebruik", "gebruiksdoel": "woonfunctie", "oppervlakte_min": 165, "oppervlakte_max": 165, "aantal_verblijfsobjecten": 1 } } GetFeature - response -
{ "type": "Feature", "id": "pand.fid--2e3a32e5_13db0af9ceb_65e", "geometry": { "type": "Polygon", "coordinates":
[ [ [ 4.159718956737263, 51.84527267018053 ], [ ... ] ] ] } "geometry_name": "geometrie", "properties": { "gid": 6571756, "identificatie": 530100002242340, "bouwjaar": 2002, "status": "Pand in gebruik", "gebruiksdoel": "woonfunctie", "oppervlakte_min": 165, "oppervlakte_max": 165, "aantal_verblijfsobjecten": 1 } } GetFeature in lon/lat -
Get, transform, save QGIS reads WFS QGIS transforms coordinates QGIS
saves to disk QGIS handles many file types -
Images vs features WMS Data display Images only Predefined styling
No analysis WFS Data download Access to geometries Not styled, up to you Analysis -
WMS + WFS - Serve tiles with WMS - Get
object id through GetFeatureInfo - Get object geometry through WFS GetFeature http://example.com/geoserver/wfs? service=wfs& version=2.0.0& request=GetFeature& typeName=namespace:featuretype& featureID=feature http://bagviewer.geodan.nl/index.html -
Make me a map 1. Extract data from WFS endpoint
2. Style with TileMill / CartoDB 3. Host on web through mapbox.com, own server and CartoDB -
TileMill + MapProxy open-source cartography toolbox by MapBox open-source proxy
for geo data - Style GIS data with CartoCSS (revolution! Google "SLD") - Bake .mbtiles - pre-rendered tile cache in SQLite - Host on mapbox.com - Cache, accelerate and transform data from existing data sources: WMS, .mbtiles and others - Disseminate through WMS, WMTS, TMS and KML - View with OpenLayers MapProxy or TileStache or Django TileMill -
Data and tools CBS census data http://geodata.nationaalgeoregister.nl/wijkenbuurten2011/wfs - Municipalities >>
inhabitants - Neighbourhoods >> inhabitants Road network http://geodata.nationaalgeoregister.nl/nwbwegen/wfs - Street geometries and names QGIS and gdal/ogr -
Data and tools CBS census data http://geodata.nationaalgeoregister.nl/wijkenbuurten2011/wfs - Municipalities >>
inhabitants - Neighbourhoods >> inhabitants Road network http://geodata.nationaalgeoregister.nl/nwbwegen/wfs - Street geometries and names QGIS and gdal/ogr Somehow QGIS saves WFS numerical fields as text... ... GDAL/OGR to the rescue! -
gdal/ogr Swiss army knife for querying, transforming and manipulating GIS
data shapefile, csv, gml, gpx, kml, geojson, sqlite, postgresql, mysql, svg, couchdb, wfs ogrinfo - investigate data source ogr2ogr - interact with data [-f format_name] - write output in specified format [-s_srs] - source coordinate system [-t_srs] - target coordinate system [-sql] - execute sql query -
ogrinfo investigate WFS endpoint ogrinfo -so WFS:"url" investigate gemeenten2011 layer
ogrinfo -so WFS:"url" gemeenten2011 url = http://geodata.nationaalgeoregister.nl/wijkenbuurten2011/wfs -
ogr2ogr - get - ogr2ogr -f format_name destination source get
gemeenten2011 layer in native SRS as GeoJSON ogr2ogr -f GeoJSON gemeenten2011.geojson WFS:"url" gemeenten2011 url = http://geodata.nationaalgeoregister.nl/wijkenbuurten2011/wfs -
ogr2ogr - get + transform - ogr2ogr -f format_name destination
source target_srs get gemeenten2011 layer in lat/lng as GeoJSON ogr2ogr -f GeoJSON gemeenten2011.geojson WFS:"url" -t_srs EPSG: 4326 gemeenten2011 url = http://geodata.nationaalgeoregister.nl/wijkenbuurten2011/wfs -
ogr2ogr - sql ogr2ogr -f format_name destination source target_srs sql
ogr2ogr -f GeoJSON gemeenten_2011_84.geojson WFS:" url" -sql "SELECT 'gemeentenaam', cast('aantal_inwoners' AS integer) AS 'aantal_inwoners' FROM 'wijkenbuurten2011:gemeenten2011'" -t_srs EPSG:4326 Load in TileMill -
CartoDB "Web platform for creating dynamic maps [...]" - Style
GIS data with CartoCSS (revolution! Google "SLD") - Store data in PostGIS (revolution! Gazillions of spatial functions) - SQL queries directly from JS (revolution! NoSQL is dead) - Real-time visualisations (revolution!) - Hosting at cartodb.com is expensive... - ... but it's on github -
Prepare shp - ogr2ogr -f format_name destination source ogr2ogr -f
"ESRI Shapefile" gemeenten2011.shp gemeenten2011.geojson - Zip gemeente2011.* and upload to CartoDB -
Neat Thanks for your time! (Code) examples at @simeonnedkov http://ndkv.nl
bl.ocks.org/ndkv waalweelde.ndkv.nl