Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Open geo data in The Netherlands (and beyond)

Open geo data in The Netherlands (and beyond)

A brief introduction to the Dutch open geo data scene.

Provides answers to:
- why it is perceived as being crap
- why it is not crap
- where's the data
- how to get it
- how to make a map using TileMill / CartoDB

Go to Google Drive for working links: https://docs.google.com/presentation/d/1lgoJ4R6CGEcBKrfzs2dn5eGswsjoSpbqgntU8TuywI0/edit#slide=id.p

Simeon Nedkov

April 02, 2013
Tweet

More Decks by Simeon Nedkov

Other Decks in Technology

Transcript

  1. Bio and outlook @simeonnedkov Past: MSc Geomatics at TU Delft

    Current: PhD research at Vrije Universiteit on geo tech use in the city/people/government triad - redefine the smart city Today - Dutch/European open geo data and stores - View and get data - Display (in fancy ways) and host your own data - No D3 and OpenGeo Suite
  2. 1. Out of sight, out of mind.. well, almost. 2.

    Gradually... 3. Easy access to pan-European data
  3. It sucks. Verbose Unpolished and obscure docs SHP? EPSG: 28992??

    GIS? WMS? WFS? Reference servers and implementations are mostly Java (and C++) Geoportals.. ugh! Lack of decent geo data sources and services Politics
  4. It needs to get better And it is! Better mapping

    tech: OpenLayers 3, MapBox, CartoDB, Shapely, Fiona More tutorials by non-geo developers Favourable legislation: Wet Basisregistraties Publieke Dienstverlening op de Kaart (PDOK) Nationaal Georegister 2... INSPIRE
  5. Why bother? Well established and powerful standards Extensive infrastructure Access

    to European open geo data Interoperability and ease of use Use it to make it better Encourage cities / open data initiatives to adopt standards You have no choice... :)
  6. Client Data Discover View Download Spatial Data Infrastructures? w e

    b s i t e c a t a l o g machine readable geo services
  7. Geo data in theory Client Metadata - location URL -

    format - extent - accuracy - temporal info - coordinate system - owner - access conditions Data store I Data store III Data store II Data store IV Catalog Discover data Location of data URL Query for data Download Data - georeferenced - vector (shp) - raster (geotiff) View
  8. Geo data in theory Client (Machine) searchable catalog NLR Universities

    Schools Governmental bodies Provinces Cities Research institutes Aerial images? NLR has them at this URL Geo extent json, svg .png, jpg
  9. Geo data in practice Client NLR Universities Schools Provinces Research

    institutes Data store V Provinciaal GeoRegister Governmental bodies Cities Most stores CKAN flat files
  10. Geo data in practice Client NLR Universities Schools Provinces Research

    institutes Data store V Provinciaal GeoRegister Governmental bodies Cities Most stores CKAN flat files Politics and organisational issues
  11. Geo data in practice Client NLR Universities Schools Provinces Research

    institutes Data store V Provinciaal GeoRegister Governmental bodies Cities Most stores CKAN flat files Politics and organisational issues The tech works!
  12. nationaalgeoregister.nl Needs to grow... Back to reality pdok.nl national data

    sets robust up-to-date great use it! provinciaalgeoregister.nl provinces, no services KNMI / NLR
  13. Even more data INSPIRE "... an EU initiative to establish

    an infrastructure for spatial information in Europe ..." 1. Data from all EU member states 2. More Dutch data
  14. Geo services View Web Map Service (WMS) Raster: images Web

    Map Tile Service (WMTS) Tiles Download Web Feature Service (WFS) Vector: geometry and attributes Web Coverage Service (WCS) Raw raster data
  15. http://geo.serv.er/wms Server View vector and raster Download vector From data

    to services Tile Cache geometry sttyling http://geo.serv.er/wfs - (Spatial) filters - (Spatial) operations - Coordinate transformations vector raster values
  16. Client Server - Description of service - Layers - Requests

    - GetMap - GetFeatureInfo - GetLegendGraphic Client GetCapabilities Capabilities document (XML) Server Subsequent requests .png, .jpg and text Parse
  17. Viewing data: WMS PDOK kaart wizard kaart.pdok.nl QGIS - open

    source GIS OpenLayers 2 >> WMS made easy
  18. OpenLayers - WMS var map = new OpenLayers.Map('map', { maxExtent:

    new OpenLayers.Bounds(10000.0, 305000.0, 280000.0, 619000.0), projection: new OpenLayers.Projection("EPSG:28992") }); var wms_url = "http://geodata.nationaalgeoregister.nl/ahn25m/wms" var layer = new OpenLayers.Layer.WMS( "Actueel Hoogtebestand Nederland 25 m", wms_url, { layers: "ahn25m", format: "image/png", srs: "EPSG:28992", styles: "" }, { transitionEffect: 'resize' } ); map.addLayer(layer); map.setCenter(new OpenLayers.LonLat(445463, 156271), 1); http://bl.ocks.org/ndkv/
  19. OpenLayers - Feature info var getFeatureControl = new OpenLayers.Control.WMSGetFeatureInfo({ url:

    proxy + wms_url, layers: [layer], queryVisible: true, eventListeners: { getfeatureinfo: function (event) { var pop = new OpenLayers.Popup.FramedCloud( "ahn25m", map.getLonLatFromPixel(event.xy), null, event.text, null, true) pop.panMapIfOutOfView = false; map.addPopup(pop); } } });
  20. WMS - a note on tiling WMS: single images, no

    tiling, no caching WMS-C: tiling, caching - same endpoint as "pure" WMS - invoke with parameter: tiled=true - OSM and Leaflet do this automagically TMS / WMTS - used for the Basic Registry Topography service - TMS docs: http://wiki.osgeo.org/wiki/Tile_Map_Service_Specification - WMTS docs: ...
  21. Anatomy of WFS Delivers geometry Geographical Markup Language GeoJSON Anatomy

    of WFS BAG http://geodata.nationaalgeoregister.nl/bagviewer/wfs?
  22. Client Server - Description of service - Filters - Functions

    - Feature types i.e. layers - Requests - GetFeature [ filter ] - DescribeFeatureType - ... Client GetCapabilities Capabilities document (XML) Server Subsequent requests .gml, .json, .shp Parse
  23. { "type": "Feature", "id": "pand.fid--2e3a32e5_13db0af9ceb_65e", "geometry": { "type": "Polygon", "coordinates":

    [ [ [ 70418.262, 429235.086 ], [ ... ] ] ] } "geometry_name": "geometrie", "properties": { "gid": 6571756, "identificatie": 530100002242340, "bouwjaar": 2002, "status": "Pand in gebruik", "gebruiksdoel": "woonfunctie", "oppervlakte_min": 165, "oppervlakte_max": 165, "aantal_verblijfsobjecten": 1 } } GetFeature - response
  24. { "type": "Feature", "id": "pand.fid--2e3a32e5_13db0af9ceb_65e", "geometry": { "type": "Polygon", "coordinates":

    [ [ [ 4.159718956737263, 51.84527267018053 ], [ ... ] ] ] } "geometry_name": "geometrie", "properties": { "gid": 6571756, "identificatie": 530100002242340, "bouwjaar": 2002, "status": "Pand in gebruik", "gebruiksdoel": "woonfunctie", "oppervlakte_min": 165, "oppervlakte_max": 165, "aantal_verblijfsobjecten": 1 } } GetFeature in lon/lat
  25. Images vs features WMS Data display Images only Predefined styling

    No analysis WFS Data download Access to geometries Not styled, up to you Analysis
  26. WMS + WFS - Serve tiles with WMS - Get

    object id through GetFeatureInfo - Get object geometry through WFS GetFeature http://example.com/geoserver/wfs? service=wfs& version=2.0.0& request=GetFeature& typeName=namespace:featuretype& featureID=feature http://bagviewer.geodan.nl/index.html
  27. Make me a map 1. Extract data from WFS endpoint

    2. Style with TileMill / CartoDB 3. Host on web through mapbox.com, own server and CartoDB
  28. TileMill + MapProxy open-source cartography toolbox by MapBox open-source proxy

    for geo data - Style GIS data with CartoCSS (revolution! Google "SLD") - Bake .mbtiles - pre-rendered tile cache in SQLite - Host on mapbox.com - Cache, accelerate and transform data from existing data sources: WMS, .mbtiles and others - Disseminate through WMS, WMTS, TMS and KML - View with OpenLayers MapProxy or TileStache or Django TileMill
  29. Data and tools CBS census data http://geodata.nationaalgeoregister.nl/wijkenbuurten2011/wfs - Municipalities >>

    inhabitants - Neighbourhoods >> inhabitants Road network http://geodata.nationaalgeoregister.nl/nwbwegen/wfs - Street geometries and names QGIS and gdal/ogr
  30. Data and tools CBS census data http://geodata.nationaalgeoregister.nl/wijkenbuurten2011/wfs - Municipalities >>

    inhabitants - Neighbourhoods >> inhabitants Road network http://geodata.nationaalgeoregister.nl/nwbwegen/wfs - Street geometries and names QGIS and gdal/ogr Somehow QGIS saves WFS numerical fields as text... ... GDAL/OGR to the rescue!
  31. gdal/ogr Swiss army knife for querying, transforming and manipulating GIS

    data shapefile, csv, gml, gpx, kml, geojson, sqlite, postgresql, mysql, svg, couchdb, wfs ogrinfo - investigate data source ogr2ogr - interact with data [-f format_name] - write output in specified format [-s_srs] - source coordinate system [-t_srs] - target coordinate system [-sql] - execute sql query
  32. ogrinfo investigate WFS endpoint ogrinfo -so WFS:"url" investigate gemeenten2011 layer

    ogrinfo -so WFS:"url" gemeenten2011 url = http://geodata.nationaalgeoregister.nl/wijkenbuurten2011/wfs
  33. ogr2ogr - get - ogr2ogr -f format_name destination source get

    gemeenten2011 layer in native SRS as GeoJSON ogr2ogr -f GeoJSON gemeenten2011.geojson WFS:"url" gemeenten2011 url = http://geodata.nationaalgeoregister.nl/wijkenbuurten2011/wfs
  34. ogr2ogr - get + transform - ogr2ogr -f format_name destination

    source target_srs get gemeenten2011 layer in lat/lng as GeoJSON ogr2ogr -f GeoJSON gemeenten2011.geojson WFS:"url" -t_srs EPSG: 4326 gemeenten2011 url = http://geodata.nationaalgeoregister.nl/wijkenbuurten2011/wfs
  35. ogr2ogr - sql ogr2ogr -f format_name destination source target_srs sql

    ogr2ogr -f GeoJSON gemeenten_2011_84.geojson WFS:" url" -sql "SELECT 'gemeentenaam', cast('aantal_inwoners' AS integer) AS 'aantal_inwoners' FROM 'wijkenbuurten2011:gemeenten2011'" -t_srs EPSG:4326 Load in TileMill
  36. CartoDB "Web platform for creating dynamic maps [...]" - Style

    GIS data with CartoCSS (revolution! Google "SLD") - Store data in PostGIS (revolution! Gazillions of spatial functions) - SQL queries directly from JS (revolution! NoSQL is dead) - Real-time visualisations (revolution!) - Hosting at cartodb.com is expensive... - ... but it's on github
  37. Prepare shp - ogr2ogr -f format_name destination source ogr2ogr -f

    "ESRI Shapefile" gemeenten2011.shp gemeenten2011.geojson - Zip gemeente2011.* and upload to CartoDB