Building a National Tile Server

8d6d22bb8ae016ca2e041a439194b41b?s=47 Matt Walker
September 05, 2012

Building a National Tile Server

How Astun Technology (http://astuntechnology.com) built a national (Great Britain) web map tile server and WMS using PostgreSQL, PostGIS, MapServer and MapProxy all running on AWS. Presented at OSGIS 2012.(http://www.nottingham.ac.uk/osgis/)

8d6d22bb8ae016ca2e041a439194b41b?s=128

Matt Walker

September 05, 2012
Tweet

Transcript

  1. Building a National Tile Server Matt Walker mattwalker@astuntechnology.com @_walkermatt

  2. What is it? Great Britain base map tile service (TMS)

    and WMS OS Open (Strategy, Meridian 2, VectorMap District) OS Premium (OS Open + VectorMap Local & OS MasterMap) Smaller areas of Aerial Photography and other base mapping
  3. © crown copyright 2012

  4. Who's using it? Exactrak Vehicle Tracking and Fleet Management Mendip

    District Council Public Website Windsor and Maidenhead Public Website Surrey County Council Public Website South Norfolk Council Public Website
  5. Who's using it? Stockport MBC Public Website Surrey Heath Borough

    Council Public Website / Intranet GIS Chichester District Council Public Website
  6. Software

  7. Infrastructure

  8. Loader • Python script for loading GML and KML data

    using OGR • Originally for loading OS GML data (but has been used with other GML and KML data) • Allows the source GML to be prepared and enhanced during loading via Python
  9. Loader • Fairly fast (national cover OS MasterMap in 2

    days) • Concurrent loading via 4 processes into PostgreSQL via PGDump with COPY statements, create indexes once when all data loaded
  10. PostgreSQL/PostGIS • PostgreSQL v9 with PostGIS v1.5 • 210GB of

    OS MasterMap and VectorMap Local data and indexes • 10GB for Meridian, Strategi etc. • Separate tablespace for OSMM data; OSMM index; VML data; VML index
  11. PostgreSQL/PostGIS • During load – fsync = off to avoid

    flushing to disk too frequently – work_mem = 1GB when building indexes • Effectively read-only once in production so configured for read: – shared_buffers = 2GB – effecitive_cache_size = 4GB
  12. MapServer • Fast and reliable • Simple configuration • Mature

    feature set • All data read direct from PostgreSQL
  13. MapServer • PROCESSING "CLOSE_CONNECTION=DEFER" to share connections • OS MasterMap

    and VectorMap Local generally styled based on featurecode • Using some attributes derived by Loader for CartoText such as orientdeg and anchorposition for better text placement
  14. MapProxy • Provides caching of map tiles for performance •

    Exposes WMS, TMS services for your application, serving WMS direct from the cache
  15. MapProxy

  16. MapProxy • Nice features like only storing one tile for

    single colour tiles and mixed format caches • Sophisticated cache seeding and management tools • We seed popular areas (urban areas) for our customers and cache is only about 20GB compared to hundreds of GB if pre-seeded
  17. Infrastructure

  18. • ~8 EC2 instances all running Ubuntu • Seperate EBS

    Volumes for PostgreSQL Data, Indexes and Tile Cache • One Load Balancer in front of the production servers • AWS monitoring for health as well as seperate web monitoring Infrastructure
  19. • Shared, High Performance, Redundant Cache, possibly S3 SimpleDB or

    similar • Improved cartography, possibly Mapnik • Co-located hosting for imporved redundency What's next?
  20. astuntechnology.com AstunTechnology walkermatt @isharemaps @_walkermatt Thanks