Upgrade to Pro — share decks privately, control downloads, hide ads and more …

State of GeoServer Q1 2021, 2.18 and beyond

State of GeoServer Q1 2021, 2.18 and beyond

Webinar from GeoSolutions on GeoServer 2.18 and the coming 2.19: Improvements on raster analysis, GeoPackage metadata and styles, S3 COG support and more. We will have a 30 minutes presentation and 30 minutes of Q&A with our lead GeoServer developer, Andrea Aime.

Simone Giannecchini

February 03, 2021
Tweet

More Decks by Simone Giannecchini

Other Decks in Technology

Transcript

  1. State of GeoServer Q1 2021, 2.18 and beyond Feb 3,

    2021 Luis E. Bermudez, Ph.D., Andrea Aime Virtual
  2. Agenda and Q&A • 11:00 - 11:10 Introduction – Luis

    Bermudez • 11:10 - 12:00 State of GeoServer – Andrea Aime • 12:00 Poll • 12:00 - 12:30 AAA (Ask Anything to Andrea)
  3. Robust Geospatial Server • GeoServer is the lead open source

    server to publish geospatial data at scale. • GeoServer is a robust solution for management and disseminating geospatial data according to the OGC, ISO and INSPIRE standards. • Organizations use GeoServer to serve petabytes of data, from point data (e.g. real-time world-wide ship positions) to massive earth observation data.
  4. GeoServer WFS 1.0, 1.1, 2.0 WMS 1.1.1 1.3.0 PostGIS Oracle

    H2 DB2 SQL Server MySql Spatialite GeoCouch Shapefile GeoPackage ---------- ---------- --------- ---------- ---------- ---------- --------- ---------- ---------- ---------- --------- ---------- ArcSDE WFS PNG, GIF JPEG TIFF, GeoTIFF SVG, PDF KML/KMZ Shapefile GML2 GML3 GeoRSS GeoJSON CSV/XLS Raw vector data Servers Styled maps DBMS Vector files WCS 1.0,1.1.1 2.0.1 GeoTIFF WMS ArcGrid GTopo30 Img+world Mosaic MrSID JPEG 2000 ECW,Pyramid Oracle GeoRaster, PostGis Raster NetCDF GeoPackage Raster files Raw raster data GeoTIFF ArcGrid GTopo30 Img+World Google WPS 1.0.0 Vector Tiles Vector tiles format MapBox Vector (MVT) GeoJSON TopoJSON OGC Open APIs Mostly from directory Others GeoWebCache Authentication ( OKTA, LDAP, CAS, WSO2, OAUTH..) GeoFence Active Clustering (Hazelcast) …
  5. World leader in the development and maintenance of open source

    enterprise software for geospatial information management
  6. UN FAO (CIOK, FIGIS, NRL, FORESTRY, ESTG), UN WFP, World

    Bank, DLR, EUMETSAT, JRC, ARPAT, NATO CMRE, UNESCO, IGAD, UNEP, ... BAYER, BASF, DigitalGlobe, MDA, TOPCON, SwissRE, e- GEOS, Halliburton, .. More than 200 clients since 2006
  7. GeoSolutions Support • Advice regarding a geospatial challenge • Architect,

    help configure and deploy clusters of GeoServers in the cloud • Automate deployments from different environments • Performance monitoring • Develop custom application (server or client slide) • Optimize ingesting and publishing geospatial data • Training • Other technical support
  8. Subscription Services Example Premium Plan 4 hours – response time

    16 # of incidents per year 20 Additional Support hours 24/7 support Oracle, SQL Server and PostGIS Support
  9. A Solution for Data Management Identify how and where products,

    services, and data are utilized The monitor extension tracks requests made against a GeoServer instance. With the extension request data can be persisted to a database, used to generate simple reports , and routed to a customized request audit log. 10 Most Accessed Layers 10 Most Active Users 10 Most Active IPs Request types
  10. GeoSolutions Contact Information Luis E. Bermudez, Ph.D. CEO GeoSolutions USA

    https://www.geosolutionsgroup.com/contacts/ +1 (301) 337-7055 [email protected] https://www.linkedin.com/in/bermudez https://www.geosolutionsgroup.com https://twitter.com/geosolutions_it
  11. GeoServer 2.18.x l Releases already performed: l 2.18.0, released September

    2020 l 2.18.1, released November 2020 l 2.18.2, released January 2021 l Planned releases: l 2.18.3, April 2021 l 2.18.4, June 2021 l 2.18.5, August 2021
  12. Small GUI improvements l "Apply" button on most pages l

    Better visual separation for "cancel" l Save/Apply/Cancel bar always visible
  13. Jiffle now supports multiband l GeoServer uses "Jiffle" as its

    map algebra language l In 2.18.0 improved to support multi-band output l Very simple before/after/after-before comparison script, generating a 3-band image from two separate one-banded images: before = src[0]; after = src[1]; dest[0] = before; dest[1] = after; dest[2] = after - before;
  14. Projection subsytem improvements l Goode's interrupted homolosine l Full "advanced

    projection handling" support l Polygon sliced automatically at interruptions, dynamic densification
  15. Projection subsytem improvements l Azimuthal equidistant support improved l Set

    of improvements to support Equi7Grid (method to store raster data with low distortion)
  16. Features template community module l Old "JSON-LD" community module l

    Can now generate GeoJSON too Simple features Complex features JSON-LD GeoJSON JSON template
  17. GeoPackage improvements l GeoServer "gs-geopkg" community module contains a WPS

    process that can produce GeoPackages l During OGC Testbed 16 it has been improved significantly: l Embeddig metadata l Embedding styles l Increased writing speed (2x or more) l Enumeration support l Generalized tables l GeoHash sorting
  18. GeoPackage, storing styles l OS Mastermap Topography l Comes with

    6 layers, and 4 different ways to present them ("layer groups") Backdrop Light Outdoor Standard
  19. GeoPackage, storing styles l GeoPackage "portrayal" extension stores: l Style

    bodies (eventually in multiple languages, e.g., MBStyles and SLD) l Style icons (fully self contained!) l Semantic annotation extension l Links styles to tables (tags both with the same annotation) l Metadata extension l Stores "operational pictures", stacks of layers and the associated styles l Stored as OWS context documents in the metadata table
  20. GeoPackage writing speed l Writing OS Master Topography, full UK

    l Six tables, 500 million records! l 300GB as stored in PostgreSQL (roughly the size of a world wide OSM database!) Before TB16 After TB16 Cut it off at 24H, did not finish Terminates in 9 hours
  21. GeoPackage enumerations l Added support for the GeoPackage "schema extension"

    (one of the core extensions) l Ability to map integer numbers to value (enumeration) l Lots of enumerated fields in Mastermap
  22. GeoPackage, arrays l Storing them as JSON arrays l Using

    the JSON1 SQLite extension for querying l Eventually enumerated l TB16 specific extension, not part of the std
  23. GeoPackage, GeoHash sorting l Ability to write records sorted by

    Geohash l Up to 100x speed on Mastermap topography (topographicline table, 330 million records) https://postgis.net/workshops/postgis-intro/clusterindex.html Not an extension, just improved physical layout!
  24. GeoPackage, generalized tables l Speed up multi-scale representation l OSM

    style, secondary tables with simpler geometries and less records Table name Record count waterlines 2.431.848,00 waterlines_g1 164.208,00 waterlines_g2 12.752,00 waterlines_g3 2.481,00 Up to 100x speedup (depending on dataset)
  25. GeoPackage, more info l Get more information about this work!

    l 10 minutes video presentation l Testbed 16 GeoPackage Engineering Report
  26. Modules: going up, going down Extension Community GWC S3 WPS

    download WPS JDBC Params extractor WMTS multidim COG ArcSDE Script DGGS S3 GeoTiff Imagemosaic JDBC
  27. GWC – S3 l GWC blobstore l Stores GWC tiles

    on S3. l Cheaper alternative to local file system for cloud deploys. GWC GWC GWC WMTS OGC API - Tiles
  28. WPS – Download l Set of WPS processes to replace

    WFS/WCS/WMS for large download workloads l Asynch request support l gs:Download à Vector and raster large download l gs:DownloadMap à Large map download (with eventual external layers) l gs:DownloadAnimation à MP4 generator based on maps and time series Execute wps:Download (asynch) GetStatus Download sesult
  29. WPS - JDBC l WPS status storage l Stores status

    of async requests in a database of choice l Allows clustered usage of WPS async requests LOAD BALANCER
  30. Params-extractor l Tool to smuggle vendor params into capabilities URLs

    l Makes un-aware shrink wrapped GIS software use vendor parameters (e.g., CQL_FILTER, VIEW_PARAMS, ENV, …) /geoserver/tiger/wms/H11 ?SERVICE=WMS &VERSION=1.1.1 &REQUEST=GetMap &… /geoserver/tiger/wms ?SERVICE=WMS &VERSION=1.1.1 &REQUEST=GetMap &CQL_FILTER=CFCC%3D%27H11%27 &…
  31. WMTS multidim l Extensions to the WMTS protocol developed in

    Testbed12 l Extra requests allowing to explore dimensions (time, elevation and others) l DescribeDomains: list of dimensions and their description l GetDomainValues: list values in a dimension, eventually filtering on other dimension ranges (e.g., "what are the times available in this bbox"?) l GetFeature: get tuples of all actual dimension value combinations l GetHistogram: produce histogram of values along a given dimension, with filtering
  32. WMTS multidim l MapStore Timeline plugin, powered by WMTS multidim

    Animation controls Histogram Filtering by current view
  33. COG replaces S3-GeoTiff l The COG module provides improved support

    for Cloud Optimized GeoTiffs l Works both on HTTP servers and S3 l Leverages the COG structure to minimize the download l Integration with REST mosaic harvest operations in GeoServer l S3-GeoTIFF instead saved chunks of the remote TIFF on disk, without really leveraging its structure
  34. DGGS support l Set of community modules, under the GeoServer

    OGC API umbrella, to support Discrete Global Grid Systems (hierarchical, equal-ish area tassellations of the word)
  35. DGGS geometry store l No data per se, but allows

    to explore the DGGS and learn its structure rHealPix H3
  36. DGGS ClickHouse l Allows for actual data storage l Uses

    ClickHouse OLAP database l DGGS zones (aka cells) are stored by cell identifier l Fast lookup and aggregation using multiple cores/nodes
  37. DGGS API l Grab data, explore relationships between cells l

    E.g., children/parents, neighbors Neighbors Grand-children of a cell
  38. DGGS for analysis l Multi-resolution database l Perform aggregations at

    the desired zoom level l Choose between speed and accuracy
  39. DGGS DAPA API l Data Access and Processing API l

    Multi-aggregations on a given layer, based on space-time filter l Aggregate on time only or space only: series of value l Aggregate on both time and space: single values l DGGS implementation allows to choose desired resolution l Fast ClickHouse aggregation l Space/time aggregation l 16 attributes l 8.9 million records aggregated in less than a second l On spinning disks (RAID)
  40. DGGS: learn more l Dedicated 10 minutes presentation l OGC

    Testbed 16 - DGGS AND DGGS API engineering report l OGC Testbed-16: Data Access and Processing Engineering Report
  41. Modules demoted and removed l The script modules allowed to

    script certain GeoServer plugins in Python, Javascript, Groovy l The module fell un-maintained for years, we eventually had to drop it l ArcSDE is no longer supported by ESRI, and the module was also un-maintained l Imagemosaic-JDBC allows to read rasters from databases. No longer maintained, dropped to "community" status for the time being.