State of GeoServer Q1 2021, 2.18 and beyond

Webinar from GeoSolutions on GeoServer 2.18 and the coming 2.19: Improvements on raster analysis, GeoPackage metadata and styles, S3 COG support and more. We will have a 30 minutes presentation and 30 minutes of Q&A with our lead GeoServer developer, Andrea Aime.

Transcript

1. State of GeoServer Q1 2021, 2.18 and beyond Feb 3, 2021 Luis E. Bermudez, Ph.D., Andrea Aime Virtual
2. Agenda and Q&A • 11:00 - 11:10 Introduction – Luis Bermudez • 11:10 - 12:00 State of GeoServer – Andrea Aime • 12:00 Poll • 12:00 - 12:30 AAA (Ask Anything to Andrea)
3. Robust Geospatial Server • GeoServer is the lead open source server to publish geospatial data at scale. • GeoServer is a robust solution for management and disseminating geospatial data according to the OGC, ISO and INSPIRE standards. • Organizations use GeoServer to serve petabytes of data, from point data (e.g. real-time world-wide ship positions) to massive earth observation data.
4. GeoServer WFS 1.0, 1.1, 2.0 WMS 1.1.1 1.3.0 PostGIS Oracle H2 DB2 SQL Server MySql Spatialite GeoCouch Shapefile GeoPackage ---------- ---------- --------- ---------- ---------- ---------- --------- ---------- ---------- ---------- --------- ---------- ArcSDE WFS PNG, GIF JPEG TIFF, GeoTIFF SVG, PDF KML/KMZ Shapefile GML2 GML3 GeoRSS GeoJSON CSV/XLS Raw vector data Servers Styled maps DBMS Vector files WCS 1.0,1.1.1 2.0.1 GeoTIFF WMS ArcGrid GTopo30 Img+world Mosaic MrSID JPEG 2000 ECW,Pyramid Oracle GeoRaster, PostGis Raster NetCDF GeoPackage Raster files Raw raster data GeoTIFF ArcGrid GTopo30 Img+World Google WPS 1.0.0 Vector Tiles Vector tiles format MapBox Vector (MVT) GeoJSON TopoJSON OGC Open APIs Mostly from directory Others GeoWebCache Authentication ( OKTA, LDAP, CAS, WSO2, OAUTH..) GeoFence Active Clustering (Hazelcast) …
5. World leader in the development and maintenance of open source enterprise software for geospatial information management
6. UN FAO (CIOK, FIGIS, NRL, FORESTRY, ESTG), UN WFP, World Bank, DLR, EUMETSAT, JRC, ARPAT, NATO CMRE, UNESCO, IGAD, UNEP, ... BAYER, BASF, DigitalGlobe, MDA, TOPCON, SwissRE, e- GEOS, Halliburton, .. More than 200 clients since 2006
7. GeoSolutions Support • Advice regarding a geospatial challenge • Architect, help configure and deploy clusters of GeoServers in the cloud • Automate deployments from different environments • Performance monitoring • Develop custom application (server or client slide) • Optimize ingesting and publishing geospatial data • Training • Other technical support
8. Enterprise Support Services
9. Subscription Services Example Premium Plan 4 hours – response time 16 # of incidents per year 20 Additional Support hours 24/7 support Oracle, SQL Server and PostGIS Support
10. Training
11. Customized Solutions
12. A Solution for Data Management Identify how and where products, services, and data are utilized The monitor extension tracks requests made against a GeoServer instance. With the extension request data can be persisted to a database, used to generate simple reports , and routed to a customized request audit log. 10 Most Accessed Layers 10 Most Active Users 10 Most Active IPs Request types
13. GeoSolutions Contact Information Luis E. Bermudez, Ph.D. CEO GeoSolutions USA https://www.geosolutionsgroup.com/contacts/ +1 (301) 337-7055 luis.bermudez@geosolutionsgroup.com https://www.linkedin.com/in/bermudez https://www.geosolutionsgroup.com https://twitter.com/geosolutions_it
14. Andrea Aime GeoSolutions State of GeoServer Q1 2021
15. GeoServer 2.18.x
16. GeoServer 2.18.x l Releases already performed: l 2.18.0, released September 2020 l 2.18.1, released November 2020 l 2.18.2, released January 2021 l Planned releases: l 2.18.3, April 2021 l 2.18.4, June 2021 l 2.18.5, August 2021
17. GeoServer 2.18.0 - extensions
18. GeoServer 2.18.0 - community https://build.geoserver.org/geoserver/2.18.x/community-latest/
19. Small GUI improvements l "Apply" button on most pages l Better visual separation for "cancel" l Save/Apply/Cancel bar always visible
20. Jiffle now supports multiband l GeoServer uses "Jiffle" as its map algebra language l In 2.18.0 improved to support multi-band output l Very simple before/after/after-before comparison script, generating a 3-band image from two separate one-banded images: before = src[0]; after = src[1]; dest[0] = before; dest[1] = after; dest[2] = after - before;
21. Ship density in the Black Sea T1 T2 T1/T2/DIFFERENCE
22. Projection subsytem improvements l Goode's interrupted homolosine l Full "advanced projection handling" support l Polygon sliced automatically at interruptions, dynamic densification
23. Projection subsytem improvements l Azimuthal equidistant support improved l Set of improvements to support Equi7Grid (method to store raster data with low distortion)
24. Projection subsytem improvements l Azimuthal equidistant polar area improvements l Example source is vector, 3857 (hence hole)
25. Features template community module l Old "JSON-LD" community module l Can now generate GeoJSON too Simple features Complex features JSON-LD GeoJSON JSON template
26. GeoPackage improvements l GeoServer "gs-geopkg" community module contains a WPS process that can produce GeoPackages l During OGC Testbed 16 it has been improved significantly: l Embeddig metadata l Embedding styles l Increased writing speed (2x or more) l Enumeration support l Generalized tables l GeoHash sorting
27. GeoPackage metadata embedding l Metadata links target embedded in generated GeoPackage
28. GeoPackage, storing styles l OS Mastermap Topography l Comes with 6 layers, and 4 different ways to present them ("layer groups") Backdrop Light Outdoor Standard
29. GeoPackage, storing styles l GeoPackage "portrayal" extension stores: l Style bodies (eventually in multiple languages, e.g., MBStyles and SLD) l Style icons (fully self contained!) l Semantic annotation extension l Links styles to tables (tags both with the same annotation) l Metadata extension l Stores "operational pictures", stacks of layers and the associated styles l Stored as OWS context documents in the metadata table
30. GeoPackage writing speed l Writing OS Master Topography, full UK l Six tables, 500 million records! l 300GB as stored in PostgreSQL (roughly the size of a world wide OSM database!) Before TB16 After TB16 Cut it off at 24H, did not finish Terminates in 9 hours
31. GeoPackage enumerations l Added support for the GeoPackage "schema extension" (one of the core extensions) l Ability to map integer numbers to value (enumeration) l Lots of enumerated fields in Mastermap
32. GeoPackage enumerations l Part of the standard l Yet generally not well supported in clients
33. GeoPackage, arrays l Storing them as JSON arrays l Using the JSON1 SQLite extension for querying l Eventually enumerated l TB16 specific extension, not part of the std
34. GeoPackage, GeoHash sorting l Ability to write records sorted by Geohash l Up to 100x speed on Mastermap topography (topographicline table, 330 million records) https://postgis.net/workshops/postgis-intro/clusterindex.html Not an extension, just improved physical layout!
35. GeoPackage, generalized tables l Speed up multi-scale representation l OS ZoomStack, free 10GB GeoPackage download
36. GeoPackage, generalized tables l Speed up multi-scale representation l OSM style, secondary tables with simpler geometries and less records Table name Record count waterlines 2.431.848,00 waterlines_g1 164.208,00 waterlines_g2 12.752,00 waterlines_g3 2.481,00 Up to 100x speedup (depending on dataset)
37. GeoPackage, more info l Get more information about this work! l 10 minutes video presentation l Testbed 16 GeoPackage Engineering Report
38. GeoServer 2.19.x
39. Modules: going up, going down Extension Community GWC S3 WPS download WPS JDBC Params extractor WMTS multidim COG ArcSDE Script DGGS S3 GeoTiff Imagemosaic JDBC
40. GWC – S3 l GWC blobstore l Stores GWC tiles on S3. l Cheaper alternative to local file system for cloud deploys. GWC GWC GWC WMTS OGC API - Tiles
41. WPS – Download l Set of WPS processes to replace WFS/WCS/WMS for large download workloads l Asynch request support l gs:Download à Vector and raster large download l gs:DownloadMap à Large map download (with eventual external layers) l gs:DownloadAnimation à MP4 generator based on maps and time series Execute wps:Download (asynch) GetStatus Download sesult
42. WPS - JDBC l WPS status storage l Stores status of async requests in a database of choice l Allows clustered usage of WPS async requests LOAD BALANCER
43. Params-extractor l Tool to smuggle vendor params into capabilities URLs l Makes un-aware shrink wrapped GIS software use vendor parameters (e.g., CQL_FILTER, VIEW_PARAMS, ENV, …) /geoserver/tiger/wms/H11 ?SERVICE=WMS &VERSION=1.1.1 &REQUEST=GetMap &… /geoserver/tiger/wms ?SERVICE=WMS &VERSION=1.1.1 &REQUEST=GetMap &CQL_FILTER=CFCC%3D%27H11%27 &…
44. WMTS multidim l Extensions to the WMTS protocol developed in Testbed12 l Extra requests allowing to explore dimensions (time, elevation and others) l DescribeDomains: list of dimensions and their description l GetDomainValues: list values in a dimension, eventually filtering on other dimension ranges (e.g., "what are the times available in this bbox"?) l GetFeature: get tuples of all actual dimension value combinations l GetHistogram: produce histogram of values along a given dimension, with filtering
45. WMTS multidim l MapStore Timeline plugin, powered by WMTS multidim Animation controls Histogram Filtering by current view
46. COG replaces S3-GeoTiff l The COG module provides improved support for Cloud Optimized GeoTiffs l Works both on HTTP servers and S3 l Leverages the COG structure to minimize the download l Integration with REST mosaic harvest operations in GeoServer l S3-GeoTIFF instead saved chunks of the remote TIFF on disk, without really leveraging its structure
47. DGGS support l Set of community modules, under the GeoServer OGC API umbrella, to support Discrete Global Grid Systems (hierarchical, equal-ish area tassellations of the word)
48. DGGS geometry store l No data per se, but allows to explore the DGGS and learn its structure rHealPix H3
49. DGGS ClickHouse l Allows for actual data storage l Uses ClickHouse OLAP database l DGGS zones (aka cells) are stored by cell identifier l Fast lookup and aggregation using multiple cores/nodes
50. DGGS API l Grab data, explore relationships between cells l E.g., children/parents, neighbors Neighbors Grand-children of a cell
51. DGGS API l Map polygons to set of DGGS cells at a given resolution
52. DGGS for analysis l Multi-resolution database l Perform aggregations at the desired zoom level l Choose between speed and accuracy
53. DGGS DAPA API l Data Access and Processing API l Multi-aggregations on a given layer, based on space-time filter l Aggregate on time only or space only: series of value l Aggregate on both time and space: single values l DGGS implementation allows to choose desired resolution l Fast ClickHouse aggregation l Space/time aggregation l 16 attributes l 8.9 million records aggregated in less than a second l On spinning disks (RAID)
54. DGGS: learn more l Dedicated 10 minutes presentation l OGC Testbed 16 - DGGS AND DGGS API engineering report l OGC Testbed-16: Data Access and Processing Engineering Report
55. Modules demoted and removed l The script modules allowed to script certain GeoServer plugins in Python, Javascript, Groovy l The module fell un-maintained for years, we eventually had to drop it l ArcSDE is no longer supported by ESRI, and the module was also un-maintained l Imagemosaic-JDBC allows to read rasters from databases. No longer maintained, dropped to "community" status for the time being.
56. That’s all! https://www.geosolutionsgroup.com/contacts/