State of GeoServer - FOSS4G 2021 Edition

This presentation provides an update on our community as well as reviews of the new and noteworthy features for the latest releases. Attend this talk for a cheerful update on what is happening with this popular OSGeo project, whether you are an expert user, a developer, or simply curious what GeoServer can do for you.

Transcript

1. State of GeoServer 2.19 and 2.20
2. Jody Garnett jody.garnett@geocat.net @jodygarnett GeoCat Government Geographic Data publishing Andrea Aime Technical Lead andrea.aime@geo-solutions.it @geowolf GeoSolutions Innovative, robust, cost-effective solutions leveraging best-of-breed Open Source products. This presentation is brought to you by
3. GeoServer at a Glance Java Web Application to share and edit geospatial data. Publish data from any major spatial data source using open standards. Core Protocols WMS – maps WFS – vector WFS-T – editing WCS – coverage WMTS – tiles TMS – tiles WMS-C – tiles Extension/community protocols WPS – process CSW – search OpenSearch for Earth Observation – search OGC API - prototype json + rest standards
4. GeoServer Community 2021 Update
5. Core committers are nominated by their peers and trusted with the ability to approve pull-requests. We also support community commit access by request (often to work on a specific research and development topic): • 21 committers • Marco Volpini (new core committer) Project Steering Committee serves to guide the project drawing from the various groups with a stake in the success of the project. • Alessio Fabiani • Andrea Aime • Ian Turton • Jody Garnett • Jukka Rahkonen • Kevin Smith • Nuno Oliveira • Simone Giannecchini • Torben Barsballe GeoServer Team
6. GeoServer maintains a list service providers: • Core Contributors Ongoing commitment to the project devoting resources to security fixes, releases and maintenance activities. • Experienced Providers Successfully contribute functionality to the project on behalf of their customers. • Additional Services Providers Provide training, setup and integration support and assistance using GeoServer. (request: please take part in community) GeoServer Service Providers Policy change recognize participation: Core contributors directly contribute to project sustainability and are recognized for: • core, committer, support Experienced providers participate in aspects of the project as community members: • development, coding, documentation, outreach, translation, training, service, product
7. • Infrastructure ◦ geoserver.org (GitHub Pages) geoserver.org/blog ◦ repo.osgeo.org (OSGeo) ◦ downloads (SourceForge) ◦ email (SourceForge) ◦ build.geoserver.org (GeoCat) ◦ github.com/geoserver (GitHub) ◦ web map (GeoSolutions) • Transition from Boundless → OSGeo ◦ blog.geoserver.org (wordpress) ◦ docs.geoserver.org (s3) ◦ domain names (namecheap) GeoServer Infrastructure
8. Incoming: • Cloud Native GeoServer (distribution) • cov-json • rest-openapi-java-client • gwc-mbtiles • schemaless-mongodb • smart data loader • stac bindings for opensearch eo • ogcapi-maps • ogcapi-coverages • cog-geotiff Outgoing: • ArcSDE (datastore) • gs-scripts Community modules GeoServer maintains a “community space” for experiments and new developers to work. • Developers propose a community module idea and request commit access • We make sure they compile! • May be proposed as an extension when both the author and functionality is ready Spot a community module of interest, reach out to see how you can help.
9. GeoServer Releases 2021 Update
10. stable maintenance development September 2020 March 2021 September 2021 March 2022 2.18.x 2.19.x 2.20.x Releases covered by this presentation YOU ARE HERE
11. • 2.19.0 ◦ Release March 2021 ◦ First stable release of the 2.19.x series • 2.20.0 ◦ Released September 2021 ◦ First stable release of 2.20.x series GeoServer 2.19 and 2.20 March 2021 September 2021 March 2022 September 2022 development stable maintenance • 2.20.x get 6 more releases ◦ In "stable" status until March 2122, with some new features getting backported ◦ In "maintenance" until September 2022, mostly with bug fixes • Can run on Java 8 or Java 11 ◦ OpenJDK 8 LTS recommended (AdoptOpenJDK → Adoptium) YOU ARE HERE
12. Are you using a older version? Upgrade! • User list and stack exchange typically covers only supported versions ◦ In September 2021 “supported” means 2.19.x and 2.20.x ◦ In March 2022 “supported” will mean 2.20.x and 2.21.x • Security fixes added to supported versions only • Please upgrade your GeoServer installations! Photo by SpaceX on Unsplash
13. Upgrade? What’s in it for me? • Much! • Check what’s new in 2.20 and 2.19 • Check the bottom of each slide to see who sponsored a certain feature, who implemented it, and what version contains it • This icon marks activities done without any sponsoring Version Author Sponsor ♥
14. Mapping Data making a scene!
15. SLD contextual selection • We often need to create complex SLD to get the right output • The resulting legend is complex, difficult to read • Options available? ◦ Pre-rendered legend ( one loses all dynamic abilities, count, removing rules not shown in current area, JSON legend is still complex) ◦ Some systems allow to write a completely separate stylesheet for legend → duplication ◦ Our approach: label rules/symbolizers for map or legend usage only instead! 2.19 Marco Volpini GeoSolutions Province of Bolzano, Italy <VendorOption name=”inclusion”>legendOnly</VendorOption> <VendorOption name=”inclusion”>mapOnly</VendorOption>
16. SLD contextual selection example 2.19 Marco Volpini GeoSolutions Province of Bolzano, Italy That “boundary” rule does help making the legend readable. Yet, we need borders and labels. Eliminating it from the legend with mapOnly
17. Add colormap labels in GetFeatureInfo 2.19 Marco Volpini GeoSolutions UNEP • Classified raster. Numbers only may make feature info hard to understand • New option to include the ColorMapEntry in the output • Adding label keeping number, or replacing it <sld:RasterSymbolizer> <sld:ColorMap> <sld:ColorMapEntry color="#0000FF" quantity="1.0" label="low" /> <sld:ColorMapEntry color="#FFFF00" quantity="124" label="mid" /> <sld:ColorMapEntry color="#FF7F00" quantity="559" label="high" /> <sld:ColorMapEntry color="#FF0000" quantity="999" label="veryhigh" /> </sld:ColorMap> <sld:ContrastEnhancement/> <VendorOption name="labelInFeatureInfo" >add</VendorOption> </sld:RasterSymbolizer> { "type": "FeatureCollection", "features": [ { "type": "Feature", "id": "", "geometry": null, "properties": { "GRAY_INDEX": 124, "Label_GRAY_INDEX": "mid" } } ], "totalFeatures": "unknown", "numberReturned": 1, "timeStamp": "...", "crs": null }
18. MapML extensions graduation • Proposed extension to HTML (w3c.org) ◦ Think of what <video> did for playing videos ◦ Native support for maps in browsers ◦ Supports maps (WMS), tiles (WMTS), features (WFS) • Accessible, simple and extensible map semantics is a key goal 2.19 Peter Rushforth Canada Centre for Mapping and Earth Observation https://{serverName}/geoserver/mapml/{layerName}/{projectionName} https://example.org/geoserver/mapml/topp:states/osmtile/
19. MapML extensions graduation 2.19 Peter Rushforth • Geo community is warmly invited to help us improve the proposal (and the GeoServer module)! HTML <map…> element <map is="web-map" projection="osmtile"zoom ="2" lat="61.209125" lon="-90.850837" controls> <layer- label="US States" src ="http://localhost:8080/geoserver/mapml/topp:states/osmtile?style=population " </layer-> </map> Canada Centre for Mapping and Earth Observation
20. Services News and Updates
21. WPS-JDBC graduation 2.19 Ian Turton Andrea Aime GeoSolutions • WPS can do async requests • Clients call back to the server, inquiring about progress • Problem if there is a cluster, and async request status is stored in memory • “Status store” extension point, implemented as a simple relational DB storage WPS WPS WPS Relational DB Load balancer WPS client Get Status
22. WPS-download graduation 2.19 Alessio Fabiani Daniele Romagnoli Andrea Aime GeoSolutions • Set of WPS processes to download large data sets • Meant to be used in asynchronous mode ◦ Vector and raster, with clip and reprojection ◦ Native resolution downloads ◦ Large maps ◦ Time based animations • Used, among others, by MapStore and GeoNode
23. WMTS multi-dimensional graduation 2.19 Nu Andrea Aime GeoSolutions • Extends WMTS with new operations • Allows to drill into time/elevation/custom dimensions • E.g. ◦ “what times are available in this area?” ◦ “What forecast run-times are available for these predicted times”? • Evolved in OGC Testbed 12 • Used by MapStore timeline extension, to drive map animation
24. Params-extractor graduation 2.19 Alessio Fabiani Daniele Romagnoli Andrea Aime GeoSolutions • Problem: want to use GeoServer vendor options, but a client does not know how to use them • Solution: embed params in the path or query string that are mapped to specific vendor option values • Use params in GetCapabilities, have them reflected in all backlinks • Client uses extensions without realizing Desktop client supporting basic OGC protocols GeoServer with vendor options ???
25. Params-extractor graduation 2.19 GeoSolutions KVP param echo-ing Simple, not all clients support query params in caps backlinks Path value extraction Stick the value of a param in the path .../geoserver/1023/wms?... → .../geoserver/wms?cql_filter=CFCC=’1023’ Nuno Oliveira
26. GeoWebCache
27. GWC S3 graduated to extension 2.19 Kevin Smith Andrea Aime Boundless/GeoSolutions • GeoServer extension module • Storing S3 tiles into GeoServer • Already available in GWC stand-alone • Used to be a community module in GeoServer • Options: ◦ Dynamic caching ◦ Seed and use S3 bucket directly (TMS tile layout)
28. OGC Two Dimensional Tile Matrix Set 2.20 Andrea Aime GeoSolutions • Out of the box support for the OGC 2D TMS well known tile matrix sets • Both 256 and 512px versions • Use numbers for tile matrix… compatible with Mapbox clients expectations
29. Configuration Improvements
30. Can you say... internationalization? 2.20 Marco Volpini GeoSolutions • Improved support for multi-language in GeoServer • Title and abstract i18n for layer, group, service • Contact information (different for different languages) • Reacts to AcceptLanguages and Language • INSPIRE compatibility (e.g. “ger” → “de”)
31. SLD i18n 2.20 Marco Volpini GeoSolutions • Title rule with multiple languages • language() function to access current language used for language specific map render <Title>This is the default title <Localized lang="en">English title </Localized> <Localized lang="it">Titolo in italiano </Localized> </Title> <TextSymbolizer> <Label> <ogc:Function name="Recode"> <ogc:Function name="language"/> <ogc:Literal/> <ogc:PropertyName>name_default</ogc:PropertyName> <ogc:Literal>en</ogc:Literal> <ogc:PropertyName>name_en</ogc:PropertyName> <ogc:Literal>it</ogc:Literal> <ogc:PropertyName>name_it</ogc:PropertyName> <ogc:Literal>fr</ogc:Literal> <ogc:PropertyName>name_fr</ogc:PropertyName> </ogc:Function> </Label> <Fill> <CssParameter name="fill">#000000</CssParameter> </Fill> </TextSymbolizer>
32. SLD i18n 2.20 Marco Volpini GeoSolutions <Rule> <ogc:Filter> <ogc:PropertyIsEqualTo> <ogc:Function name="language" /> <ogc:Literal>it</ogc:Literal> </ogc:PropertyIsEqualTo> </ogc:Filter> <PointSymbolizer> <Graphic> <ExternalGraphic> <OnlineResource xlink:type="simple" xlink:href="it_symbol.png" /> <Format>image/png </Format> </ExternalGraphic> <Size>32</Size> </Graphic> </PointSymbolizer> </Rule> <Rule> <ogc:Filter> <ogc:PropertyIsEqualTo> <ogc:Function name="language" /> <ogc:Literal>de</ogc:Literal> </ogc:PropertyIsEqualTo> </ogc:Filter> <PointSymbolizer> <Graphic> <ExternalGraphic> <OnlineResource xlink:type="simple" xlink:href="de_symbol.png" /> <Format>image/png </Format> </ExternalGraphic> <Size>32</Size> </Graphic> </PointSymbolizer> </Rule> <Rule> <PointSymbolizer> <Graphic> <ExternalGraphic> <OnlineResource xlink:type="simple" xlink:href=" ${language()}_symbol.png "/> <Format>image/png </Format> </ExternalGraphic> <Size>32</Size> </Graphic> </PointSymbolizer> </Rule>
33. GetLegendGraphics i18n vector legends 2.20 Marco Volpini GeoSolutions • Leveraging i18n title for layers • Also use rule title i18n (when present) &language=it &language=en
34. GetLegendGraphics i18n raster legends 2.20 Marco Volpini GeoSolutions • “Default” language also available for cases where no language was selected &language=it &language=de no language param (default language)
35. Community Building
36. • Keep exploit details out of issue report • Mark the issue as a vulnerability. • Be prepared to work with Project Steering Committee (PSC) on a solution • Keep in mind PSC members are volunteers and an extensive fix may require fundraising / resources If you are not in position to communicate in public please consider commercial support, contacting a PSC member, or reaching us via the Open Source Geospatial Foundation at info@osgeo.org. In case you stumble into a vulnerability: Responsible Disclosure
37. Developer Participation Concerns: • Project care and feeding relies on a low number of people Actions: • Reach out to “downstream projects” to test release candidates • Revamp service providers page to highlight participation and experience (encourage service providers to participate) • Exploring small contracts, and “cost recovery” code-sprint model
38. Road Map Coming soon to a GeoServer near you
39. Landing in 2.21: layer group styles • A layer group style is a style name offering a different list of layers and styles to build the group • Use cases: ◦ Different representation of same layers ◦ Simplified representation of same map (less layers) • Example here: Ordnance Survey MasterMap (6 layers) in 4 alternative styles 2.21 Marco Volpini GeoSolutions SWAM
40. Landing in 2.21: symbology factories control • Point Vector symbology in SLD comes from “mark factories” • Control order of execution and availability in styles • For complex point maps, big speed improvements by putting the most used factories first 2.21 Marco Volpini EMSA Above: real time tracking of 150k ships in the EU area
41. Landing in 2.21: Windows Installer • Last released for GeoServer 2.14.3 ◦ “Lost” in infrastructure vandalism • Manual process has been in dev guide (for organizations requiring installer) • GSIP-198: small contract with GeoCat ◦ New NSIS installer ◦ New Service wrapper ◦ Automation for build.geoserver.org 2.21 Sander Schaminee GeoCat GeoServer
42. Scheduled for 2.21: Welcome Page Layout GSIP-202 Welcome Page Layout - Inspired by ogc-api community module - Use title and description to make a heading for each service - List capabilities and tools for each section - Default data directory headings updated - GeoServer WMS → Maps - GeoServer WMTS → Tiles - Change workspace - A welcome page for each workspace - Click to explore virtual web services 2.21 Jody Garnett GeoCat GeoServer Enterprise GeoCat
43. Looking Ahead Community modules RnD seeking participation and funding
44. • Supports efficient access to Cloud Optimized GeoTiff (COG) • HTTP/S3 (Google storage incoming) • Minimizes reads on blob storage Cloud Optimized GeoTIFF community module Planet GeoSolutions Community Header Tile Tile Tile Tile Tile Tile Read 1 Read 2 Josh Fix Daniele Romagnoli • Contact GeoSolutions to sponsor: ◦ More blob storage options ◦ Caching ◦ Other improvements
45. • New OGC Standards ◦ Self describing OpenAPI ◦ REST / JSON ◦ Building on from W3C collaboration and STAC / WFS3 progress • Contact Andrea Aime if you are interested in helping with time or funding! OGC API community module Andrea Aime GeoSolutions GeoSolutions/OGC API Common Maps API Coverages API Implementing (as part of TB14/tb15) Looking for volunteers/sponsors Community Process API Records API Features API Tiles API Styles API STAC API
46. GeoPackage community module improvements • Core include geopackage as a data format, community module focuses on geopackage output formats for WPS and WFS • Faster SQLite database creation with insert batching • Extending GeoPackage - yes you can ◦ Include in-line linked metadata ◦ Include styles ◦ Included generalized tables (store pre-simplified geometries) • Contact Andrea Aime if you are interested in helping with time or funding! Community Andrea Aime GeoSolutions GeoSolutions/OGC
47. DGGS community module • Supports Discrete Global Grid Systems • Attach data to DGGS zones, store in ClickHouse OLAP db • Map on WMS, download with WFS • Use OGC API - DGGS Community Andrea Aime rHealPix H3 GeoSolutions/OGC
48. DGGS community module • Discrete Global Grid System • Perform quick summarization by time/area with DAPA extensions (Data Access and Processing API) • Multi-resolution database: choose between fast and accurate analysis Community Andrea Aime GeoSolutions/OGC
49. Features templating Community module • Templating system to customize WFS/OGC API Features responses • GeoJSON, GML, HTML templates • Templates are valid JSON/XML/XHTML documents, with expansion directives • Template Includes/inheritance supported • Can back-map user visible properties to data source ones (for efficient filtering) Community Marco Volpini GeoSolutions BRGM { "$merge": "items-base.json", "type": "FeatureCollection", "features": [ { "stac_version": "$${env('stac_version', '1.0.0-rc2')}", "stac_extensions": [ // skip SAR sensors "$filter{eop:parentIdentifier LIKE 'SENTINEL%' or eop:parentIdentifier LIKE 'LANDSAT%'}, https://stac-extensions.github.io/eo/v1.0.0/schema.json", "https://stac-extensions.github.io/view/v1.0.0/schema.json" ], "id": "${eop:identifier}", "type": "Feature", "geometry": "${footprint}", "bbox": [ "$${minx(footprint)}", "$${miny(footprint)}", "$${maxx(footprint)}", "$${maxy(footprint)}" ], "$include": "sarProperties.json", ....
50. STAC/OpenSearch EO Community module • OpenSearch for EO dusted off from 2017 implementation • GeoJSON for OpenSearch EO encoding added • STAC API beta2 implemented on top of same database (HTML/JSON encodings) • Database columns customizable (add your own, including arrays and JSON fields) • All encoding driven by “features templating”, fully customizable Collection Product/ Item Granule Community Andrea Aime GeoSolutions DLR
51. STAC/OpenSearch EO Community module • See “Serving earth observation data with GeoServer: COG, STAC, OpenSearch and more” for more info • Screenshots from DLR's EOC Geoservice (STAC support in development / Beta) Community Andrea Aime GeoSolutions DLR
52. Smart data loader community module • Complex features without the hassle of app-schema setup • Crawls a relational database structure from a root table • Generates an XSD schema and app-schema mappings for you. Instant complex features Community Marco Volpini GeoSolutions Fraunhofer IOSB
53. Schemaless community module • GeoTools/GeoServer features need a fixed target schema • The “schemaless” community module explores having no target schema • Source data should be in MongoDB • Won’t work with schema oriented formats (GML, shapefile) but does the job with GeoJSON • Dynamic filtering based on properties actually found on features Community Marco Volpini GeoSolutions Vianova
54. Official Docker Distribution • Official geoserver docker image: ◦ building from many in-the-wild examples ◦ Manage extensions ◦ Marlin ◦ Initial draft ready for review ◦ github.com/geoserver/docker/pull/1 • Contact Nils Bühner/Alessandro Parma if you are interested in helping with time or funding! Community Nils Bühner Alessandro Parma Community
55. Cloud Native GeoServer • GSIP 201 - Cloud Native GeoServer Project ◦ New community distribution for the cloud! ◦ github.com/geoserver/geoserver-cloud ◦ September 29th: Release 1.0-RC1 • Community experiment in microservices : ◦ building on prior clustering experience ◦ spring-boot microservices ◦ shared catalogue / config • Interested? ◦ Contact Gabriel Roldan to help with with time or funding! Cloud Native GeoServer Gabriel Roldan camptocamp
56. CITE Sprint (why you should care) Compliance Interoperability & Testing Evaluation
57. GeoServer progress towards recertification: • GeoServer 2.6.1 (Boundless 2012) • GeoServer 2.9.1 (Boundless 2016) • Phase I: CITE Test Automation RFP ◦ GSIP-176 Contact with GeoSolutions ◦ Test framework completed in 2021 • Phase II: Code Sprint to pass tests ◦ Fundraising at FOSS4G 2021 ◦ Sprint supported with cost recovery model (based on participation) • Phase III: Certification ◦ Provide test results to OGC ◦ OSGeo agreement covers certification Do you rely on GeoServer to share data: • Government contracts and standards often require that web services be certified to meet data provider obligations • Previously folks assumed GeoServer was certified by Boundless • We request and appreciate your support in addressing CITE compliance Solid value beyond “marketing” sticker: • Provide “black box” test confidence • A “second opinion” beyond our own QA About CITE Recertification Community OSGeo Community YOU ARE HERE
58. GeoServer CITE Sprint Financial support via OSGeo: directly, github sponsorship, or paypal donation. Sponsorship recognized on OSGeo and GeoServer website, and future “State of GeoServer” presentations. Prior to attending participants are asked to read the standard and be set up for development with the docker cite test containers for local debugging. Participation is encouraged a cost-recovery model and each-friday-in-november schedule for a healthy work/life/code balance. Sprint Sprint Sprint Sprint
59. Thanks!