In this presentation, our GeoServer Lead Andrea Aime walks us through the key points of the GeoServer release 2.22 covering also what to expect for the next release.
Jody Garnett [email protected] @jodygarnett GeoCat Government Geographic Data publishing Andrea Aime Technical Lead [email protected] @geowolf GeoSolutions Innovative, robust, cost-effective solutions leveraging best-of-breed Open Source products. This presentation is brought to you by
GeoServer at a Glance Java Web Application to share and edit geospatial data. Publish data from any major spatial data source using open standards. Core Protocols WMS – maps WFS – vector WFS-T – editing WCS – coverage WMTS – tiles TMS – tiles WMS-C – tiles Extension/community protocols WPS – process CSW – search OpenSearch for Earth Observation – search OGC API - prototype json + rest standards
Core committers are nominated by their peers and trusted with the ability to approve pull-requests. We also support community commit access by request (often to work on a specific research and development topic): ● 22 committers New committer this week: Joseph Miller! Project Steering Committee serves to guide the project drawing from the various groups with a stake in the success of the project. ● Alessio Fabiani ● Andrea Aime ● Ian Turton ● Jody Garnett ● Jukka Rahkonen ● Kevin Smith ● Nuno Oliveira ● Simone Giannecchini ● Torben Barsballe GeoServer Team
GeoServer maintains a list service providers: ● Core Contributors Ongoing commitment to the project devoting resources to security fixes, releases and maintenance activities. ● Experienced Providers Successfully contribute functionality to the project on behalf of their customers. ● Additional Services Providers Provide training, setup and integration support and assistance using GeoServer. (request: please take part in community) GeoServer Service Providers Policy change recognize participation: Core contributors directly contribute to project sustainability and are recognized for: ● core, committer, support Experienced providers participate in aspects of the project as community members: ● development, coding, documentation, outreach, translation, training, service, product
Incoming: ● gt-json-core and gt-json-store ● gwc-mbtiles ● LibDeflate ● VSI (Virtual File system GDAL reader) Outgoing: ● gt-geojson (soon) ● epsg-oracle ● Ingres datastore ● geogig ● importer-bdb ● nsg-wfs-profiles ● qos ● wfs3 Community modules GeoServer maintains a “community space” for experiments and new developers to work. ● Developers propose a community module idea and request commit access ● We make sure they compile! ● May be proposed as an extension when both the author and functionality is ready Spot a community module of interest, reach out to see how you can help. Graduating ● csw-iso and metadata ● kml-ppio Downgraded ● Teradata store
stable maintenance development May 2022 November 2022 March 2023 2.21.x 2.22.x Releases covered by this presentation 2.23.x September 2023 Java 11 minimum! YOU ARE HERE
Are you using a older version? Upgrade! ● User list and stack exchange typically covers only supported versions ○ In September 2022 “supported” means 2.21.x and 2.22.x ○ In March 2023 “supported” will mean 2.22.x and 2.23.x ● Security fixes added to supported versions only ● Please upgrade your GeoServer installations! Photo by SpaceX on Unsplash
Upgrade? What’s in it for me? ● Much! ● Check what’s new in 2.21, 2.22 and incoming in 2.23 ● Check the bottom of each slide to see who sponsored a certain feature, who implemented it, and what version contains it ● This icon marks activities done without any sponsoring Version Author Sponsor ♥
Official Docker Image ● Breakout session after FOSS4G ○ GSIP 192 - Official Docker Image ○ https://github.com/geoserver/docker ● Proposal 1: Luca Pasquali ○ custom_build.sh ○ option to use local geoserver.war ○ download and install extensions ○ option to include local data dir ● Proposal 2: Nils Bühner ○ enable / disable CORS ○ Install extensions from external folder ○ Install libs from external folder ● Bolsena 2022 ○ Reboot at Bolsena code sprint ○ Combine both proposal to some extent ○ First official release with 2.21.1 2.21.1 Nils Bühner Luca Pasquali ♥ $ docker pull docker.osgeo.org/geoserver:2.21.1 $ docker run -it -p8080:8080 \ docker.osgeo.org/geoserver:2.21.1
Windows Installer ● Last released for GeoServer 2.14.3 ○ “Lost” in infrastructure vandalism ● Manual process has been in dev guide (for organizations requiring installer) ● GSIP-198: small contract with GeoCat ○ New NSIS installer ○ New Service wrapper ○ Automation for build.geoserver.org 2.21 Sander Schaminee GeoCat GeoServer OSGeo
Layer group styles ● A layer group style is a style name offering a different list of layers and styles to build the group ● Use cases: ○ Different representation of same layers ○ Simplified representation of same map (less layers) ● Example here: Ordnance Survey MasterMap (6 layers) in 4 alternative styles 2.21 Marco Volpini GeoSolutions SWAM
Symbology providers control ● Point Vector symbology in SLD comes from “mark factories” ● Control order of execution and availability in styles ● For complex point maps, big speed improvements by putting the most used factories first 2.21 Fernando Mino EMSA Above: EMSA, real time tracking of 150k ships in the EU area
Improved mosaicking performance - Trying to mosaic several hundreds of images in the same output? - The index can have millions mind, we’re talking hundreds images to build a single GetMap output - Performance and memory usage for this use case have been improved by a couple of orders of magnitude (GetMap built with 300 images) 2.22 Andrea Aime GeoSolutions USGS
Improved hyperspectral performance - Hyperspectral images: those having hundreds of bands - Typically stored in band-interleaved structure (one data bank for all pixels in the same tile) - GS is now orders of magnitude faster at reading them 2.22 Andrea Aime GeoSolutions DLR Rows Cols Bands
New Natural Earth 1:50m demo layers - Simple map based on 1:50m data sets - Included in the built-in data directory 2.22/various Jody/Andrea GeoCat/GeoSolutions - For full layers support up to 1:10m, check this out: https://github.com/geosolutions-it/ne-styles
Feature Type Customisation ● Rename attributes ● Change attribute order ● Change attribute type ● Generate attributes using expressions 2.21 Andrea Aime (GeoSolutions) Canton of Solothurn
● GeoServer now works and is tested with ○ GDAL 3.x Series ○ GDAL 3.2 and 3.4 tested ○ (GDAL 3.5 build was not yet stable) ● Primarily update to ImageIO-EXT GDAL 3.x Support 2.21 David Blasby GeoCat Rijkswaterstaat
COG for Google Cloud Storage - Cloud Optimized GeoTIFF native support for Google Cloud Storage - Allows access to authenticated sources - Native access, improved performance 2.21 Andrea Aime GeoSolutions Farmers Edge
libdeflate support for GeoTIFF - libdeflate is a “Heavily optimized library for DEFLATE/zlib/gzip compression and decompression” - Currently used to decompress/compress GeoTIFF files - Community module adds out of the box support for Linux (for other OS, add native lib) - PNG encoding support in the roadmap 2.21 Andrea Aime GeoSolutions DLR Zlib Implementation CPU Time Original 1m 17s Intel 1m 15s CloudFlare 1m 4s Libdeflate 0m 43s http://www.htslib.org/benchmarks/zlib.html
STAC Datastore and Mosaic ● Pull STAC items from a STAC API ● Render footprints on WMS ● Mosaic images on the fly ● Community module! 2.22 Andrea Aime GeoSolutions DLR STAC API STAC datastore COGs Image mosaic World Settlements Footprints, STAC API + COG
Vector mosaicking store ● Useful to handle thousands of files with similar structure ● Database used only to index them ● Much cheaper than storing everything in a database (on cloud) ● Not suitable if on the fly aggregations are needed ● Great if the end user application mostly pulls a few files (one?) at a time 2.22 Joseph Miller GeoSolutions TROO Community module!
GeoPackage output extension (for WMS and WFS) GeoPackage output for: ● WFS ● WMS - produces a rendered tileset Aside: ● WPS download process can produce very simple geopackage output similar to WMS and WFS above ● geopackage community module still a work-in-progress supporting experimental geopkg extensions like generalization 2.21 David Blasby GeoCat Rijkswaterstaat wfs?version=2.0.0&request=GetFeature& typeNames=topp:states&outputFormat=geopkg wms?version=1.1.0&request=GetMap&layers=...& styles=&bbox=...&srs=EPSG:4326&& width=1536&height=984&format=geopkg
CSW-ISO module graduation ● The CSW-ISO module allows to generate ISO metadata records around GeoServer layer and services. This is a rare come back from community (used to be extension already) 2.22 @fileIdentifier.CharacterString=prefixedName identificationInfo.AbstractMD_Identification.citation.CI_Citation.title.CharacterString=title identificationInfo.AbstractMD_Identification.descriptiveKeywords.MD_Keywords.keyword.CharacterString=keywords identificationInfo.AbstractMD_Identification.abstract.CharacterString=abstract $dateStamp.Date= if_then_else ( isNull("metadata.date") , 'Unknown', "metadata.date")
Metadata module graduation ● The metadata module adds an editor for extra metadata information (to be used in CSW-ISO) The editor structure can be customized using YAML configs 2.22 Niels Charlier Scitus - attributes: - key: metadata-identifier fieldType: UUID - key: metadata-datestamp label: Date fieldType: DATETIME - key: data-language fieldType: DROPDOWN values: - dut - eng - fre - ger - key: topic-category fieldType: SUGGESTBOX occurrence: REPEAT values: - farming - …
KML output for WPS ● The KML-PPIO (Process-Parameter-IO) module graduated, part of the WPS package now ● Adds KML output to every process generating a feature collection ● Better alignment for WFS vs WPS Download Process 2.21 Alessio Fabiani GeoSolutions GeoSolutions
MBTiles as GWC layer ● Use a pre-seeded MBTiles file as a GWC layer ● Serve the contents over all GWC tile protocols ● Core in GWC, community module in GS 2.20 Daniele Romagnoli GeoSolutions DLR MBTiles
Descriptions in DescribeFeatureType ● If a database has column described with comments, they can be advertised in the WFS XML schema returned by DescribeFeatureType ● They will be soon also editable from GUI as well (for all vector data sources) ● PR in review today! 2.22.3 Joseph Miller GeoSolutions GeoScience Australia
New Welcome Page Layout GSIP-202 Welcome Page Layout - Available now in 2.22-M0 Milestone - Inspired by ogc-api community module - Use title and description to make a heading for each service - List capabilities and tools for each section - Change workspace - A welcome page for each workspace - Click to explore virtual web services - Change layer/group - A welcome page specific to a given layer/group - Layer specific services 2.22 Jody Garnett GeoCat GeoServer Enterprise GeoCat
Speed-up for many layers/many rules servers ● The new home-page was very slow on some server ● Reference server, 20k layers, 40k security rules! ● Made GUI pay more attention to large setups, fall back on simple text boxes ● Sped up security filtering in home page, capabilities, preview page by 2 orders of magnitude (yes, 200 times faster) 2.22 Andrea Aime GeoSolutioons Undisclosed customer Graceful degradation
Language Chooser and INSPIRE language support ● Language chooser added ● Internationalization updated (thanks to Alexandre Gacon and others) 2.21 Andrea Aime ♥
Language specific service/layer metadata ● Abstract, title ● Contact information ● Service descriptions ● In multiple languages ● Support INSPIRE “LANGUAGE” request parameter (e.g. GetMap with language specific labels) ● Drive styles using the “language” function 2.20 Marco Volpini Provincia di Bolzano
Style format in styles page ● Playing around with multiple styling languages? ● SLD 1.0, SLD 1.1, CSS, YSLD, MBStyles, oh my ● The styles page now indicates the format for each style 2.22 Mohammad Mohiuddin Ahmed
Server Status Cleanup ● Status tab now uses units ○ GB, threads, milliseconds, … ● Resource cache count ○ Indicates resources in “ObjectPool” cache ○ Provides feedback for Clear button ● Documentation updated ○ Clarified what the “clear” button actually clears (e.g. use clear to reload user projections) 2.21.1 Jody Garnett GeoCat Live
Server Status JVM Console ● User interface access and download ○ Java memory use ○ Java thread use ● Troubleshooting docs updated 2.21.1 Nuno Oliveira GeoSolutioons
Reset of single store/layer via REST API ● Currently one can do a system wide “reset” ● Clears up feature type caches, connection pools and so on, from the whole GeoServer ● Add the ability to perform reset of caches/pools on a single store and on a single layer 2.22 Andrea Aime GeoSolutions GeoSolutions
● Keep exploit details out of issue report ● Mark the issue as a vulnerability. ● Be prepared to work with Project Steering Committee (PSC) on a solution ● Keep in mind PSC members are volunteers and an extensive fix may require fundraising / resources If you are not in position to communicate in public please consider commercial support, contacting a PSC member, or reaching us via the Open Source Geospatial Foundation at [email protected]. In case you stumble into a vulnerability: Responsible Disclosure
● Log4Shell ○ Published immediate assessment (that GeoServer was not vulnerable) ○ Released GeoServer w/ patched log4k 1.2 (with no JMS Appender) ○ Instructions for patching prior versions ○ Range of community response from demands to offering funding ● Spring4Shell ○ Could not find exploit ○ Mitigation: Java 8 ○ GeoServer Release with Spring update ● RCE in Jiffle ○ CVE-2022-24847 ← our first ever CVE! ■ Mitigation: janino.jar Security Vulnerabilities ● RCE in JNDI lookups ○ Similar to Log4Shell ○ Made safe lookup methods in GeoTools ● Log4J 2 Upgrade ○ Did a small fundraising activity ○ Public proposal process ○ Funds raised (and effort to raise funds) not sufficient for these activities ○ GeoServer 2.21 Log4j 2 Update ● 2023 SQL injection ○ Through filters. Been there since 2005, pretty hard to spot. ○ No mitigation, upgrade! ● Website: tag “security vulnerability” posts GeoServer PSC 🔥🔥🔥 Various versions
Developer Participation Concerns: ● Project care and feeding relies on a low number of people Actions: ● Reach out to “downstream projects” to test release candidates ● Revamp service providers page to highlight participation and experience (encourage service providers to participate) ● Exploring small contracts, and “cost recovery” code-sprint model
● New OGC Standards ○ Self describing OpenAPI ○ REST / JSON ○ Building on from W3C collaboration and STAC / WFS3 progress ● OGC API - Features moving towards CITE/INSPIRE compliance ● Contact Andrea Aime if you are interested in helping with development or funding! OGC API community module Andrea Aime GeoSolutions GeoSolutions/OGC API Common Maps API Coverages API Implementing (as part of TB14/tb15) Looking for volunteers/sponsors Process API Records API Features API Tiles API Styles API STAC API Community
OGC API - Features ● Has been around for a while as a community module ● Core specification (space/time filtering) and CRS support finalized ● Advanced filtering (CQL2) still in progress ● Fully customizable appearance ● Moving towards graduation of the core portion 2.23 Andrea Aime GeoSolutions Geonovum Various others
GeoPackage community module improvements ● core: geopackage a data format ● geopkg output extension: WFS and WMS ● geopkg community module: WPS ● Reading grids other than png/jpeg tiles: TBD ● Extending GeoPackage - yes you can ○ Include in-line linked metadata ○ Include styles ○ Included generalized tables (store pre-simplified geometries) ● Contact Andrea Aime if you are interested in helping with time or funding! Community Andrea Aime GeoSolutions GeoSolutions/OGC
Webp output format ● Output format for WMS ● Well supported across modern browsers ● Compared to PNG ○ Up to 2 times smaller file ○ Up to 3 times slower to produce on topp:states (this can be improved) ● The actual speed difference depends on the map production time (less noticeable on more complex maps with rich symbology) ● For now, best suited for tile caches Community OpenDEM ♥
● The community really does not really have a “road map” ● The core developers are mostly employed in companies providing commercial services for GeoServer, or using it in some hosting solution → customer driven (no other significant source of funding) ● Other developers pop up occasionally to provide improvements, fixes and new features they need Whatever else you want to push for Community
simboss
sansantech
kei_s
h1kita
bun913
orimanabu
baseballyama
oldbigbuddha
yokawasa
rilmayer
route06
oracle4engineer
kzykmyzw
mza
zakiyama
erikaheidi
frogandcode
morganepeng
malarkey
jonrohan
kastner
jensimmons
eileencodes
lara
jacobian