Upgrade to Pro — share decks privately, control downloads, hide ads and more …

State of GeoServer - FOSS4G 2023 Edition

State of GeoServer - FOSS4G 2023 Edition

GeoServer is a web service for publishing your geospatial data using industry standards for vector, raster and mapping. Choose additional extensions to process data (either in batch or on the fly) and catalog records.

GeoServer is widely used by organizations throughout the world to manage, disseminate and analyze data at scale. GeoServer web services power a number of open source projects like GeoNode and geOrchestra.

This presentation provides an update on our community as well as reviews of the new and noteworthy features for the latest releases. In particular, we will showcase new features landed in 2.22 and 2.23, as well as a preview of what we have in store for 2.24 (to be released in September 2023).

Attend this talk for a cheerful update on what is happening with this popular OSGeo project, whether you are an expert user, a developer, or simply curious what GeoServer can do for you.

Simone Giannecchini

October 20, 2023
Tweet

More Decks by Simone Giannecchini

Other Decks in Technology

Transcript

  1. State of
    GeoServer
    2.23

    View full-size slide

  2. Jody Garnett
    [email protected]
    @[email protected]
    GeoCat
    Government Geographic Data publishing
    Andrea Aime
    Technical Lead
    [email protected]
    @geowolf
    GeoSolutions
    Innovative, robust, cost-effective
    solutions leveraging best-of-breed
    Open Source products.
    This presentation is brought to you by

    View full-size slide

  3. GeoServer at a Glance
    Java Web Application to share and
    edit geospatial data.
    Publish data from any major spatial
    data source using open standards.
    Core Protocols
    WMS – maps
    WFS – vector
    WFS-T – editing
    WCS – coverage
    WMTS – tiles
    TMS – tiles
    WMS-C – tiles
    Extension/community protocols
    WPS – process
    CSW – search
    OGC API - json + rest standards
    OGC STAC - spatio-temporal asset catalogue

    View full-size slide

  4. GeoServer Team Update
    2023 Update

    View full-size slide

  5. Core committers are nominated by their peers
    and trusted with the ability to approve
    pull-requests.
    We also support community commit access by
    request (often to work on a specific research
    and development topic):
    ● 23 committers
    Project Steering Committee serves to guide
    the project drawing from the various groups with
    a stake in the success of the project.
    ● Alessio Fabiani
    ● Andrea Aime
    ● Ian Turton
    ● Jody Garnett
    ● Jukka Rahkonen
    ● Kevin Smith
    ● Nuno Oliveira
    ● Simone Giannecchini
    ● Torben Barsballe
    GeoServer Team

    View full-size slide

  6. GeoServer maintains a list service providers:
    ● Core Contributors
    Ongoing commitment to the project
    devoting resources to security fixes,
    releases and maintenance activities.
    ● Experienced Providers
    Successfully contribute functionality to the
    project on behalf of their customers.
    ● Additional Services Providers
    Provide training, setup and integration
    support and assistance using GeoServer.
    (request: please take part in community)
    GeoServer Service Providers
    Policy change recognize participation:
    Core contributors directly contribute to project
    sustainability and are recognized for:
    ● core, committer, support
    Experienced providers participate in aspects of
    the project as community members:
    ● development, coding, documentation,
    outreach, translation, training, service,
    product

    View full-size slide

  7. ● Transition from Boundless → Completed!
    ○ geoserver.org (GitHub Pages)
    geoserver.org/blog
    ○ github.com/geoserver (GitHub)
    ○ blog.geoserver.org (github)
    ○ repo.osgeo.org (OSGeo)
    ○ domain names (OSGeo)
    ○ downloads (SourceForge)
    ○ email (SourceForge)
    ○ build.geoserver.org (GeoSolutions)
    ○ web map (GeoSolutions)
    ○ docs.geoserver.org (OSGeo)
    ● New
    ○ docker (OSGeo Nexus)
    GeoServer Infrastructure

    View full-size slide

  8. experiment
    Community modules
    GeoServer maintains a “community space” for
    experiments and new developers to work.
    ● Developers propose a community module
    idea and request commit access
    ● We make sure they compile!
    ● May be proposed as an extension when
    both the author and functionality is ready
    Spot a community module of interest, reach out
    to see how you can help.
    Graduating extension
    ● (not yet)
    Outgoing Community
    ● ows-simulate
    ● nsg-wmts
    ● wmts-styles
    ● teradata
    ● importer-fgdb
    ● saml
    ● wms-eo
    ● geostyler
    Incoming Community
    ● proxy-base
    ● webp-wms-output
    ● spatialJSON-wfs-output
    Downgraded extension
    ● xslt
    ● imagemap
    supported
    end-of-life un-supported

    View full-size slide

  9. GeoServer Releases
    2023 Update

    View full-size slide

  10. March 2023 September 2023 March 2024
    2.22.x
    2.23.x 2.23.x
    Releases covered by this presentation
    YOU
    ARE
    HERE
    2.24.x 2.24.x
    September 2024
    2.24.x
    Java 11 minimum!
    Last Java 8 release

    View full-size slide

  11. Are you using a older version? Upgrade!
    ● Easier to get answers. User list and stack exchange typically cover only
    supported versions
    ○ In September 2023 “supported” means 2.24.x and 2.23.x
    ○ In March 2024 “supported” will mean 2.24.x and 2.25.x
    ● Security fixes added to
    supported versions only
    ● Please upgrade your
    GeoServer installations!
    Photo by SpaceX on Unsplash

    View full-size slide

  12. Upgrade? What’s in it for me?
    ● Much!
    ● Check what’s new in 2.22, 2.23
    and incoming in 2.24
    ● Check the bottom of each slide to
    see who sponsored a certain
    feature, who implemented it, and
    what version contains it
    ● This icon marks activities done
    without any sponsoring
    Version
    Author
    Sponsor

    View full-size slide

  13. Distribution

    View full-size slide

  14. ● OGC API Code sprint activity
    ○ Goal to get feedback on the new ogcapi
    services under development
    ○ Use of osgeo docker nexus
    Docker for Nightly Builds (with community modules)
    ● Download from build server
    ○ 2.24-SNAPSHOT
    ○ community modules also
    2.24.x
    Jody Garnett

    docker run -it -p 80:8080 \
    --env INSTALL_EXTENSIONS=true \
    --env STABLE_EXTENSIONS="wps,css" \
    --env COMMUNITY_EXTENSIONS="ogcapi-features" \
    -t geoserver-docker.osgeo.org/geoserver:2.24.x
    docker pull \
    geoserver-docker.osgeo.org/geoserver:2.24.x

    View full-size slide

  15. Mapping
    Data making a scene!

    View full-size slide

  16. Improved mosaicking performance
    - Trying to mosaic several hundreds of
    images in the same output?
    - The index can have millions mind,
    we’re talking hundreds images to
    build a single GetMap output
    - Performance and memory usage for
    this use case have been improved
    by a couple of orders of magnitude
    (GeMap built with 300 images)
    2.22
    Andrea Aime
    GeoSolutions
    USGS

    View full-size slide

  17. Improved hyperspectral performance
    - Hyperspectral images: those having
    hundreds of bands
    - Typically stored in band-interleaved
    structure (one data bank for all pixels
    in the same tile)
    - GS is now orders of magnitude
    faster at reading them
    2.22
    Andrea Aime
    GeoSolutions
    DLR
    Rows
    Cols
    Bands

    View full-size slide

  18. Raster Attribute Table support
    - .aux.xml sidecar file
    associating pixel values with
    various classifications and
    colors
    - Generate SLDs out of them
    and associate with layers
    - Mimick QGIS
    RasterAttributeTable plugin
    - Development in progress,
    screenshot from QGIS!
    Community
    Andrea Aime
    GeoSolutions
    NOAA

    View full-size slide

  19. Data sources and formats
    News and Updates

    View full-size slide

  20. Feature Type Customisation : Description
    ● Previously in 2.21.x
    ○ Rename attributes
    ○ Change attribute order
    ○ Change attribute type
    ○ Generate attributes using expressions
    ● Now in 2.23.x
    ○ Description for human readable name
    2.23
    Joseph Miller
    (GeoSolutions)
    ?

    View full-size slide

  21. ● Supports efficient access to Cloud
    Optimized GeoTiff (COG)
    ● HTTP/S3/Azure/Google storage
    ● Minimizes reads on blob storage
    Cloud Optimized GeoTIFF community module
    Planet
    GeoSolutions
    Header Tile Tile Tile Tile Tile Tile
    Read 1 Read 2
    Josh Fix
    Daniele Romagnoli
    ● Contact GeoSolutions to sponsor:
    ○ More blob storage options
    ○ More authentication types support
    ○ Caching
    ○ Other improvements and fixes
    Community

    View full-size slide

  22. COG for Azure
    - Cloud Optimized GeoTIFF native
    support for Azure blob storage
    - Allows access to authenticated
    sources
    - Native access, improved
    performance
    - Community module!
    Community
    Daniele Romagnoli
    GeoSolutions
    TOTAL

    View full-size slide

  23. STAC Datastore and Mosaic
    ● Pull STAC items from a STAC API
    ● Render footprints on WMS
    ● Mosaic images on the fly
    ● Community module!
    Community
    Andrea Aime
    GeoSolutions
    DLR
    STAC
    API
    STAC
    datastore
    COGs Image mosaic
    World Settlements Footprints, STAC API + COG

    View full-size slide

  24. Vector mosaicking store
    ● Useful to handle thousands
    of files with similar structure
    ● Database used only to index
    them
    ● Much cheaper than storing
    everything in a database (on
    cloud)
    ● Not suitable if on the fly
    aggregations are needed
    ● Great if the end user
    application mostly pulls a
    few files (one?) at a time
    Community
    Joseph Miller
    GeoSolutions
    TROO
    Community
    module!

    View full-size slide

  25. WMS, WFS and WMTS
    cascading improvements
    ● Steady stream of fixes from
    Roar Brænden
    ● New contributor, 50+ tickets
    closed in the last year
    ○ Strong focus on WFS,
    WMS and WMTS stores
    ○ Feedback and
    improvement of internal
    “ResourceStore”
    ● Activity in GeoTools that
    positively reflects in
    GeoServer
    ● Live link
    2.24
    Roar Brænden
    Norwegian Institute for
    Water Research
    2.23

    View full-size slide

  26. Services
    News and Updates

    View full-size slide

  27. CSW-ISO module graduation
    ● The CSW-ISO module allows to generate ISO metadata records around GeoServer layer and
    services. This is a rare come back from community (used to be extension already)
    2.22
    @fileIdentifier.CharacterString=prefixedName
    identificationInfo.AbstractMD_Identification.citation.CI_Citation.title.CharacterString=title
    identificationInfo.AbstractMD_Identification.descriptiveKeywords.MD_Keywords.keyword.CharacterString=keywords
    identificationInfo.AbstractMD_Identification.abstract.CharacterString=abstract
    $dateStamp.Date= if_then_else ( isNull("metadata.date") , 'Unknown', "metadata.date")


    CoverageInfoImpl--4a9eec43:132d48aac79:-8000


    Unknown




    Niels Charlier
    Scitus
    -

    View full-size slide

  28. Metadata module graduation
    ● The metadata module adds an editor for extra metadata information (to be used in CSW-ISO)
    The editor structure can be customized using YAML configs
    2.22
    Niels Charlier
    Scitus
    -
    attributes:
    - key: metadata-identifier
    fieldType: UUID
    - key: metadata-datestamp
    label: Date
    fieldType: DATETIME
    - key: data-language
    fieldType: DROPDOWN
    values:
    - dut
    - eng
    - fre
    - ger
    - key: topic-category
    fieldType: SUGGESTBOX
    occurrence: REPEAT
    values:
    - farming
    - …

    View full-size slide

  29. ● New OGC Standards
    ○ Self describing OpenAPI
    ○ REST / JSON
    ○ Building on from W3C collaboration
    and STAC / WFS3 progress
    ● Contact Andrea Aime if you are interested
    in helping with development or funding!
    Check out dedicated presentation
    Demystifying OGC APIs with GeoServer:
    introduction and status of implementation
    OGC API community module
    Andrea Aime
    GeoSolutions
    GeoSolutions OGC
    GeoNovum
    API
    Common
    Maps
    API
    Coverages
    API
    Implementing (as part of TB14/tb15)
    Looking for volunteers/sponsors
    Process
    API
    Records
    API
    Features
    API
    Tiles
    API
    Styles
    API
    STAC
    API
    Community
    CITE
    compliant!

    View full-size slide

  30. Configuration and Setup

    View full-size slide

  31. New Welcome Page Layout
    GSIP-202 Welcome Page Layout
    - Available now in 2.22-M0 Milestone
    - Inspired by ogc-api community module
    - Use title and description to make a
    heading for each service
    - List capabilities and tools for each section
    - Change workspace
    - A welcome page for each workspace
    - Click to explore virtual web services
    - Change layer/group
    - A welcome page specific to a given
    layer/group
    - Layer specific services
    2.22
    Jody Garnett
    GeoCat
    GeoServer Enterprise
    GeoCat

    View full-size slide

  32. Configure Welcome page selectors
    ● Configure for use with large catalogues
    ○ GeoServerHomePage.selectionMode
    ■ dropdown - auto complete
    ■ text - simple text fields
    ■ auto - based on response time
    ○ GeoServerHomePage.selectionTimeout
    ○ GeoServerHomePage.selectionMaxItems
    When in ‘text’ mode summary does not try and count
    available layers
    2.23
    Andrea Aime
    GeoSolutions
    GeoSolutions

    View full-size slide

  33. Style format in styles page
    ● Playing around with multiple
    styling languages?
    ● SLD 1.0, SLD 1.1, CSS,
    YSLD, MBStyles, oh my
    ● The styles page now
    indicates the format for each
    style
    2.22
    Mohammad
    Mohiuddin Ahmed

    View full-size slide

  34. Reset of single store/layer via REST API
    ● Currently one can do a system wide
    “reset”
    ● Clears up feature type caches, connection
    pools and so on, from the whole
    GeoServer
    ● Add the ability to perform reset of
    caches/pools on a single store and on a
    single layer
    2.22
    Andrea Aime
    GeoSolutions
    GeoSolutions

    View full-size slide

  35. proxy-base-ext
    ● When proxy-base is not powerful enough….
    ● Allows deeper URL changes, e.g., expose
    different services on different host-names
    ● Can expand HTTP header placeholders from the
    proxy itself
    Community
    Joseph Miller
    GeoSolutions
    DLR
    Proxy
    (e.g. nginx)
    wms.myhost.com wfs.myhost.com
    GeoServer
    http://ip/geoserver/wms http://ip/geoserver/wfs
    Backlinks in responses with the original host and path
    (e..g, Capabilities)

    View full-size slide

  36. Space, the next frontier

    View full-size slide

  37. Getting off the EPSG assumption
    ● GeoServer 2.24 will be able to handle
    more CRS authorities
    ● In particular, IAU is being added, which
    covers planetary CRSs
    ● The work opens the possibility to have
    more authorities as well (e.g., ESRI, IGNF,
    NKG, and more)
    2.24
    Andrea Aime
    GeoSolutions
    USGS

    View full-size slide

  38. Mars, original dataset and north polar reprojection
    2.24
    Andrea Aime
    GeoSolutions
    USGS

    View full-size slide

  39. Work ongoing
    2.24
    Andrea Aime
    GeoSolutions
    USGS
    ● Can be tested right now on the developer
    nightly build
    ● It’s still far from complete!
    ○ Capabilities generators
    ○ WMS/WFS/WCS/WPS
    ○ Retain CRS in GIS output formats
    ○ PostGIS/GeoPackage
    ● Aiming at work completion in time for the
    2.24.0 release

    View full-size slide

  40. Security Vulnerabilities

    View full-size slide

  41. ● Keep exploit details out of issue report
    ● Mark the issue as a vulnerability.
    ● Be prepared to work with Project Steering Committee (PSC) on a solution
    ● Keep in mind PSC members are volunteers and an extensive fix may require
    fundraising / resources
    Report via [email protected] or github private vulnerability
    reporting. If you are not in position to communicate in public please consider
    commercial support, contacting a PSC member, or reaching us via the Open
    Source Geospatial Foundation at [email protected].
    In case you stumble into a vulnerability:
    Responsible Disclosure

    View full-size slide

  42. ● OGC Filter Injection
    ○ CVE-2023-25157 (GeoServer)
    ○ CVE-2023-25158 (GeoTools)
    ○ Archived releases patched on behalf of
    customers and projects not in a position to
    upgrade
    ● Ongoing management of dependencies
    ○ Vulnerability not often verified, as these
    reports can be based on automated scan
    ● New: GitHub vulnerability reporting
    ○ Secure communication
    ○ Automate CVE assignment
    Security Vulnerabilities
    2.23
    GeoServer PSC
    🔥🔥🔥

    View full-size slide

  43. Control remote HTTP requests (GSIP-218)
    One of the longest outstanding vulnerability
    requests has been to limit the HTTP requests
    made by GeoServer using user provided
    locations.
    However some protocols require access access
    to external web resources …
    ● Config: Check remote location
    ● WMS dynamic SLD
    ● WMS feature portrayal of remote WFS
    ● WPS remote inputs
    ● More? Funding/volunteer required!

    View full-size slide

  44. ● H2 is an embedded database we started
    using many years ago
    ● Convenient, ended up powering some
    core functionality:
    ○ GWC disk quota (by default)
    ○ KML super-overlays index
    ● And some non core functionality
    ○ Default DB for GeoFence, JDBCConfig
    (can use external db), WPS JDBC
    ○ Index DB for NetCDF/Grib files
    ● Upgrade to H2 version 2 is hard, different
    binary format
    ● Couple of CVEs against v1, we did not find
    a way to use them
    H2 version 1 removal
    2.23
    GeoSolutions
    ● In GeoServer 2.23.2, removing core usage
    ● Replaced by HSLQDB
    ● Usages in optional plugins will be removed
    in time (are you interested?)
    linz.gov.nz

    View full-size slide

  45. But what about CVE-2023-35042?
    This is a duplicate of the “Jiffle” vulnerability
    CVE-2022-24816 patched last year.
    Researched just saw automated attacks come in
    and stood up an older unpatched GeoServer to
    see if something would happen.
    Not sure why it got a new CVE number so we
    have asked :)
    Update: Now marked as “disputed”
    (which is not quite right)

    View full-size slide

  46. Community Building

    View full-size slide

  47. Participation required for Sustainable Open Source
    Thanks to new release volunteers, very much appreciated:
    ● Gabriel Roldan (camptocamp)
    ● Peter Smythe
    GeoServer increasingly relies on a small development team:
    ● Big thanks to our core contributors for making this project happen
    ● Experiment: with sponsorship and small contracts?
    ○ Tried with Log4j - amount of overhead in fundraising not worth the time commitment
    ● Experiment: “cost recovery” code-sprint model?
    ○ Trying with “remove opengis” sprint - helping cover costs for participants

    View full-size slide

  48. ● The community really does not really have a “road
    map”
    ● The core developers are mostly employed in
    companies providing commercial services for
    GeoServer, or using it in some hosting solution →
    customer driven (no other significant source of
    funding)
    ● The other developers pop up occasionally to
    provide improvements, fixes and new features
    they need
    Roadmap - whatever else you want to push for
    Community

    View full-size slide