State of GeoServer - FOSS4G 2018

State of GeoServer provides an update on our community and reviews the new and noteworthy features for the Project. The community keeps an aggressive six month release cycle.

Each releases bring together exciting new features. View this presentation for a cheerful update on what is happening with this popular OSGeo project. Whether you are an expert user, a developer, or simply curious what these projects can do for you, this talk is for you.

Transcript

1. State of
   GeoServer
   FOSS4G 2018 edition!
   

   View Slide

2. Andrea Aime
   Technical Lead
   andrea.aime@geo-solutions.it
   @geowolf
   GeoSolutions
   Innovative, robust and cost-effective
   solutions leveraging best-of-breed
   Open Source products.
   This presentation is brought to you by
   Jody Garnett
   Community Lead
   jgarnett@boundlessgeo.com
   @jodygarnett
   Boundless
   Provides geospatial tools and
   services for managing data and
   building applications.
   Ian Turon
   Open Source Evangelist
   ian@ianturton.com
   @ijturton
   Astun technologies
   Passionate about geography and
   passionate about open (source,
   standards and data).
   

   View Slide

3. GeoServer at a Glance
   Java Web Application to share and
   edit geospatial data.
   Publish data from any major spatial
   data source using open standards.
   Core Protocols
   WMS – maps
   WFS – vector
   WFS-T – editing
   WCS – coverage
   WMTS – tiles
   TMS – tiles
   WMS-C – tiles
   Extension/community protocols
   WPS – process
   CSW – search
   OpenSearch for Earth Observation – search
   

   View Slide

4. GeoServer 2018
   What’s in a year?
   

   View Slide

5. stable maintenance
   development
   September 2017 March 2018 September 2018 March 2019
   2.12.x
   2.13.x
   2.14.x
   Releases covered by this presentation
   

   View Slide

6. Are you using a older version? Upgrade!
   ● User list and stack exchange typically covers only supported versions
   ○ Today it means 2.13.x and 2.12.x
   ○ In September it will mean 2.14.x and 2.13.x
   ● Security fixes being added to
   supported versions only
   ● Please upgrade your GeoServer
   installations!
   

   View Slide

7. Upgrade? What’s in it for me?
   ● Much!
   ● Let’s check what’s new in 2.12,
   2.13 and in the upcoming 2.14
   releases
   ● Check the bottom of each slide to
   see who sponsored a certain
   feature, who implemented it, and
   what version contains it
   Version
   Author
   Sponsor
   

   View Slide

8. Vector data sources
   Precisely placed and attributed!
   

   View Slide

9. GeoPackage graduated from community module to “core”:
   ● Provides an sqlite database containing vector or raster data
   ○ Initial idea from spatialite project
   ○ Now an OGC Standard to replace shapefile and personal file geodatabase
   ● GeoPackage improvements:
   ○ Bug fix - raster mosaic geopackage had y-axis inverted.
   ○ Support for spatial indexes an RnD topic
   ● Additional GeoPackage community modules
   ○ GeoPackage WMS output available as a community module
   ○ GeoPackage WFS output format
   ○ GeoPackage WPS output format
   GeoPackage support built-in!
   2.12
   GeoSolutions
   Boundless
   Community
   Boundless Suite
   

   View Slide

10. GeoPackage performance improvements
    GeoPackage reading and rendering:
    ● Up to two times faster
    on large datasets
    ● Up to 50% faster as you zoom in
    GeoPackage performance is
    now on par with PostGIS
    (Shapefile remains fastest / most optimized)
    2.13
    Andrea Aime
    GeoSolutions
    Community
    

    View Slide

11. MongoDB
    Graduated to extension status!
    ● Makes use of the supported GeoTools
    MongoDB DataStore
    ● Publish information stored in the MongoDB
    Document Database, via a mapping configuration
    2.14
    Nuno Oliveira
    GeoSolutions
    NPRA
    

    View Slide

12. Support for more PostgreSQL data types
    ● Basic HStore and JSON data types support.
    Both render as a JSON formatted string field in
    WFS output (2.13.x), read only, no filtering
    support
    ● Array support in progress (2.14.x), read, write,
    limited filtering
    ● Geometry XYZM support (2.14.x)
    2.13 - 2.14
    Daniele Romagnoli - Andrea Aime
    GeoSolutions
    MDA / DLR
    

    View Slide

13. Raster data sources
    Pixels, pixels, billions of them!
    

    View Slide

14. ● Initial version in 2.11
    ● Several fixes in 2.12, e.g.
    accuracy, performance,
    dateline crossing handling
    ● Being tested against global
    mosaics of Sentinel2 (each
    image in a different UTM
    zone), more
    improvements/fixes incoming
    Heterogeneous CRS image mosaics
    2.11/2.12
    Boundless/GeoSolutions
    RnD/DLR
    UTM60 image footprints UTM1 image footprints
    Dateline wrapped
    image mosaic
    

    View Slide

15. Coverage views from heterogeneous bands
    2.13
    Source data
    (e.g. Sentinel 2)
    A new feature in 2.13 is the ability to define a coverage view using mix bands from
    seperate files, at different resolutions, into a single multiband coverage view.
    Daniele Romagnoli - Andrea Aime
    GeoSolutions
    DLR
    Virtual multiband coverage Select 3 bands
    and contrast stretch
    

    View Slide

16. S3 GeoTIFF Community Module + Mosaic
    Support for GeoTiffs hosted on Amazon S3
    ● In 2.14.x, can be coupled with image mosaic to support mosaics of AWS
    images (experimental!)
    2.12 - 2.14
    Devon / Niels
    Boundless
    Boundless
    s3://landsat-pds/L8/001/002/LC80010022016230LGN00/LC8001
    0022016230LGN00_B1.TIF?useAnon=true&awsRegion=US_WEST_2
    

    View Slide

17. Web Map Service
    Got maps?
    

    View Slide

18. Nearest match in time dimension
    ● Time dimension now supports “nearest match”
    ● Map will return the closest time to the one selected
    ● Possibility to set a search “radius”
    ● The actual time used is returned
    in a HTTP response header
    2.14
    Andrea Aime
    GeoSolutions
    EUMETSAT
    

    View Slide

19. WMS sortBy vendor param
    ● Sort by recency, cloud cover, and so on
    ● Supported by vector data and image mosaic
    2.12
    Andrea Aime
    GeoSolutions
    DLR
    Mosaic sorted by recency
    Mosaic sorted by cloud cover
    

    View Slide

20. Better centroid / label placement in “vector” outputs
    KML
    ● generate placemarks that are
    always inside polygon
    (for concave polygons)
    Vector Tiles
    ● Pregenerate label points in
    tiles
    ○ Reduce effort on client
    2.12
    Kevin Smith
    Justin Deoliveira
    RnD
    Label Label
    vs
    

    View Slide

21. Web Feature Service
    To the source of vector data!
    

    View Slide

22. Almost WFS 2.0 certified compliant
    ● Lot of work went into making GeoServer pass the
    WFS 2.0 certification on simple features
    ● 48 issues fixed
    ● Almost there, still a couple of minor fixes to do (but
    so many other urgent things to work on…)
    2.12 - 2.13 - 2.14
    Andrea Aime
    GeoSolutions
    OGC and GeoSolutions
    

    View Slide

23. 
    srsDimension="4"
    gml:id="postgres_lines_zm.1.geometry">
    
    1 -2 10 0 1 3 15 1 5 3 20 2 5 -2 25 3
    
    
    Nuno / GeoSolutions
    Jody / Boundless
    Encode coordinates measures
    - Enable to encode coordinate
    measures (“M”) in WFS output
    formats that support measures.
    - PostGIS is supported today!
    WFS Measure Support
    Community 2.14
    

    View Slide

24. App-schema improvements
    ● Improvements to connection pool usage (only one connection used
    per request now)
    ● Some improvements helping while using MongoDB as a source for
    app-schema records
    ● Hale mapping support being extended to support MongoDB
    ● Support for SOLR as a data source for MongoDB, as well as
    secondary index for full text search engines
    2.12 - 2.13 - 2.14
    Nuno Oliveira
    GeoSolutions
    NPRA
    

    View Slide

25. Web Coverage Service
    To the source of raster data, too!
    

    View Slide

26. CQL_FILTER and SORT_BY on mosaics
    ● Sort by recency, cloud cover,
    and so on
    ● Filter on the same
    ● Maps to image mosaic read
    parameters
    ● Uses attributes in the mosaic
    index
    ● Consistent with WMS now,
    first see and then download
    2.12
    Andrea Aime
    GeoSolutions
    DLR
    

    View Slide

27. NetCDF input/output improvements
    2.12
    Niels (Scitus)
    Ben (Transient)
    Community
    ● Can configure NCML files as sources
    ● Support variable attribute definition in
    NetCDF output
    ● Copy scalars from NetCDF/GRIB source
    ● Copy scalars from source granules to
    make vector in NetCDF output
    

    View Slide

28. GHRSST Community Module
    ● WCS output format plugin
    ● Builds a NetCDF file complaint with the
    “Group for High Resolution Sea Surface
    Temperature” specification
    ● It’s the first multi-coverage output, e.g.
    ○ Sea surface temperature
    ○ Sea ice fraction
    ○ Quality level
    ○ Pixels per bin
    ○ ...
    2.13
    Andrea Aime
    GeoSolutions
    EUMETSAT
    

    View Slide

29. Tiles
    My alphabet: X,Y and Z
    

    View Slide

30. ● GeoServer supports “cascading”
    remote WMTS services
    ○ GeoServer acts as a client, rather than
    just rerouting each request
    ○ Same as WMS cascading
    WMTS cascading
    2.12
    Ian Turton - Emanuele Tajariol
    GeoSolutions
    GeoInfo
    

    View Slide

31. Parameter filter cache drop fixes/improvements
    ● GeoWebCache can cache based on request
    parameters like style, time, or elevation.
    ○ Could not truncate (remove cached tiles) across all
    parameter values, each must be known and truncated
    separately.
    ○ Now truncates all possible values
    ● Added operations for
    ○ Cross parameter truncate
    ○ Orphan purge
    ● Enabled cross parameter truncate on data update
    GWC
    1.12
    GS
    2.12
    Kevin Smith
    Boundless
    MapStory
    

    View Slide

32. Manage and configure “blob store” used to hold cached tiles,
    and “gridsets” to define tile grids associated with a spatial reference system.
    GeoWebCache REST API - new endpoints!
    2.13
    Boundless
    Torben Barsballe
    Boundless
    

    View Slide

33. Styling
    Your cartographic tailor
    

    View Slide

34. Labelling improvements
    ● Kerning on by default
    (adjusts spacing between chars, e.g.
    Check space between T and r in Trimble)
    ● Underline and Strikethrough
    ● Char and word spacing control
    ● Labelling border of polygon
    (instead of the centroid)
    2.11
    2.12
    Andrea Aime
    Nuno Oliveria
    Community/Geoinfo/
    OpenGeoGroep
    

    View Slide

35. Determining a good location for a label can be tricky.
    Use of maxDisplacement indicates how far a label is allowed to move
    ● Previously we searched in a circular area for a good location
    ● Now you can supply your preference
    using displacementMode
    Labelling improvements
    2.13
    Nikolaos
    Pringouris
    

    View Slide

36. ● Allow using SVG as
    shapes for marks
    ● Filling and stroking
    can be controlled
    from SLD
    SVGs as fillable/strokable marks
    2.14
    Andrea Aime
    GeoSolutions
    Community
    
    
    
    file://firestation.svg
    
    #FF0000
    
    
    32
    
    
    

    View Slide

37. ● JSON representation of styling
    ○ More limited than SLD/YSLD/CSS
    ● Works well with vector tiles
    ○ Cross platform styling (OpenLayers, MapBoxGL)
    ● Work in progress:
    ○ Currently a GeoTools community module
    ○ Should package and go to extension if we get time!
    MapBox Style Community Module
    2.12
    David Vick
    Boundless
    Boundless Suite
    MapBox Style in GeoServer and
    OpenLayers
    

    View Slide

38. ● Usage of units for more
    readable scale dependencies
    ● LessCSS functions to alter
    colors
    ● Convenient syntax to use env
    variables, @varName
    ● Much faster CSS -> SLD
    translation
    CSS improvements
    2.12
    Andrea Aime
    Community
    [class = 'major'][@sd < 25M] {
    [@scale > 3M] {
    mark: url('./symbols/place/place-6.svg');
    mark-mime: 'image/svg+xml';
    label-anchor: 0.5 0;
    label-offset: 0 2
    };
    label: [name];
    font-size: [interpolate(@sd, 400k, 15, 25M, 12)];
    font-color: lighten(red, 10%);
    }
    

    View Slide

39. ● ChannelSelection allows for env
    variables now
    ● Build pseudo-color styles with a
    default band layout
    ● Change the selected bands in
    GetMap request
    ● No need to prepare tens of
    different styles
    ● What if you want to do band math?
    You wait just 1 minute!
    Dynamic channel selection
    2.14
    Andrea Aime
    GeoSolutions
    DLR
    ...&env=r:4;g:3,b:2 ...&env=r:8;g:11,b:4
    Pseudo color Vegetation analysis
    

    View Slide

40. Web Processing Service
    When the going gets tough the processes get going
    

    View Slide

41. Unleash map algebra with Jiffle
    ● Powerful and fast map algebra language
    ● Can be used as a rendering transformation
    ● Example, computing NDVI on the fly:
    2.14
    Andrea Aime
    GeoSolutions
    Bayer
    nir = src[7];
    vir = src[3];
    dest = (nir-vir)/(nir+vir);
    Sentinel 2 Jiffle script On the fly NDVI index display
    

    View Slide

42. WPS Download (Community module)
    ● Download large raw rasters
    ● Download large raw vector
    ● Download large static maps with
    remote maps inclusion
    ● Build time based animations
    ● Allows usage of async calls for
    expensive map rendering and
    long animation building
    2.13
    GeoSolutions
    EUMETSAT
    

    View Slide

43. CSV input/output for WPS
    ● Allows the input and output of CSV for use in WPS
    processes
    ● This will lead to GeoTools CSV Datastore being
    upgraded to community (and hopefully to supported)
    2.12
    Ian Turton
    Astun
    RnD
    

    View Slide

44. Configuration and management
    

    View Slide

45. User Interface - Autocompletion
    Entering in URLs for data files has been improved with autocomplete
    ● Entering URLs for data files,
    will scan the path suggest
    existing files
    ● Long lists of stores or layers
    also feature autocomplete
    2.13.0
    Andrea Aime
    Community
    

    View Slide

46. User Interface - Raster
    Parameters
    Filling in raster layer, parameters is
    now easier and less error prone
    ● checkboxes
    ● dropdowns
    2.13.0
    Andrea Aime
    Community
    

    View Slide

47. Full-screen style editor
    Andrea & Torben
    Community 2.14
    

    View Slide

48. External graphic chooser in style editor
    Niels
    -
    ● Security fixes being added to
    supported versions only
    ● Please upgrade your
    GeoServer installations!
    2.14
    

    View Slide

49. Color picker in style editor
    Andrea Aime
    Community
    You know…. just in case you
    don’t remember the hex code
    of the color you want!
    2.14
    

    View Slide

50. ● Label export, most shared configurations exported
    ● Some in the works (word/char spacing, strikethrough, polygon border labelling)
    ● Some changes needed in GeoServer SE 1.1 parser too
    If you prefer point and click: QGIS 3.0 style export
    2.12
    Andrea Aime
    GeoSolutions
    OpenGeoGroep
    

    View Slide

51. Isolated Workspaces
    Workspaces are used to partition what is
    published for ease of management:
    ● A workspace has a namespace
    (used to keep XML content from conflict)
    ● Workspace published as a web service
    Problem:
    ● INSPIRE standard requires the use of a
    specific namespace
    ● Cannot use same namespace,
    preventing reuse of INSPIRE namespace
    Isolated workspace:
    ● Allows one workspace to publish the
    common layers
    ● The remaining workspace (each one
    marked isolated) can reuse these common
    layer
    Allows the same application schema to be
    published from multiple workspaces, each
    mapped onto different datasets
    2.13.0
    Nuno Oliveira
    GeoSolutions
    SYKE
    

    View Slide

52. JDBCConfig and JDBCStore (Community)
    Use a database, rather than data directory, for clustering:
    ● GeoServer “Catalog” configuration
    ● GeoServer support files (styles, icons, fonts, config files)
    Work done for 2.13:
    ● Reduce number of database queries to improve performance
    ● Consistently log configuration queries for analysis
    2.13.0
    Niels Charlier
    Steve Ikeoka
    -
    

    View Slide

53. Search
    Where did I put that satellite image again??
    

    View Slide

54. OpenSearch for Earth Observations
    ● Recent addition to OGC protocol for search of earth imagery
    ● Simpler alternative to CSW with ebRIM
    ● Two steps search
    ○ First find collection of similar products (by satellite/sensor)
    ○ Then search products in it (by other attributes, e.g., cloud cover, off nadir)
    ● Keeps database of searchable metadata
    ● Links to OGC services providing said data
    ● Allows downloading ISO/O&M metadata
    2.12
    GeoSolutions
    DLR/EVO-ODAS
    

    View Slide

55. Help us help you
    

    View Slide

56. User List Participation
    ● Answering users questions relies on a low number of people
    Action:
    ● We are sending reminder on effective use of the user list on subscription and
    at the bottom of each mail
    ● Community building activity - remind developers about the definition of
    “supported”
    (not only reviewing pull requests, also answering user questions, used to be
    obvious, but it’s not obvious to devs hired into open source)
    

    View Slide

57. ● Keep exploit details out of issue report
    ● Mark the issue as a vulnerability.
    ● Be prepared to work with Project Steering Committee (PSC) on a solution
    ● Keep in mind PSC members are volunteers and an extensive fix may require
    fundraising / resources
    If you are not in position to communicate in public please consider commercial
    support, contacting a PSC member, or reaching us via the Open Source Geospatial
    Foundation at info@osgeo.org.
    In case you stumble into a vulnerability:
    Responsible Disclosure
    

    View Slide

58. ● Bug stomp introduced in 2016
    ○ Relying on volunteer time not working out
    ○ allowed developers to reserve time
    ● Moved to fixed schedule in 2017
    ○ Arranging day each month too random
    ● Last Friday of each month
    ○ Check blog post for tips on participation
    You can join and help, too!
    Standing “last friday of the month” bug stomp
    

    View Slide

59. Thanks!
    

    View Slide