Share
Embed
Slide 1
Slide 1 text
Serving earth observation data
with GeoServer:
COG, STAC, OpenSearch
and more
Ing. Simone Giannecchini
Ing. Daniele Romagnoli
Ing. Andrea Aime
GeoSolutions
Slide 2
Slide 2 text
GeoSolutions
Enterprise Support
Services
Deployment
Subscription
Professional
Training
Customized
Solutions
GeoNode
• Offices in Italy & US, Global Clients/Team
• 40+ collaborators, 30+ Engineers
• Our products
• Our Offer
Slide 3
Slide 3 text
Affiliations
We strongly support Open
Source, it Is in our core
We actively participate in OGC
working groups and get funded
to advance new open
standards
We support standards critical to
GEOINT
Slide 4
Slide 4 text
Reference scenario
REP/COP
InSitu
Obs
MetOc
Models
Remote
Sensing
Slide 5
Slide 5 text
Locating raster sources
Slide 6
Slide 6 text
OpenSearch for EO (2017)
●
Use case
●
Lots of satellite imagery
●
Different satellites
●
Different sensor
●
Deep time and space
distribution
●
How to find the right
image?
●
By time and space
●
By attributes (cloud cover,
snow cover, off nadir, ….)
OpenSearch
OGC OpenSearch Geo and
Time
OGC OpenSearch for EO
●
“OpenSearch for EO“ to the rescue
Slide 7
Slide 7 text
General information schema
Collection
Product or
Item
Granule
Search
Search
Mosaic/OGC
Slide 8
Slide 8 text
Two steps search
●
First find collection of
similar products (by
satellite/sensor), e.g.
●
Optical or SAR?
●
Supported resolutions
●
Availability of data in
particular timeframe
●
Then search products in it,
by
●
Time
●
Space
●
Cloud cover
●
…
Slide 9
Slide 9 text
Sample search
●
http://cloudsdi.
geo-solutions.it/
●
geoserver/oseo/search?
parentId=SENTINEL2
&cloudCover=30]
●
Output is RSS with
description,
thumb/quicklook and
links
● Since 2021, output can
also be GeoJSON
Slide 10
Slide 10 text
Metadata and cross linking
●
Links to metadata:
●
ISO metadata for collection
●
GML O&M for products
●
Links to OGC services (cross linking)
●
WMS/WMTS to see collection/product
●
WFS to get “masks” (validity, snow, cloud, …)
●
WCS to extract raster
●
Direct download to get original package
Slide 11
Slide 11 text
STAC API (RC1)
● Same use case as
OpenSearch for EO
● Based on OGC API,
JSON based
● Collections and items
● Cross collection
search at /search
○ filter, sort, field
selection
Slide 12
Slide 12 text
STAC API (RC1)
● Screenshots from DLR's EOC Geoservice (STAC
support in development / Beta)
● Customized database, HTML and JSON templates
Landing page
Slide 13
Slide 13 text
STAC API (RC1) - Collections
Collections
Slide 14
Slide 14 text
STAC API (RC1) - Collection
Slide 15
Slide 15 text
STAC API (RC1) - items
Slide 16
Slide 16 text
STAC API (RC1) - item
Slide 17
Slide 17 text
REST API to manage search index
Swagger documentation
Slide 18
Slide 18 text
Accessing raster data
Slide 19
Slide 19 text
Finding rasters
● STAC has “assets” in Item pointing to the
imagery files
● OpenSearch can point to the raw package,
and the “granules” table allow to get to the
single images
● Where can they be located?
○ Local file system (classic approach)
○ HTTP server
○ S3 or other blob storage
Slide 20
Slide 20 text
Reading from blobstore
● COG plugin
○ Leverages COG structure
● But also, old S3 GeoTIFF plugin (Boundless)
○ Reads any GeoTIFF (non COG) from S3
○ Reads in 5MB blocks, caches them
○ Deprecated soon
Slide 21
Slide 21 text
COG plugin
Hea
der
● Leverages COG structure
● Reads header first
● HTTP Range read of the tiles needed (coaleshe
nearby tiles into single read)
● Works against S3, HTTP(s) and Google Storage
● More blob storages incoming (Azure blob)
● May get caching down the way (interested?)
Tile Tile Tile Tile Tile Tile
R1 R2
Slide 22
Slide 22 text
Mosaicking rasters
Slide 23
Slide 23 text
Terminology
●
Granule/Tile
●
The individual raster element composing the
mosaic
●
(Granule) Index
●
The collection of metadata records describing
the location, spatial coverage and other
attributes of each single granule
Slide 24
Slide 24 text
Complete freedom
●
Granules can:
●
Overlap as they please, can have different
resolutions
●
Be in different file formats (faster if the format is
uniform)
●
Have different color models, RGB, gray, paletted
(needs JAI-EXT enabled)
●
Be in different coordinate reference systems
Slide 25
Slide 25 text
The mosaic index
●
The file lookup system
●
A table of granules, their footprint, location,
and extra attributes
●
Currently supported/tested sources:
●
PostGIS, Oracle, SQL Server
●
Shapefile (no attr indexing, no timestamp)
●
Other GeoTools supported stores
●
Can be customized to support custom
granule indexes (e.g. legacy catalog)
Slide 26
Slide 26 text
The Dimensions
●
Maps to alphanumeric attributes in the index
●
TIME and ELEVATION receive special
treatment for WMS and WCS
●
Custom/Additional dimensions
●
Everything besides TIME & ELEVATION
●
They can be dynamically discovered
from GetCapabilities/DescribeCoverage
Slide 27
Slide 27 text
STAC/OpenSearch as an index
・ The OpenSearch/STAC database can be
used as a mosaic index
・ SQL Views selecting a particular collection
sub/set
・ REST API to automatically create them
Collection
Product or
Item
Granule
Slide 28
Slide 28 text
External STAC API as index
・ Community module for STAC API store
landing just landed on 2.22.x
・ Adapting mosaic code to use STAC items as
we speak
STAC
API
STAC
store
GeoTools
ImageMosaic
COG
Slide 29
Slide 29 text
Documentation links
●
GeoServer reference docs:
●
http://docs.geoserver.org/stable/en/user/data/raster/image
mosaic/index.html
●
http://docs.geoserver.org/stable/en/user/data/raster/image
mosaic/tutorial.html
●
http://docs.geoserver.org/api/#/1.0.0/structuredcoverages.y
aml
●
GeoSolutions training:
●
http://geoserver.geo-solutions.it/edu/en/multidim/imagemo
saic/index.html
●
http://geoserver.geo-solutions.it/edu/en/multidim/accessing
_multidim/index.html
●
http://geoserver.geo-solutions.it/edu/en/multidim/rest/index
.html
Slide 30
Slide 30 text
ImageMosaic – the fun stuff
Slide 31
Slide 31 text
Filtering and sorting
●
The mosaic index can be filtered upon with CQL
●
Can be sorted on using WFS like syntax
●
More possibilities than just Time and Elevation
SENSOR = SAR AND SATELLITE=XYZ
SORTBY = TIME D
Slide 32
Slide 32 text
Coverage Views
●
NetCDF/Grib/Mosaic: one reader, many coverages
●
Sometimes the coverages are strictly related
●
E.g., U and V components of wind/current vectors
●
Merge them back into a single coverage as bands
●
Allows rendering engine to work with them both as one
Slide 33
Slide 33 text
Native multi-coverage support
●
Satellite images often
comes in one file per band
●
Requirement, as they have
different resolutions
●
Grab the band name from
file, build a “band specific”
coverage
●
Use coverage views to
merge them together in a
virtual multiband coverage
●
Smarts to read only the
desired files when doing
band selection
Slide 34
Slide 34 text
Documentation links
●
GeoServer reference docs:
●
http://docs.geoserver.org/latest/en/user/data/raster/image
mosaic/index.html
●
http://docs.geoserver.org/latest/en/user/data/raster/covera
geview.html
●
http://docs.geoserver.org/latest/en/user/tutorials/imagemos
aic_footprint/imagemosaic_footprint.html
●
GeoSolutions training:
●
https://geoserver.geo-solutions.it/edu/en/multidim/accessin
g_multidim/coverageview.html
Slide 35
Slide 35 text
The services - visualize
Slide 36
Slide 36 text
WMS-T (time) support
Slide 37
Slide 37 text
Filtering via WMS-T support
TIME, ELEVATION & Custom dimensions
http://localhost:8080/geoserver/geosolutions/wms?...
&time=2013-03-1T00:00:00.000Z
&elevation=35.0
&DIM_FILEDATE=2013-03-01T00:00:00.000Z
&DIM_UPDATED=2013-04-08T08:18:41.597Z
Slide 38
Slide 38 text
Filtering and sorting via vendor options
&CQL_FILTER=platform=SENTINEL2
&sortBy=time D
&sortBy=cloudCover
Slide 39
Slide 39 text
Rendering transformations
●
Style based transformations
●
Applied on the fly, yet fast enough for
interactive use
●
Pluggable: add your own
contouring
Slide 40
Slide 40 text
Rendering transformations examples
windbarbs
currents
Slide 41
Slide 41 text
On the fly map algebra with Jiffle
nir = src[7];
vir = src[3];
dest = (nir-vir)/
(nir+vir);
●
Jiffle: powerful and fast
map algebra language
●
Can be used as a
rendering transformation
Slide 42
Slide 42 text
Documentation links
●
GeoServer documentation
●
http://docs.geoserver.org/latest/en/user/services/wms
/time.html
●
http://docs.geoserver.org/stable/en/user/data/webad
min/layers.html#edit-layer-dimensions
●
http://docs.geoserver.org/latest/en/user/styling/sld/ex
tensions/rendering-transform.html
●
GeoSolutions training:
●
https://geoserver.geo-solutions.it/edu/en/wps/renderi
ng_tx.html
●
Jiffle documentation:
●
https://github.com/geosolutions-it/jai-ext/wiki/Jiffle
Slide 43
Slide 43 text
Download via WCS
Slide 44
Slide 44 text
WCS 2.0
●
Basics
●
Core service
●
KVP binding
●
XML binding
●
Common extensions
●
CRS
●
Scaling
●
Interpolation
●
Range subsetting
●
GeoTiff
●
GML
●
NetCDF
●
Add the output format extensions
●
GeoTIFF
●
GML Grid
●
NetCDF
Slide 45
Slide 45 text
WCS 2.0 DescribeCoverage
4D BBOX
TIME details
as extension
ELEVATION
details as
extension
CUSTOM
Slide 46
Slide 46 text
WCS – custom filtering and sorting
request=GetCoverage&...
&CQL_FILTER=platform=SENTINEL2
&sortBy=time D
&sortBy=cloudCover
Slide 47
Slide 47 text
Download via WPS
Slide 48
Slide 48 text
Asynchronous download
● WCS 2.0 is a fine protocol
● But it’s synchronous
● Not well suited for large downloads
● WPS execute can be
asynchronous
● WPS download
plugin
● Dedicated to large
extractions
Slide 49
Slide 49 text
WPS download - large raw download
25GB
GeoTIFF
2 meters res
Some time later
Slide 50
Slide 50 text
WPS download - large rendered map
10GB
NDVI
GeoTIFF
2 meters res
Some time later
Slide 51
Slide 51 text
WPS download - long animation
3GB
.MP4 file
Some time later