Share
Embed
Slide 1
Slide 1 text
Serving earth observation data
with GeoServer:
COG, STAC, OpenSearch
and more
Simone Giannecchini
Daniele Romagnoli
Andrea Aime
GeoSolutions
Slide 2
Slide 2 text
GeoSolutions
Enterprise Support
Services
Deployment
Subscription
Professional
Training
Customized
Solutions
GeoNode
• Offices in Italy & US, Global Clients/Team
• 30+ collaborators, 25+ Engineers
• Our products
• Our Offer
Slide 3
Slide 3 text
Affiliations
We strongly support Open
Source, it is in our core
We participate in OGC
testbeds and get funded to
advance new open
standards
We support standards
critical to GEOINT
Slide 4
Slide 4 text
Locating raster sources
Slide 5
Slide 5 text
Reference scenario
REP/COP
InSitu
Obs
MetOc
Models
Remote
Sensing
Slide 6
Slide 6 text
STAC API - v1.0
● Same use case as
OpenSearch for EO
● Based on OGC API,
JSON based
● Collections and items
● Cross collection
search at /search
○ filter, sort, field
selection
Slide 7
Slide 7 text
General information schema
Collection
Product or
Item
Asset
Search
Search
Mosaic/OGC
Slide 8
Slide 8 text
STAC API
● Screenshots from DLR's EOC Geoservice (STAC
support in development / Beta)
● Customized database, HTML and JSON templates
Landing page
Slide 9
Slide 9 text
STAC API - Collections
Collections
Slide 10
Slide 10 text
STAC API - Collection
Slide 11
Slide 11 text
STAC API - items
Slide 12
Slide 12 text
STAC API - item
Slide 13
Slide 13 text
OpenSearch for EO (2017)
●
Earlier OGC specification
to search for satellite
imagery
●
Based on OpenSearch
and OpenSearch Geo
and Time
OpenSearch
OGC OpenSearch Geo and
Time
OGC OpenSearch for EO
Slide 14
Slide 14 text
Backend: ingest/config REST API
Swagger documentation
Managing the shared database behind STAC and
OpenSearch for EO
Slide 15
Slide 15 text
Accessing raster data
Slide 16
Slide 16 text
Finding rasters
● STAC has “assets” in Item pointing to the
imagery files
● OpenSearch can point to the raw package
● Where can they be located?
○ Local file system (classic approach)
○ HTTP server
○ S3 or other blob storage
Slide 17
Slide 17 text
Reading from blobstore
● COG plugin
○ cog-http-plugin
○ cog-azure-plugin
○ cog-google-plugin
○ cog-s3-plugin
● But also, old S3 GeoTIFF plugin (Boundless)
○ Reads any GeoTIFF (non COG) from S3
○ Reads in 5MB blocks, caches them
○ Deprecation planned
Slide 18
Slide 18 text
COG plugin
Hea
der
● Leverages COG structure
● Reads header first
● HTTP Range read of the tiles needed (coalesce
nearby tiles into single read)
● May get caching down the way (looking for
sponsors!)
Tile Tile Tile Tile Tile Tile
R1 R2
Slide 19
Slide 19 text
Mosaicking rasters
Slide 20
Slide 20 text
Mosaic Terminology
●
Granule/Tile
●
The individual raster element composing the
mosaic (asset in STAC)
●
(Granule) Index
●
The collection of metadata records describing
the location, spatial coverage and other
attributes of each single granule
Slide 21
Slide 21 text
Complete freedom
●
Granules can:
●
Overlap as they please, can have different
resolutions
●
Be in different file formats (faster if the format is
uniform)
●
Have different color models, RGB, gray, paletted
(needs JAI-EXT enabled)
●
Be in different coordinate reference systems
Slide 22
Slide 22 text
The mosaic index
●
The granule lookup system
●
A “table” of granules, their footprint, location,
and extra attributes
●
Currently supported/tested sources:
●
PostGIS, Oracle, SQL Server, Shapefile
●
STAC API (local or remote)
●
Can be customized to support custom
granule indexes (e.g. legacy catalog)
Slide 23
Slide 23 text
The Dimensions
●
Maps to alphanumeric attributes in the index
●
TIME and ELEVATION receive special
treatment for WMS and WCS
●
Custom/Additional dimensions
●
Everything besides TIME & ELEVATION
●
They can be dynamically discovered
from GetCapabilities/DescribeCoverage
Slide 24
Slide 24 text
Internal STAC as an index
・ The OpenSearch/STAC database can be
used as a mosaic index
・ SQL Views selecting a particular collection
sub/set
・ REST API to automatically create them
Collection
Product or
Item
Granule
Slide 25
Slide 25 text
External STAC API as index
・ Community module for STAC API store
・ Integrates well with image mosaic
・ Tricky issues: each system uses different
auth mechanisms to
actually download data
STAC
API
STAC
store
GeoTools
ImageMosaic
COG
Slide 26
Slide 26 text
Documentation links
●
GeoServer reference docs:
●
http://docs.geoserver.org/stable/en/user/data/raster/image
mosaic/index.html
●
http://docs.geoserver.org/stable/en/user/data/raster/image
mosaic/tutorial.html
●
http://docs.geoserver.org/api/#/1.0.0/structuredcoverages.y
aml
●
GeoSolutions training:
●
http://geoserver.geo-solutions.it/edu/en/multidim/imagemo
saic/index.html
●
http://geoserver.geo-solutions.it/edu/en/multidim/accessing
_multidim/index.html
●
http://geoserver.geo-solutions.it/edu/en/multidim/rest/index
.html
Slide 27
Slide 27 text
ImageMosaic – the fun stuff
Slide 28
Slide 28 text
Filtering and sorting
●
The mosaic index can be filtered upon with CQL
●
Can be sorted on using WFS like syntax
●
More possibilities than just Time and Elevation
SENSOR = SAR AND SATELLITE=XYZ
SORTBY = TIME D
Slide 29
Slide 29 text
Coverage Views
●
NetCDF/Grib/Mosaic: one reader, many coverages
●
Sometimes the coverages are strictly related
●
E.g., U and V components of wind/current vectors
●
Merge them back into a single coverage as bands
●
Allows rendering engine to work with them both as one
Slide 30
Slide 30 text
Native multi-coverage support
●
Satellite images often
comes in one file per band
●
Requirement, as they have
different resolutions
●
Grab the band name from
file, build a “band specific”
coverage
●
Use coverage views to
merge them together in a
virtual multiband coverage
●
Smarts to read only the
desired files when doing
band selection
Slide 31
Slide 31 text
Hyperspectral imagery support
● Hyperspectral images:
those having hundreds of
bands
● Typically stored in
band-interleaved structure
(one data bank for all pixels
in the same band)
Rows
Cols
Bands
Slide 32
Slide 32 text
Coverage Views
●
New in 2.27.x
●
Jiffle, raster algebra
●
E.g. on the fly agronomic index computation
●
Transparently available on all protocols
○
WMS/WCS/WPS/OGC APIs
Slide 33
Slide 33 text
Documentation links
●
GeoServer reference docs:
●
http://docs.geoserver.org/latest/en/user/data/raster/image
mosaic/index.html
●
http://docs.geoserver.org/latest/en/user/data/raster/covera
geview.html
●
http://docs.geoserver.org/latest/en/user/tutorials/imagemos
aic_footprint/imagemosaic_footprint.html
●
GeoSolutions training:
●
https://geoserver.geo-solutions.it/edu/en/multidim/accessin
g_multidim/coverageview.html
Slide 34
Slide 34 text
The services - visualize
Slide 35
Slide 35 text
WMS-T (time) support
Slide 36
Slide 36 text
Filtering via WMS-T support
TIME, ELEVATION & Custom dimensions
http://localhost:8080/geoserver/geosolutions/wms?...
&time=2013-03-1T00:00:00.000Z
&elevation=35.0
&DIM_FILEDATE=2013-03-01T00:00:00.000Z
&DIM_UPDATED=2013-04-08T08:18:41.597Z
Slide 37
Slide 37 text
Filtering and sorting via vendor options
&CQL_FILTER=platform=SENTINEL2
&sortBy=time D
&sortBy=cloudCover
Slide 38
Slide 38 text
Rendering transformations
●
Style based transformations
●
Applied on the fly, yet fast enough for
interactive use
●
Pluggable: add your own
contouring
Slide 39
Slide 39 text
Rendering transformations examples
windbarbs
currents
Slide 40
Slide 40 text
Documentation links
●
GeoServer documentation
●
http://docs.geoserver.org/latest/en/user/services/wms
/time.html
●
http://docs.geoserver.org/stable/en/user/data/webad
min/layers.html#edit-layer-dimensions
●
http://docs.geoserver.org/latest/en/user/styling/sld/ex
tensions/rendering-transform.html
●
GeoSolutions training:
●
https://geoserver.geo-solutions.it/edu/en/wps/renderi
ng_tx.html
●
Jiffle documentation:
●
https://github.com/geosolutions-it/jai-ext/wiki/Jiffle
Slide 41
Slide 41 text
Download via WCS
Slide 42
Slide 42 text
WCS 2.0
●
Basics
●
Core service
●
KVP binding
●
XML binding
●
Common extensions
●
CRS
●
Scaling
●
Interpolation
●
Range subsetting
●
GeoTiff
●
GML
●
NetCDF
●
Add the output format extensions
●
GeoTIFF
●
GML Grid
●
NetCDF
Slide 43
Slide 43 text
WCS 2.0 DescribeCoverage
4D BBOX
TIME details
as extension
ELEVATION
details as
extension
CUSTOM
Slide 44
Slide 44 text
WCS – custom filtering and sorting
request=GetCoverage&...
&CQL_FILTER=platform=SENTINEL2
&sortBy=time D
&sortBy=cloudCover
Slide 45
Slide 45 text
Download via WPS
Slide 46
Slide 46 text
Asynchronous download
● WCS 2.0 is a fine protocol
● But it’s synchronous
● Not well suited for large downloads
● WPS execute can be
asynchronous
● WPS download
plugin
● Dedicated to large
extractions
Slide 47
Slide 47 text
WPS download - large raw download
25GB
GeoTIFF
2 meters res
Some time later
Slide 48
Slide 48 text
WPS download - large rendered map
10GB
NDVI
GeoTIFF
2 meters res
Some time later
Coverage view with
Jiffle + SLD
Slide 49
Slide 49 text
WPS download - long animation
3GB
.MP4 file
Some time later
Slide 50
Slide 50 text
Want to see all this in action?
https://view.eumetsat.int/productviewer