ERA5 climate reanalysis in Earth Engine

Slide 1

Slide 1 text

ERA5 climate reanalysis in Earth Engine Julia Wagemann PhD candidate at University of Marburg Visiting Scientist at ECMWF @JuliaWagemann #EarthEngineVirtualMeetup 4 Dec 2019

Slide 2

Slide 2 text

That’s me Intersection between data provider and data users Making open data better accessible to users

Slide 3

Slide 3 text

5 PBs temporal resolution atmospheric parameters spatial resolution FULL, FREE and OPEN data policy ERA5

Slide 4

Slide 4 text

Principle components of a Big Earth Data workﬂow Access Processing Visualization

Slide 5

Slide 5 text

Reproducibility challenge - DATA ACCESSIBILITY ● different data are accessible via different data access systems ● it is still about downloading data ● community-speciﬁc data formats (GRIB, NetCDF, GeoTiff) ● data structure and complexity (analyses vs forecast, multiple dimensions) Access

Slide 6

Slide 6 text

Islands of Open Big Earth Data

Slide 7

Slide 7 text

Islands of Open Big Earth Data Meteorological / climate community Earth Observation community

Slide 8

Slide 8 text

Islands of Open Big Earth Data Meteorological / climate community Earth Observation community ● Copernicus Climate Data Store ● GRIB, NetCDF ● Google Earth Engine ● GeoTiff, JPEG2000

Slide 9

Slide 9 text

ERA5 in GEE in numbers 7 TBs 104,000 243,113 3 IMAGE COLLECTIONS 9 VARIABLES HOURLY | DAILY | MONTHLY ● 2m air temperature (min, mean, max) ● 2m dewpoint temperature ● Total precipitation ● Surface Pressure ● Mean sea-level pressure ● 10m u- and v-component of wind

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

Manifest upload in Earth Engine ● JSON ﬁle ● Deﬁnition of the EE asset ● Links to GCP URI ● 1 manifest per time stamp ○ 9 ‘bands’ Upload via CLI earthengine --use_cloud_api upload image --manifest manifest.json

Slide 12

Slide 12 text

Interoperability of data systems - A CHALLENGE BOTTLENECK DATA DOWNLOAD Non-modiﬁable EE assets

Slide 13

Slide 13 text

https://developers.google.com/earth-engine/datasets/catalog/ECMWF_ERA5_DAILY https://developers.google.com/earth-engine/datasets/catalog/ECMWF_ERA5_MONTHLY

Slide 14

Slide 14 text

ERA5 surface pressure

Slide 15

Slide 15 text

What’s next? ● Ingest of hourly ﬁles → 359,160 assets with 9 bands!!! ● Making workﬂow and python code available on Github (after a clean-up)

Slide 16

Slide 16 text

Resources ● Climate Data Store (https://cds.climate.copernicus.eu/#!/home) ● ERA5 climate reanalysis (https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=overview) ● Python library ‘cdsapi’ (https://pypi.org/project/cdsapi/) ● Python library ‘xarray’ (http://xarray.pydata.org/en/stable/index.html) ● Google Cloud Platform (https://cloud.google.com/) ● google-cloud-storage API (https://cloud.google.com/storage/docs/reference/libraries) ● earthengine-api (https://github.com/google/earthengine-api) ● Manifest upload (https://developers.google.com/earth-engine/image_manifest)

Slide 17

Slide 17 text

Thank you! Questions? Julia Wagemann PhD candidate at University of Marburg Visiting Scientist at ECMWF @JuliaWagemann #EarthEngineVirtualMeetup 4 Dec 2019