CMIP6
WITHOUT THE
INTERPOLATION
GRID-NATIVE ANALYSIS WITH PANGEO IN THE CLOUD
JULIUS BUSECKE | RYAN ABERNATHEY
Slide 2
Slide 2 text
EYRING ET AL.,
BOM 2003.
…
Each “MIP” consists of
many numerical models
Slide 3
Slide 3 text
SLOW AND STORAGE INTENSIVE
TYPICAL WORKFLOW
Download Data ⏰ Apply Analysis Discovery
Regrid Data ⏰
Homogenize Data ⏰
Slide 4
Slide 4 text
SLOW AND STORAGE INTENSIVE
TYPICAL WORKFLOW
Download Data ⏰ Apply Analysis Discovery
Regrid Data ⏰
Homogenize Data ⏰
Pangeo’s CMIP6
Google Cloud
Public Dataset (~600TB)
Slide 5
Slide 5 text
SLOW AND STORAGE INTENSIVE
TYPICAL WORKFLOW
Download Data ⏰ Apply Analysis Discovery
Regrid Data ⏰
Homogenize Data ⏰
Pangeo’s CMIP6
Google Cloud
Public Dataset (~600TB)
Slide 6
Slide 6 text
CURVILINEAR GRIDS
DELANDMETER AND VAN SEBILLE, 2019
X0
, Y0
X1
, Y1
X2
, Y2
X3
, Y3
u0
u1
v0
v1
T0
F0
F1
F2
F3
x,y
x
y
⇠
⌘
0, 0 1, 0
1, 1
0, 1
F0 F1
F2
F3
U0 U1
V0
V1
T0
⇠,⌘
(a) (b)
ENES.ORG
Slide 7
Slide 7 text
COMPUTING A DERIVATIVE
XGCM
DELANDMETER AND VAN SEBILLE, 2019
u, v, T u
v
T
&
` ' A ep
g
b
j 1
j 1
j
j
j + 1
j + 1
j + 2
i 1
i 1
i
i
i + 1
i + 1
Axis:Y
axis:X
Slide 8
Slide 8 text
COMPUTING A DERIVATIVE
XGCM
DELANDMETER AND VAN SEBILLE, 2019
u, v, T u
v
T
&
` ' A ep
g
b
j 1
j 1
j
j
j + 1
j + 1
j + 2
i 1
i 1
i
i
i + 1
i + 1
Axis:Y
axis:X
Slide 9
Slide 9 text
COMPUTING A DERIVATIVE
XGCM
DELANDMETER AND VAN SEBILLE, 2019
u, v, T u
v
T
&
` ' A ep
g
b
j 1
j 1
j
j
j + 1
j + 1
j + 2
i 1
i 1
i
i
i + 1
i + 1
Axis:Y
axis:X
Slide 10
Slide 10 text
COMPUTING A DERIVATIVE
XGCM
DELANDMETER AND VAN SEBILLE, 2019
u, v, T u
v
T
&
` ' A ep
g
b
j 1
j 1
j
j
j + 1
j + 1
j + 2
i 1
i 1
i
i
i + 1
i + 1
Axis:Y
axis:X
Slide 11
Slide 11 text
COMPUTING A DERIVATIVE
XGCM
DELANDMETER AND VAN SEBILLE, 2019
u, v, T u
v
T
&
` ' A ep
g
b
j 1
j 1
j
j
j + 1
j + 1
j + 2
i 1
i 1
i
i
i + 1
i + 1
Axis:Y
axis:X
Slide 12
Slide 12 text
COMPUTING A DERIVATIVE
XGCM
DELANDMETER AND VAN SEBILLE, 2019
u, v, T u
v
T
&
` ' A ep
g
b
j 1
j 1
j
j
j + 1
j + 1
j + 2
i 1
i 1
i
i
i + 1
i + 1
Axis:Y
axis:X
Not too complicated but tedious and error prone
Slide 13
Slide 13 text
COMPUTING A DERIVATIVE
XGCM
DELANDMETER AND VAN SEBILLE, 2019
u, v, T u
v
T
&
` ' A ep
g
b
j 1
j 1
j
j
j + 1
j + 1
j + 2
i 1
i 1
i
i
i + 1
i + 1
Axis:Y
axis:X
Slide 14
Slide 14 text
COMPUTING A DERIVATIVE
XGCM Starting from xarray dataset
DELANDMETER AND VAN SEBILLE, 2019
u, v, T u
v
T
&
` ' A ep
g
b
j 1
j 1
j
j
j + 1
j + 1
j + 2
i 1
i 1
i
i
i + 1
i + 1
Axis:Y
axis:X
Slide 15
Slide 15 text
COMPUTING A DERIVATIVE
XGCM Starting from xarray dataset
Create a `grid` object
DELANDMETER AND VAN SEBILLE, 2019
u, v, T u
v
T
&
` ' A ep
g
b
j 1
j 1
j
j
j + 1
j + 1
j + 2
i 1
i 1
i
i
i + 1
i + 1
Axis:Y
axis:X
Slide 16
Slide 16 text
DELANDMETER AND VAN SEBILLE, 2019
u, v, T u
v
T
&
` ' A ep
g
b
j 1
j 1
j
j
j + 1
j + 1
j + 2
i 1
i 1
i
i
i + 1
i + 1
Axis:Y
axis:X
COMPUTING A DERIVATIVE
XGCM
Slide 17
Slide 17 text
DELANDMETER AND VAN SEBILLE, 2019
u, v, T u
v
T
&
` ' A ep
g
b
j 1
j 1
j
j
j + 1
j + 1
j + 2
i 1
i 1
i
i
i + 1
i + 1
Axis:Y
axis:X
COMPUTING A DERIVATIVE
XGCM
Currently supported operations:
difference
interpolation
cumulative sum
min/max
average
integral
cumulative integral
derivative
Slide 18
Slide 18 text
CMIP6_PREPROCESSING
Slide 19
Slide 19 text
CMIP6_PREPROCESSING
consistent renaming
Slide 20
Slide 20 text
CMIP6_PREPROCESSING
consistent renaming
Slide 21
Slide 21 text
CMIP6_PREPROCESSING
consistent renaming
Slide 22
Slide 22 text
CMIP6_PREPROCESSING
consistent renaming
Slide 23
Slide 23 text
CMIP6_PREPROCESSING
consistent renaming
homogenize units
Slide 24
Slide 24 text
CMIP6_PREPROCESSING
consistent renaming
homogenize units
consistent longitude convention
Slide 25
Slide 25 text
CMIP6_PREPROCESSING
consistent renaming
homogenize units xgcm grids
consistent longitude convention
Slide 26
Slide 26 text
Sea Surface Temperature Gradient Magnitude
Slide 27
Slide 27 text
Sea Surface Temperature Gradient Magnitude
Slide 28
Slide 28 text
SPENDING TIME ON DISCOVERY NOT PREPROCESSING
Download Data ⏰ Apply Analysis Discovery
Regrid Data ⏰
Homogenize Data ⏰
Pangeo’s CMIP6
Google Cloud
Public Dataset
Slide 29
Slide 29 text
SPENDING TIME ON DISCOVERY NOT PREPROCESSING
Download Data ⏰ Apply Analysis Discovery
Regrid Data ⏰
Homogenize Data ⏰
Pangeo’s CMIP6
Google Cloud
Public Dataset
Slide 30
Slide 30 text
SPENDING TIME ON DISCOVERY NOT PREPROCESSING
Download Data ⏰ Apply Analysis Discovery
Regrid Data ⏰
Homogenize Data ⏰
Pangeo’s CMIP6
Google Cloud
Public Dataset
cmip6_preprocessing
Slide 31
Slide 31 text
SPENDING TIME ON DISCOVERY NOT PREPROCESSING
Download Data ⏰ Apply Analysis Discovery
Regrid Data ⏰
Homogenize Data ⏰
Pangeo’s CMIP6
Google Cloud
Public Dataset
cmip6_preprocessing
❤
Slide 32
Slide 32 text
@JULIUSBUSECKE [email protected]
INTERESTED?
JOIN US ON GITHUB!
HTTPS://XGCM.READTHEDOCS.IO/
HTTPS://GITHUB.COM/JBUSECKE/CMIP6_PREPROCESSING
HTTPS://GITHUB.COM/XGCM/XGCM
Check out the full tutorial in the
Earthcube notebook:
HTTPS://GITHUB.COM/EARTHCUBE2020/EC20_BUSECKE_ETAL