CMIP6 in the Cloud: Open, Fast, and Accessible Climate Science with Pangeo

Slide 1

Slide 1 text

CMIP6 IN THE CLOUD OPEN, FAST, AND ACCESSIBLE CLIMATE SCIENCE WITH PANGEO | JULIUS BUSECKE | RYAN ABERNATHEY

Slide 2

Slide 2 text

BIG DATA CLIMATE SCIENCE CMIP 6 Custom Analysis applied to each model and member

Slide 3

Slide 3 text

DOWNLOAD FTP / OPeNDAP / etc.

Slide 4

Slide 4 text

MB 😀 FTP / OPeNDAP / etc.

Slide 5

Slide 5 text

GB 😐 FTP / OPeNDAP / etc.

Slide 6

Slide 6 text

TB 😖 FTP / OPeNDAP / etc.

Slide 7

Slide 7 text

PB 😱 FTP / OPeNDAP / etc.

Slide 8

Slide 8 text

BRING THE COMPUTE TO THE DATA! Jupyter for interactive access remote systems Cloud / HPC Xarray provides data structures and intuitive interface for interacting with datasets Parallel computing system allows users deploy clusters of compute nodes for data processing. Dask tells the nodes what to do. Distributed storage “Analysis Ready Data”   stored on globally-available distributed storage. PANGEO ARCHITECTURE

Slide 9

Slide 9 text

CLIMATE SCIENCE IN THE CLOUD

Slide 10

Slide 10 text

CLIMATE SCIENCE IN THE CLOUD EASY COLLABORATION + PUBLIC ACCESS = REPRODUCIBLE SCIENCE

Slide 11

Slide 11 text

CLIMATE SCIENCE IN THE CLOUD ACCESSIBLE BY ANYONE WITH AN INTERNET CONNECTION = MORE DIVERSE SCIENCE COMMUNITY

Slide 12

Slide 12 text

CLIMATE SCIENCE IN THE CLOUD PERFORMANCE + DIVERSE COMMUNITY + OPEN SCIENCE = FAST SCIENTIFIC PROGRESS

Slide 13

Slide 13 text

IT ALL STARTS WITH THE DATA Pangeo/ESGF is hosting a large part of the CMIP6 archive in public cloud buckets. Check out pangeo-data.github.io/pangeo-cmip6-cloud for more info. Wondering how to bring your data to the cloud? Check out Pangeo Forge Later today (2:30 EST) - Pangeo Forge: Crowdsourcing Analysis-Ready, Cloud Optimized Data for Ocean, Weather, and Climate Science Ocean Science Meeting [IN08C Open Ocean Science] - Pangeo Forge Mini-Hackathon

Slide 14

Slide 14 text

PANGEO CMIP6 STACK Custom Code + Other packages Pangeo Foundation CMIP6 speci f ic Packages Project speci f ic Packages

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

But CMIP6 is not quite clean enough

Slide 17

Slide 17 text

😖

Slide 18

Slide 18 text

😖 😎

Slide 19

Slide 19 text

DEMO

Slide 20

Slide 20 text

SAY GOODBYE TO DOWNLOADS AND HELLO TO THE OPEN, FAST, AND FUN FUTURE OF CLIMATE SCIENCE FOR EVERYBODY.

Slide 21

Slide 21 text

I ❤ FEEDBACK, QUESTIONS, CONTRIBUTIONS. CMIP data in the cloud Pangeo Cloud Deployments - https://pangeo.io/cloud.html Docs Github @JuliusBusecke jbusecke [email protected]