Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CMIP6 in the Cloud: Open, Fast, and Accessible ...

Julius Busecke
January 26, 2022

CMIP6 in the Cloud: Open, Fast, and Accessible Climate Science with Pangeo

Slides to my AMS presentation in the "Joint J10B - Cloud Computing for Big Data in Atmosphere, Ocean, and Climate" session (https://ams.confex.com/ams/102ANNUAL/meetingapp.cgi/Session/59409)

Demo Video: https://youtu.be/bJNpfuW_ypU

Julius Busecke

January 26, 2022
Tweet

More Decks by Julius Busecke

Other Decks in Science

Transcript

  1. CMIP6 IN THE CLOUD OPEN, FAST, AND ACCESSIBLE CLIMATE SCIENCE

    WITH PANGEO | JULIUS BUSECKE | RYAN ABERNATHEY
  2. BRING THE COMPUTE TO THE DATA! Jupyter for interactive access

    remote systems Cloud / HPC Xarray provides data structures and intuitive interface for interacting with datasets Parallel computing system allows users deploy clusters of compute nodes for data processing. Dask tells the nodes what to do. Distributed storage “Analysis Ready Data” 
 stored on globally-available distributed storage. PANGEO ARCHITECTURE
  3. CLIMATE SCIENCE IN THE CLOUD ACCESSIBLE BY ANYONE WITH AN

    INTERNET CONNECTION = MORE DIVERSE SCIENCE COMMUNITY
  4. CLIMATE SCIENCE IN THE CLOUD PERFORMANCE + DIVERSE COMMUNITY +

    OPEN SCIENCE = FAST SCIENTIFIC PROGRESS
  5. IT ALL STARTS WITH THE DATA Pangeo/ESGF is hosting a

    large part of the CMIP6 archive in public cloud buckets. Check out pangeo-data.github.io/pangeo-cmip6-cloud for more info. Wondering how to bring your data to the cloud? Check out Pangeo Forge Later today (2:30 EST) - Pangeo Forge: Crowdsourcing Analysis-Ready, Cloud Optimized Data for Ocean, Weather, and Climate Science Ocean Science Meeting [IN08C Open Ocean Science] - Pangeo Forge Mini-Hackathon
  6. PANGEO CMIP6 STACK Custom Code + Other packages Pangeo Foundation

    CMIP6 speci f ic Packages Project speci f ic Packages
  7. SAY GOODBYE TO DOWNLOADS AND HELLO TO THE OPEN, FAST,

    AND FUN FUTURE OF CLIMATE SCIENCE FOR EVERYBODY.
  8. I ❤ FEEDBACK, QUESTIONS, CONTRIBUTIONS. CMIP data in the cloud

    Pangeo Cloud Deployments - https://pangeo.io/cloud.html Docs Github @JuliusBusecke jbusecke [email protected]