Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Pangeo European Meeting 2019 - A history of science on the cloud at the Met Office

Pangeo European Meeting 2019 - A history of science on the cloud at the Met Office

The Informatics Lab is a technology, science and design research group at the Met Office. For the last four years we've been exploring, among other things, how we can do more science using the power of cloud computing. Using tools such as Jupyter, Dask and Kubernetes we've built a number of prototypes of cloud based scientific platforms starting with our JADE project through to current implementations of Pangeo. In this session we will cover how we evolved these systems, produced tools like dask-kubernetes and the Pangeo helm chart, helped build a community, and pushed our science and the Met Office forward.

Jacob Tomlinson

May 23, 2019
Tweet

More Decks by Jacob Tomlinson

Other Decks in Science

Transcript

  1. JACOB TOMLINSON
    TECHNOLOGIST
    PETER KILLICK
    TECHNOLOGIST

    View Slide

  2. ABOUT
    US

    View Slide

  3. THE
    OFFICE

    View Slide

  4. THE
    LAB

    View Slide

  5. HISTORY OF SCIENCE
    ON THE CLOUD

    View Slide

  6. 3D
    VISUALISATION

    View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. JADE
    JUPYTER AND DASK
    ENVIRONMENT

    View Slide

  11. A TYPICAL WORKFLOW

    View Slide

  12. Iris

    View Slide

  13. IRIS AND
    DASK

    View Slide

  14. Iris

    View Slide

  15. PANGEO
    PROJECT AND COMMUNITY

    View Slide

  16. View Slide

  17. Rebuilt JADE as
    Pangeo helm chart

    View Slide

  18. And many more…

    View Slide

  19. View Slide

  20. FUTURE
    WORK

    View Slide

  21. View Slide

  22. Forecasters

    View Slide

  23. View Slide

  24. THE ENVIRONMENTAL
    FUTURES & BIG DATA
    IMPACT LAB

    View Slide

  25. View Slide

  26. CLOUD
    PANGEO
    and quick demo...

    View Slide

  27. View Slide

  28. View Slide

  29. DASK
    KUBERNETES

    View Slide

  30. SHARED VS
    INDIVIDUAL CLUSTERS

    View Slide

  31. JADE Dask (Aug 2016)
    JASMIN Dask (Aug 2017)
    Daskernetes (Jan 2018)
    Dask Kubernetes (Mar 2018)

    View Slide

  32. JADE Dask (Aug 2016)
    JASMIN Dask (Aug 2017)
    Daskernetes (Jan 2018)
    Dask Kubernetes (Mar 2018)

    View Slide

  33. JADE Dask (Aug 2016)
    JASMIN Dask (Aug 2017)
    Daskernetes (Jan 2018)
    Dask Kubernetes (Mar 2018)

    View Slide

  34. JADE Dask (Aug 2016)
    JASMIN Dask (Aug 2017)
    Daskernetes (Jan 2018)
    Dask Kubernetes (Mar 2018)

    View Slide

  35. SERVERLESS
    PANGEO

    View Slide

  36. View Slide

  37. View Slide

  38. View Slide

  39. CaaS

    View Slide

  40. View Slide

  41. FUTURE
    WORK

    View Slide

  42. View Slide

  43. ANY
    QUESTIONS?
    Anything else? Get in touch:
    www.informaticslab.co.uk / @informatics_lab / [email protected]

    View Slide