Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Pangeo European Meeting 2019 - A history of science on the cloud at the Met Office

Pangeo European Meeting 2019 - A history of science on the cloud at the Met Office

The Informatics Lab is a technology, science and design research group at the Met Office. For the last four years we've been exploring, among other things, how we can do more science using the power of cloud computing. Using tools such as Jupyter, Dask and Kubernetes we've built a number of prototypes of cloud based scientific platforms starting with our JADE project through to current implementations of Pangeo. In this session we will cover how we evolved these systems, produced tools like dask-kubernetes and the Pangeo helm chart, helped build a community, and pushed our science and the Met Office forward.

Ca3d0556227d66b3c15be1eadf69473b?s=128

Jacob Tomlinson

May 23, 2019
Tweet

Transcript

  1. JACOB TOMLINSON TECHNOLOGIST PETER KILLICK TECHNOLOGIST

  2. ABOUT US

  3. THE OFFICE

  4. THE LAB

  5. HISTORY OF SCIENCE ON THE CLOUD

  6. 3D VISUALISATION

  7. None
  8. None
  9. None
  10. JADE JUPYTER AND DASK ENVIRONMENT

  11. A TYPICAL WORKFLOW

  12. Iris

  13. IRIS AND DASK

  14. Iris

  15. PANGEO PROJECT AND COMMUNITY

  16. None
  17. Rebuilt JADE as Pangeo helm chart

  18. And many more…

  19. None
  20. FUTURE WORK

  21. None
  22. Forecasters

  23. None
  24. THE ENVIRONMENTAL FUTURES & BIG DATA IMPACT LAB

  25. None
  26. CLOUD PANGEO and quick demo...

  27. None
  28. None
  29. DASK KUBERNETES

  30. SHARED VS INDIVIDUAL CLUSTERS

  31. JADE Dask (Aug 2016) JASMIN Dask (Aug 2017) Daskernetes (Jan

    2018) Dask Kubernetes (Mar 2018)
  32. JADE Dask (Aug 2016) JASMIN Dask (Aug 2017) Daskernetes (Jan

    2018) Dask Kubernetes (Mar 2018)
  33. JADE Dask (Aug 2016) JASMIN Dask (Aug 2017) Daskernetes (Jan

    2018) Dask Kubernetes (Mar 2018)
  34. JADE Dask (Aug 2016) JASMIN Dask (Aug 2017) Daskernetes (Jan

    2018) Dask Kubernetes (Mar 2018)
  35. SERVERLESS PANGEO

  36. None
  37. None
  38. None
  39. CaaS

  40. None
  41. FUTURE WORK

  42. None
  43. ANY QUESTIONS? Anything else? Get in touch: www.informaticslab.co.uk / @informatics_lab

    / hello@informaticslab.co.uk