Slide 1

Slide 1 text

Distributed compute with Dask, Kubernetes and AWS Jacob Tomlinson, Alex Hilson Met Office Informatics Lab

Slide 2

Slide 2 text

Who

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Our Problem

Slide 6

Slide 6 text

A typical workflow...

Slide 7

Slide 7 text

A typical workflow...

Slide 8

Slide 8 text

Distributed Compute

Slide 9

Slide 9 text

http://dask.pydata.org

Slide 10

Slide 10 text

http://dask.pydata.org

Slide 11

Slide 11 text

Adaptive Clusters

Slide 12

Slide 12 text

$ dask-scheduler --preload adaptive.py

Slide 13

Slide 13 text

IaaS PaaS

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

www.informaticslab.co.uk data.informaticslab.co.uk github.com/met-office-lab @informatics_lab