Dask Gateway Dask Proxy Service Worker Scheduler Scheduler Worker Worker Node Node Node The user connects to the gateway and requests cluster resources. Dask gateway launches the cluster on their behalf and proxies traffic through.
DaskHub Helm Chart 💻 Dask Gateway Service Service Ingress Jupyter Worker Service Lines omitted for clarity. Things got a bit crazy. Just imagine lines from basically everything to everything. Jupyter Auth Jupyter Spawner Database
a Jupyter Server and Dask cluster via Kubernetes deployments. Can be manually scaled via kubectl/helm/dask_kubernetes.HelmCluster(). https://github.com/dask/helm-chart dask_kubernetes.KubeCluster() Ephemeral Dynamically launch Dask clusters onto Kubernetes and scale adaptively. Gets garbage collected when idle. https://github.com/dask/dask-kubernetes Dask Gateway Ephemeral/Fixed/Centralized Central hub for spawning Dask Clusters. Great for teams and organizations who want to run many clusters for many users. https://gateway.dask.org/