Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Accelerate Docker Image delivery with Varnish c...

Accelerate Docker Image delivery with Varnish caching

Slides for my Confoo 2026 presentation about the acceleration of Docker image delivery using Varnish caching.

A "docker pull" command triggers a series of HTTP requests that are sent to the container registry. By putting a cache in front of the registry and caching the container images, the delivery of these images can be accelerated and scaled.

Faster container image retrieval means faster CI/CD pipelines, but also means faster deployments.

See https://feryn.eu/presentations/accelerate-docker-image-delivery-with-varnish-caching-confoo-2026 for more information.

Avatar for Thijs Feryn

Thijs Feryn PRO

September 30, 2026

More Decks by Thijs Feryn

Other Decks in Technology

Transcript

  1. $ docker pull ubuntu Using default tag: latest latest: Pulling

    from library/ubuntu b380bbd43752: Pulling fs layer 5a5bd11a791e: Pulling fs layer d4d1cd1a70f3: Pulling fs layer b380bbd43752: Download complete d4d1cd1a70f3: Download complete 5a5bd11a791e: Download complete b380bbd43752: Pull complete 5a5bd11a791e: Pull complete d4d1cd1a70f3: Pull complete Digest: sha256:72f6db1e83b7a178d32b6a73a1cfda5f2cb236d25844b76b1af68cd8c47e4c1a Status: Downloaded newer image for ubuntu:latest docker.io/library/ubuntu:latest
  2. $ docker pull ubuntu Using default tag: latest latest: Pulling

    from library/ubuntu b380bbd43752: Pulling fs layer 5a5bd11a791e: Pulling fs layer d4d1cd1a70f3: Pulling fs layer b380bbd43752: Download complete d4d1cd1a70f3: Download complete 5a5bd11a791e: Download complete b380bbd43752: Pull complete 5a5bd11a791e: Pull complete d4d1cd1a70f3: Pull complete Digest: sha256:72f6db1e83b7a178d32b6a73a1cfda5f2cb236d25844b76b1af68cd8c47e4c1a Status: Downloaded newer image for ubuntu:latest docker.io/library/ubuntu:latest USES HTTP CAN BE SLOW CAN BE CACHED
  3. Public Private Azure Container Registry Azure Artifacts Google Artifact Registry

    AWS ECR Registry Docker Hub GitHub Container Registry GitLab Container Registry CD Workers ArgoCD FluxCD Octopus Deploy CI Workers GitHub Actions GitLab Actions CircleCI Quay.io Developers
  4. WHAT'S THE PROBLEM WITH PUBLIC REGISTRIES? ✓ RATE LIMITS ✓

    SLOW CONTAINER REGISTRIES ✓ CLOUD OUTAGES ✓ EGRESS CHARGES
  5. Public Azure Container Registry Azure Artifacts Google Artifact Registry AWS

    ECR Registry Private Repository Manager ArgoCD JFrog Artifactory CloudSmith Sonatype Nexus Docker Hub GitHub Container Registry GitLab Container Registry CD Workers Harbor FluxCD Octopus Deploy CI Workers GitHub Actions GitLab Actions CircleCI Quay.io Developers
  6. Public Azure Container Registry Azure Artifacts Google Artifact Registry AWS

    ECR Registry Private Repository Manager ArgoCD JFrog Artifactory CloudSmith Sonatype Nexus Docker Hub GitHub Container Registry GitLab Container Registry CD Workers Harbor FluxCD Octopus Deploy CI Workers GitHub Actions GitLab Actions CircleCI Quay.io Developers
  7. WHAT'S THE PROBLEM WITH PRIVATE REGISTRIES? ✓ LATENCY ✓ HARD

    TO SCALE ✓ EGRESS CHARGES ✓ INFRASTRUCTURE COST ✓ LICENSE COST
  8. name: ci on: push: branches: - main jobs: docker: runs-on:

    ubuntu-latest steps: - name: Login to Docker Hub uses: docker/login-action@v3 with: username: ${{ vars.DOCKERHUB_USERNAME }} password: ${{ secrets.DOCKERHUB_TOKEN }} - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - name: Build Docker image working-directory: app run: docker build --platform=linux/amd64 --load --pull -t app/image:latest . - name: Push Docker image run: docker push app/image:latest GITHUB ACTIONS EXAMPLE
  9. name: ci on: push: branches: - main jobs: docker: runs-on:

    ubuntu-latest steps: - name: Login to Docker Hub uses: docker/login-action@v3 with: username: ${{ vars.DOCKERHUB_USERNAME }} password: ${{ secrets.DOCKERHUB_TOKEN }} - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - name: Build Docker image working-directory: app run: docker build --platform=linux/amd64 --load --pull -t app/image:latest . - name: Push Docker image run: docker push app/image:latest
  10. FROM node:latest ENV PORT=80 WORKDIR /app EXPOSE 80 COPY ./src/package.json

    . COPY ./src/app.js . COPY ./src/node_modules node_modules CMD ["npm","start"] DOCKERFILE FOR APP
  11. FROM node:latest ENV PORT=80 WORKDIR /app EXPOSE 80 COPY ./src/package.json

    . COPY ./src/app.js . COPY ./src/node_modules node_modules CMD ["npm","start"]
  12. apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app namespace: argocd spec:

    project: default sources: - repoURL: https://github.com/my-user/my-repo targetRevision: main ref: valuesRepo path: my-app directory: exclude: app.yaml - repoURL: https://my-helm-repo.com/app-helm chart: my-app targetRevision: 0.1.2 helm: passCredentials: true valueFiles: - $valuesRepo/my-app/values.yaml destination: server: 'https://kubernetes.default.svc' namespace: my-app syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true - ServerSideApply=true DOCKER IMAGE PULLED FROM HELM CHART HELM CHART DEPLOYED WITH ARGOCD
  13. R E K C ? O E D K I

    A L HOW DOEYOU CACHE S OK O O D L A ADOCKER IMAGE? T GE H A W IM
  14. . ├── blobs │ └── sha256 │ ├── 0b1fad950f54a1d6f9e8e580205c157b43315d2c4231c3a0b78137d87fb928fa │

    ├── 1c036b4ac03eae628080e4b172cbddc4f169887aebe81b1d979fcca796cb79c1 │ ├── 2123190679e81d983648da92f1bb9ddc74383512edb00ad64f93d24d00d8807a │ ├── 260d5cf808df32b294d6eb34f31fef285b2e7fdc0986989517607b37aa40495f │ ├── 32885a2b0a589e832bf6b250bd35a528b268360f166af2cd7094d3a14993fcc1 │ ├── 5cdc8ed4f59b12fd00bf3cf863adeb51841e9c215f3d2b03842d77f750d8b914 │ ├── 5d93aea697980315f27f81c68582d14f63dd3579c2d3a27dc495a588279eda20 │ ├── a9521666e9d84e13efe850468f5c3be3e068522ff9e8b9ba2158707f695221bf │ ├── bb445e472b1bad54f5a28edd51b11aec79eca8513394866a261891be9da6a343 │ ├── d12117b46e66d25e585fad6bb9cb05ab0cedfff0a9cf634b7d0bb4a6fe8f0848 │ └── d56bd99507f882da74ece6b9f7c4aeee969118df49d83e47a3de443998cf3ed4 ├── index.json ├── manifest.json └── oci-layout
  15. $ cat manifest.json | jq [ { "Config": "blobs/sha256/5cdc8ed4f59b12fd00bf3cf863adeb51841e9c215f3d2b03842d77f750d8b914", "RepoTags":

    [ "node:latest" ], "Layers": [ "blobs/sha256/5d93aea697980315f27f81c68582d14f63dd3579c2d3a27dc495a588279eda20", "blobs/sha256/bb445e472b1bad54f5a28edd51b11aec79eca8513394866a261891be9da6a343", "blobs/sha256/2123190679e81d983648da92f1bb9ddc74383512edb00ad64f93d24d00d8807a", "blobs/sha256/32885a2b0a589e832bf6b250bd35a528b268360f166af2cd7094d3a14993fcc1", "blobs/sha256/d12117b46e66d25e585fad6bb9cb05ab0cedfff0a9cf634b7d0bb4a6fe8f0848", "blobs/sha256/1c036b4ac03eae628080e4b172cbddc4f169887aebe81b1d979fcca796cb79c1", "blobs/sha256/a9521666e9d84e13efe850468f5c3be3e068522ff9e8b9ba2158707f695221bf", "blobs/sha256/d56bd99507f882da74ece6b9f7c4aeee969118df49d83e47a3de443998cf3ed4" ] } ]
  16. /v2/ /v2/library/node/manifests/latest AUTHENTICATION /v2/library/node/blobs/sha256:b81c3047c0240876c5be21e30ab0bb3930d31a1fc064a5cfe3b73eaec871a74c ENDPOINT /v2/library/node/blobs/sha256:7511be7a1302b9b97420810021d83f1515e65ec589859a529d82c1c20618bca3 /v2/library/node/blobs/sha256:5463896571d3ff5317461a64229e9e4cb27d6d877114079419cf8b4fc96b0c02 EVEN /v2/library/node/blobs/sha256:e5bba4efb848c0bb1ec0e2278507ca2e3a6d4788f8cb73daf7b1066ce9d7fbb7 PUBLIC

    IMAGES /v2/library/node/blobs/sha256:4edb356e361bad3660b7a9d36f1affd76cd4c25fee229d500dffecab9ccd5bb1 REQUIRE THE USE /v2/library/node/blobs/sha256:27c39b635f712c13cd8a6c70efb4dfbf8569b0b090ace4f5a45bb66ddb18ae45 OF A TOKEN /v2/library/node/blobs/sha256:020f0f7f102dcd1ca7603a86d7398adbe5369a820cc6f32954c0b3b5e2ac7403 /v2/library/node/blobs/sha256:4d8dd1a372f666998a8fdd318ae6205992db5777f2eebb26965b2904918b17a6 /v2/library/node/blobs/sha256:2d02d6605c0c3d633d95aa907a1fbd0f4d99f95c0a225aecce3d1aba720dd772
  17. /v2/ /v2/library/node/manifests/latest /v2/library/node/blobs/sha256:b81c3047c0240876c5be21e30ab0bb3930d31a1fc064a5cfe3b73eaec871a74c MANIFEST FILE /v2/library/node/blobs/sha256:7511be7a1302b9b97420810021d83f1515e65ec589859a529d82c1c20618bca3 FOR THE IMAGE /v2/library/node/blobs/sha256:5463896571d3ff5317461a64229e9e4cb27d6d877114079419cf8b4fc96b0c02

    TAG /v2/library/node/blobs/sha256:020f0f7f102dcd1ca7603a86d7398adbe5369a820cc6f32954c0b3b5e2ac7403 /v2/library/node/blobs/sha256:e5bba4efb848c0bb1ec0e2278507ca2e3a6d4788f8cb73daf7b1066ce9d7fbb7 /v2/library/node/blobs/sha256:4edb356e361bad3660b7a9d36f1affd76cd4c25fee229d500dffecab9ccd5bb1 /v2/library/node/blobs/sha256:27c39b635f712c13cd8a6c70efb4dfbf8569b0b090ace4f5a45bb66ddb18ae45 /v2/library/node/blobs/sha256:4d8dd1a372f666998a8fdd318ae6205992db5777f2eebb26965b2904918b17a6 /v2/library/node/blobs/sha256:2d02d6605c0c3d633d95aa907a1fbd0f4d99f95c0a225aecce3d1aba720dd772
  18. Public Azure Container Registry Azure Artifacts Google Artifact Registry AWS

    ECR Registry Private Repository Manager ArgoCD JFrog Artifactory CloudSmith Sonatype Nexus Docker Hub GitHub Container Registry GitLab Container Registry CD Workers Harbor FluxCD Octopus Deploy CI Workers GitHub Actions GitLab Actions CircleCI Quay.io Developers
  19. Public Azure Container Registry Azure Artifacts Google Artifact Registry AWS

    ECR Registry Private Repository Manager GitHub Container Registry GitLab Container Registry CD Workers ArgoCD JFrog Artifactory CloudSmith Sonatype Nexus Docker Hub Artifact Cache Harbor FluxCD Octopus Deploy CI Workers GitHub Actions GitLab Actions CircleCI Quay.io Developers
  20. BuildX Alpine Linux Ansible Bower Buildkit/Buildctl (using Docker or OCI

    repositories) Cargo Conan Conda CRAN Chef Chocolatey Debian Docker Generic Git LFS Go Modules Gradle Helm (Helm OCI) Hex Hugging Face Models & Data Sets Machine Learning Repositories Maven npm OCI OpenTofu Opkg ORAS NuGet (.NET) Nvidia CocoaPods P2 (Eclipse) PHP Composer podman Poetry Powershell Pub/Dart Puppet PyPI (Python) RPM (yum) RubyGems SBT Swift Terraform Twine Vagrant VCS WASM Yarn
  21. WHY NOT USE A CDN ✓ LACK OF COVERAGE ✓

    PRICE (EGRESS) ✓ NOT (ALWAYS) ABLE TO HANDLE AUTHENTICATION
  22. $ time docker pull node Using default tag: latest latest:

    Pulling from library/node b81c3047c024: Pull complete 5463896571d3: Pull complete 020f0f7f102d: Pull complete e5bba4efb848: Pull complete 4edb356e361b: Pull complete 27c39b635f71: Pull complete 4d8dd1a372f6: Pull complete 2d02d6605c0c: Pull complete Digest: sha256:0b1fad950f54a1d6f9e8e580205c157b43315d2c4231c3a0b78137d87fb928fa Status: Downloaded newer image for node:latest docker.io/library/node:latest What's next: View a summary of image vulnerabilities and recommendations → docker scout quickview node docker pull node 0,15s user 0,15s system 0% cpu 51,381 total
  23. $ time docker pull docker.localhost/library/node Using default tag: latest latest:

    Pulling from library/node b81c3047c024: Pull complete 5463896571d3: Pull complete 020f0f7f102d: Pull complete e5bba4efb848: Pull complete 4edb356e361b: Pull complete 27c39b635f71: Pull complete 4d8dd1a372f6: Pull complete 2d02d6605c0c: Pull complete Digest: sha256:0b1fad950f54a1d6f9e8e580205c157b43315d2c4231c3a0b78137d87fb928fa Status: Downloaded newer image for docker.localhost/library/node:latest CACHE MISS docker.localhost/library/node:latest What's next: View a summary of image vulnerabilities and recommendations → docker scout quickview docker.localhost/library/node docker pull docker.localhost/library/node 0,15s user 0,15s system 0% cpu 50,781 total
  24. $ time docker pull docker.localhost/library/node Using default tag: latest latest:

    Pulling from library/node b81c3047c024: Pull complete 5463896571d3: Pull complete 020f0f7f102d: Pull complete e5bba4efb848: Pull complete 4edb356e361b: Pull complete 27c39b635f71: Pull complete 4d8dd1a372f6: Pull complete 2d02d6605c0c: Pull complete Digest: sha256:0b1fad950f54a1d6f9e8e580205c157b43315d2c4231c3a0b78137d87fb928fa Status: Downloaded newer image for docker.localhost/library/node:latest CACHE HIT docker.localhost/library/node:latest What's next: View a summary of image vulnerabilities and recommendations → docker scout quickview docker.localhost/library/node docker pull docker.localhost/library/node 0,11s user 0,11s system 1% cpu 12,113 total
  25. name: ci on: push: branches: - main jobs: docker: #runs-on:

    ubuntu-latest runs-on: actions-runner-set steps: - name: Login to Docker Hub uses: docker/login-action@v3 with: username: ${{ vars.DOCKERHUB_USERNAME }} password: ${{ secrets.DOCKERHUB_TOKEN }} - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - name: Build Docker image working-directory: app run: docker build --platform=linux/amd64 --load --pull -t varnish.default.svc.cluster.local/app/ image:latest . - name: Push Docker image run: docker push varnish.svc.cluster.local/app/image:latest LOCAL CI RUNNER PUSH & PULL WITHIN THE K8S CLUSTER SUPPORT AUTHENTICATION
  26. FROM varnish.default.svc.cluster.local/node:latest ENV PORT=80 WORKDIR /app EXPOSE 80 COPY ./src/package.json

    . COPY ./src/app.js . COPY ./src/node_modules node_modules CMD ["npm","start"]
  27. apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app namespace: argocd spec:

    project: default sources: - repoURL: https://github.com/my-user/my-repo targetRevision: main ref: valuesRepo path: my-app directory: exclude: app.yaml - repoURL: https://varnish.default.svc.cluster.local/app-helm chart: my-app targetRevision: 0.1.2 helm: passCredentials: true valueFiles: - $valuesRepo/my-app/values.yaml destination: server: 'https://kubernetes.default.svc' namespace: my-app syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true - ServerSideApply=true PULL HELM CHART THROUGH VARNISH DURING DEPLOYMENT ARGOCD CONFIG
  28. server = "https://registry-1.docker.io" [host."http://<ARTIFACT_CACHE_ENDPOINT>"] capabilities = ["pull", "resolve"] skip_verify =

    true [host."http://<ARTIFACT_ENDPOINT>".header] Host = ["dockerhub.localhost"] K8S CLUSTER CONFIGURATION
  29. server = "https://registry-1.docker.io" [host."http://<ARTIFACT_CACHE_ENDPOINT>"] capabilities = ["pull", "resolve"] skip_verify =

    true /ETC/ CONTAINERD/ CERTS.D/DOCKER.IO/ HOSTS.TOML [host."http://<ARTIFACT_CACHE_ENDPOINT>".header] Host = ["dockerhub.localhost"] K8S CLUSTER CONFIGURATION REGISTER CONTAINERD CONFIG PATCHES IN K8S CLUSTER
  30. CACHING PRIVATE IMAGES ✓ AUTHORIZATIONS ARE CACHED PER USER ✓

    LAYERS AND MANIFESTS ARE ONLY CACHED ONCE ✓ LAYERS ARE CACHE ACROSS IMAGES
  31. node:latest ⌞ sha256:563cc2194f6bdc30d9006e0224d485dc73002e2faee758925aac4597afd1e02b ⌞ sha256:f680c9b2c572647c0dab239b6355da8f22cb2c4c96c7308eef455683642d24d0 ⌞ sha256:61a723bcedf75b9c82a2dfb22901ba33bf69f65fd2339420d71430a95813fbd6 ⌞ sha256:936d81443473ef4413e668241f76fdff86f7ac4e649eb67f3cffef19db5b4cd7 ⌞

    sha256:cff50e8435859cc7399677158d512f1449a44efd066dab489230cfa5c60d14b1 ⌞ sha256:a1c124f459202b17eb8c65739eb5376287868fcd62db9051b92ca852f54f7817 ⌞ sha256:86ecc984c2c435fa64cea80449360e66de074ab7c4ddb1ce905f47a7207aef95 ⌞ sha256:6ca93875d52e5ec211c4e182bf4bdf483542ec7c6fa4155150df356ce1bb89de buildpack-deps:bookworm ⌞ sha256:563cc2194f6bdc30d9006e0224d485dc73002e2faee758925aac4597afd1e02b ⌞ sha256:f680c9b2c572647c0dab239b6355da8f22cb2c4c96c7308eef455683642d24d0 ⌞ sha256:61a723bcedf75b9c82a2dfb22901ba33bf69f65fd2339420d71430a95813fbd6 ⌞ sha256:936d81443473ef4413e668241f76fdff86f7ac4e649eb67f3cffef19db5b4cd7
  32. VIRTUAL REGISTRY ✓ CACHING ✓ SECURITY ✓ RESILIENCE ✓ ROUTING

    FLEXIBILITY ✓ SINGLE POINT OF ENTRY:OBSERVABILITY & CONTROL
  33. Public Azure Container Registry Azure Artifacts Google Artifact Registry AWS

    ECR Registry Private Repository Manager GitHub Container Registry GitLab Container Registry CD Workers ArgoCD JFrog Artifactory CloudSmith Sonatype Nexus Docker Hub Artifact Cache Harbor FluxCD Octopus Deploy CI Workers GitHub Actions GitLab Actions CircleCI Quay.io Developers
  34. Public Azure Container Registry Azure Artifacts Google Artifact Registry AWS

    ECR Registry Private Repository Manager GitHub Container Registry GitLab Container Registry CD Workers ArgoCD JFrog Artifactory CloudSmith Sonatype Nexus Docker Hub Virtual Registry Harbor FluxCD Octopus Deploy CI Workers GitHub Actions GitLab Actions CircleCI Quay.io Developers
  35. varnish: http: - port: 80 virtual_registry: registries: - name: dockerhub

    default: true remotes: - url: https://docker.io - url: https://mirror.gcr.io - name: quay remotes: - url: https://quay.io - name: ghcr remotes: - url: https://ghcr.io - name: k8s remotes: - url: https://registry.k8s.io CONFIG FILE
  36. $ docker run --rm -p 80:80 \ --name orca --platform

    linux/amd64 \ -v $(pwd)/config.yaml:/app/config.yaml:ro \ varnish/orca --config /app/config.yaml
  37. services: orca: image: varnish/orca platform: linux/amd64 ports: - "80:80" volumes:

    - ./config.yaml:/app/config.yaml:ro command: --config /app/config.yaml $ docker compose up
  38. orca: varnish: http: - port: 80 virtual_registry: registries: - name:

    dockerhub default: true remotes: - url: https://docker.io - url: https://mirror.gcr.io - name: quay remotes: - url: https://quay.io - name: ghcr VALUES.YAML
  39. $ curl -s https://packagecloud.io/install/ repositories/varnishplus/60-enterprise/ script.deb.sh | sudo bash $

    sudo apt-get install -y varnish-supervisor $ sudo vim /etc/varnish-supervisor/default.yaml $ sudo systemctl restart varnish-supervisor
  40. $ curl -s https://packagecloud.io/install/ repositories/varnishplus/60-enterprise/ script.deb.sh | sudo bash $

    sudo apt-get install -y varnish-supervisor $ sudo vim /etc/varnish-supervisor/default.yaml $ sudo systemctl restart varnish-supervisor
  41. $ curl -s https://packagecloud.io/install/ repositories/varnishplus/60-enterprise/ script.rpm.sh | sudo bash $

    sudo yum install -y varnish-supervisor $ sudo vim /etc/varnish-supervisor/default.yaml $ sudo systemctl restart varnish-supervisor
  42. USING VARNISH ORCA docker pull docker.localhost/library/ubuntu npm install express --registry=http://npmjs.localhost

    GOPROXY=http://go.localhost go mod tidy helm pull oci://ghcr.localhost/prometheus-community/ charts/prometheus --plain-http
  43. ✓ PARSES ARTIFACT MANIFESTS ✓ PULLS POLICY RULESETS FROM CENTRALIZED

    REPO ARTIFACT FIREWALL ✓ ACTIVELY MODIFIES MANIFESTS ✓ DENIES OR HIDES UNAPPROVED VERSIONS BASED ON POLICIES ✓ TURNS LATEST VERSION INTO LATEST APPROVED VERSION
  44. ✓ RULES BASED ON PACKAGE URL & VERSION MATCHING *

    ALLOW, DENY, LOG & HIDE ✓ ✓ FETCH RULES FROM CENTRALIZED (GIT) REPO WITH DOORBELL API ARTIFACT FIREWALL RULESET ✓ INTEGRATION WITH CVE SERVICES, OWASP & OSV.DEV FUTURE ✓ API FOR DEVSECOPS ALLOWING FURTHER CONTROL FUTURE