Accelerate Docker Image delivery with Varnish caching

by Thijs Feryn

Embed

Start on current slide

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Slow websites SUCK

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

HI, I'M THIJS

Slide 5

Slide 5 text

I'M THE TECH AT VARNISH EVANGELIST SOFTWARE

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

WE BUILD SOFTWARE-DEFINED WEB ACCELERATION & CONTENT DELIVERY SOLUTIONS

Slide 10

Slide 10 text

YOU CAN RUN VARNISH ANYWHERE

Slide 11

Slide 11 text

AN ACCELERATE ANYTHING THAT SPEAKS HTTP

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

USER VARNISH SERVER

Slide 15

Slide 15 text

HOW'S THAT RELATED TO DOCKER?

Slide 16

Slide 16 text

$ docker pull ubuntu Using default tag: latest latest: Pulling from library/ubuntu b380bbd43752: Pulling fs layer 5a5bd11a791e: Pulling fs layer d4d1cd1a70f3: Pulling fs layer b380bbd43752: Download complete d4d1cd1a70f3: Download complete 5a5bd11a791e: Download complete b380bbd43752: Pull complete 5a5bd11a791e: Pull complete d4d1cd1a70f3: Pull complete Digest: sha256:72f6db1e83b7a178d32b6a73a1cfda5f2cb236d25844b76b1af68cd8c47e4c1a Status: Downloaded newer image for ubuntu:latest docker.io/library/ubuntu:latest

Slide 17

Slide 17 text

Slide 18

Slide 18 text

20–40% OF TOTAL CI/CD PIPELINE TIME CONSUMED BY DEPENDENCY DOWNLOADS

Slide 19

Slide 19 text

WHY CACHE DOCKER IMAGES? ✓ FASTER DELIVERY ✓ CHEAPER ✓ MORE SCALABLE ✓ MORE RESILIENT

Slide 20

Slide 20 text

Public Private Azure Container Registry Azure Artifacts Google Artifact Registry AWS ECR Registry Docker Hub GitHub Container Registry GitLab Container Registry CD Workers ArgoCD FluxCD Octopus Deploy CI Workers GitHub Actions GitLab Actions CircleCI Quay.io Developers

Slide 21

Slide 21 text

WHAT'S THE PROBLEM WITH PUBLIC REGISTRIES? ✓ RATE LIMITS ✓ SLOW CONTAINER REGISTRIES ✓ CLOUD OUTAGES ✓ EGRESS CHARGES

Slide 22

Slide 22 text

Public Azure Container Registry Azure Artifacts Google Artifact Registry AWS ECR Registry Private Repository Manager ArgoCD JFrog Artifactory CloudSmith Sonatype Nexus Docker Hub GitHub Container Registry GitLab Container Registry CD Workers Harbor FluxCD Octopus Deploy CI Workers GitHub Actions GitLab Actions CircleCI Quay.io Developers

Slide 23

Slide 23 text

Slide 24

Slide 24 text

WHAT'S THE PROBLEM WITH PRIVATE REGISTRIES? ✓ LATENCY ✓ HARD TO SCALE ✓ EGRESS CHARGES ✓ INFRASTRUCTURE COST ✓ LICENSE COST

Slide 25

Slide 25 text

CI RUNNER CD RUNNER DEVELOPER

Slide 26

Slide 26 text

FASTER BUILDS CI RUNNER CD RUNNER DEVELOPER

Slide 27

Slide 27 text

name: ci on: push: branches: - main jobs: docker: runs-on: ubuntu-latest steps: - name: Login to Docker Hub uses: docker/login-action@v3 with: username: ${{ vars.DOCKERHUB_USERNAME }} password: ${{ secrets.DOCKERHUB_TOKEN }} - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - name: Build Docker image working-directory: app run: docker build --platform=linux/amd64 --load --pull -t app/image:latest . - name: Push Docker image run: docker push app/image:latest GITHUB ACTIONS EXAMPLE

Slide 28

Slide 28 text

Slide 29

Slide 29 text

FROM node:latest ENV PORT=80 WORKDIR /app EXPOSE 80 COPY ./src/package.json . COPY ./src/app.js . COPY ./src/node_modules node_modules CMD ["npm","start"] DOCKERFILE FOR APP

Slide 30

Slide 30 text

FROM node:latest ENV PORT=80 WORKDIR /app EXPOSE 80 COPY ./src/package.json . COPY ./src/app.js . COPY ./src/node_modules node_modules CMD ["npm","start"]

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

FASTER DEPLOYMENTS CI RUNNER CD RUNNER DEVELOPER

Slide 33

Slide 33 text

apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app namespace: argocd spec: project: default sources: - repoURL: https://github.com/my-user/my-repo targetRevision: main ref: valuesRepo path: my-app directory: exclude: app.yaml - repoURL: https://my-helm-repo.com/app-helm chart: my-app targetRevision: 0.1.2 helm: passCredentials: true valueFiles: - $valuesRepo/my-app/values.yaml destination: server: 'https://kubernetes.default.svc' namespace: my-app syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true - ServerSideApply=true DOCKER IMAGE PULLED FROM HELM CHART HELM CHART DEPLOYED WITH ARGOCD

Slide 34

Slide 34 text

FASTER IMAGE PULLS CI RUNNER CD RUNNER DEVELOPER

Slide 35

Slide 35 text

https://xkcd.com/303/

Slide 36

Slide 36 text

DOCKER REGISTRY REMOTE DEVELOPERS CI/CD RUNNER APPLICATIONS

Slide 37

Slide 37 text

WE NEED SOME SORT OF CACHING

Slide 38

Slide 38 text

HOW DO YOU CACHE A DOCKER IMAGE?

Slide 39

Slide 39 text

R E K C ? O E D K I A L HOW DOEYOU CACHE S OK O O D L A ADOCKER IMAGE? T GE H A W IM

Slide 40

Slide 40 text

$ docker pull node $ docker save node > node.tar $ tar xvf node.tar $ tree

Slide 41

Slide 41 text

. ├── blobs │ └── sha256 │ ├── 0b1fad950f54a1d6f9e8e580205c157b43315d2c4231c3a0b78137d87fb928fa │ ├── 1c036b4ac03eae628080e4b172cbddc4f169887aebe81b1d979fcca796cb79c1 │ ├── 2123190679e81d983648da92f1bb9ddc74383512edb00ad64f93d24d00d8807a │ ├── 260d5cf808df32b294d6eb34f31fef285b2e7fdc0986989517607b37aa40495f │ ├── 32885a2b0a589e832bf6b250bd35a528b268360f166af2cd7094d3a14993fcc1 │ ├── 5cdc8ed4f59b12fd00bf3cf863adeb51841e9c215f3d2b03842d77f750d8b914 │ ├── 5d93aea697980315f27f81c68582d14f63dd3579c2d3a27dc495a588279eda20 │ ├── a9521666e9d84e13efe850468f5c3be3e068522ff9e8b9ba2158707f695221bf │ ├── bb445e472b1bad54f5a28edd51b11aec79eca8513394866a261891be9da6a343 │ ├── d12117b46e66d25e585fad6bb9cb05ab0cedfff0a9cf634b7d0bb4a6fe8f0848 │ └── d56bd99507f882da74ece6b9f7c4aeee969118df49d83e47a3de443998cf3ed4 ├── index.json ├── manifest.json └── oci-layout

Slide 42

Slide 42 text

$ cat manifest.json | jq [ { "Config": "blobs/sha256/5cdc8ed4f59b12fd00bf3cf863adeb51841e9c215f3d2b03842d77f750d8b914", "RepoTags": [ "node:latest" ], "Layers": [ "blobs/sha256/5d93aea697980315f27f81c68582d14f63dd3579c2d3a27dc495a588279eda20", "blobs/sha256/bb445e472b1bad54f5a28edd51b11aec79eca8513394866a261891be9da6a343", "blobs/sha256/2123190679e81d983648da92f1bb9ddc74383512edb00ad64f93d24d00d8807a", "blobs/sha256/32885a2b0a589e832bf6b250bd35a528b268360f166af2cd7094d3a14993fcc1", "blobs/sha256/d12117b46e66d25e585fad6bb9cb05ab0cedfff0a9cf634b7d0bb4a6fe8f0848", "blobs/sha256/1c036b4ac03eae628080e4b172cbddc4f169887aebe81b1d979fcca796cb79c1", "blobs/sha256/a9521666e9d84e13efe850468f5c3be3e068522ff9e8b9ba2158707f695221bf", "blobs/sha256/d56bd99507f882da74ece6b9f7c4aeee969118df49d83e47a3de443998cf3ed4" ] } ]

Slide 43

Slide 43 text

/v2/ /v2/library/node/manifests/latest /v2/library/node/blobs/sha256:b81c3047c0240876c5be21e30ab0bb3930d31a1fc064a5cfe3b73eaec871a74c /v2/library/node/blobs/sha256:7511be7a1302b9b97420810021d83f1515e65ec589859a529d82c1c20618bca3 /v2/library/node/blobs/sha256:5463896571d3ff5317461a64229e9e4cb27d6d877114079419cf8b4fc96b0c02 /v2/library/node/blobs/sha256:020f0f7f102dcd1ca7603a86d7398adbe5369a820cc6f32954c0b3b5e2ac7403 /v2/library/node/blobs/sha256:e5bba4efb848c0bb1ec0e2278507ca2e3a6d4788f8cb73daf7b1066ce9d7fbb7 /v2/library/node/blobs/sha256:4edb356e361bad3660b7a9d36f1affd76cd4c25fee229d500dffecab9ccd5bb1 /v2/library/node/blobs/sha256:27c39b635f712c13cd8a6c70efb4dfbf8569b0b090ace4f5a45bb66ddb18ae45 /v2/library/node/blobs/sha256:4d8dd1a372f666998a8fdd318ae6205992db5777f2eebb26965b2904918b17a6 /v2/library/node/blobs/sha256:2d02d6605c0c3d633d95aa907a1fbd0f4d99f95c0a225aecce3d1aba720dd772

Slide 44

Slide 44 text

/v2/ /v2/library/node/manifests/latest AUTHENTICATION /v2/library/node/blobs/sha256:b81c3047c0240876c5be21e30ab0bb3930d31a1fc064a5cfe3b73eaec871a74c ENDPOINT /v2/library/node/blobs/sha256:7511be7a1302b9b97420810021d83f1515e65ec589859a529d82c1c20618bca3 /v2/library/node/blobs/sha256:5463896571d3ff5317461a64229e9e4cb27d6d877114079419cf8b4fc96b0c02 EVEN /v2/library/node/blobs/sha256:e5bba4efb848c0bb1ec0e2278507ca2e3a6d4788f8cb73daf7b1066ce9d7fbb7 PUBLIC IMAGES /v2/library/node/blobs/sha256:4edb356e361bad3660b7a9d36f1affd76cd4c25fee229d500dffecab9ccd5bb1 REQUIRE THE USE /v2/library/node/blobs/sha256:27c39b635f712c13cd8a6c70efb4dfbf8569b0b090ace4f5a45bb66ddb18ae45 OF A TOKEN /v2/library/node/blobs/sha256:020f0f7f102dcd1ca7603a86d7398adbe5369a820cc6f32954c0b3b5e2ac7403 /v2/library/node/blobs/sha256:4d8dd1a372f666998a8fdd318ae6205992db5777f2eebb26965b2904918b17a6 /v2/library/node/blobs/sha256:2d02d6605c0c3d633d95aa907a1fbd0f4d99f95c0a225aecce3d1aba720dd772

Slide 45

Slide 45 text

/v2/ /v2/library/node/manifests/latest /v2/library/node/blobs/sha256:b81c3047c0240876c5be21e30ab0bb3930d31a1fc064a5cfe3b73eaec871a74c MANIFEST FILE /v2/library/node/blobs/sha256:7511be7a1302b9b97420810021d83f1515e65ec589859a529d82c1c20618bca3 FOR THE IMAGE /v2/library/node/blobs/sha256:5463896571d3ff5317461a64229e9e4cb27d6d877114079419cf8b4fc96b0c02 TAG /v2/library/node/blobs/sha256:020f0f7f102dcd1ca7603a86d7398adbe5369a820cc6f32954c0b3b5e2ac7403 /v2/library/node/blobs/sha256:e5bba4efb848c0bb1ec0e2278507ca2e3a6d4788f8cb73daf7b1066ce9d7fbb7 /v2/library/node/blobs/sha256:4edb356e361bad3660b7a9d36f1affd76cd4c25fee229d500dffecab9ccd5bb1 /v2/library/node/blobs/sha256:27c39b635f712c13cd8a6c70efb4dfbf8569b0b090ace4f5a45bb66ddb18ae45 /v2/library/node/blobs/sha256:4d8dd1a372f666998a8fdd318ae6205992db5777f2eebb26965b2904918b17a6 /v2/library/node/blobs/sha256:2d02d6605c0c3d633d95aa907a1fbd0f4d99f95c0a225aecce3d1aba720dd772

Slide 46

Slide 46 text

/v2/ /v2/library/node/manifests/latest IMAGE LAYERS /v2/library/node/blobs/sha256:b81c3047c0240876c5be21e30ab0bb3930d31a1fc064a5cfe3b73eaec871a74c /v2/library/node/blobs/sha256:7511be7a1302b9b97420810021d83f1515e65ec589859a529d82c1c20618bca3 /v2/library/node/blobs/sha256:5463896571d3ff5317461a64229e9e4cb27d6d877114079419cf8b4fc96b0c02 /v2/library/node/blobs/sha256:020f0f7f102dcd1ca7603a86d7398adbe5369a820cc6f32954c0b3b5e2ac7403 /v2/library/node/blobs/sha256:e5bba4efb848c0bb1ec0e2278507ca2e3a6d4788f8cb73daf7b1066ce9d7fbb7 /v2/library/node/blobs/sha256:4edb356e361bad3660b7a9d36f1affd76cd4c25fee229d500dffecab9ccd5bb1 /v2/library/node/blobs/sha256:27c39b635f712c13cd8a6c70efb4dfbf8569b0b090ace4f5a45bb66ddb18ae45 /v2/library/node/blobs/sha256:4d8dd1a372f666998a8fdd318ae6205992db5777f2eebb26965b2904918b17a6 /v2/library/node/blobs/sha256:2d02d6605c0c3d633d95aa907a1fbd0f4d99f95c0a225aecce3d1aba720dd772

Slide 47

Slide 47 text

Slide 48

Slide 48 text

Public Azure Container Registry Azure Artifacts Google Artifact Registry AWS ECR Registry Private Repository Manager GitHub Container Registry GitLab Container Registry CD Workers ArgoCD JFrog Artifactory CloudSmith Sonatype Nexus Docker Hub Artifact Cache Harbor FluxCD Octopus Deploy CI Workers GitHub Actions GitLab Actions CircleCI Quay.io Developers

Slide 49

Slide 49 text

ARTIFACT CACHE? BECAUSE WE CAN ALSO CACHE OTHER DEPENDENCIES OF THE CI/D PIPELINE

Slide 50

Slide 50 text

BuildX Alpine Linux Ansible Bower Buildkit/Buildctl (using Docker or OCI repositories) Cargo Conan Conda CRAN Chef Chocolatey Debian Docker Generic Git LFS Go Modules Gradle Helm (Helm OCI) Hex Hugging Face Models & Data Sets Machine Learning Repositories Maven npm OCI OpenTofu Opkg ORAS NuGet (.NET) Nvidia CocoaPods P2 (Eclipse) PHP Composer podman Poetry Powershell Pub/Dart Puppet PyPI (Python) RPM (yum) RubyGems SBT Swift Terraform Twine Vagrant VCS WASM Yarn

Slide 51

Slide 51 text

REMOTE DEVELOPERS ARTIFACT CACHE DOCKER REGISTRY CI/CD RUNNER ARTIFACT CACHE APPLICATIONS

Slide 52

Slide 52 text

WHY NOT USE A CDN?

Slide 53

Slide 53 text

HYPERLOCALIZATION

Slide 54

Slide 54 text

WHY NOT USE A CDN ✓ LACK OF COVERAGE ✓ PRICE (EGRESS) ✓ NOT (ALWAYS) ABLE TO HANDLE AUTHENTICATION

Slide 55

Slide 55 text

$ time docker pull node Using default tag: latest latest: Pulling from library/node b81c3047c024: Pull complete 5463896571d3: Pull complete 020f0f7f102d: Pull complete e5bba4efb848: Pull complete 4edb356e361b: Pull complete 27c39b635f71: Pull complete 4d8dd1a372f6: Pull complete 2d02d6605c0c: Pull complete Digest: sha256:0b1fad950f54a1d6f9e8e580205c157b43315d2c4231c3a0b78137d87fb928fa Status: Downloaded newer image for node:latest docker.io/library/node:latest What's next: View a summary of image vulnerabilities and recommendations → docker scout quickview node docker pull node 0,15s user 0,15s system 0% cpu 51,381 total

Slide 56

Slide 56 text

$ time docker pull docker.localhost/library/node Using default tag: latest latest: Pulling from library/node b81c3047c024: Pull complete 5463896571d3: Pull complete 020f0f7f102d: Pull complete e5bba4efb848: Pull complete 4edb356e361b: Pull complete 27c39b635f71: Pull complete 4d8dd1a372f6: Pull complete 2d02d6605c0c: Pull complete Digest: sha256:0b1fad950f54a1d6f9e8e580205c157b43315d2c4231c3a0b78137d87fb928fa Status: Downloaded newer image for docker.localhost/library/node:latest CACHE MISS docker.localhost/library/node:latest What's next: View a summary of image vulnerabilities and recommendations → docker scout quickview docker.localhost/library/node docker pull docker.localhost/library/node 0,15s user 0,15s system 0% cpu 50,781 total

Slide 57

Slide 57 text

$ time docker pull docker.localhost/library/node Using default tag: latest latest: Pulling from library/node b81c3047c024: Pull complete 5463896571d3: Pull complete 020f0f7f102d: Pull complete e5bba4efb848: Pull complete 4edb356e361b: Pull complete 27c39b635f71: Pull complete 4d8dd1a372f6: Pull complete 2d02d6605c0c: Pull complete Digest: sha256:0b1fad950f54a1d6f9e8e580205c157b43315d2c4231c3a0b78137d87fb928fa Status: Downloaded newer image for docker.localhost/library/node:latest CACHE HIT docker.localhost/library/node:latest What's next: View a summary of image vulnerabilities and recommendations → docker scout quickview docker.localhost/library/node docker pull docker.localhost/library/node 0,11s user 0,11s system 1% cpu 12,113 total

Slide 58

Slide 58 text

name: ci on: push: branches: - main jobs: docker: #runs-on: ubuntu-latest runs-on: actions-runner-set steps: - name: Login to Docker Hub uses: docker/login-action@v3 with: username: ${{ vars.DOCKERHUB_USERNAME }} password: ${{ secrets.DOCKERHUB_TOKEN }} - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - name: Build Docker image working-directory: app run: docker build --platform=linux/amd64 --load --pull -t varnish.default.svc.cluster.local/app/ image:latest . - name: Push Docker image run: docker push varnish.svc.cluster.local/app/image:latest LOCAL CI RUNNER PUSH & PULL WITHIN THE K8S CLUSTER SUPPORT AUTHENTICATION

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

FROM varnish.default.svc.cluster.local/node:latest ENV PORT=80 WORKDIR /app EXPOSE 80 COPY ./src/package.json . COPY ./src/app.js . COPY ./src/node_modules node_modules CMD ["npm","start"]

Slide 61

Slide 61 text

apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app namespace: argocd spec: project: default sources: - repoURL: https://github.com/my-user/my-repo targetRevision: main ref: valuesRepo path: my-app directory: exclude: app.yaml - repoURL: https://varnish.default.svc.cluster.local/app-helm chart: my-app targetRevision: 0.1.2 helm: passCredentials: true valueFiles: - $valuesRepo/my-app/values.yaml destination: server: 'https://kubernetes.default.svc' namespace: my-app syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true - ServerSideApply=true PULL HELM CHART THROUGH VARNISH DURING DEPLOYMENT ARGOCD CONFIG

Slide 62

Slide 62 text

image: repository: varnish.default.svc.cluster.local/app tag: latest pullPolicy: "Always" HELM CHART DOCKER IMAGE LOCATION

Slide 63

Slide 63 text

server = "https://registry-1.docker.io" [host."http://"] capabilities = ["pull", "resolve"] skip_verify = true [host."http://".header] Host = ["dockerhub.localhost"] K8S CLUSTER CONFIGURATION

Slide 64

Slide 64 text

server = "https://registry-1.docker.io" [host."http://"] capabilities = ["pull", "resolve"] skip_verify = true /ETC/ CONTAINERD/ CERTS.D/DOCKER.IO/ HOSTS.TOML [host."http://".header] Host = ["dockerhub.localhost"] K8S CLUSTER CONFIGURATION REGISTER CONTAINERD CONFIG PATCHES IN K8S CLUSTER

Slide 65

Slide 65 text

WHAT ABOUT PRIVATE PACKAGES?

Slide 66

Slide 66 text

AUTHENTICATED REQUEST DOCKER CLIENT VARNISH DOCKER REPOSITORY

Slide 67

Slide 67 text

WE FORWARD THE TOKEN TO THE REPO DOCKER CLIENT VARNISH DOCKER REPOSITORY

Slide 68

Slide 68 text

IF THE ORIGIN DENIES ACCESS, VARNISH DENIES ACCESS DOCKER CLIENT VARNISH DOCKER REPOSITORY

Slide 69

Slide 69 text

SUCCESSFUL AUTHORIZATIONS ARE CACHED AND FORWARDED DOCKER CLIENT VARNISH DOCKER REPOSITORY

Slide 70

Slide 70 text

SUBSEQUENT REQUESTS ARE DIRECTLY PULLED FROM VARNISH DOCKER CLIENT VARNISH DOCKER REPOSITORY

Slide 71

Slide 71 text

CACHING PRIVATE IMAGES ✓ AUTHORIZATIONS ARE CACHED PER USER ✓ LAYERS AND MANIFESTS ARE ONLY CACHED ONCE ✓ LAYERS ARE CACHE ACROSS IMAGES

Slide 72

Slide 72 text

node:latest ⌞ sha256:563cc2194f6bdc30d9006e0224d485dc73002e2faee758925aac4597afd1e02b ⌞ sha256:f680c9b2c572647c0dab239b6355da8f22cb2c4c96c7308eef455683642d24d0 ⌞ sha256:61a723bcedf75b9c82a2dfb22901ba33bf69f65fd2339420d71430a95813fbd6 ⌞ sha256:936d81443473ef4413e668241f76fdff86f7ac4e649eb67f3cffef19db5b4cd7 ⌞ sha256:cff50e8435859cc7399677158d512f1449a44efd066dab489230cfa5c60d14b1 ⌞ sha256:a1c124f459202b17eb8c65739eb5376287868fcd62db9051b92ca852f54f7817 ⌞ sha256:86ecc984c2c435fa64cea80449360e66de074ab7c4ddb1ce905f47a7207aef95 ⌞ sha256:6ca93875d52e5ec211c4e182bf4bdf483542ec7c6fa4155150df356ce1bb89de buildpack-deps:bookworm ⌞ sha256:563cc2194f6bdc30d9006e0224d485dc73002e2faee758925aac4597afd1e02b ⌞ sha256:f680c9b2c572647c0dab239b6355da8f22cb2c4c96c7308eef455683642d24d0 ⌞ sha256:61a723bcedf75b9c82a2dfb22901ba33bf69f65fd2339420d71430a95813fbd6 ⌞ sha256:936d81443473ef4413e668241f76fdff86f7ac4e649eb67f3cffef19db5b4cd7

Slide 73

Slide 73 text

ARTIFACT CACHE? VIRTUAL REGISTRY

Slide 74

Slide 74 text

CI/CD VIRTUAL REGISTRY REGISTRY

Slide 75

Slide 75 text

VIRTUAL REGISTRY ✓ CACHING ✓ SECURITY ✓ RESILIENCE ✓ ROUTING FLEXIBILITY ✓ SINGLE POINT OF ENTRY:OBSERVABILITY & CONTROL

Slide 76

Slide 76 text

Slide 77

Slide 77 text

Public Azure Container Registry Azure Artifacts Google Artifact Registry AWS ECR Registry Private Repository Manager GitHub Container Registry GitLab Container Registry CD Workers ArgoCD JFrog Artifactory CloudSmith Sonatype Nexus Docker Hub Virtual Registry Harbor FluxCD Octopus Deploy CI Workers GitHub Actions GitLab Actions CircleCI Quay.io Developers

Slide 78

Slide 78 text

INTRODUCING VARNISH ORCA

Slide 79

Slide 79 text

WWW.VARNISH-SOFTWARE.COM/ORCA

Slide 80

Slide 80 text

GITHUB.COM/VARNISH/ORCA

Slide 81

Slide 81 text

$ docker run --rm -p 80:80 \ --name orca --platform linux/amd64 \ varnish/orca

Slide 82

Slide 82 text

services: orca: image: varnish/orca platform: linux/amd64 ports: - "80:80" $ docker compose up

Slide 83

Slide 83 text

varnish: http: - port: 80 virtual_registry: registries: - name: dockerhub default: true remotes: - url: https://docker.io - url: https://mirror.gcr.io - name: quay remotes: - url: https://quay.io - name: ghcr remotes: - url: https://ghcr.io - name: k8s remotes: - url: https://registry.k8s.io CONFIG FILE

Slide 84

Slide 84 text

$ docker run --rm -p 80:80 \ --name orca --platform linux/amd64 \ -v $(pwd)/config.yaml:/app/config.yaml:ro \ varnish/orca --config /app/config.yaml

Slide 85

Slide 85 text

services: orca: image: varnish/orca platform: linux/amd64 ports: - "80:80" volumes: - ./config.yaml:/app/config.yaml:ro command: --config /app/config.yaml $ docker compose up

Slide 86

Slide 86 text

orca: varnish: http: - port: 80 virtual_registry: registries: - name: dockerhub default: true remotes: - url: https://docker.io - url: https://mirror.gcr.io - name: quay remotes: - url: https://quay.io - name: ghcr VALUES.YAML

Slide 87

Slide 87 text

$ helm install varnish-orca -f values.yaml oci://docker.io/varnish/ orca-chart

Slide 88

Slide 88 text

$ curl -s https://packagecloud.io/install/ repositories/varnishplus/60-enterprise/ script.deb.sh | sudo bash $ sudo apt-get install -y varnish-supervisor $ sudo vim /etc/varnish-supervisor/default.yaml $ sudo systemctl restart varnish-supervisor

Slide 89

Slide 89 text

Slide 90

Slide 90 text

$ curl -s https://packagecloud.io/install/ repositories/varnishplus/60-enterprise/ script.rpm.sh | sudo bash $ sudo yum install -y varnish-supervisor $ sudo vim /etc/varnish-supervisor/default.yaml $ sudo systemctl restart varnish-supervisor

Slide 91

Slide 91 text

USING VARNISH ORCA docker pull docker.localhost/library/ubuntu npm install express --registry=http://npmjs.localhost GOPROXY=http://go.localhost go mod tidy helm pull oci://ghcr.localhost/prometheus-community/ charts/prometheus --plain-http

Slide 92

Slide 92 text

THIS IS JUST THE TIP OF THE ICEBERG

Slide 93

Slide 93 text

ARTIFACT FIREWALL

Slide 94

Slide 94 text

✓ PARSES ARTIFACT MANIFESTS ✓ PULLS POLICY RULESETS FROM CENTRALIZED REPO ARTIFACT FIREWALL ✓ ACTIVELY MODIFIES MANIFESTS ✓ DENIES OR HIDES UNAPPROVED VERSIONS BASED ON POLICIES ✓ TURNS LATEST VERSION INTO LATEST APPROVED VERSION

Slide 95

Slide 95 text

ENFORCEMENT AT REQUEST TIME CI/CD VIRTUAL REGISTRY ARTIFACT FIREWALL REGISTRY

Slide 96

Slide 96 text

✓ RULES BASED ON PACKAGE URL & VERSION MATCHING * ALLOW, DENY, LOG & HIDE ✓ ✓ FETCH RULES FROM CENTRALIZED (GIT) REPO WITH DOORBELL API ARTIFACT FIREWALL RULESET ✓ INTEGRATION WITH CVE SERVICES, OWASP & OSV.DEV FUTURE ✓ API FOR DEVSECOPS ALLOWING FURTHER CONTROL FUTURE