Supercharge Your Kubernetes Platform With Carvel (KCD Munich 2023)

Working with Kubernetes and building a secure platform on top of it is challenging. First, you’ll need a way to group Kubernetes resources and apply changes predictably. Are you sure you approved all the changes reconciled by Kubernetes? I’ll demonstrate how that’s often not true.

Your apps will require a different configuration across environments. Wouldn’t it be great to have a way to use both templating and overlays that is also maintainable and YAML-native? What about air-gapped environments? You’ll need to distribute and relocate manifests and OCI artifacts safely.

And when building an internal platform, you’ll need to package all the different capabilities in an efficient and composable API, supporting GitOps and providing extensibility. This session will present a modular strategy with Carvel to solve those issues and help you improve your cloud native strategy on Kubernetes.

Transcript

1. Thomas Vitale
   KCD Munich
   July 18th, 2023
   Supercharge Your Kubernetes
   Platform With Carvel
   @vitalethomas
   

   View Slide

2. Systematic
   • Software Engineer and Cloud
   Architect.
   
   • Author of “Cloud Native Spring
   in Action” (Manning).
   
   • OSS contributor (Java, Spring,
   Cloud Native Technologies)
   Thomas Vitale
   thomasvitale.com @vitalethomas
   

   View Slide

3. Cloud Native Platform
   Enabling product teams to deliver software better, faster, safer
   @vitalethomas
   End-User
   PLATFORM API
   PLATFORM BACKEND
   Certi
   fi
   cate Management
   Continuous Deployment
   Serverless Runtime
   Supply Chain Choreographer
   Ingress and Load Balancing
   …
   Continuous Deployment
   

   View Slide

4. https://carvel.dev
   

   View Slide

5. https://carvel.dev
   

   View Slide

6. 1. Upstream Dependencies
   @vitalethomas
   

   View Slide

7. @vitalethomas
   1
   Upstream
   Dependencies 3
   Con
   fi
   guration
   2
   Deployment
   5
   Artifact
   Bundles
   4
   Image
   Resolution
   7
   Platform
   6
   Package
   Management
   The Journey
   

   View Slide

8. vendir
   Upstream dependencies
   Sync upstream
   
   dependencies
   Support various
   sources
   Lock dependencies
   
   for repeatability
   @vitalethomas
   

   View Slide

9. vendir
   Upstream dependencies
   @vitalethomas
   apiVersion: vendir.k14s.io/v1alpha1
   
   
   kind: Config
   
   
   directories:
   
   
   - contents:
   
   
   - git:
   
   
   ref: v2.7.7
   
   
   url: https://github.com/argoproj/argo-cd
   
   
   includePaths:
   
   
   - manifests/install.yaml
   
   
   newRootPath: manifests
   
   
   path: .
   
   
   path: config/upstream
   vendir.yml
   $ vendir sync
   

   View Slide

10. 2. Deployment
    @vitalethomas
    

    View Slide

11. @vitalethomas
    1
    Upstream
    Dependencies 3
    Con
    fi
    guration
    2
    Deployment
    5
    Artifact
    Bundles
    4
    Image
    Resolution
    7
    Platform
    6
    Package
    Management
    The Journey
    

    View Slide

12. kapp
    Deployments
    Group resources as
    “application”
    Predictable and
    safe deployments
    Wait for
    reconciliation
    @vitalethomas
    Order resources
    based on
    dependencies
    Record application
    deployment history
    

    View Slide

13. @vitalethomas
    apiVersion: kapp.k14s.io/v1alpha1
    
    
    kind: Config
    
    
    rebaseRules:
    
    
    - path: [data]
    
    
    type: copy
    
    
    sources: [new, existing]
    
    
    resourceMatchers:
    
    
    - kindNamespaceNameMatcher:
    
    
    kind: Secret
    
    
    namespace: argocd
    
    
    name: argocd-secret
    kapp-con
    fi
    g.yml
    $ kapp deploy -a argo-cd -f install.yml -f kapp-config.yml kapp
    Deployments
    

    View Slide

14. @vitalethomas
    $ kapp deploy -a argo-cd -f install.yml -f kapp-config.yml
    kapp
    Deployments
    $ kapp list -A
    $ kapp inspect -a argo-cd
    $ kapp delete -a argo-cd
    

    View Slide

15. 3. Configuration
    @vitalethomas
    

    View Slide

16. @vitalethomas
    1
    Upstream
    Dependencies 3
    Con
    fi
    guration
    2
    Deployment
    5
    Artifact
    Bundles
    4
    Image
    Resolution
    7
    Platform
    6
    Package
    Management
    The Journey
    

    View Slide

17. ytt
    Con
    fi
    guration
    Con
    fi
    guration
    
    via templates
    Structure-oriented
    Con
    fi
    guration
    
    via overlays
    @vitalethomas
    Schema validation Any YAML
    

    View Slide

18. @vitalethomas
    $ ytt -f config ytt
    Con
    fi
    guration
    #@ load("@ytt:data", "data")
    
    
    ---
    
    
    apiVersion: v1
    
    
    kind: ConfigMap
    
    
    metadata:
    
    
    name: argo-demo-info
    
    
    namespace: argocd
    
    
    data:
    
    
    conference: #@ data.values.conference_name
    template.yml
    

    View Slide

19. @vitalethomas
    #@ load("@ytt:data", "data")
    
    
    #@ load("@ytt:overlay", "overlay")
    
    
    #@ def argo_service():
    
    
    kind: Service
    
    
    metadata:
    
    
    name: argocd-server
    
    
    #@ end
    
    
    #@overlay/match by=overlay.subset(argo_service())
    
    
    ---
    
    
    spec:
    
    
    #@overlay/match missing_ok=True
    
    
    type: #@ data.values.service_type
    overlay.yml
    $ ytt -f config ytt
    Con
    fi
    guration
    

    View Slide

20. @vitalethomas
    $ ytt -f config ytt
    Con
    fi
    guration
    #@data/values-schema
    
    
    ---
    
    
    conference_name: “KCD Munich”
    
    
    #@schema/desc "The service type for the ArgoCD Server."
    
    
    #@schema/validation one_of=["ClusterIP", "LoadBalancer", "NodePort"]
    
    
    service_type: ClusterIP
    schema.yml
    

    View Slide

21. 4. Image Resolution
    @vitalethomas
    

    View Slide

22. @vitalethomas
    1
    Upstream
    Dependencies 3
    Con
    fi
    guration
    2
    Deployment
    5
    Artifact
    Bundles
    4
    Image
    Resolution
    7
    Platform
    6
    Package
    Management
    The Journey
    

    View Slide

23. kbld
    Image resolution
    @vitalethomas
    Resolve image
    references
    Orchestrate
    
    OCI builds
    Lock references
    
    for repeatability
    

    View Slide

24. @vitalethomas
    apiVersion: kbld.k14s.io/v1alpha1
    
    
    kind: Config
    
    
    overrides:
    
    
    - image: ghcr.io/dexidp/dex:v2.37.0
    
    
    newImage: ghcr.io/dexidp/dex@sha256:f579d00721…
    
    
    preresolved: true
    
    
    - image: quay.io/argoproj/argocd:v2.7.7
    
    
    newImage: quay.io/argoproj/argocd@sha256:6a5d0e909b…
    
    
    preresolved: true
    
    
    - image: redis:7.0.11-alpine
    
    
    newImage: index.docker.io/library/redis@sha256:121bac949f…
    
    
    preresolved: true
    kbld.lock.yml
    $ kbld -f config --lock-output kbld.lock.yml kbld
    Image resolution
    

    View Slide

25. 5. Artifact Bundles
    @vitalethomas
    

    View Slide

26. @vitalethomas
    1
    Upstream
    Dependencies 3
    Con
    fi
    guration
    2
    Deployment
    5
    Artifact
    Bundles
    4
    Image
    Resolution
    7
    Platform
    6
    Package
    Management
    The Journey
    

    View Slide

27. imgpkg
    Artifact bundles
    @vitalethomas
    Bundle con
    fi
    g
    
    as OCI artifacts
    Uni
    fi
    ed distribution
    for apps+con
    fi
    g
    Support air-gapped
    environments
    

    View Slide

28. @vitalethomas
    $ kbld -f config --imgpkg-lock-output .imgpkg/images.yml
    $ imgpkg push -b :argo-cd-bundle:v2.7.7 -f config
    imgpkg
    Artifact bundles
    $ imgpkg pull -b :argo-cd-bundle:v2.7.7 -o pulled-config
    $ imgpkg copy -b :argo-cd-bundle:v2.7.7 \
    
    
    --to-repo :argo-cd-bundle
    

    View Slide

29. 6. Package Management
    @vitalethomas
    

    View Slide

30. @vitalethomas
    1
    Upstream
    Dependencies 3
    Con
    fi
    guration
    2
    Deployment
    5
    Artifact
    Bundles
    4
    Image
    Resolution
    7
    Platform
    6
    Package
    Management
    The Journey
    

    View Slide

31. Working with Packages
    Kubernetes-native implementation with Carvel
    @vitalethomas
    Template/Patch
    Apply templates
    
    
    and overlays
    
    
    (ytt and kbld).
    2
    Deploy
    Deploy the resulting
    application
    
    
    (kapp).
    3
    Fetch
    Fetch con
    fi
    guration
    bundle and OCI images
    (vendir).
    1
    

    View Slide

32. @vitalethomas
    apiVersion: data.packaging.carvel.dev/v1alpha1
    
    
    kind: Package
    
    
    metadata:
    
    
    name: argo-cd.thomasvitale.dev.2.7.7
    
    
    spec:
    
    
    refName: argo-cd.thomasvitale.dev
    
    
    template:
    
    
    spec:
    
    
    deploy:
    
    
    - kapp: {}
    
    
    fetch:
    
    
    - imgpkgBundle:
    
    
    image: ghcr.io/thomasvitale/argo-cd-package-demo@sha256:600b566703…
    
    
    template:
    
    
    - ytt:
    
    
    paths:
    
    
    - config
    
    
    - kbld:
    
    
    paths:
    
    
    - '-'
    
    
    - .imgpkg/images.yml
    
    
    version: 2.7.7
    1
    2
    3
    

    View Slide

33. kapp-controller
    Package management
    @vitalethomas
    Continuous
    deployment via
    GitOps
    Automated
    work
    fl
    ow to build
    packages
    Kubernetes-native
    package
    management
    Package and
    distribute platforms
    Support multiple
    sources and
    con
    fi
    guration tools
    

    View Slide

34. Implement and release a package
    Kubernetes-native implementation with Carvel
    @vitalethomas
    $ kctrl package init
    $ kctrl package release
    Bootstrap package
    Release package
    Package Metadata
    Non version-speci
    fi
    c
    attributes describing a
    package.
    CRD
    Package
    Versioned combination
    of con
    fi
    guration and
    OCI images.
    CRD
    

    View Slide

35. Release a package repository
    Kubernetes-native implementation with Carvel
    @vitalethomas
    $ kctrl package repo release
    Release package repository
    Package Repository
    A collection of
    packages and their
    metadata.
    CRD
    

    View Slide

36. Install a package in a cluster
    Kubernetes-native implementation with Carvel
    @vitalethomas
    $ kctrl package repo add
    Add package repository to cluster
    $ kctrl package install
    Install package in a cluster
    Package Install
    Installation of a
    package instance in a
    cluster.
    CRD
    

    View Slide

37. 7. Platform
    @vitalethomas
    

    View Slide

38. @vitalethomas
    1
    Upstream
    Dependencies 3
    Con
    fi
    guration
    2
    Deployment
    5
    Artifact
    Bundles
    4
    Image
    Resolution
    7
    Platform
    6
    Package
    Management
    The Journey
    

    View Slide

39. Cloud Native Platform in 3 Steps
    Composable platform with Carvel
    @vitalethomas
    $ kapp deploy -a kapp-controller -y \
    
    
    -f https://github.com/carvel-dev/kapp-controller/releases/latest/download/release.yml
    1. Deploy kapp-controller
    $ kctrl package repository add -r kadras-packages \
    
    
    --url ghcr.io/kadras-io/kadras-packages:0.12.0 \
    
    
    -n kadras-packages --create-namespace
    2. Add package repository
    $ kctrl package install -i engineering-platform \
    
    
    -p engineering-platform.packages.kadras.io \
    
    
    -v 0.10.0 \
    
    
    -n kadras-packages \
    
    
    --values-file values.yml
    3. Install platform
    

    View Slide

40. Bonus. Secrets Management
    @vitalethomas
    

    View Slide

41. secretgen-controller
    Secrets management
    @vitalethomas
    Generate
    credentials and
    keys
    Automated
    distribution of
    image pull Secrets
    across namespaces
    Export and import
    Secrets across
    namespaces
    

    View Slide

42. @vitalethomas
    1
    Upstream
    Dependencies 3
    Con
    fi
    guration
    2
    Deployment
    5
    Artifact
    Bundles
    4
    Image
    Resolution
    7
    Platform
    6
    Package
    Management
    The Journey
    

    View Slide

43. thomasvitale.com @vitalethomas
    https://tag-app-delivery.cncf.io/whitepapers/platforms @vitalethomas
    

    View Slide

44. Resources
    @vitalethomas
    

    View Slide

45. Resources
    Source code on GitHub
    • Carvel 101 - Streamline Your Kubernetes Work
    fl
    ows
    
    • Enlightning: Carvel Tools and kapp-controller
    
    • Stop forking Helm charts and use Carvel ytt
    
    • Carvel kapp: kubectl on steroids
    
    • Kubernetes package management with Carvel
    
    • Carvel as a uni
    fi
    ed package manager for Kubernetes
    
    • Kadras: Cloud Native Platforms Toolkit
    @vitalethomas
    

    View Slide

46. https://github.com/kadras-io
    

    View Slide

47. Thomas Vitale
    KCD Munich
    July 18th, 2023
    Supercharge Your Kubernetes
    Platform With Carvel
    @vitalethomas
    

    View Slide