Continuous Delivery with Bazel, Kubernes and Gitops

Overview of a simple and effective deployment pipeline based off Bazel used to deploy to Kubernetes using a GitOps approach.

https://rlg.io/talks/2018/05/ci/cd-deploying-go-to-kubernetes-with-bazel--gitops/

Transcript

1. CI/CD: Go/Bazel/Kubernetes A simple & effective deployment pipeline
2. Deployment Automation
3. Manual Deployments - Tedious/Toil - Delay the delivery of completed features or fixes - Easy to mess up - Difficult to debug intentions
4. Automate Deployment - Goals - Fast - Idiot Proof - Remove all humans Bonus: - Work with local uncommitted changes - Need to have fast Edit -> Run -> Test cycle
5. Automate Deployment - How? Two Key Changes 1. Switch to GitOps 2. Move to Bazel
6. GitOps - Operations by Pull Request Git as the source of truth Works really well with Declarative orchestration (Kubernetes) Deploy to Production by simply merging a PR, machines take care of the rest github.com/box/kube-applier is a simple solution to mirror Git -> Kubernetes
7. Bazel {fast, correct} - Choose Two Build and test software of any size, quickly and reliably
8. Deployment Pipeline
9. Why Bazel? - Reproducible - Fast - Polyglot - Flexible
10. Deployment Pipeline
11. Deployment Pipeline
12. The Deployment Pipeline
13. Where do the Kubernetes definitions live? If we keep code and .yaml separate we make it hard for local testing We include basic resources in the source repository itself
14. Step 2 & 3: Build and Unit Test with Bazel Keeps track of every input over time Sandbox isolates build & tests from surrounding environment No more “The tests pass on my machine”
15. Step 3: Packaging with Bazel - Supports a variety of packaging formats - Deb (pkg_deb) - Tar (pkg_tar) - Docker Images (image) - Easy to add support for your own - Produces identical packages months later
16. Step 3: Docker Images pkg_tar( name = "bundle", srcs = [":profile"], package_dir = "/opt/pyli", ) image( name = "image", base = "@distroless_base//image", tars = [":bundle"], visibility = [ "//deployment/app:__pkg__", ], )
17. Step 3: Docker Images Can lead to strange looking timestamps $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE bazel/cmd/profile image 78789f09c502 48 years ago 26.8MB
18. Step 5: Deploy Deployment with GitOps is a PR to a operations repository Bazel is used to generate the Kubernetes files CD opens a PR to OPs repo with Bazel’s help
19. Step 5: Deploy github.com/bazelbuild/rules_k8s Bazel Kubernetes rules allow you to build Kubernetes resource definitions.
20. K8s_object rule Teaches Bazel about the relationship between a pod and a docker image rule
21. Step 5: Deploy: Kubernetes Rule k8s_object( name = "profile", images = { "profile:dev": "//cmd/profile:image", }, template = ":profile.yaml", image_chroot = “gcr.io/<project>/”, kind = “nonsense”, cluster = “default”, ) apiVersion: apps/v1beta1 kind: Deployment metadata: name: profile spec: [...] template: [...] spec: containers: - name: profile image: profile:dev
22. Step 5: Deploy: Kubernetes Rule $ bazel run //deployment:profile apiVersion: apps/v1beta1 kind: Deployment metadata: name: profile spec: [...] template: [...] spec: containers: - name: profile image: gcr.io/<project>/cmd/profile@sha256:<sha>
23. Step 5: Deployment $ bazel run //deployment:profile > path/to/deployment/repo/deployment.yaml $ cd path/to/deployment/repo $ git checkout -b deployment-branch $ git commit -m “deployment commit” $ git push origin deployment-branch
24. Step 5: Human Conformation Once the PR is up a Human checks the PR and merges it
25. Step 5: Local Development Every k8s_object rule with a kind and cluster comes with a .apply target $ bazel run //deployment:profile.apply [...] Edit some more code $ bazel run //deployment:profile.apply [...]
26. Goals (Revisited) - Fast - Idiot Proof Bonus: - Work with local uncommitted changes - Need to have fast Edit -> Run -> Test cycle
27. Bazel and Go Some Tips
28. Tip 0: Use Gazelle $ cat BUILD load("@bazel_gazelle//:def.bzl", "gazelle") gazelle( name = "gazelle", prefix = "github.com/example/project", ) $ bazel run //:gazelle Project: github.com/bazelbuild/bazel-gazelle Auto-Generate Bazel build for existing go code
29. Tip 1: Continue to use Dep Bazel does not understand semver. Use Gopkg.lock to populate your WORKSPACE $ bazel run //:gazelle -- update-repos -from_file=Gopkg.lock
30. Tip 2: Use dep to update deps Bazel does not understand semver. Use dep update to update dependencies and the sync with Bazel $ dep ensure -update $ bazel run //:gazelle -- update-repos -from_file=Gopkg.lock
31. Tip 3: Use Bazel Remote Caching Remote caching can dramatically speed up build times If you’re building vendoring large projects (like Kubernetes) this save a lot of time Google Cloud Storage caching is easy and works within 5 minutes
32. Tip 4: Manually Cross Compile packages go_binary( name = "bin-linux-arm64", goarch = "arm64", goos = "linux", ) go_binary( name = "bin-darwin-amd64", goarch = "amd64", goos = "darwin", )
33. Thank you! @0xRLG rlg.io
34. Step 5: Deploy: Defining Defaults Kubernetes Rules let you define default in your WORKSPACE k8s_defaults( name = "k8s_deploy", kind = "nonsense", image_chroot = "gcr.io/<project>/", cluster = "default", )