Preventing Kubernetes Misconfigurations

Joint webinar co-hosted by Datree and Komodor, featuring Komodor's CTO Itiel Shwartz and CEO of Datree Shimon Tolts

Transcript

4. Prevent Kubernetes misconfigurations from reaching production https://github.com/datreeio/datree
6. Agenda 3. Why policies are important 4. Cool tools you can use to prevent it from happening to you 1. Kubernetes failures stories 2. Post mortems & Lessons learned
7. Top Kubernetes failures | EVENT 1 failing CronJob created 4320 “Restarting” pods X LESSON Verify concurrencyPolicy is always set to either “Forbid” or “Replace” ✓ TARGET 5 #4 apiVersion: batch/v1beta1 kind: CronJob metadata: name: gke-cron-job spec: schedule: '*/1 * * * *' startingDeadlineSeconds: 10 concurrencyPolicy: Allow
8. Top Kubernetes failures | EVENT Their API server was down due to OOM issues X LESSON Always ensure that Kubernetes YAML structure is valid, ideally through automated checks. ✓ concurrencyPolicy: Forbid successfulJobsHistoryLimit: 1 failedJobsHistoryLimit: 1 ZALANDO 5 #3
9. Top Kubernetes failures | EVENT Entire cluster was down due to OOM issues and high memory usage X LESSON Specify requests and limits. Especially for third-party services and apps ✓ BLUE MATADOR 5 #2
10. Top Kubernetes failures | EVENT An entire cluster was down due to Ingress misconfiguration X LESSON Prevent users from specifying host as “*” ✓ TARGET 5 #1
11. Top Kubernetes failures | Lessons learned 1 Missing YAML structure validation Zalando 2 Enforce policy on requests and limits Blue Matador Enforce concurrencyPolicy set to Allow Target 3 Conclusions Enforce ‘*’ in ingress resource Target 4
12. Top Kubernetes failures |
13. Cool tools that you can use | Where in the pipeline? datree gatekeeper confTest
14. Cool tools that you can use | Where in the pipeline? datree confTest gatekeeper
15. Cool tools that you can use | Conftest Helps writing tests against structured files It’s specially designed to be used with CI or local testing. Built on top of OPA so all the policies should be written in Rego.
16. Datree CLI solution for policies enforcement It’s specially designed to be used with CI or local testing or even as a pre-commit hook Built in policies & Best practices It’s comes with built-in policies ansl enables K8s admis create their own policies and centralized management.
17. Datree
18. Datree
19. Datree
20. Cool tools that you can use | Datree Open source It’s free, and open for PRs 😎
21. Summary