Introduction to K8up - July 2023

Introduction to K8up - presentation for CNCF TAG Storage

Transcript

1. VSHN – The DevOps Company Tobias Brunner, CTO Introduction to K8up 1
2. VSHN – The DevOps Company Pronounced /keɪtæpp/ or simply "ketchup" Started early 2018 as BaaS Operator - Backup as a Service Need to back up PVCs and application consistent data Grew into todays K8up - always based on real world needs Uses under the hood Current version: 2.7.1 (May 12th, 2023) and K8up K8up is a Kubernetes backup operator based on Restic that will handle PVC and application backups on a Kubernetes or OpenShift cluster. restic k8up.io github.com/k8up-io 2
3. VSHN – The DevOps Company K8up is a CNCF Sandbox project since November 2021 Annual review: github.com/cncf/toc/pull/1067 3
4. VSHN – The DevOps Company Used by its creator VSHN in all products and services Integral part of - A product of amazee.io by Mirantis Adoption Lagoon 4
5. VSHN – The DevOps Company Any S3-compatible backend Any restic-compatible backend  Always fully encrypted at rest and in transit Where does it store backups? 5
6. VSHN – The DevOps Company K8up saves all PVCs in a given namespace 1. Configure backup encryption secret 2. Trigger a backup or set up a backup schedule 3. No step 3! How does it work? 6
7. VSHN – The DevOps Company 0) PVC Resource kind: PersistentVolumeClaim apiVersion: v1 metadata: name: app-data spec: accessModes: - ReadWriteMany resources: requests: storage: "1Gi" 7
8. VSHN – The DevOps Company 1 A secure password to encrypt the backup with 1) Credentials apiVersion: v1 kind: Secret metadata: name: backup-encryption type: Opaque stringData: password: RoGhahbaix3Eekee-dont-use-me 1 apiVersion: v1 kind: Secret metadata: name: objectbucket-creds type: Opaque stringData: AWS_ACCESS_KEY_ID: from-your-provider1 AWS_SECRET_ACCESS_KEY: from-your-provider2 8
9. VSHN – The DevOps Company 1 A backup at 04:05, every day (Cron syntax) 2) Schedule apiVersion: k8up.io/v1 kind: Schedule metadata: name: backup-my-data spec: backend: repoPasswordSecretRef: name: backup-encryption key: password s3: endpoint: https://sos-ch-gva-2.exo.io bucket: my-bucket-change-name accessKeyIDSecretRef: name: objectbucket-creds key: AWS_ACCESS_KEY_ID secretAccessKeySecretRef: name: objectbucket-creds key: AWS_SECRET_ACCESS_KEY backup: schedule: '5 4 * * *' 1 9
10. VSHN – The DevOps Company Source: twitter.com/nixcraft/status/613636528439345152 10
11. VSHN – The DevOps Company 1 PVC where the restoration takes place Restore apiVersion: k8up.io/v1 kind: Restore metadata: name: restore-my-data spec: snapshot: RESTIC_SNAPSHOT_ID restoreMethod: folder: claimName: my-data-pvc backend: repoPasswordSecretRef: name: backup-encryption key: password s3: endpoint: https://sos-ch-gva-2.exo.io bucket: my-bucket-change-name accessKeyIDSecretRef: name: objectbucket-creds key: AWS_ACCESS_KEY_ID secretAccessKeySecretRef: name: objectbucket-creds key: AWS_SECRET_ACCESS_KEY 1 11
12. VSHN – The DevOps Company Manual Restore via restic $ export RESTIC_REPOSITORY=s3:https://sos-ch-gva-2.exo.io/my-bucket-change-name $ export RESTIC_PASSWORD=RoGhahbaix3Eekee-dont-use-me $ export AWS_ACCESS_KEY_ID=from-your-provider1 $ export AWS_SECRET_ACCESS_KEY=from-your-provider2 $ restic snapshots repository dec6d66c opened successfully, password is correct ID Date Host Tags Directory ---------------------------------------------------------------------- 5ed64a2d 2018-06-08 09:18:34 mydata.local /data ---------------------------------------------------------------------- 1 snapshots $ restic restore 5ed64a2d --target /restore 12
13. VSHN – The DevOps Company Pre-Backup Pods apiVersion: k8up.io/v1 kind: PreBackupPod metadata: name: mysqldump spec: backupCommand: sh -c 'mysqldump -u$USER -p$PW -h $DB_HOST --all-databases' pod: spec: containers: - env: - name: USER value: dumper - name: PW value: topsecret - name: DB_HOST value: mariadb.example.com image: mariadb command: - 'sleep' - 'infinity' imagePullPolicy: Always name: mysqldump 13
14. VSHN – The DevOps Company Application-Aware Backups [...] template: metadata: labels: app: mariadb annotations: k8up.io/backupcommand: mysqldump -uroot -psecure --all-databases [...] 14
15. VSHN – The DevOps Company Regularly checks for data sanity using restic check Backup rotation with restic prune Backup archival on a dedicated location (for example AWS Glacier) Prometheus metrics (lots of!) Webhooks with backup information Other Features 15
16. VSHN – The DevOps Company List of available snapshots in Kubernetes resources for easy restore Restore via the new K8up CLI Support for backing up RWO PVCs Internal rewrites of queueing Newest Features 16
17. VSHN – The DevOps Company Onboard more maintainers from other companies than VSHN Increase adoption to gain traction for the project Progress stability and reliability to be rock-solid and a fully reliable data protection tool Fix annoying bugs which are being discovered while the adoption rises Plans - Project 17
18. VSHN – The DevOps Company Renew operator patterns to be on-par with current best practices (e.g. replace controller configuration via environment variables with a configuration CRD) Use native Kubernetes CronJobs for scheduling backup jobs, rather than using our home-grown solution Refresh Restic repository configuration Further enhance the newly built CLI tool Support multiple backups per namespace Plans - Features 18
19. VSHN – The DevOps Company Tobias Brunner, CTO – VSHN AG – Neugasse 10 – CH-8005 Zürich – +41 44 545 53 00 – – Thanks! tobias.brunner@vshn.ch vshn.ch info@vshn.ch 19