Data Management in Kubernetes Using Kanister

Kubernetes is the most popular container orchestrator and is enabling enterprises to rapidly containerize their application stacks. Kubernetes' adoption still faces many challenges, particularly when it comes to stateful applications.

The engineers at Kasten have open sourced Kanister to allow ops teams to incorporate their existing tools into Kubernetes. Kanister is a framework for domain experts to write blueprints specifying how to perform data management in Kubernetes. Each blueprint is specific to a data service, like MySQL and can be modified to integrate with your infrastructure. The talk will conclude with demos of backup and restore of MySQL and MongoDB using example blueprints included with Kanister.

This talk will be targeted towards anyone interesting in running stateful applications in Kubernetes. The audience will learn why the current primitives exposed by Kubernetes aren't sufficient for data operations and how Kanister fills in the gaps.

Transcript

1. Tom Manville | April 25th, 2018 Data Management in Kubernetes Using Kanister
5. 5 yes*
6. 6 Database Operations in Kubernetes kanister.io
7. 7 Agenda • Background • DB in a container • DB in Kubernetes • Kanister • Use Cases • Components • Walkthrough • Demo
8. 8 whoami
9. 9 DB in a Container $ docker run -d \ percona/percona-server:5.7
10. 10 DB in a Container $ docker run -d \ –v /local/datadir:/var/lib/mysql \ percona/percona-server:5.7
11. 11 Cloud Native • Container Package • Dynamically Managed • Micro-Services Oriented
15. 15 Requirements https://kubernetes.io/ https://github.com/kubernetes/kubernetes https://www.helm.sh/ https://github.com/kubernetes/helm
16. 16 DB in Kubernetes $ helm install stable/percona
17. 17 Application Centric Use Cases • Portability between cloud providers • Testing on real data • Filtering • Masking • Backup/Recover • Distributed point-in-time consistency • Incremental backups
18. 18 Install Kanister $ helm install stable/kanister-operator $ helm repo add kanister http://charts.kanister.io
19. 19 Kanister DB Charts $ helm install kanister/kanister-mysql \ --set kanister.s3_bucket="mysql-backup-bucket" \ --set kanister.s3_api_key="${AWS_ACCESS_KEY_ID}" \ --set kanister.s3_api_secret="${AWS_SECRET_ACCESS_KEY}" \
20. 20 Kanister Components API Objects • ActionSet • Blueprints • Profiles (coming soon!) • Locations • Credentials Controller • Watches for new API objects • Launches actions • Creates events / logs status
21. 21 Controller Blueprint Database
22. 22 Controller Blueprint CLI Database
23. 23 ActionSet Controller Blueprint CLI Database
24. 24 ActionSet Controller Blueprint CLI Database
25. 25 ActionSet Controller Blueprint CLI Database
26. 26 ActionSet Controller Blueprint Action CLI Database
27. 27 ActionSet Controller Blueprint Action CLI Database
28. 28 ActionSet Controller Blueprint Action CLI Database
29. Demos Click to add text
30. 30 An Abridged ActionSet spec: actions: - name: backup blueprint: mysql-blueprint object: kind: Deployment name: mysql-sakila namespace: video-store-app status: ...
31. 31 backup: phases: - func: KubeExec args: - "{{ .Deployment.Namespace }}" - "{{ index .Deployment.Pods 0 }}" - kanister-sidecar - bash - -c - | s3_path="s3://${S3_BUCKET}/{{ .ArtifactsOut.mysqlCloudDump.path }}" s3_cmd=(aws "${S3_VERIFY_SSL}" s3 cp - "${s3_path}") mysqldump --password="${MYSQL_ROOT_PASSWORD}" | gzip - | ${s3_cmd[@]} Blueprint: Backup
32. 32 restore: phases: - func: KubeExec args: - "{{ .Deployment.Namespace }}" - "{{ index .Deployment.Pods 0 }}" - kanister-sidecar - bash - -c - | s3_path="s3://${S3_BUCKET}/{{ .ArtifactsIn.mysqlCloudDump.path }}" s3_cmd=(aws "${S3_VERIFY_SSL}" s3 cp "${s3_path}" -) ${s3_cmd[@]} | gunzip -c - | mysql --password="${MYSQL_ROOT_PWD}" Blueprint: Restore
33. 33 Chaining ActionSets $ kanctl perform restore --from backup-mysql-04-23
34. 34 Database Operations in Kubernetes kanister.io
35. 35 Try Kanister Today! • Site: https://kanister.io/ • Github: https://github.com/kanisterio/kanister • Docs: https://docs.kanister.io/ • Slack: https://kasten.typeform.com/to/QBcw8T
36. Backup Slides Click to add text
40. 40 Common Databases