Day 2 with Stateful Applications - Implementing a Data Protection Strategy

As teams start to onboard mission-critical applications into production, there’s a need to address day-2 concerns. Dealing with regulatory requirements, user error, ransomware and cluster upgrades - requires safeguarding of data and state. For this reason, Volume snapshots have been one of the most requested features from the storage team and now show up in the CSI spec. Snapshots alone, however, are often not enough. Application consistency, local storage, archival, retirement and compliance are just some of the other concerns.

In this talk, the speakers describe the challenges associated with implementing a robust data protection strategy in a containerized environment, open source tools available to help build this and a framework to evaluate what your environment needs. This talk targets DevOps teams looking to improve the reliability of their production environments.

Transcript

1. Day 2 With Stateful Applications Implementing a Data Protection Strategy Vaibhav Kamra Deepika Dixit @vaibhavkamra @deepikadixit
2. about us page 02 Vaibhav Kamra CTO & Co-Founder @ Kasten https://github.com/kanisterio Previously @ Dell EMC, Maginatics, Microsoft @vaibhavkamra Deepika Dixit MTS @ Kasten https://github.com/kanisterio Previously @ Tintri, ASU @deepikadixit
3. agenda what we’ll cover Where is the Data? Adoption patterns of Stateful Applications in Kubernetes Data Protection Strategy What, Why, Misconceptions Getting it Right Implementing Data Protection in Kubernetes Tools available Demo page 03
4. show of hands where is the data Who is running stateful applications in Kubernetes? Who is running applications that store data in services outside of Kubernetes? page 04
5. kubernetes stateful applications wide variety of patterns page 05 Application uses data services outside of Kubernetes Data services in Kubernetes – separate from Application Application includes data services – all in Kubernetes
6. data protection strategy what and why page 06 Accidental or Malicious Data Loss Infrastructure or Hardware Failure Application Misconfiguration Regulatory Compliance Systems in place to recover applications and data if things go bad
7. data protection strategy key elements Automated Backup and Recovery Scheduling and Retirement Policies Security and Encryption Recovery SLAs page 07
8. data protection strategy key elements Automated Backup and Recovery Scheduling and Retirement Policies Security and Encryption Recovery SLAs “Operate At Scale” page 08
9. data protection strategy misconceptions “I don’t have any Stateful Applications in Kubernetes” “My data stores are replicated and resilient” “My underlying infrastructure already takes care of this” page 09
10. anatomy of a cloud-native app kubernetes resources and persistent state page 010
11. implementing data protection page 011
12. implementation capture application config Application Definition (Kubernetes Resources) • From Kubernetes API Server • From Source Code (infra-as-code) • From Helm Repo Other State • Pipeline state/Release information • Environment config page 012
13. implementation capture persistent data Unstructured Data from PVCs • Volume Snapshots • File System backups • A combination of both Data services in the application • Snapshot underlying volumes (crash-consistent) • Application-level tools (app-consistent) • A combination of both Managed services outside K8s (self- hosted or cloud) • Application-level tools • Managed Service APIs page 013
14. implementation workflow orchestration • Application requirements • Ordering across microservices • Quiescing • Pre/Post steps • Kubernetes/Container interactions • Getting access to application data and volumes • Shutting down/Starting services page 014
15. implementation orchestration example Shutdown PostgreSQL Restore DB files + WALs Run PostgreSQL recovery Start PostgreSQL ... ENTRYPOINT ["docker-entrypoint.sh"] EXPOSE 5432 CMD ["postgres"] Scale Down PostgreSQL Create Recovery Pod or Job Restore DB files + WALs Run PostgreSQL recovery Shutdown Recovery Pod Scale Up PostgreSQL Recovery Playbook for PostgreSQL Orchestrating on Kubernetes Use container image with Postgres + Tools Run custom commands Attach PostgreSQL volumes (PVCs) Pod will restart on PG shutdown page 015
16. implementation backup storage and format • Where will backups be stored • Object Storage tends to be a good choice • References to underlying data service snapshots • Durability • Portability • Security and Encryption • Who has access to the data • Who can restore • Key management page 016
17. demo and tools page 017
18. demo picture gallery demo app Picture Gallery • Deployment with 1 replica • 3 Persistent Volumes • MySQL • Unstructured File Data • Config page 018
19. page 019 kanister: Data management workflows in Kubernetes • Describe data protection workflows using Kubernetes Custom Resources (CR) • Primitives for data capture from (and into) a variety of data sources • Workflow Orchestration https://github.com/kanisterio
20. demo backup workflow -> blueprint CR apiVersion: cr.kanister.io/v1alpha1 kind: Blueprint metadata: name: snapshot-blueprint namespace: demo actions: backup: type: Deployment outputArtifacts: backupInfo: ... phases: - func: CreateVolumeSnapshot name: backupVolumes Backup • Discover PVCs • Snapshot underlying Volumes • Push Snapshot Info to Backup Storage page 020
21. demo restore workflow -> blueprint CR apiVersion: cr.kanister.io/v1alpha1 kind: Blueprint metadata: name: snapshot-blueprint namespace: demo actions: backup: ... restore: type: Deployment inputArtifactNames: - backupInfo phases: - func: ScaleWorkload name: shutdownPods - func: CreateVolumeFromSnapshot name: restoreVolumes args: snapshots: "{{ .ArtifactsIn.backupInfo }}" - func: ScaleWorkload name: bringupPods Restore • Scale down application • Delete existing PVCs • Create new PVCs from snapshots • Scale up application page 021
22. tools • Kanister • https://github.com/kanisterio/kanister • Kasten K10 • https://kasten.io • Ark • https://github.com/heptio/ark • ReShifter • https://github.com/mhausenblas/reshifter • k8s-snapshots • https://github.com/miracle2k/k8s-snapshots • Stash • https://github.com/appscode/stash • Others • https://stateful.kubernetes.sh/#backup-and-restore page 022
23. implementation additional topics page 023 Backup Catalog Search, Discovery, Reporting, Auditing Scheduling and Retirement Restore Validation and Testing Integrating into CI/CD Look for slides/recording soon from talk in the CI/CD track!
24. thank you Image is the cover art from Better Together, a Jack Johnson song Questions? You can also find us at: Booth S/E15 www.kasten.io @kastenhq @vaibhavkamra @deepikadixit