Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Day 2 with Stateful Applications - Implementing a Data Protection Strategy

89fd37e5dcabc98563ae7b7e05513e36?s=47 Kasten
December 12, 2018

Day 2 with Stateful Applications - Implementing a Data Protection Strategy

As teams start to onboard mission-critical applications into production, there’s a need to address day-2 concerns. Dealing with regulatory requirements, user error, ransomware and cluster upgrades - requires safeguarding of data and state. For this reason, Volume snapshots have been one of the most requested features from the storage team and now show up in the CSI spec. Snapshots alone, however, are often not enough. Application consistency, local storage, archival, retirement and compliance are just some of the other concerns.

In this talk, the speakers describe the challenges associated with implementing a robust data protection strategy in a containerized environment, open source tools available to help build this and a framework to evaluate what your environment needs. This talk targets DevOps teams looking to improve the reliability of their production environments.



December 12, 2018


  1. Day 2 With Stateful Applications Implementing a Data Protection Strategy

    Vaibhav Kamra Deepika Dixit @vaibhavkamra @deepikadixit
  2. about us page 02 Vaibhav Kamra CTO & Co-Founder @

    Kasten https://github.com/kanisterio Previously @ Dell EMC, Maginatics, Microsoft @vaibhavkamra Deepika Dixit MTS @ Kasten https://github.com/kanisterio Previously @ Tintri, ASU @deepikadixit
  3. agenda what we’ll cover Where is the Data? Adoption patterns

    of Stateful Applications in Kubernetes Data Protection Strategy What, Why, Misconceptions Getting it Right Implementing Data Protection in Kubernetes Tools available Demo page 03
  4. show of hands where is the data Who is running

    stateful applications in Kubernetes? Who is running applications that store data in services outside of Kubernetes? page 04
  5. kubernetes stateful applications wide variety of patterns page 05 Application

    uses data services outside of Kubernetes Data services in Kubernetes – separate from Application Application includes data services – all in Kubernetes
  6. data protection strategy what and why page 06 Accidental or

    Malicious Data Loss Infrastructure or Hardware Failure Application Misconfiguration Regulatory Compliance Systems in place to recover applications and data if things go bad
  7. data protection strategy key elements Automated Backup and Recovery Scheduling

    and Retirement Policies Security and Encryption Recovery SLAs page 07
  8. data protection strategy key elements Automated Backup and Recovery Scheduling

    and Retirement Policies Security and Encryption Recovery SLAs “Operate At Scale” page 08
  9. data protection strategy misconceptions “I don’t have any Stateful Applications

    in Kubernetes” “My data stores are replicated and resilient” “My underlying infrastructure already takes care of this” page 09
  10. anatomy of a cloud-native app kubernetes resources and persistent state

    page 010
  11. implementing data protection page 011

  12. implementation capture application config Application Definition (Kubernetes Resources) • From

    Kubernetes API Server • From Source Code (infra-as-code) • From Helm Repo Other State • Pipeline state/Release information • Environment config page 012
  13. implementation capture persistent data Unstructured Data from PVCs • Volume

    Snapshots • File System backups • A combination of both Data services in the application • Snapshot underlying volumes (crash-consistent) • Application-level tools (app-consistent) • A combination of both Managed services outside K8s (self- hosted or cloud) • Application-level tools • Managed Service APIs page 013
  14. implementation workflow orchestration • Application requirements • Ordering across microservices

    • Quiescing • Pre/Post steps • Kubernetes/Container interactions • Getting access to application data and volumes • Shutting down/Starting services page 014
  15. implementation orchestration example Shutdown PostgreSQL Restore DB files + WALs

    Run PostgreSQL recovery Start PostgreSQL ... ENTRYPOINT ["docker-entrypoint.sh"] EXPOSE 5432 CMD ["postgres"] Scale Down PostgreSQL Create Recovery Pod or Job Restore DB files + WALs Run PostgreSQL recovery Shutdown Recovery Pod Scale Up PostgreSQL Recovery Playbook for PostgreSQL Orchestrating on Kubernetes Use container image with Postgres + Tools Run custom commands Attach PostgreSQL volumes (PVCs) Pod will restart on PG shutdown page 015
  16. implementation backup storage and format • Where will backups be

    stored • Object Storage tends to be a good choice • References to underlying data service snapshots • Durability • Portability • Security and Encryption • Who has access to the data • Who can restore • Key management page 016
  17. demo and tools page 017

  18. demo picture gallery demo app Picture Gallery • Deployment with

    1 replica • 3 Persistent Volumes • MySQL • Unstructured File Data • Config page 018
  19. page 019 kanister: Data management workflows in Kubernetes • Describe

    data protection workflows using Kubernetes Custom Resources (CR) • Primitives for data capture from (and into) a variety of data sources • Workflow Orchestration https://github.com/kanisterio
  20. demo backup workflow -> blueprint CR apiVersion: cr.kanister.io/v1alpha1 kind: Blueprint

    metadata: name: snapshot-blueprint namespace: demo actions: backup: type: Deployment outputArtifacts: backupInfo: ... phases: - func: CreateVolumeSnapshot name: backupVolumes Backup • Discover PVCs • Snapshot underlying Volumes • Push Snapshot Info to Backup Storage page 020
  21. demo restore workflow -> blueprint CR apiVersion: cr.kanister.io/v1alpha1 kind: Blueprint

    metadata: name: snapshot-blueprint namespace: demo actions: backup: ... restore: type: Deployment inputArtifactNames: - backupInfo phases: - func: ScaleWorkload name: shutdownPods - func: CreateVolumeFromSnapshot name: restoreVolumes args: snapshots: "{{ .ArtifactsIn.backupInfo }}" - func: ScaleWorkload name: bringupPods Restore • Scale down application • Delete existing PVCs • Create new PVCs from snapshots • Scale up application page 021
  22. tools • Kanister • https://github.com/kanisterio/kanister • Kasten K10 • https://kasten.io

    • Ark • https://github.com/heptio/ark • ReShifter • https://github.com/mhausenblas/reshifter • k8s-snapshots • https://github.com/miracle2k/k8s-snapshots • Stash • https://github.com/appscode/stash • Others • https://stateful.kubernetes.sh/#backup-and-restore page 022
  23. implementation additional topics page 023 Backup Catalog Search, Discovery, Reporting,

    Auditing Scheduling and Retirement Restore Validation and Testing Integrating into CI/CD Look for slides/recording soon from talk in the CI/CD track!
  24. thank you Image is the cover art from Better Together,

    a Jack Johnson song Questions? You can also find us at: Booth S/E15 www.kasten.io @kastenhq @vaibhavkamra @deepikadixit