Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to K8up - July 2023

Introduction to K8up - July 2023

Introduction to K8up - presentation for CNCF TAG Storage

Tobias Brunner

July 13, 2023
Tweet

More Decks by Tobias Brunner

Other Decks in Technology

Transcript

  1. VSHN – The DevOps Company
    Tobias Brunner, CTO
    Introduction to K8up
    1

    View full-size slide

  2. VSHN – The DevOps Company
    Pronounced /keɪtæpp/ or simply "ketchup"
    Started early 2018 as BaaS Operator - Backup as a Service
    Need to back up PVCs and application consistent data
    Grew into todays K8up - always based on real world needs
    Uses under the hood
    Current version: 2.7.1 (May 12th, 2023)
    and
    K8up
    K8up is a Kubernetes backup operator based on
    Restic that will handle PVC and application
    backups on a Kubernetes or OpenShift cluster.
    restic
    k8up.io github.com/k8up-io
    2

    View full-size slide

  3. VSHN – The DevOps Company
    K8up is a CNCF Sandbox project since November 2021
    Annual review: github.com/cncf/toc/pull/1067
    3

    View full-size slide

  4. VSHN – The DevOps Company
    Used by its creator VSHN in all products and services
    Integral part of - A product of amazee.io by Mirantis
    Adoption
    Lagoon
    4

    View full-size slide

  5. VSHN – The DevOps Company
    Any S3-compatible backend
    Any restic-compatible backend
     Always fully encrypted at rest and in transit
    Where does it store backups?
    5

    View full-size slide

  6. VSHN – The DevOps Company
    K8up saves all PVCs in a given namespace
    1. Configure backup encryption secret
    2. Trigger a backup or set up a backup schedule
    3. No step 3!
    How does it work?
    6

    View full-size slide

  7. VSHN – The DevOps Company
    0) PVC Resource
    kind: PersistentVolumeClaim

    apiVersion: v1

    metadata:

    name: app-data

    spec:

    accessModes:

    - ReadWriteMany

    resources:

    requests:

    storage: "1Gi"
    7

    View full-size slide

  8. VSHN – The DevOps Company
    1 A secure password to encrypt the backup with
    1) Credentials
    apiVersion: v1

    kind: Secret

    metadata:

    name: backup-encryption

    type: Opaque

    stringData:

    password: RoGhahbaix3Eekee-dont-use-me 1
    apiVersion: v1

    kind: Secret

    metadata:

    name: objectbucket-creds

    type: Opaque

    stringData:

    AWS_ACCESS_KEY_ID: from-your-provider1

    AWS_SECRET_ACCESS_KEY: from-your-provider2
    8

    View full-size slide

  9. VSHN – The DevOps Company
    1 A backup at 04:05, every day (Cron syntax)
    2) Schedule
    apiVersion: k8up.io/v1

    kind: Schedule

    metadata:

    name: backup-my-data

    spec:

    backend:

    repoPasswordSecretRef:

    name: backup-encryption

    key: password

    s3:

    endpoint: https://sos-ch-gva-2.exo.io

    bucket: my-bucket-change-name

    accessKeyIDSecretRef:

    name: objectbucket-creds

    key: AWS_ACCESS_KEY_ID

    secretAccessKeySecretRef:

    name: objectbucket-creds

    key: AWS_SECRET_ACCESS_KEY

    backup:

    schedule: '5 4 * * *' 1
    9

    View full-size slide

  10. VSHN – The DevOps Company
    Source: twitter.com/nixcraft/status/613636528439345152
    10

    View full-size slide

  11. VSHN – The DevOps Company
    1 PVC where the restoration takes place
    Restore
    apiVersion: k8up.io/v1

    kind: Restore

    metadata:

    name: restore-my-data

    spec:

    snapshot: RESTIC_SNAPSHOT_ID

    restoreMethod:

    folder:

    claimName: my-data-pvc

    backend:

    repoPasswordSecretRef:

    name: backup-encryption

    key: password

    s3:

    endpoint: https://sos-ch-gva-2.exo.io

    bucket: my-bucket-change-name

    accessKeyIDSecretRef:

    name: objectbucket-creds

    key: AWS_ACCESS_KEY_ID

    secretAccessKeySecretRef:

    name: objectbucket-creds

    key: AWS_SECRET_ACCESS_KEY
    1
    11

    View full-size slide

  12. VSHN – The DevOps Company
    Manual Restore via restic
    $ export RESTIC_REPOSITORY=s3:https://sos-ch-gva-2.exo.io/my-bucket-change-name

    $ export RESTIC_PASSWORD=RoGhahbaix3Eekee-dont-use-me

    $ export AWS_ACCESS_KEY_ID=from-your-provider1

    $ export AWS_SECRET_ACCESS_KEY=from-your-provider2



    $ restic snapshots

    repository dec6d66c opened successfully, password is correct

    ID Date Host Tags Directory

    ----------------------------------------------------------------------

    5ed64a2d 2018-06-08 09:18:34 mydata.local /data

    ----------------------------------------------------------------------

    1 snapshots



    $ restic restore 5ed64a2d --target /restore
    12

    View full-size slide

  13. VSHN – The DevOps Company
    Pre-Backup Pods
    apiVersion: k8up.io/v1

    kind: PreBackupPod

    metadata:

    name: mysqldump

    spec:

    backupCommand: sh -c 'mysqldump -u$USER -p$PW -h $DB_HOST --all-databases'

    pod:

    spec:

    containers:

    - env:

    - name: USER

    value: dumper

    - name: PW

    value: topsecret

    - name: DB_HOST

    value: mariadb.example.com

    image: mariadb

    command:

    - 'sleep'

    - 'infinity'

    imagePullPolicy: Always

    name: mysqldump
    13

    View full-size slide

  14. VSHN – The DevOps Company
    Application-Aware Backups
    [...]

    template:

    metadata:

    labels:

    app: mariadb

    annotations:

    k8up.io/backupcommand: mysqldump -uroot -psecure --all-databases

    [...]
    14

    View full-size slide

  15. VSHN – The DevOps Company
    Regularly checks for data sanity using restic check
    Backup rotation with restic prune
    Backup archival on a dedicated location (for example AWS Glacier)
    Prometheus metrics (lots of!)
    Webhooks with backup information
    Other Features
    15

    View full-size slide

  16. VSHN – The DevOps Company
    List of available snapshots in Kubernetes resources for easy restore
    Restore via the new K8up CLI
    Support for backing up RWO PVCs
    Internal rewrites of queueing
    Newest Features
    16

    View full-size slide

  17. VSHN – The DevOps Company
    Onboard more maintainers from other companies than VSHN
    Increase adoption to gain traction for the project
    Progress stability and reliability to be rock-solid and a fully reliable
    data protection tool
    Fix annoying bugs which are being discovered while the adoption
    rises
    Plans - Project
    17

    View full-size slide

  18. VSHN – The DevOps Company
    Renew operator patterns to be on-par with current best practices
    (e.g. replace controller configuration via environment variables
    with a configuration CRD)
    Use native Kubernetes CronJobs for scheduling backup jobs,
    rather than using our home-grown solution
    Refresh Restic repository configuration
    Further enhance the newly built CLI tool
    Support multiple backups per namespace
    Plans - Features
    18

    View full-size slide

  19. VSHN – The DevOps Company
    Tobias Brunner, CTO –
    VSHN AG – Neugasse 10 – CH-8005 Zürich – +41 44 545 53 00 – –
    Thanks!
    [email protected]
    vshn.ch [email protected]
    19

    View full-size slide