Karbor - An Ecosystem for Data Protection Providers

Karbor is a new OpenStack Big Tent project, focused on protecting the Data and Metadata that comprises an OpenStack-deployed Application against loss/damage (e.g. backup, replication) by providing a standard framework of APIs and services that allows vendors to provide plugins through a unified interface.
In this talk, we will present Karbor's vision of an ecosystem for data protection providers, using Karbor to orchestrate backup and restore providers.
We will also show how Karbor can be used inside a Disaster Recovery vendor's stack.

Transcript

1. Karbor An Ecosystem for Data Protection Providers
2. Why Data Protection? Power outage Hardware fault Network issue Human error Natural disaster
3. When is Data Protection Crucial? Pets • A mix of non-aligned entities • Probably monolithic • Data protection is crucial! Cattle • Template generated (HEAT, TOSCA, etc.) • Orchestrated, built for Cloud • Data protection is beneficial
4. Data protection is a set of measures taken to ensure data is reliably recoverable on demand Data Protection • Key component of business continuity / disaster recovery plans • Protection is measured at the time of recovery In our OpenStack world, what is data?
5. Data > Storage Storage (volume, image, share) Resource data (volume, network, server, etc) > Resource links and dependencies Resource metadata
6. Typical 3-tier Application
7. Underlying Resources and Dependencies Project Web Server 1 Web Server 2 Linux Image Web Net Security Grp. Security Grp. Router DB Net App Net App Server DB Server DB Image DB Volumes App Image
8. Protection Aspects • RPO (Recovery Point Objective): Maximum allowed “lost” data, measured in time • RTO (Recovery Time Objective): Maximum allowed time between a failure and recovery • Geo-location • Cost (There Ain't No Such Thing As A Free Lunch) RTO Protect Restore begins Restore ends RPO Protect
9. Solution Diversity Backup vs Replication Differential backup vs Incremental backup Storage array replication vs Guest assisted replication On-premise vs Off-premise
10. Karbor Highlights Pluggable: what, where, and how to protect Versatile: different protections for different use-cases Open architecture: support diverse vendor solutions Protect any OpenStack resource, as a Service
11. How can you benefit from Karbor? • Provide Data Protection - as a Service • Create tiers, SLA • Offer a variety of data protection solutions • Protect whole applications • Choose matching protection for groups of resources • Consistent snapshot (and recovery) of all the system Operators Users • Integrate existing or new data protection solutions • Fast time-to-market • Painless adaptation to OpenStack cloud environment Vendors
12. Karbor Components
13. Karbor Pluggable Architecture Bank Plugin Where to store Protectable Plugin What can be protected Protection Plugin How to protect
14. Bank Karbor Components & Flow Protection Plan Checkpoint Restore Protect Operation Restore Operation
15. Bank Bank is a pluggable generalization of an object storage Defines “where to protect” Responsible for: • Storing metadata and possibly data • Implementation defines replication strategy Example: OpenStack Swift, Ceph, Amazon S3
16. Checkpoint Checkpoint represents a restorable state of resources at a point in time • Stored in a Bank • Hold all the information (or references to the information) sufficient to restore the protected data
17. Restore Restore is the object representing the process of a checkpoint recovery Restore is likely to be parameterized - where to restore to, custom network configuration, etc.
18. Bank Karbor Components & Flow Protection Plan Checkpoint Restore Protect Operation Restore Operation Provider Resources (Protectables) Parameters
19. Protection Plan Protection plan is a recipe for creating a Checkpoint • Sufficient for creating a Checkpoint with no additional parameters Contains: • Resources - what to protect • Provider - how and where to protect • Parameters
20. Protectable Plugin Protectable is a type of resource (probably an OpenStack resource) Karbor can protect Defines “what can be protected” Responsible for: • How to find resources of this type • Dependency on other protectables Examples: Image, Volume, Server, Network, Share
21. Bank Karbor Components & Flow Protection Plan Checkpoint Restore Protect Operation Restore Operation Provider Resources (Protectables) Parameters Bank plugin Protection plugins
22. Protection Plugin Protection plugin is responsible for protecting and restoring a specific Protectable Defines “how to protect and restore a resource” Responsible for: • Actual implementation of protect and restore Example: backup an image into the bank (Swift, for example)
23. Protection Provider Required Plugins Cinder::Volume Glance::Image Nova::Server Neutron::Network Keystone::Project Bank Reference Provider Volume plugin Image plugin Server plugin Network plugin Project plugin Swift Bank plugin Vendor Provider Vendor Volume plugin Vendor Image plugin Vendor Server plugin Network plugin Project plugin S3 Bank plugin Volume plugins Image plugins Network plugins Server plugins Project plugins Bank plugins Available Plugins
24. Example: Pure OpenStack Protection Provider Required Plugins Cinder::Volume Glance::Image Nova::Server Neutron::Network Keystone::Project Bank Reference Provider Volume plugin: Cinder Backup Image plugin: Glance based Server plugin: Nova based Network plugin: Neutron based Project plugin: Keystone based Bank plugin: Swift
25. An Ecosystem of Providers
26. Before Karbor: Data Protection “Silos” Cloud User Backup System B Volume Server DR System Backup System A Volume Image Application Backup Backup System C Share Database
27. After Karbor: Ecosystem of Providers Cloud User Backup System B Volume Server DR System Backup System A Volume Image Application Backup Backup System C Share Database Karbor
28. v Protection Service DB Operation Engine Service Trigger Engine Bank Plugin Operation Workflow Time Event Providers Manager RPC Protectable Resource Type Checkpoint Restore API Service Plan API Protectable API Scheduled Operation API Providers API Checkpoint API Restoration API Karbor Architecture Trigger API Checkpoints Workflow Engine Resource Plugin Protection Plugin
29. Karbor Cross-site Recovery Protection Service Protection Service Bank Site A Site B Orchestrator API Service
30. Karbor Cross-site Recovery Bank Protection Service Protection Service Site A Site B Orchestrator API Service Protect
31. Karbor Cross-site Recovery Protection Service Protection Service Checkpoint Bank Site A Site B Orchestrator API Service
32. Karbor Cross-site Recovery Protection Service Protection Service Checkpoint Bank Site A Site B Orchestrator API Service Restore
33. Karbor Cross-site Recovery Checkpoint Bank Site A Protection Service Site B Orchestrator API Service Protection Service
34. Demo
35. Help Us Build Karbor openstack.org/wiki/Karbor