Persistent Linux Storage software: LINBIT SDS + LINSTOR

1. Resilient and Fast Persistent Container Storage Leveraging Linux’s Storage Functionalities Philipp Reisner, CEO LINBIT 1
2. 2 COMPANY OVERVIEW REFERENCES • Developer of DRBD • 100% founder owned • Offices in Europe and US • Team of 30 highly experienced Linux experts • Partner in Japan TECHNOLOGY OVERVIEW LINBIT - the company behind it
3. 3 Linux Storage Gems LVM, RAID, SSD cache tiers, deduplication, targets & initiators
4. 4 Linbit Software defined Storage - Linbit HA NFS / CIFS / iSCSI KVM / VMWare /Xen Databases Fileservers Webservers NagiosXI Messaging (MQ) - Container-native OpenShift Kubernetes Docker Cloud-native OpenNebula OpenStack Proxmox VE
5. 5 Linux's LVM • based on device mapper • original objects • PVs, VGs, LVs, snapshots • LVs can scatter over PVs in multiple segments • thinlv • thinpools = LVs • thin LVs live in thinpools • multiple snapshots became efficient!
6. 6 Linux's LVM Linux already provides several storage gems: LVM• RAID• SSD cache tiers• De-duplication• Targets & initiators

7. 7 Linux's RAID • original MD code • mdadm command • Raid Levels: 0,1,4,5,6,10 • Now available in LVM as well • device mapper interface for MD code • do not call it ‘dmraid’; that is software for hardware fake-raid • lvcreate --type raid6 --size 100G VG_name
8. 8 SSD cache for HDD • dm-cache • device mapper module • accessible via LVM tools • bcache • generic Linux block device • slightly ahead in the performance game
9. 9 Linux’s DeDupe • Virtual Data Optimizer (VDO) since RHEL 7.5 • Red hat acquired Permabit and is GPLing VDO • Linux upstreaming is in preparation • in-line data deduplication • kernel part is a device mapper module • indexing service runs in user-space • async or synchronous writeback • Recommended to be used below LVM
10. Linux LVM Based on device mapper• Original objects•
PVs, VGs, LVs, snapshots• LVs can scatter over PVs in multiple segments• thinlv•
thinpools = LVs• Thin LVs live in thinpools• Multiple snapshots are efficient!•

11. 11 ZFS on Linux • Ubuntu eco-system only • has its own • logic volume manager (zVols) • thin provisioning • RAID (RAIDz) • caching for SSDs (ZIL, SLOG) • and a file system!
12. dm-cache• device mapper module •accessible via LVM tools• bcache•
generic Linux block device • slightly ahead in the performance game

13. 13 LINUX Dedupe: Virtual Data Optimizer (VDO) since RHEL 7.5•
Red hat acquired Permabit and is GPLing VDO• Linux upstreaming is in preparation•
In-line data deduplication• Kernel part is a device mapper module

14. 14 Targets and Initiators: Open-ISCSI initiator
Ietd, STGT, SCST
mostly historical IO iSCSI• iSER, SRP, FC, FCoE, SCSI pass through, block IO, file IO, user-specific-IO• NVMe-OF, target & initiator

15. Drbd Main Line Linux kernel 1000’s of Nodes
Up to 32 Synchronous or async replicas per volume • Automatic partial resync after connection outage• Multiple resources per node possible (1000s)• Diskless nodes•
Intentional diskless (no change tracking bitmap)• Disks can fail
Reliable• A node knows the version of the data is exposes• Checksum-based verify & resync• Split brain detection & resolution policies• Fencing Quorum•
Dual Primary for live migration of VMs only!
16. 16 DRBD – up to 32 replicas • each may be synchronous or async
17. 17 DRBD – Diskless nodes • intentional diskless (no change tracking bitmap) • disks can fail
18. 18 DRBD - more about • a node knows the version of the data is exposes • automatic partial resync after connection outage • checksum-based verify & resync • split brain detection & resolution policies • fencing • quorum • multiple resouces per node possible (1000s) • dual Primary for live migration of VMs only!
19. Controls LVM/ZFS• Snapshots• Thin •Multiple VGs
•For caching SSDs•Different pools•Controls DRBD

20. LINSTOR features complete
•Snapshot Support •Multiple Sites with DRBD Proxy •Swordfish API •Access via NVMe-oF•Scheduler Support •OpenShift •Kubernetes •OpenStack •OpenNebula
•ProxmoxVE •Cloud Platform Support •AWS •Google Cloud •IBM Cloud •Azure

LINSTOR Road map North bound drivers
•Autoplace policies as LINSTOR objects (Q2 2019)
•Management of PMEM/NVDIMM storage (Q2 2019)
•DRBD: Erasure Coding (Raid 5 support) (Q4 2019)

Transcript

1. Software Defined Storage (SDS) EFFICIENT AVAILABLE ON ALWAYS
2. The obvious approach to enterprise grade SDS
3. Partners References Global Presence Fast, Reliable & Agnostic HA DR SDS Open Source In Linux Kernel LINBIT at a Glance
4. LINBIT Software-Defined Storage Highly reliable Cost effective Easy to provision Easy to scale LINBIT SDS LINBIT HA Must be NFS / CIFS / iSCSI KVM / VMWare /Xen Databases Fileservers Webservers NagiosXI Messaging (MQ) Container-native OpenShift Kubernetes Docker Cloud-native OpenNebula OpenStack Proxmox VE
5. The Challenge or Scheduler • • • • • • • • • • • • • • App App App App
6. OS-Based Storage Technology • Linux already provides several storage gems: • LVM • RAID • SSD cache tiers • De-duplication • Targets & initiators Native Storage Management Capabilities
7. Cloud Storage
8. Container Storage
9. Virtualization Storage
10. Capabilities Linux LVM LVM • Based on device mapper • Original objects • PVs, VGs, LVs, snapshots • LVs can scatter over PVs in multiple segments • thinlv • thinpools = LVs • Thin LVs live in thinpools • Multiple snapshots are efficient!
11. Capabilities Linux RAID RAID • Original MD code • mdadm command • Raid Levels: 0,1,4,5,6,10 • Now available in LVM as well • Device mapper interface for MD code • Do not call it ‘dmraid’; that is software for hardware fake-raid • lvcreate --type raid6 --size 100G VG_name RAID1 A4 A3 A2 A1 A4 A3 A2 A1
12. SSD Cache for HDD • dm-cache • device mapper module • accessible via LVM tools • bcache • generic Linux block device • slightly ahead in the performance game
13. Linux’s DeDupe • Virtual Data Optimizer (VDO) since RHEL 7.5 • Red hat acquired Permabit and is GPLing VDO • Linux upstreaming is in preparation • In-line data deduplication • Kernel part is a device mapper module • Indexing service runs in user-space • Asynch or synchronous writeback • Recommended to be used below LVM
14. Capabilities Targets & initiators Targets & Initiators • Open-ISCSI initiator • Ietd, STGT, SCST • mostly historical • LIO • iSCSI, iSER, SRP, FC, FCoE • SCSI pass through, block IO, file IO, user- specific-IO • NVMe-OF • target & initiator
15. ZFS on Linux • Ubuntu eco-system only • Has its own • logic volume manager (zVols) • thin provisioning • RAID (RAIDz) • caching for SSDs (ZIL, SLOG) • and a file system!
16. Capabilities DRBD DRBD – Mainline Linux Kernel • 1000’s of Nodes • Up to 32 Synchronous or async replicas per volume • Automatic partial resync after connection outage • Multiple resources per node possible (1000s) • Diskless nodes • Intentional diskless (no change tracking bitmap) • Disks can fail • Reliable • A node knows the version of the data is exposes • Checksum-based verify & resync • Split brain detection & resolution policies • Fencing • Quorum • Dual Primary for live migration of VMs only! •
17. LINBIT SDS Architecture • Stacked clusters vs Mesh clusters • Clusters larger than 4 nodes drbdctrl A B drbdctrl A B D drbdctrl E B C D Satellite E C D
18. Capabilities à Products à Solution
19. Goals Approach LINSTOR - Goals Build storage from generic (x86) nodes • Serve SDS consumers (OpenStack Cinder, Kubernetes, Custom platforms) • Allow muliti-tenancy • Enable multiple Deployment architectures • Distinct storage nodes • Hyperconverged with hypervisors / container hosts • Don’t recreate the wheel • Use existing Linux storage components
20. Capabilities LINSTOR LINSTOR • Controls LVM/ZFS • Snapshots • Thin • Multiple VGs • For caching SSDs • Different pools • Controls DRBD
21. Block Storage Kubernetes Scheduler • • • • • • • • • • • • • • volume volume volume volume volume Scheduler volume diskless diskless volume volume volume volume or
22. Complete Roadmap LINSTOR Features • Snapshot Support • Multiple Sites with DRBD Proxy • Swordfish API • Access via NVMe-oF • Scheduler Support • OpenShift • Kubernetes • OpenStack • OpenNebula • ProxmoxVE • Cloud Platform Support • AWS • Google Cloud • IBM Cloud • Azure • LINSTOR: North bound drivers • XenServer (In Progress) • Multi-user capability (Q1 2019) • Encrypted communication (Q1 2019) • Decoupling of LINSTOR and DRBD (Q1 2019) • Autoplace policies as LINSTOR objects (Q2 2019) • Management of PMEM/NVDIMM storage (Q2 2019) • Rest API (Q2 2019) • DRBD: Erasure Coding (Raid 5 support) • October 2019
23. Resources High Availability Disaster Recovery Software-Defined Storage • https://www.linbit.com/en/re sources/documentation/ • https://www.linbit.com/en/dr bd-community/drbd- download/ • Video Demo (6 minutes) https://www.youtube.com/w atch?v=Sf0lPCllDWk • Blog post with LINSTOR + LINBIT DR Video demo: https://www.linbit.com/en/d emo-extending-linstor- managed-drbd-volume-linbit- dr-node/ • Container Storage: https://www.linbit.com/en/li nbit-sds-container-storage/ • Private Cloud: https://www.linbit.com/en/li nbit-sds-private-cloud/ • Public Cloud: https://www.linbit.com/en/li nbit-sds-public-cloud/ • LINSTOR: https://www.linbit.com/en/li nstor/