StorPool - IT Press Tour #45 - Paris, France - Sept. 2022

Transcript

1. StorPool Storage Presentation @ ITPT 45 2022-09-06
2. Session 1: Introduction to StorPool Boyan Ivanov, CEO
3. StorPool Storage in a Nutshell Best-in-class, Block-Storage Software that Powers the Most Demanding Apps Crazy-fast, Feature-rich, Linearly Scalable, Multi-protocol, Primary Storage Platform Built from scratch, re-engineered entire domains: Own On-disk Format, Block Protocol, Networking calyer, Quorum management, and so much more Replaces/upgrades AFA, high-end SANs and other storage software, when building large-scale and demanding IT infrastructure.
4. The fastest public cloud CONTINUES to run StorPool Source: https://storpool.com/wp-content/uploads/2022/01/StorPool-Public-Cloud-Performance-Measurement-Report_2022.pdf
5. StorPool on Amazon AWS
6. StorPool on Amazon AWS
7. 1. Software company, latest-generation block storage software (SDS) a. Turning standard servers & network into high-end storage system 2. Focus on primary flash storage, but covers multiple tiers + backup too 3. Not a Silicon Valley “startup” -- but a solid, robust, organically grown vendor 4. SDS 2.0 - fast, feature-rich & scalable (The Holy Grail, see below) 5. Unique company: developed from scratch with tremendous IP: a. Own the storage stack: on-disk format, protocol, quorum, client, etc, etc. b. Latest generation, yet mature: in production for 9+ years; 20 major releases; Global spread of mission critical customers. c. A solid, fast growing and profitable business Best-in-class block storage software (1)
8. Best-in-class block storage software (2) 6. The best storage solution for next-generation private & public clouds: a. Robust, fast, scalable b. Low TCO, high ROI c. Latest generation, yet proven and versatile 7. Target customers - companies building public & private clouds: a. Service providers / public clouds: IaaS, PaaS, MSP, cloud & hosting b. Enterprises / private clouds: SME & larger enterprises 8. Predominantly direct Go to Market strategy 9. Licensing is Pay Per Use in customer datacenter, based on volume of data (Cloud-like, Storage as a Service)
9. The Storage Pyramid - use-cases and market segmentation by StorPool ©
10. The Storage Pyramid - use-cases and market segmentation by StorPool © 34x diff.
11. Use Cases / Customers / Workloads Top Use Cases Selected Customers Workloads of Virtual Disks for VMs, DBs, VDI, bare metal aps Large SaaS / Web Apps Test/Dev and Production Upgrade legacy IT systems to SDDC design
12. The Hidden Gem - Getting More Visible “Storage Transformation Project of the Year”
13. Session 2: Block Storage Software Capabilities and Use-cases Boyan Krosnov, CTO
14. Agenda Capabilities & Use-cases for block storage • Performance • Data Management • Scalability
16. Performance with StorPool A modern primary storage system must be able to run demanding applications. This means millions of IOPS at NVMe-class latency. >1M IOPS per node. I.e. 3 servers = 3+ mln IOPS Writes at QD1 ~ 70 μs end-to-end (fio in VM) StorPool latency overhead is half the latency of NVMe devices. End-to-end latency approximately 1.5x local NVMe latency.
17. Performance with StorPool - how StorPool Achieves 5.6 Million Read/Write IOPS on a 12-node Cluster... How? • Parallel architecture that achieves maximum speed in two primary ways: ▪ Multiple server services perform write/read operations simultaneously ▪ Each server service works asynchronously, resulting in many parallel operations in flight • Write/read operations are performed on all drives in the storage system • StorPool Data Fastpathing removes “expensive” context switching • Every drive added to a StorPool cluster adds IOPS to the cluster, for all data • Maximum possible performance is maintained in all cases With StorPool, storage is no longer the bottleneck in the IT stack
18. The Fastest Public Cloud is Powered by StorPool https://storpool.com/wp-content/uploads/2022/01/StorPool-Public-Cloud-Performance-Measurement-Report_2022.pdf Pgbench Unfit for Databases
19. The Fastest Public Cloud is Powered by StorPool https://storpool.com/wp-content/uploads/2022/01/StorPool-Public-Cloud-Performance-Measurement-Report_2022.pdf Pgbench 2-3x lower latency at fixed load
20. The Fastest Public Cloud is Powered by StorPool https://storpool.com/wp-content/uploads/2022/01/StorPool-Public-Cloud-Performance-Measurement-Report_2022.pdf Pgbench 2.6-4x transactions with same CPU, RAM
21. Performance Use-cases 1. Public clouds recognize the difference between "general purpose" storage at $0.10/GB and storage suitable for OLTP (e.g. Amazon io2) 2. In private clouds storage latency is seen as a driver for overall application efficiency 3. Primary use cases - large/heavy web sites and apps, online games, betting, enterprise databases
22. Data Management with StorPool End-to-end data integrity protection 4k granularity - thin provisioning / reclaim - Redirect-on-write snapshots, clones - changed block tracking, incremental recovery and transfer Multi-site, multi-cluster - connect 2 or more StorPool clusters over public Internet - send snapshots between clusters for backup and DR - commonly 100TB backup once per hour - workload migration (including live) within a large DC and between DCs
23. Clones use-case #1
24. Clones use-case #2
25. Remote snapshots use-case #1 Snapshot-based backups in remote location • As service offered and billed to customer • Workload migration between locations for maintenance
26. Remote snapshots use-case #2
27. Scalability & Automation with StorPool Dynamic provisioning and API control Detailed metrics collection, monitoring. Scale-out (shared-nothing cluster) architecture Existing integrations with Kubernetes, OpenStack, OpenNebula, CloudStack & OnApp >1PB usable All-SSD & Hybrid clusters in production for years StorPool customers have multiple clusters per location and multiple locations
28. Scalability & Automation use-case • Ability to deploy storage nodes automatically is mandatory in all "Modern IT" "bare metal" IT teams. They have 100s or 1000s of identical servers which they assign different roles. • A good scale-out system capability enables a "river of hardware" with a new generation typically every 12-18 months. This ability is treasured even by smaller StorPool customers.
29. Integrations use-case • With a good cloud integration, a lot of the functionality of the storage system is exposed through the cloud platform to the user. Each virtual disk becomes a separate volume in StorPool and enables per-disk and per-VM data management functionality. • Some advanced cloud users of StorPool provide their customers with access to storage metrics per virtual disk.
31. Session 3: Redefining Software-Defined Storage
32. • Hardware vs Software • The history of SDS • The future of SDS Agenda
33. • Two distinct models of delivering block storage solutions • Storage Appliance (Array) model ◦ Inflexible solution - you are stuck with the solution you get ◦ Software/services tied to particular hardware unit - high replacement cost, low residual value, high migration cost ◦ Expensive upgrade options ◦ "Green*" (recurring fee) models are flawed ◦ "Standard" service level with warranty is low - 8x5 support Software vs Hardware
34. • The original vision: Deliver SAN functionality without the proprietary hardware • Early attempts were LeftHand Networks, ZFS/Nexenta, DataCore ◦ ZFS is still popular despite its aging architecture & implementation • Early SDS failed to deliver on the vision in a number of ways • In the meantime Storage Array "controllers" moved to x86 servers • The scale-out approach for primary storage systems was pioneered by a number of very different players around 2011. StorPool was there. • Lots of storage startups came and went. A lot of over-promising and under-delivering. • In 2022 "Software-defined storage" is (still) associated with low performance / low cost storage. Also with Object Storage. Software-defined storage - history of under-delivering
35. Software Storage solution + Standard Hardware (StorPool Storage) • Equivalent or better speed (in terms of latency, IOPS per workload) • Flexible solution - adapts to current and future use-cases • Monthly metered billing is the golden standard ◦ Perpetual license also supported. License not tied to hardware - low renewal/replacement cost • High service level - Managed Service, Hosted monitoring - default • Standard Intel/AMD server hardware - efficient "fleet" approach Software vs Hardware
36. • Scale-out architecture ◦ redundancy across storage nodes ◦ high availability ◦ simple and standard building block (server) • No special requirements ◦ no RDMA, specific NIC vendors, no niche network requirements ◦ no Optane DIMMs, NVDIMMs, Optane NVMes ◦ no DPUs, GPUs, etc ◦ no RAID controllers, LSI CacheVault ◦ Just an Intel/AMD CPU, a NIC and NVMe/SSD/HDD drives Software-defined Primary Storage Platform 2.0
37. Software-defined Primary Storage Platform 2.0 • SRE approach with Continuous Deployment / rolling upgrades • Operational Excellence: Highly efficient ops procedures, automation, useful metrics and monitoring, availability proven in production • Support HDDs, SATA and NVMe SSDs • Benefit from the latest and greatest technologies, optionally(!) • Support mixed clusters - multiple performance tiers, multiple generations of hardware • Support different storage protocols - NVMe/TCP, iSCSI, NFS, StorPool Block Protocol • … and great integrations with cloud management systems - OpenStack, Kubernetes + 3 more
38. Software-Defined Storage Primary Storage 2.0
39. Aggregate analytics / metrics collection done by StorPool Example Cluster
40. Aggregate analytics / metrics collection done by StorPool
41. Aggregate analytics / metrics collection done by StorPool More on metrics & monitoring: https://storpool.com/blog/storpool-storage-monitoring-and-analytics
42. UI/Dashboard StorPool GUI
43. UI/Dashboard StorPool GUI
44. Session 4: What's new in StorPool v20
45. 1. Added support for provisioning volumes using NVMe over Fabrics, TCP Transport - next-gen block storage protocol that works over standard Ethernet 2. Delivers high-speed, low-latency access to NVMe-based StorPool storage systems. 3. The implementation is software-only and does not require specialized hardware 4. NVMe/TCP targets are highly available - in the event of a node failure, StorPool fails over the targets on the failed storage node to a running node in the cluster. 5. Compared to iSCSI, saves CPU cycles and delivers higher performance on the same hardware. New capabilities in StorPool v20 release - NVMe/TCP (announced 2 weeks ago)
46. New capabilities in StorPool v20 release - StorPool on AWS (announced 2 weeks ago) 1. Specialized solution capable of delivering a lot of storage performance to individual compute instances. 2. Enables customers to migrate heavily-loaded monolithic workloads to AWS - this was not economically achievable with other technologies until now. 3. Leader in absolute IOPS/instance AND very good $/IOPS efficiency
47. New capabilities in StorPool v20 release - NFS on StorPool (announced 2 weeks ago) 1. Added support for running highly-available NFS Servers inside StorPool storage clusters for specific use cases. 2. NFS services delivered with StorPool are suitable for throughput-intensive file workloads shared among internal and external end-users (video rendering, video editing, heavily-loaded web applications) 3. The cumulative provisioned storage of all shares exposed from each NFS Server can be up to 50 TB. 4. The high availability of each NFS service with the proven resilience of the StorPool block storage layer - data is distributed across all nodes in the cluster, and StorPool maintains data and service integrity in case of hardware failures
48. Simultaneous Multi-Stack Support
49. Session 5: Use cases & Case studies
50. Typical Customers and Primary Use Cases Typical Customers: • Follow Modern Cloud Infrastructure Practices API-driven, Automated, DevOps, SREs, Cloud Management Platforms • Offer Public Cloud Services IaaS, PaaS, hosted SaaS, virtual private servers, ecommerce, shared hosting, desktop as a service, BC/DR • Offer Private Cloud Services Tailored solutions for SMBs and Enterprises - virtual private clouds, managed/hosted private clouds Primary Use Cases: • Media: SATA/SAS/NVMe SSD-based storage served to virtualized, container, and bare-metal environments • Workloads: Databases, VM disks, VDI, large apps or SaaS, video game hosting, persistent K8s volumes, etc • Scale: 100TB to 10s of PB stored across multiple data centers • Example Projects: Convert Legacy IT Stack to SDDC Design; Consolidate Мultiple IT systems to One Stack
51. Case Study #1 - Atos The Project: • Reduce vendor lock-in by replacing legacy Oracle storage appliances with an API-driven primary storage system. The goals: • Maintain service quality while optimizing prices. • Align storage layer pricing with the OpEx, pay-per-use model offered to end customers. • Remove maintenance windows. • Modernize platform’s hardware to be competitive. • Optimize platform life cycle and costing management. The solution: • StorPool storage system that serves the platform over iSCSI, resolves known storage bottlenecks, enables automation efforts, and simplifies platform lifecycle management. • All-NVMe setup with consistent <0.2ms front-end latency. Company Profile: • Publicly listed IT services and consulting company • Revenue: €11 Billion • Employees: 104,000+ • 216 locations globally • Infrastructure using traditional enterprise technology - appliances for both storage and compute Case study: https://storpool.com/news/ato s-success-story
52. Case Study #2 - Dustin The Project: • IT unification & consolidation: standardizing on a New-Age IT stack, company wide. • Reduce complexity and vendor lock-in: used HPE, IBM, EMC, NetApp, Cisco, VMware, Microsoft, and more. The goals: • Consolidate 10s of deployments from legacy platforms (acquired MSPs) to one new, future-proof platform. • Benefit from modern IT practices - automation, scalability, hardware independence. • Streamline business operations and improve profitability. The solution: • New-Age IT IaaS platform: Software-Defined, SDDC design. • The selected stack: KVM, StorPool, Mellanox, Cumulus Linux. Company Profile: • A publicly listed Managed Service Provider (MSP) company • Revenue: €1.28 Billion (~$1.4B) • Employees: 1800+ • 6 locations in Europe • Grown through acquisitions • Had multiple IT stacks (VMware, Hyper-V, Linux) Case study: https://storpool.com/storpool_ case-study_dustin_2020
53. The Project: • Virtual IaaS platform built for extreme performance and simplicity • Build the fastest public cloud using standard hardware The goals: • Modern linearly scalable storage solution • Unmatched performance • High availability • A high level of flexibility • Guaranteed data integrity The solution: • Tailored IaaS platform, focused on extreme performance • Automation and API-first approach • Extremely high level of data protection offered by StorPool’s end-to-end data integrity and 3x copies of data. Case Study #3 - Katapult Company Profile: • Katapult by Krystal Hosting • One of the largest independent UK web hosting companies • Founded in 2002 and steadily grown over the last 20 years • Comprehensive portfolio of hosting, cloud and VPS services Case study: https://storpool.com/krystal-kat apult-case-study
54. The ideal foundation for large-scale clouds running diverse, mission-critical workloads www.storpool.com | info@storpool.com Thank you and Q&A