StorPool - IT Press Tour Dec 2020

Transcript

1. StorPool Storage @ IT Press Tour 2020 ( ITPT #37) #ITPT37 | @storpool
2. Boyan Ivanov • CEO & Co-founder at StorPool • Multi-industry experience in companies big and small • Serial Entrepreneur https://www.linkedin.com/in/boyanivanov/ @bhivanov Today’s Presenters
3. Boyan Krosnov • Chief of Product & Co-founder at StorPool • 20+ years in ISPs, SDN, SDS • IT Infrastructure with a focus on invention, performance & efficiency https://www.linkedin.com/in/krosnov/ @bkrosnov Today’s Presenters
4. 4 1. Introduction to StorPool Storage 2. Architecture 3. Performance 4. Features and Capabilities 5. Use Cases & Case Studies 6. The Future Agenda
5. Introduction to StorPool
6. Software Defined Storage (SDS) Vendor of the Year at 2020 Storage Awards “The heated battle for the SDS vendor of the year award between the tech giant IBM and the storage market disruptor, StorPool, has ended with a well-deserved victory for the innovative scale-up company. IBM took the place of runner-up for the category and this showed us that modern business has become more accepting in the adoption of new-age technologies. ” Source: https://containerjournal.com/news/news-releases/storpool-named-software-defined-storage-sds-vendor-of-the-year-at-2020-storage-awards/
7. StorPool in a Nutshell Best-in-class Software-defined Scale-out Block Storage Platform SDS 2.0 – Feature Rich Shared Storage System Faster than Local SSD Purpose Built Fully Distributed System: own on-disk formats, protocols, quorum management, data management and more… Replace SAN, AFA and other SDS storage software
8. 1. A software company. Latest-generation block-storage software (SDS 2.0) a. Turning standard servers & network into high-end storage system b. Shared storage, faster than local SSD 2. Focus on Tier 0 / Tier 1 (primary, NVMe). At scale (100 TB+). 3. European vendor, since Nov 2011. 4. Unique company - developed from scratch with tremendous IP: a. Owns the storage stack: on-disk format, protocols, quorum, client, etc, etc. b. Modern, yet mature: in production for 8+ years; 20 major releases; Global spread of mission critical customers. c. Scaled up to a global vendor with customers on 4 continents. Solid, profitable & fast growing business. About StorPool (1)
9. About StorPool (2) 6. The best storage solution for demanding New-Age public & private clouds: a. Extremely robust, blazingly fast, seamlessly scalable b. Latest generation solution, cloud native: API driven, feature rich, SDS 2.0 c. Multi-stack support: KVM (OpenStack, K8s), VMware/Hyper-V, bare metal 7. Target customers - companies building demanding public & private clouds: a. Service providers / public clouds: IaaS, PaaS, SaaS, MSP, cloud & hosting b. Enterprises / private clouds: SME & larger enterprises 8. Typical use cases: DBs, VM disks, VDI, bare metal, containers, etc. 9. Replacing/upgrading high-end SAN, AFA or other storage software. PAYG. 10. A mix of direct and channel GTM (Go To Market) strategy
10. NVMe shared storage system with: 1. Latency: < 100 μs 2. Throughput: >1M IOPS per server. a. Scale-out: 10 servers = 10M IOPS @ ~ 100-300μs. 3. Powerful Data Management: API, end-to-end data integrity, self-healing, online everything, thin provisioning, snaps & clones, QoS, backup & DR, etc. 4. Fully managed: 24/7 Support; SLA; proactive monitoring & issue resolution 5. 5 Year NVMe TCO: < $0.04 /GB provisioned /month; < $0.002 /IOPS/month. a. Including: a) software, b) hardware and c) 24/7 support, SLA and Fully Managed Services (FMS). Best-in-class Block-storage SDS - Example:
11. The Storage Pyramid - Use-cases and Market Segmentation by StorPool
12. Use Cases / Customers / Workloads Top Use Cases Selected Customers Workloads of Shared storage for VMs, DBs, VDI, bare metal Large SaaS / WebApps. Test/Dev and Production Refresh legacy IT systems to SDDC design
13. Architecture
14. • SDS - runs on Linux • No special hardware requirements ◦ Just NICs and NVMes ◦ Also supports SAS & SATA drives, controllers • Scale-out system, active/active multi-controller, pooling of capacity, pooling of performance • Copy-on-write on-disk format • End-to-end data integrity • 3-way replication Architecture 14 • Low latency <100 μs end-to-end (measured in VM) • High throughput - 1M IOPS per storage node (10M+ IOPS in a 10-node system) • Highly efficient storage protocol and implementation, Kernel bypass • API control. Common deployments are with 10000+ volumes, snapshots • Integrations with OpenStack, CloudStack, OpenNebula, Kubernetes, etc. • Snapshots and multi-site capabilities • Monitoring and fine-grained metrics, analytics
15. Architecture - Sergerated Shown (separate storage nodes)
16. Architecture: Hyper-Converged Infrastructure (HCI) Redundant Ethernet Network
17. StorPool - related to hardware and IT stacks
18. mgmt bridge iscsi target block client server beacon ↔ ↔ ↔ ↔ ↔ ↔ ↔ ↔ ↔ ↔ Components of the StorPool storage system - All StorPool • • • • • • •
20. storpool_server instance 1 CPU thread 2-4 GB RAM NIC storpool_server instance 1 CPU thread 2-4 GB RAM storpool_server instance 1 CPU thread 2-4 GB RAM storpool_block instance 1 CPU thread NVMe SSD NVMe SSD NVMe SSD NVMe SSD NVMe SSD NVMe SSD KVM Virtual Machine KVM Virtual Machine • Highly scalable & efficient parallel architecture • Scales up in each storage node & out with multiple nodes 25GbE . . . 25GbE 20
21. Components of the StorPool Storage System → → → → ↔ ↔ ↔ ↔ ↔ ↔
22. Aggregate Analytics / Metrics Collection Done by StorPool
23. Aggregate Analytics / Metrics Collection Done by StorPool
24. Aggregate Analytics / Metrics Collection Done by StorPool More on metrics & monitoring: https://storpool.com/blog/storpool-storage-monitoring-and-analytics
25. UI/Dashboard StorPool GUI
26. Performance
27. • Storage performance drives application performance • Applications, running on seemingly the same infrastructure can have orders of magnitude difference in performance • Applications perform measurably better on StorPool-powered clouds: ◦ up to 2.5 times better than the second-best public cloud; ◦ more than 10 times better than on other leading clouds, in some cases WHY?! Storage Performance = Application Performance 27
28. 1. Top performance, Top availability, Top support 2. Adopting best-in-class technology allows better business metrics (profit margin, TCO, ROI) 3. StorPool manages the storage layer, so top talent at our customers can grow their business How StorPool-powered Clouds Excell 28
29. * - ramdisk used to reduce usable RAM to 16 GB VMs and Block Storage Provider Service name $/GiB/month IOPS limit Size of volume [GiB] Monthly cost Katapult Shared disk NVMe $0.15 unlimited 1024 $154 StorPool Private Cloud 3 NVMe unlimited 500 n/a AWS EBS gp2 $0.10 3,072 1024 $102 Google Cloud SSD Persistent Disk $0.17 15,000 1024 $174 Google Cloud Balanced Persistent Disk $0.10 6,144 1024 $102 Microsoft Azure Premium SSD $0.12 3,500 1024 $123 Digital Ocean Block Storage $0.10 10,000 1024 $102 Provider Instance name region monthly cost (with 12 month commitment) vCPUs RAM Katapult ROCK-24 London $120 8 24GB* StorPool Lab one-8vcpu-16gb Sofia n/a 8 16GB AWS Compute optimized: c5.2xlarge us-east-2 $245 8 16GB Google Cloud General purpose: n2-8vcpu-16gb us-central1 $197 8 32GB* Microsoft Azure Compute optimized: Standard_F8s_v2 - 8 vcpus, 16 GiB memory East US $235 8 16GB Digital Ocean CPU Optimized Droplet: 16GB sfo2 $160 8 16GB 29
30. • Download the full report ◦ https://storpool.com/public-cloud-performance-measure ment-report Benchmarks 30
31. • Storage heavy, a little CPU ◦ FIO, rsync • Storage & CPU ◦ pgbench, sysbench • CPU, RAM* ◦ coremark • Network* * - future additions to our suite Tools Used 31
33. 33 Unfit for databases!
34. 34 Unfit for databases!
36. 2.5-3x lower latency at fixed load 36
37. 2.3x transactions with same CPU, RAM / same cost 37
38. pgbench small database 38
39. sysbench / MySQL 39
43. Features and Capabilities
44. CONFIDENTIAL Performance - Low Latency and Extreme Efficiency • NVMe-class performance where needed - for transaction databases • NVMe-class latency - sub-100 microsecond under moderate load • StorPool is faster than local SSD - aggregates the performance of all drives in a cluster and has very low latency overhead • Good performance fit for OLTP and OLAP (transactional databases and business intelligence) • Good fit for hyper-converged architectures due to extreme CPU efficiency - 250k IOPS per CPU core, 1M IOPS+ even from a small cluster
45. CONFIDENTIAL Scale-Оut Аrchitecture • Datacenter-scale (multi-PB) deployments • Support for simple flat Ethernet networks and routed datacenter-scale networks • Pooling of capacity and performance • HA by design • In-service software upgrades, hardware upgrades
46. CONFIDENTIAL Unique Reliable Data Storage Platform • End-to-end data integrity and protection -- particularly needed for high data or transaction intensive environments -- e,g, comprehensive 64-bit checksum across the software defined IT stack (https://storpool.com/features/end-to-end-data-integrity) • Data redundancy across storage nodes • Fine-grained copy-on-write - 4k granularity, efficient snapshots, clones, thin provisioning, changed block tracking • Fast mesh recovery
47. CONFIDENTIAL Data Мanagement • Online data mobility - between performance tiers, availability zones, between locations • LUN per virtual disk (modern architecture allows you LUN per virtual disk - hard for traditional SAN to dynamically provision LUNs and support 1000s of LUNs) • Snapshot-based backups ◦ integrated backup schedule/retention policies, or driven by customer's system ◦ in the same datacenter ◦ or in a different datacenter • Good fit for DR use-cases, business continuity ◦ Run from backup - run straight from backup data in DR location ◦ or Backup Streaming - start VM/container now and stream backup data to primary location in the background
48. CONFIDENTIAL API Control, DevOps • StorPool is very well-suited for "New IT" workflows which are characterized by scalability and automation • REST-full API • Integration with common container/VM orchestration systems - Kubernetes CSI, OpenStack Cinder,+3 more • Deploy storage nodes and hosts with Ansible roles and playbooks • Easy to build custom automation and integrations. Evidenced by many StorPool customers using the StorPool API for provisioning, billing, monitoring • Monitoring - hosted by StorPool or on-prem • Fine-grained metrics collection and analytics - deep visibility into the storage behavior of each container or VM, hosted by StorPool or on-prem
49. Use Cases & Case Studies
50. Typical Customers: • New-Age / Modern IT: Cloud native - automation, APIs, CMPs, DevOps & SRE • Public Cloud: IaaS/PaaS/SaaS, MSP, cloud, hosting, dedicated/bare metal servers, etc. • Private Cloud: Transitioning Enterprises as part of their private / hybrid clouds Use cases: • Flash shared storage for virtualized / container / bare metal environments • DBs, VM disks, VDI, large apps or SaaS, persistent K8s volumes, etc. • At scale: 100TB+ to 10s of PB of data, multiple datacenters • Convert legacy IT stacks to SDDC designs / consolidate multiple IT systems Typical Customers & Use Cases
51. The Project: • IT unification & consolidation: standardizing on a New-Age IT stack, company wide. • Reduce complexity and vendors: used HPE, IBM, EMC, NetApp, Cisco, VMware, Microsoft, and more. The goals: • Consolidate 10s of deployments from legacy platforms (acquired MSPs) to one new, future-proof platform. • Benefit from modern IT practices - automation, scalability, hardware independence. • Streamline business operations and improve profitability. The solution: • New-Age IT IaaS platform: Software-Defined, SDDC design. • The selected stack: KVM+OpenNebula, StorPool, Mellanox, Cumulus Linux. Case Study #1 - Dustin Group 51 Company Profile: • A publicly listed Managed Service Provider (MSP) company • Sales: €1.28 Billion (~$1.4B) • Employees: 1800+ • 6 locations in Europe • Grown through acquisitions • Had multiple IT stacks (VMware, Hyper-V, Linux) Case study: https://storpool.com/storpool_ case-study_dustin_2020
52. Case Study #2 - Amito 52 Company Profile: • UK-based Cloud and Managed Service Provider • Founded in 2000 • Presence in 28 data centers worldwide The Project: • "Тhe most difficult part of the infrastructure stack for high availability virtualized servers is storage. Amito was using quite a few different solutions, from big-name vendors to in house developed HA SANs based on ZFS. None of these solutions brought the expected results." The goals: • Implement a reliable and high-performance shared storage, across multiple data centers to support a broad range of applications. The solution: • “It’s probably not surprising that our shortlist didn’t contain any of the vendors with older legacy solutions – the approach to storage has moved on, and they have not all kept up.”, Ed Butler, CEO, Amito. • KVM, CloudStack, StorPool, supporting a large number of Windows and Linux virtual machines. Case study: https://storpool.com/amito-cas e-study-storpool
53. Case Study #3 - Katapult by Krystal 53 Company Profile: • Katapult by Krystal Hosting • One of the largest independent UK web hosting companies • Founded in 2002 and steadily grown over the last 18 years • Comprehensive portfolio of hosting, cloud and VPS services The Project: • Virtual Infrastructure as a Service platform built for extreme performance and simplicity with several high-performance NVMe storage clusters running on StorPool Storage. The goals: • A storage solution which keeps the promise of a modern SDS; high availability, extreme performance, a high level of flexibility and guaranteed data integrity. The solution: • Virtual Infrastructure as a Service platform, ensuring high performance, robust API, unique space savings features and the extremely high level of data protection offered from StorPool’s triple data replication. Case study: https://storpool.com/krystal-kat apult-case-study
54. Case Study #4 - High Traffic US Website 54 Company Profile: • High traffic website - Alexa top 300 in US. Top site in their category • 60m+ monthly views • Publicly traded company • Sales: ~$600m • Employees: 800+ The Project: • Persistent storage for DevOps workflow (build, test/dev, staging, CI) The goals: • Optimize, automate and accelerate software test/dev cycles The solution: • A fully integrated and automated IT stack, using cloud-native technologies: OpenNebula, StorPool. • A storage solution which keeps the promise of a modern SDS - extreme performance, a high level of flexibility, online everything, scalability. • End-to-end data integrity.
55. The Future
56. “Public Cloud will eat the World”. Not true. The Future is Hybrid. 1. IT infrastructure [both public and private] spending to reach $171.9 billion in 2024 2. Public cloud datacenters will account for 44.1% of above amount ($75.9B) 3. Private cloud will account for 19.5% ($33.4B) 4. Non-cloud IT (Traditional IT) will account for 36.4% ($62.6B) https://www.idc.com/getdoc.jsp?containerId=prUS46639820 44.1% 19.5% 36.4%
57. IT Infra / Cloud Commoditization Source: Adir Ron, Open Source Lead for Middle East & Africa at Microsoft (from 7 months ago) https://www.linkedin.com/posts/adirron_linuxonazure-progress-activity-6665502370795003905-DY1G/
58. Competition and Future Direction Source: https://gigaom.com/report/gigaom-radar-for-software-defined-storage/
59. StorPool Storage The best storage solution when building a New-Age Cloud www.storpool.com | info@storpool.com Thank you | Q&A