Speedb - IT Press Tour Israel #43 April 2022

Transcript

1. Next Generation Data Engine as a service 1
2. 20 Years in management, commercialization and executive sales positions, leading global software technology companies to outstanding growth. Double academic degree in Mathematics & Computer Science CRO SQream, Infinidat, Spot, XIV Adi Gelvan Co-Founder & CEO 35 years in developing enterprise software technologies, data structures and algorithms Bsc in Math&CS Holds over 100 patents in storage software Chief Scientist Infinidat, EMC Hilik Yochai Co-Founder & CSO 25 years of building enterprise systems - specializing in scalable software architectures Bsc in Math&CS Holds over 20 patents in storage technologies Senior Architect Infinidat, Cadence, Verisity Mike Dorfman Co-Founder & CTO The Founders 2 | • Team: 18 Mostly algorithm developers & software engineers
3. Would you Drive a Brand-New Car with an Old Engine? | 2 The world revolves around data. However, existing database architectures are based on data engines that were not built for today’s massive, hyperscale data operations.
4. Our Vision Speedb is revolutionizing the key-value database market with a next-generation data engine that removes the capacity, scale and performance limitations of existing solutions. | 3
5. Metadata: The Silent Killer BEFORE CLOUD ANNO DATUM 1980 1990 2000 2010 2020 Structured Unstructured | 6 Big Data needs a revolution As unstructured data continues to grow
6. The Price of Scaling Scalability/ Management Performance drops as data grows Scaling done through sharding Performance CPU & RAM utilization overhead IO overhead Substantial performance degradation Short SSD lifespan due to extensive writes IO stalls Inter node communication delays Sharding management overhead | 8
7. Storage Engine Limitations Data size limitation (~100GB) High CPU usage due to write amplification issues IO hangs due to compaction issues Performance degradation starting at ±20GB Limitations on objects larger than 500KB Number of objects limited due to RAM consumption (Millions) | 3
8. KVS storage engines have become the key bottleneck of modern databases: under- performing their tasks, hardly scale, and highly fragmented across multiple use cases 8 | Our Opportunity Replacing the Weakest Link of Hyperscale Data Operations
9. Petabyte-scale datasets | Billions of objects at high performance | Low hardware requirements | Fully compliant with incumbent leader, RocksDB 9 | Our Opportunity We Created the First KVS Data Engine that Truly Matches the Needs of Hyperscale Data Operations
10. Seamless Integration, Easy to Use | 4 Speedb is a drop-in replacement for RocksDB allowing you to get much more, with less resources - and all without making any changes to your architecture or code.
11. The Next Generation Data Engine Speedb enables RocksDB users to achieve an order of magnitude higher capacity, scale, performance and cost savings without making any changes to applications and their underlying data infrastructure. Bigger Dataset 100X More OP/S 10X Less Resources 80% | 5
12. Key Benefits | 11 Take your hyperscale operations to the next level without making any changes to your existing data infrastructure. Optimize performance for user operations and improve customer experience by eliminating user-side stalls and minimizing user latency. Free developers from having to constantly deal with sharding, database tuning and other time-consuming operational tasks, so they can focus on delivering real business value. Vendor support services and bespoke customization to support stringent use-case specific requirements and achieve faster time-to-market.
13. Technology Highlights | 12 New compaction method that dramatically reduces write amplification for large scale LSM. Flow control redesign to eliminate spikes in user latency. Eliminate the need for sharding by allowing you to grow unlimited on a single instance while maintaining low memory consumption. Revolutionary indexing method that supports the storing of index together with data for extreme performance at scale.
14. Our Technology | 17 Speedb has developed a revolutionary Log-Structured Merge (LSM)-based key-value store that supports petabyte scaling of datasets with billions of objects while maintaining high performance and low hardware requirements. Our enterprise-grade solution utilizes a range of technological breakthroughs to meet the needs of hyperscale data operations.
15. | 18 Speedb Revolutionized the LSM compaction algorithm with an innovative multi dimensional compaction. By utilizing multi dimensional compaction, Speedb can reduce the WAF (write amplification factor) by more than 80% and enable fast writes even on large datasets while keeping a B-Tree like read performance. Our Technology
16. | 14 Reduce WAF from ~24 to ~4
17. | 19 Redesigned I/O and job schedulers Our Technology Our Innovative multi-dimensional compaction technology, coupled with redesigned IO and job schedulers, guarantees high, stable performance with no stalls
18. | 10 Stable Performance, No Stalls
19. | 19 Fill up the database up to 200M Objects, stops when finished Rocksdb Speedb 16000 14000 12000 10000 8000 6000 4000 2000 0 1 361 181 541 841 902 1081 1021 1082 1201 1262 1381 1501 1681 1681 2041 2221 2401 2581 2761 2941 3121 3301 IOPS Time (in seconds)
20. | 20 Perform seek operations while performing random write operation to simulate real workload Rocksdb Speedb 14000 12000 10000 8000 6000 4000 2000 0 1 361 181 541 721 901 1081 1261 1441 1621 1801 1981 2161 2341 2521 2701 2762 2881 2942 3061 3122 IOPS Time (in seconds)
21. | 22 95% write. 5% read Rocksdb Speedb 20000 18000 16000 14000 12000 10000 8000 6000 4000 2000 0 1 361 181 541 721 901 1081 1261 1441 1621 1801 1981 2161 2341 2462 2581 2642 2761 2822 2941 3002 IOPS Time (in seconds)
22. Cloud transformation B2V - Embedded extrapolation Redis on flash Kafka Streams Metadata Store B2B - Application specific MySQL Managed Live Archive IOT collection Application Developers Use Cases Spark Flink Serverless Application | 22 Key-Value- Store Microservices Data Streams
23. | 23 Use Case 1: Effective Storage Tiering Customer Profile Redis accelerates application response time by serving frequently needed data from an in-memory (RAM) cache. While RAM provides extremely fast read/write speeds, it is an expensive resource. To help customers scale their data in a more cost-effective manner, Redis Enterprise allows for the creation of Redis on Flash databases that extend RAM capacity with SSD and persistent memory to store significantly more data with fewer resources. Challenge Performance issues exacerbate as larger portions of the dataset move outside of memory to slower flash drives, resulting in latency and user-side stalls when running large datasets. Solution By narrowing down the performance gap from RAM to flash, Speedb enables Redis’ customers to allocate more data to flash and capitalize on cost and scalability benefits. Customer testimonial “Speedb is the only technology that was able to seamlessly replace RocksDB and provide the performance and scale boost we needed to support our largest Redis on Flash deployments without having to use special hardware or SSD drives.”
24. | 26 Use Case 2: Stream Processing at Scale Customer Profile XM Cyber is a breach and attack simulation software provider that helps identify vulnerabilities in clients’ security environments. Based on metadata such as OS information, network and registry configuration, XM Cyber formulates simulated cyber attacks that allow for identifying potential threats in real-time. Challenge Solution Customer testimonial “With Speedb, we were able to achieve 10X performance improvement with less memory resources.” • XM Cyber is using Flink for real- time metadata processing to identify vulnerabilities. • As the state grows, memory consumption increases, leading to performance and scalability issues. • Using Speedb, XM Cyber was able to significantly reduce memory consumption. • XM Cyber can now handle more metadata and more tasks on the same hardware while boosting read and write response times.
25. | 25 Additional Use Cases The client was using GraphDB based on RocksDB as the underlying storage engine for all data. As the amount of data increased, performance issues became more frequent. The client tried to address this problem by adding more memory and CPU resources and limit the scale of a single database in specific environments, thus giving up on functionality. By replacing RockDB with Speedb, the client was able to achieve improved performance with no scaling limitations and leverage the full functionality of the system. High performance at scale Hardware vendors today are innovating new and faster ways to store and retrieve data. While hardware level benchmarks are showing outstanding results, customers are hindered from getting the most value from these innovations due to lack of adequate software that can utilize them effectively. Speedb’s enterprise-grade solution was designed so it can be customized to capitalize on specific hardware innovations. Unlocking hardware innovation While data is stored on disks as files/blocks/objects, metadata is stored in-memory for fast retrieval but must still be consistent and persistent. Existing in-memory key-value stores have limited capacity and high CPU utilization and memory consumption due to high write amplification, which impacts their performance when dealing with large datasets. By replacing RocksDB with speedb on the metadata layer, the client was able to achieve continuous growth while improving performance. Metadata growth
26. Replace your RocksDB with an enterprise-grade data engine built from the ground up for IO-intensive workloads, with premium support and unprecedented cost-effectiveness. Conclusion | 12
27. Case Study | 8 Speedb is the only technology that was able to seamlessly replace RocksDB and provide the performance and scale boost we needed to support our largest Redis on Flash deployments, without the need to use special hardware or SSD drives “ ” - Yiftach Shoolman Founder and CTO
28. Redis on Speedb 800 600 400 200 0 Total writes to disk: RocksDB vs Speedb Speedb wrote 50% less while doing twice as much user operations Speedb Disk Writes Aggregate (GB) Rocksdb Disk Writes Aggregate (GB) 40k 30k 20k 10k 0 Speedb IOPS (i3.8xlarge) Rocksdb IOPS (i3.8xlarge) Speedb on i3.2xlarge (25% cost) vs RocksDB on i3.8xlarge | 28 Disk Writes OPS IOPS IOPS
29. Thank You! | 29