Redis Labs Technology - Jan 19, 2017

Transcript

1. 1 Redis Labs proprietary & confidential information Redis Labs proprietary & confidential information Home of Redis Redis Labs - Redise Technology
2. 2 Redis Labs proprietary & confidential information Open Source & Enterprise Technology Fully compatible with open source commands & data types Enterprise Layer Open Source Layer Redise Node
3. 3 Redis Labs proprietary & confidential information Redise Node - Main Components REST API Cluster Manager Zero Latency Proxy Standard OSS Redis Servers
4. 4 Redis Labs proprietary & confidential information Redise Node - Main Components (2) • Requests Forwarding • Cluster hashing • Redis acceleration • Memcached support • Based-on cut through architecture • Provisioning • Watchdogs Failover • Resource management • Shard migration • Re-sharding • Auto-scaling • Re-balancing • Provisionin g • Stats • Monitoring Zero-Latency Proxy Cluster Manager RESTful API
5. 5 Redis Labs proprietary & confidential information Shared Nothing Cluster Architecture Uneven number of nodes Distributed Proxies Single or Multiple endpoints
6. 6 Redis Labs proprietary & confidential information Data Path is Decoupled From Cluster Management Cluster Management Path Proxies Node Watchdog Cluster Watchdog Node 1 Node 2 Node N (uneven number) … Redis Shards Data Path
7. 7 Redis Labs proprietary & confidential information Database is sharded by the Cluster Manager. Each shard runs on a separate process. Optimized Resource Utilization 1 core 2 cores N cores
8. 8 Redis Labs proprietary & confidential information Multi Tenancy Node 1 Node 2 Node 3 Database A shards Database B shards Database C shards
9. 9 Redis Labs proprietary & confidential information How We Achieve Infinite Scalability Useful for high throughput databases. Scale up by sharding
10. 10 Redis Labs proprietary & confidential information How We Achieve Infinite Scalability (2) Scale out Rebalancing Resharding Scale Out, Rebalancing & Resharding
11. 11 Redis Labs proprietary & confidential information How is Scaling Managed Redise Cloud Redise Pack • Fully- automated • GUI • CLI • API
12. 12 Redis Labs proprietary & confidential information Replica shards (slaves) can be in the same / different rack / datacenter. How We Achieve High-Availability M Master S Slave M 1 S2 Node 1 Node 3 Node 2 M 2 S1 Shard Replication
13. 13 Redis Labs proprietary & confidential information Auto-failover Redis Failure How We Achieve High-Availability (2) P P P P P M S M M S S M 1 2 2 1 2 1 2 1 Slave sync. Re-balancing Redis 1 fails Redis 1 recovers P P P
14. 14 Redis Labs proprietary & confidential information Auto-failover How We Achieve High-Availability (3) P M S S M S M S M 1 2 2 1 2 1 2 1 Slave sync. Re-balancing P P P Proxy fails Proxy recovers P P Proxy Failure
15. 15 Redis Labs proprietary & confidential information How We Achieve High-Availability (4) Auto-failover M S M M S 1 2 2 1 2 1 Slave sync. Node fails Node recovers P P P P Node Failure
16. 16 Redis Labs proprietary & confidential information How is Failure Recovery Managed Auto-failover Replace faulty node • Redise Cloud Fully-automated • Redise Pack Fully-automated • Redise Cloud Fully-automated • Redise Pack Thru simple GUI, CLI or API commands
17. 17 Redis Labs proprietary & confidential information Multiple, not always persistent connections How We Improve Performance High overhead in connection setup and teardown on Redis side. Application Open-source Redis
18. 18 Redis Labs proprietary & confidential information Few persistent connections Redis Labs No connection setup and teardown overhead on Redis side. Application How We Improve Performance (2) Proxy
19. 19 Redis Labs proprietary & confidential information Application How We Improve Performance (3) Proxy 1 2 3 4 5 6 7 1 2 3 4 5 6 7 Multiplexing & pipelining Redis Labs
20. 20 Redis Labs proprietary & confidential information Data Persistence and Replication Improvements Ephemeral Storage Network Attached Storage Use non-ephemeral storage in order to avoid data loss in complete outage situations M S Disk-based Replication Diskless Replication
21. 21 Redis Labs proprietary & confidential information Data Persistence Improvements Slow disk performance does not affect database performance Open-source Redis Slave writes to disk Redise M S Slow Slow M S Fast Slow
22. 22 Redis Labs proprietary & confidential information File system enhancements • Optimal AOF rewrites frequency • Eliminate Redis blocking on high ‘write’ rate • Optimal chunk size Slow Fast Data Persistence Improvements (2) Open-source Redis Redise
23. 23 Redis Labs proprietary & confidential information • Compression technology contributes to minimal replication gap • Can replicate from any type to any type (standard Redis, HA Redis, Redis Cluster, HA Redis Cluster) • Supports multiple sources to a single destination How We Replicate Over WAN S3 M1 S1 M2 S2 M3 S3 M1 S1 M2 S2 M3 WAN Compression
24. 24 Redis Labs proprietary & confidential information RLEC Requires Minimal Ops Simple cluster setup & management Drill-down metrics Seamless software upgrade • Join node • Replace node • Offline node • Remove node • Rebalancing • Per cluster • Per node • Per database • Per shard
25. 25 Redis Labs proprietary & confidential information How We Run Redise Flash • On RAM: Redis dictionary; dataset keys; hot values • On Flash: cold values • Existing data-persistence mechanisms are maintained (Flash is only used as a slow RAM) • Non-blocking, asynchronous Redis when accessing Flash • Multi-threaded architecture accelerates Flash access RAM Flash (Slow RAM)
26. 26 Redis Labs proprietary & confidential information Strong Consistency Proxy Maste r Stora ge Redise Cluster 2 7 3 6 1 8 4 5 Applicati on Slave
27. 27 Redis Labs proprietary & confidential information Weak Consistency Proxy Maste r Stora ge 2 3 5 6 1 4 7 8 Applicati on Slave Redise Cluster
28. 28 Redis Labs proprietary & confidential information Cluster Manager RESTful API Deployment—Simple Redis Application Zero-Latency Proxy
29. 29 Redis Labs proprietary & confidential information Cluster Manager RESTful API Application Zero-Latency Proxy Deployment—H/A Redis Mast er Slave -1 Slave -2
30. 30 Redis Labs proprietary & confidential information Deployment—Redis Cluster M1 M3 M2 M5 M4 Cluster Manager RESTful API Application Zero-Latency Proxy
31. 31 Redis Labs proprietary & confidential information Deployment—H/A Redis Cluster M1 M3 M2 M5 M4 S4 S3 S2 S1 S5 Cluster Manager RESTful API Application Zero-Latency Proxy
32. 32 Redis Labs proprietary & confidential information Redis Labs proprietary & confidential information Home of Redis Thank You!