Getting started with MemoryDB for Redis

Redis is one of the most popular open-source technologies out there - in fact, it's been crowned the "Most Loved" database by Stack Overflow’s developer survey (2021) for the fifth year in a row! This session is all about using Redis on AWS as a primary, durable database. You will get an overview of Amazon MemoryDB for Redis, learn how to get started along with practical examples and hands-on demos!

Transcript

1. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. I ❤ Redis
2. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Redis: One of THE most ❤ databases out there!
3. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon MemoryDB for Redis Getting Started Abhishek Gupta Principal Developer Advocate Amazon Web Services abhi_tweeter abhirockzz
4. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda • Overview: Amazon MemoryDB for Redis • Getting started, quickly! • MemoryDB superpowers • Demos • Q&A
5. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Web and mobile User data stores, session management, geospatial indexing, chat and message queues Retail Customer profiles, inventory tracking, fulfillment Gaming Player data stores, session history, leaderboards Banking and finance User transactions, fraud detection Media and entertainment User data stores, real-time streaming IoT Streaming device data, operational insights Redis use cases Microservices Highly scalable, performant, simple to complex modularity Customer 360 Profiles, history, preferences, contacts, likes/dislikes, risk category
6. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Redis data structures and APIs Data type Description Example use cases Associated commands String Sequence of bytes Fast data access APPEND,GET,SET,INCR,DECR,GETSET... List* A list of strings Work queue LSET,LLEN,LPUSH,LPOP,LTRIM,RPOP... Set* Nonrepeating, unordered collection of strings Cardinality SADD,SCARD,SDIFF,SUNION,SINTER,SMEMBERS... Sorted set* Nonrepeating, ordered collection of strings Leaderboards ZADD,ZCARD,ZCOUNT,ZRANK,ZSCORE... Hash* Map of key-value pairs Objects HGET,HGETALL,HKEYS,HVALS,HMSET,HMGET... Streams Log data structure Streaming XADD,XRANGE,XREAD,XACK,XCLAIM,XLEN... Geospatial Longitude-/latitude-based entries Maps, "nearby" GEOADD,GEODIST,GEOPOS,GEORADIUS... Bitmaps Special usage of string type GETBIT,BITCOUNT,SETBIT,SETRANGE,GETRANGE... HyperLogLogs Special usage of string type PFADD,PFCOUNT,PFMERGE
7. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Difficult to scale Online scaling can be error prone, replication performance needs to be monitored during scaling operations, data loss can occur if issues arise. Expensive Invest in people, processes, hardware, and software Hard to make robust Many in-memory databases do not have data durability, which can lead to data loss during outages or failover. Also, need to implement fast error detection and remediation. Difficult to manage Manage server provisioning, software patching, setup, configuration, and backups Challenges with Self Managed In-memory datastores
8. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hello, MemoryDB for Redis! Redis-compatible, durable, in-memory database for ultra-fast performance
9. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Durability and high availability Multi-AZ transactional log for durability and replicas for high availability Redis compatibility Flexible and friendly Redis APIs and data structures Fully managed AWS-managed hardware and software setup, configuration, monitoring, and snapshots Ultra-fast performance Lowest latency database offered by AWS Security Amazon VPC, encryption at-rest and in-transit, Access Control Lists (ACLs) Amazon MemoryDB for Redis Redis-compatible, in-memory database with Multi-AZ durability High scalability Up to 500 nodes and 128 TB of in-memory storage per cluster (with 1 replica per shard)
10. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Core Components Replica Node Primary Node Replica Node Max 5 replicas Shard Replica Node Primary Node Replica Node Max 5 replicas Shard Cluster
11. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Scalability Scale out in minutes Scale up in minutes Scale reads in minutes Scale up to 500 shards Scale from 13 to 419 GiB of RAM per node Scale up to 5 replicas
12. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Availability Zone B Availability Zone C Redis cluster Availability Zone A P R Slots 0–5454 Slots 10910–16363 R Slots 5455–10909 R R Slots 0–5454 Slots 10910–16363 P Slots 5455–10909 R P Slots 0–5454 Slots 10910–16363 R Slots 5455–10909 Shards, primaries, and read replicas Example 3 shards 2 replicas per shard Multi-AZ S S S Multi-AZ High availability architecture • Add shards to scale reads/writes, increase in-memory capacity • Add replicas to scale reads, increase availability
13. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Use open-source cluster-aware Redis client Authenticate with ACLs Use open-source Redis 6.2 API commands U S E E X I S T I N G O P E N - S O U R C E R E D I S C L U S T E R - A W A R E C L I E N T S – P Y T H O N E X A M P L E Getting connected to MemoryDB
14. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Getting started with MemoryDB, the easy way!
15. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://itnext.io/getting-started-with-redis-on-aws-the-easy-way-11afe9717dc2
16. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. MemoryDB superpowers!
17. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-AZ Transaction Log • All writes committed to Multi-AZ transactional log • Guaranteed delivery of writes to replicas Amazon MemoryDB for Redis MemoryDB cluster
18. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Data Durability and HA Redis Client SET A 100 ACK SET A 100 SET A 100 ACK SET A 100 Amazon MemoryDB for Redis Cluster Availability Zone A Primary Availability Zone B Replica Availability Zone C Replica Synchronous Asynchronous Multi-AZ Transaction Log Acknowledgement sent to client only after transaction is written. Data is replicated via the transaction log and consumed by replicas. Guaranteed delivery of writes to replicas
19. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Strong read consistency from primary GET A(nil) Data Consistency Redis Client SET A 100 ACK SET A 100 SET A 100 ACK SET A 100 Amazon MemoryDB for Redis Cluster Availability Zone A Primary Availability Zone B Replica Availability Zone C Replica Multi-AZ Transaction Log GET A(100) GET A(100) Eventual read consistency from replicas REMEMBER: Data is always in memory, so reads are super fast
20. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Primary Handling Failover Redis Client Amazon MemoryDB for Redis Cluster Availability Zone A Primary Availability Zone B Replica Availability Zone C Replica Synchronous Asynchronous Multi-AZ Transaction Log SET A,B,C Roll-up ACK x3 SET A,B,C SET A SET B SET C ACK x3 SET A SET B SET C Replica reads all transactions from the log, then is promoted to primary Replicas continue to consume from log asynchronously Client is now redirected to newly promoted primary for all writes
21. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Primary Handling Failover Redis Client Amazon MemoryDB for Redis Cluster Availability Zone A Availability Zone B Availability Zone C Replica Multi-AZ Transaction Log Zero data loss during failover! Replica
22. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. MemoryDB in action!
23. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Leaderboard app with AWS Lambda
24. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://itnext.io/build-a-twitter-leaderboard-app-with-redis-and-aws-lambda-part-1- 670e7a8c6a91
25. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Web apps with AWS App Runner
26. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://abhishek1987.medium.com/build-cloud-native-go-apps-with-aws-app-runner-redis-and-aws-cdk- e1bd7ba6ddf3
27. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Microservices with Kubernetes
28. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Best practices • High available (HA) setup § Multi-AZ – typically across 3 AZs § 3 shards and 1 primary 2 replicas (3 nodes) per shard • Redis Cluster? Pick the right client, configure for read/write • Connection pool, Pipeline • Timeouts – client and server-side • Monitoring
29. Thank you! © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Abhishek Gupta Principal Developer Advocate Amazon Web Services abhi_tweeter abhirockzz https://eventbox.dev/survey/IKIEG9U