RedisConf18 presentation on how Redis can be leveraged to build lightweight microservices frameworks using your favorite programming language. The hydra microservice library is used as a case study. https://www.npmjs.com/package/hydra
• Carlos Justiniano • 2005 World Record in Distributed Computation • Chief Software Architect at Flywheel Sports • Redis since 2011 • @cjus on Twitter, Github • More about me at http://cjus.me About me
Flywheel Sports • Founded in 2010 • Featuring high-energy Indoor Stadium Cycling and Barre classes. • Performance tracking technology • ~1200 employees • 42 Studios across the U.S.
• Three studios in the Bay Area • One right here in SF • At Market St in the Financial District Flywheel Sports • Sign up for a free class using the offer code: FLYREDIS
Redis as distributed computing glue? • Each microservice binds to a Redis cluster • Presence by key / expiration • IPC via Pub / Sub • Routes via set stores • Microservice features minimal dependencies?
• Health and presence • Service discovery • Inter-service (P2P) communication • Load balancing and routing • Self registration with near zero configuration • Job queues Hydra Features
Your NodeJS Microservice Hydra module (npmjs.org) Other Node module Other Node module Other Node module Other Node module Your JS code Other Microservice Hydra module (npmjs.org) Other Node module Other Node module Other Node module Other Node module JS code Other Microservice Hydra module (npmjs.org) Other module Other module Other module Other module JS code
Redis key space • Prefix: allows for filtering Hydra vs non-Hydra keys • Service Name: filtering keys of a particular service type • Instance ID: filtering keys for a unique service instance • Type: classifies the type of use – i.e. purpose of key Prefix Service Name Instance ID Type * * * Service Name and Instance ID don’t apply in all situations
Example key hydra:service:user-svcs:636a68fb1a34493f8fc17fe77d5a1b4b:presence Prefix Service Name Instance ID Type Hydra:service user-svcs 636a68fb1a34493f8fc17fe77d5a1b4b presence
hydra:service:user-svcs:636a68fb1a34493f8fc17fe77d5a1b4b:presence Prefix Service Name Instance ID Type Hydra:service user-svcs 636a68fb1a34493f8fc17fe77d5a1b4b presence : [STRING] GET, SETEX Presence in Redis
: [STRING] GET, SETEX Health in Redis hydra:service:auth-svcs:1556536eda5e4d9089b56d0d6dd0a081:health Prefix Service Name Instance ID Type Hydra:service auth-svcs 1556536eda5e4d9089b56d0d6dd0a081 health
Service Discovery Recap • ServiceName used to query for service information • Individual services store information using their unique IDs • Managed using Redis Hash – blazing fast!
Routes Recap • Services can publish their routes to Redis • Individual services store information using their unique IDs • Enables dynamic service aware routing • Managed using a Redis Set
Load balancing using Redis : [STRING, HASH, SET] GET, SETEX, HGET, HGETALL, SADD, SMEMBERS hydra:service:nodes Prefix Type Hydra:service nodes hydra:service:asset-svcs:service:routes Prefix Type Hydra:service service:routes Service Name asset-svcs hydra:service:user-svcs:636a68fb1a34493f8fc17fe77d5a1b4b:presence Prefix Service Name Instance ID Type Hydra:service user-svcs 636a68fb1a34493f8fc17fe77d5a1b4b presence
: [PUB/SUB] SUBSCRIBE, UNSUBSCRIBE, PUBLISH Messaging in Redis hydra:service:mc:asset-svcs:cb8c96deb5434d5b95f3cd7bcdd6851c Prefix Hydra:service Service Name asset-svcs Instance ID cb8c96deb5434d5b95f3cd7bcdd6851c hydra:service:mc:asset-svcs
: [LIST] lpush, rpush, rpoplpush, lrem Queues in Redis hydra:service:imageproc-svcs:mqrecieved Prefix Service Name Type Hydra:service imageproc-svcs mq{bin} hydra:service:imageproc-svcs:mqinprogress hydra:service:imageproc-svcs:mqincomplete
Queuing Recap • Great when we don’t need immediate message responses • Redis Lists using lpush and rpoplpush • Atomic operations FTW • Ease of abstractions at an application level
: [LIST] lrange, lpush, ltrim Logging in Redis hydra:service:imageproc-svcs:1ce584aeee24429891182180fee12f58:health:log Prefix Service Name Instance ID Type Hydra:service imageproc-svcs 1556536eda5e4d9089b56d0d6dd0a081 health:log
Logging Recap • Necessary because services are distributed • Distributed logging • Redis as a Flight Recorder! • Implemented using lists and the lpush / ltrim commands
: [HASH] hget, hset, hkeys Redis for configuration management? hydra:service:imageproc-svcs:configs Prefix Service Name Type Hydra:service imageproc-svcs configs
Configuration Management Recap • Redis can be used for configuration management • Statefulness is a concern • Redis Hash indexed by service version with a stringified JSON contents