Scalable Caching for Microservices

No content

Internet Scale Caching @ Netflix 3 “The Netflix customer experience relies heavily on caches to provide a high-volume, low latency, globally available data layer that backs our stateless services. Throughout a typical customer experience, from sitting down on the couch through playing a movie, caches play a role in every interaction.” #CachesEverywhere http://bit.ly/netflix-caching-2016

Internet Scale Caching @ Netflix 4

Internet Scale Caching @ Netflix 5

➔ ➔ ➔ ➔ ➔ ➔

Introducing Pivotal Cloud Cache (PCC) 7 • • • • • • •

No content

2 sides of a coin ● ○ ○ ● ○

Create Service Plans Define Quotas Deploy PCC Broker Quotas - Max Cluster Size - Max # of Clusters On-Demand Service Broker - OpsMan Tile - Register with Marketplace Define Plans - Define VMs - Define Memory - Define CPU - Define Disk Size

On-Demand Service ● Eliminate the “noisy neighbor” problem when one application hogs resources on a shared cluster ● No need to pre-provision a fixed quantity of IaaS resources ● Scale resource consumption linearly with need, without having to plan for pre-provisioning ● App developers get more control over resources, and do not have to acquire them through the operator

Service-Service for Devs: Create PCC Cluster ● ● ●

Binding Application to PCC Service

No content

Scalable Data and Session Caching ● ● ● ● 1 2 Cache Server Cache Server Cache Server Locator 1 2 3 3 1 Locator PCC Client Connection Pool 54.209.251.110 54.209.251.111 54.209.251.112 Locator Cache Server 2 54.209.251.113 PCF AZ 1 PCF AZ 2

PCC Topology Members Cache Server • In-Memory Storage for Data Regions • Standard PCC Process with one Cache Server per JVM Cache Server Cache Server Cache Server Locator Locator • Cluster Discovery & Config • Load Balancing for Servers • Cache Server Cache Server Cache Server Locator

Regions 17 ● Synonymous to a Table in NoSQL terminology ● Stores Data in pairs with unique Keys ● Divided into buckets across Cache Members

Regions 18 Replicated Partitioned GEM Server GEM Server GEM Server GEM Server GEM Server GEM Server HASH

Data Aware Clients Locator PCC Client Connection Pool Cache Server Cache Server Partitioned Region Cache Server 0 23 4 11 44 45 72 18 Total 113 Buckets (Configurable) 2 3 Target Bucket = Hash(Key) % Number of Buckets 1

Failing data copies are replaced transparently Data is replicated to other clusters and sites (WAN) Split Brain Detection is enabled Client and cluster disconnections are handled gracefully Data is persisted on local disk for ultimate durability “split brain”

No content

Design Pattern: Look-Aside Caching 22 https://github.com/Pivotal-Field-Engineering/pad-pcc-demo

Cross PCF Foundations Active-Active Replication 23 WAN Replication across Foundations, across Clouds

Cross PCF Foundations Active-Active Replication Pivotal Application Service Pivotal Application Service East Foundation Central Foundation Services Network 10.0.16.0/26 ERT Network 10.0.8.0/24 Services Network 10.0.24.0/26 ERT Network 10.0.8.0/24 Google VPN Gateway Secure VPN Tunnel Pivotal Application Service Google VPN Gateway

Demo: Active - Active Data Replication between PCF Foundations

Client Side Eventing Client Subscriptions ○ Client interest in server side event changes ○ Filter events based on RegEx Interest Continuous Query(CQ) ○ Register interest through server side queries ○ Suitable for creating dashboards

PCC Client Subscriptions

Front-ending a RDBMS (In-line Caching) ● ● 1000s of transactions per second 100s of transactions per second

POS Next Best Offer: Conceptual Demo

High level Architecture 30

Transaction Flow

Offers Flow 32

No content

Cluster Scaling Out cf update-service pcc-prod-cluster -c '{"num_servers": 6}'

Cluster Monitoring Dashboard

Monitoring PCC Cluster: Node Failure

No content