Scalable Caching for Microservices

As enterprises move towards hierarchical microservices architecture, Architects are often tasked to develop performant and scalable microservices which necessitates the need for data caching. A platform managed Caching solution inside Pivotal Cloud Foundry(PCF) makes it easy for these microservices to consume low latency caching while delivering self-provisioning for developers and automated Day 2 operations. In this talk, you will learn about how Pivotal Cloud Cache(PCC) helps enterprises boost microservices app performance and deliver the seamless user experiences to their customers!

Transcript

1. Scalable Caching for Microservices
3. Internet Scale Caching @ Netflix 3 “The Netflix customer experience relies heavily on caches to provide a high-volume, low latency, globally available data layer that backs our stateless services. Throughout a typical customer experience, from sitting down on the couch through playing a movie, caches play a role in every interaction.” #CachesEverywhere http://bit.ly/netflix-caching-2016
4. Internet Scale Caching @ Netflix 4
5. Internet Scale Caching @ Netflix 5
6. ➔ ➔ ➔ ➔ ➔ ➔
7. Introducing Pivotal Cloud Cache (PCC) 7 • • • • • • •
9. 2 sides of a coin • ◦ ◦ • ◦
10. Create Service Plans Define Quotas Deploy PCC Broker Quotas - Max Cluster Size - Max # of Clusters On-Demand Service Broker - OpsMan Tile - Register with Marketplace Define Plans - Define VMs - Define Memory - Define CPU - Define Disk Size
11. On-Demand Service • Eliminate the “noisy neighbor” problem when one application hogs resources on a shared cluster • No need to pre-provision a fixed quantity of IaaS resources • Scale resource consumption linearly with need, without having to plan for pre-provisioning • App developers get more control over resources, and do not have to acquire them through the operator
12. Service-Service for Devs: Create PCC Cluster • • •
13. Binding Application to PCC Service
15. Scalable Data and Session Caching • • • • 1 2 Cache Server Cache Server Cache Server Locator 1 2 3 3 1 Locator PCC Client Connection Pool 54.209.251.110 54.209.251.111 54.209.251.112 Locator Cache Server 2 54.209.251.113 PCF AZ 1 PCF AZ 2
16. PCC Topology Members Cache Server • In-Memory Storage for Data Regions • Standard PCC Process with one Cache Server per JVM Cache Server Cache Server Cache Server Locator Locator • Cluster Discovery & Config • Load Balancing for Servers • Cache Server Cache Server Cache Server Locator
17. Regions 17 • Synonymous to a Table in NoSQL terminology • Stores Data in <Key,Value> pairs with unique Keys • Divided into buckets across Cache Members
18. Regions 18 Replicated Partitioned GEM Server GEM Server GEM Server GEM Server GEM Server GEM Server HASH
19. Data Aware Clients Locator PCC Client Connection Pool Cache Server Cache Server Partitioned Region Cache Server 0 23 4 11 44 45 72 18 Total 113 Buckets (Configurable) 2 3 Target Bucket = Hash(Key) % Number of Buckets 1
20. Failing data copies are replaced transparently Data is replicated to other clusters and sites (WAN) Split Brain Detection is enabled Client and cluster disconnections are handled gracefully Data is persisted on local disk for ultimate durability “split brain”
22. Design Pattern: Look-Aside Caching 22 https://github.com/Pivotal-Field-Engineering/pad-pcc-demo
23. Cross PCF Foundations Active-Active Replication 23 WAN Replication across Foundations, across Clouds
24. Cross PCF Foundations Active-Active Replication Pivotal Application Service Pivotal Application Service East Foundation Central Foundation Services Network 10.0.16.0/26 ERT Network 10.0.8.0/24 Services Network 10.0.24.0/26 ERT Network 10.0.8.0/24 Google VPN Gateway Secure VPN Tunnel Pivotal Application Service Google VPN Gateway
25. Demo: Active - Active Data Replication between PCF Foundations
26. Client Side Eventing Client Subscriptions ◦ Client interest in server side event changes ◦ Filter events based on RegEx Interest Continuous Query(CQ) ◦ Register interest through server side queries ◦ Suitable for creating dashboards
27. PCC Client Subscriptions
28. Front-ending a RDBMS (In-line Caching) • • 1000s of transactions per second 100s of transactions per second
29. POS Next Best Offer: Conceptual Demo
30. High level Architecture 30
31. Transaction Flow
32. Offers Flow 32
34. Cluster Scaling Out cf update-service pcc-prod-cluster -c '{"num_servers": 6}'
35. Cluster Monitoring Dashboard
36. Monitoring PCC Cluster: Node Failure