Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scalable Caching for Microservices

Scalable Caching for Microservices

As enterprises move towards hierarchical microservices architecture, Architects are often tasked to develop performant and scalable microservices which necessitates the need for data caching. A platform managed Caching solution inside Pivotal Cloud Foundry(PCF) makes it easy for these microservices to consume low latency caching while delivering self-provisioning for developers and automated Day 2 operations. In this talk, you will learn about how Pivotal Cloud Cache(PCC) helps enterprises boost microservices app performance and deliver the seamless user experiences to their customers!



April 29, 2019

More Decks by AMEY BANARSE

Other Decks in Technology


  1. Scalable Caching for Microservices

  2. None
  3. Internet Scale Caching @ Netflix 3 “The Netflix customer experience

    relies heavily on caches to provide a high-volume, low latency, globally available data layer that backs our stateless services. Throughout a typical customer experience, from sitting down on the couch through playing a movie, caches play a role in every interaction.” #CachesEverywhere http://bit.ly/netflix-caching-2016
  4. Internet Scale Caching @ Netflix 4

  5. Internet Scale Caching @ Netflix 5

  6. ➔ ➔ ➔ ➔ ➔ ➔

  7. Introducing Pivotal Cloud Cache (PCC) 7 • • • •

    • • •
  8. None
  9. 2 sides of a coin • ◦ ◦ • ◦

  10. Create Service Plans Define Quotas Deploy PCC Broker Quotas -

    Max Cluster Size - Max # of Clusters On-Demand Service Broker - OpsMan Tile - Register with Marketplace Define Plans - Define VMs - Define Memory - Define CPU - Define Disk Size
  11. On-Demand Service • Eliminate the “noisy neighbor” problem when one

    application hogs resources on a shared cluster • No need to pre-provision a fixed quantity of IaaS resources • Scale resource consumption linearly with need, without having to plan for pre-provisioning • App developers get more control over resources, and do not have to acquire them through the operator
  12. Service-Service for Devs: Create PCC Cluster • • •

  13. Binding Application to PCC Service

  14. None
  15. Scalable Data and Session Caching • • • • 1

    2 Cache Server Cache Server Cache Server Locator 1 2 3 3 1 Locator PCC Client Connection Pool Locator Cache Server 2 PCF AZ 1 PCF AZ 2
  16. PCC Topology Members Cache Server • In-Memory Storage for Data

    Regions • Standard PCC Process with one Cache Server per JVM Cache Server Cache Server Cache Server Locator Locator • Cluster Discovery & Config • Load Balancing for Servers • Cache Server Cache Server Cache Server Locator
  17. Regions 17 • Synonymous to a Table in NoSQL terminology

    • Stores Data in <Key,Value> pairs with unique Keys • Divided into buckets across Cache Members
  18. Regions 18 Replicated Partitioned GEM Server GEM Server GEM Server

    GEM Server GEM Server GEM Server HASH
  19. Data Aware Clients Locator PCC Client Connection Pool Cache Server

    Cache Server Partitioned Region Cache Server 0 23 4 11 44 45 72 18 Total 113 Buckets (Configurable) 2 3 Target Bucket = Hash(Key) % Number of Buckets 1
  20. Failing data copies are replaced transparently Data is replicated to

    other clusters and sites (WAN) Split Brain Detection is enabled Client and cluster disconnections are handled gracefully Data is persisted on local disk for ultimate durability “split brain”
  21. None
  22. Design Pattern: Look-Aside Caching 22 https://github.com/Pivotal-Field-Engineering/pad-pcc-demo

  23. Cross PCF Foundations Active-Active Replication 23 WAN Replication across Foundations,

    across Clouds
  24. Cross PCF Foundations Active-Active Replication Pivotal Application Service Pivotal Application

    Service East Foundation Central Foundation Services Network ERT Network Services Network ERT Network Google VPN Gateway Secure VPN Tunnel Pivotal Application Service Google VPN Gateway
  25. Demo: Active - Active Data Replication between PCF Foundations

  26. Client Side Eventing Client Subscriptions ◦ Client interest in server

    side event changes ◦ Filter events based on RegEx Interest Continuous Query(CQ) ◦ Register interest through server side queries ◦ Suitable for creating dashboards
  27. PCC Client Subscriptions

  28. Front-ending a RDBMS (In-line Caching) • • 1000s of transactions

    per second 100s of transactions per second
  29. POS Next Best Offer: Conceptual Demo

  30. High level Architecture 30

  31. Transaction Flow

  32. Offers Flow 32

  33. None
  34. Cluster Scaling Out cf update-service pcc-prod-cluster -c '{"num_servers": 6}'

  35. Cluster Monitoring Dashboard

  36. Monitoring PCC Cluster: Node Failure

  37. None