Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scalable Caching for Microservices

Scalable Caching for Microservices

As enterprises move towards hierarchical microservices architecture, Architects are often tasked to develop performant and scalable microservices which necessitates the need for data caching. A platform managed Caching solution inside Pivotal Cloud Foundry(PCF) makes it easy for these microservices to consume low latency caching while delivering self-provisioning for developers and automated Day 2 operations. In this talk, you will learn about how Pivotal Cloud Cache(PCC) helps enterprises boost microservices app performance and deliver the seamless user experiences to their customers!

AMEY BANARSE

April 29, 2019
Tweet

More Decks by AMEY BANARSE

Other Decks in Technology

Transcript

  1. Scalable Caching for
    Microservices

    View Slide

  2. View Slide

  3. Internet Scale Caching @ Netflix
    3
    “The Netflix customer experience relies
    heavily on caches to provide a
    high-volume, low latency, globally
    available data layer that backs our
    stateless services.
    Throughout a typical customer
    experience, from sitting down on the
    couch through playing a movie, caches
    play a role in every interaction.”
    #CachesEverywhere
    http://bit.ly/netflix-caching-2016

    View Slide

  4. Internet Scale Caching @ Netflix
    4

    View Slide

  5. Internet Scale Caching @ Netflix
    5

    View Slide







  6. View Slide

  7. Introducing Pivotal Cloud Cache (PCC)
    7







    View Slide

  8. View Slide

  9. 2 sides of a coin





    View Slide

  10. Create
    Service
    Plans
    Define
    Quotas
    Deploy
    PCC
    Broker
    Quotas
    - Max Cluster Size
    - Max # of Clusters
    On-Demand Service
    Broker
    - OpsMan Tile
    - Register with
    Marketplace
    Define Plans
    - Define VMs
    - Define Memory
    - Define CPU
    - Define Disk Size

    View Slide

  11. On-Demand Service
    ● Eliminate the “noisy neighbor” problem
    when one application hogs resources on a
    shared cluster
    ● No need to pre-provision a fixed quantity
    of IaaS resources
    ● Scale resource consumption linearly with
    need, without having to plan for
    pre-provisioning
    ● App developers get more control over
    resources, and do not have to acquire them
    through the operator

    View Slide

  12. Service-Service for Devs: Create PCC Cluster



    View Slide

  13. Binding Application to PCC Service

    View Slide

  14. View Slide

  15. Scalable Data and Session Caching




    1
    2
    Cache
    Server
    Cache
    Server
    Cache
    Server
    Locator
    1 2 3
    3 1
    Locator
    PCC
    Client
    Connection Pool
    54.209.251.110 54.209.251.111 54.209.251.112
    Locator
    Cache
    Server
    2
    54.209.251.113
    PCF
    AZ 1
    PCF
    AZ 2

    View Slide

  16. PCC Topology Members
    Cache Server

    In-Memory Storage for Data Regions

    Standard PCC Process with one Cache
    Server per JVM
    Cache
    Server
    Cache
    Server
    Cache
    Server
    Locator
    Locator

    Cluster Discovery & Config

    Load Balancing for Servers

    Cache
    Server
    Cache
    Server
    Cache
    Server
    Locator

    View Slide

  17. Regions
    17
    ● Synonymous to a Table in
    NoSQL terminology
    ● Stores Data in
    pairs with unique Keys
    ● Divided into buckets across
    Cache Members

    View Slide

  18. Regions
    18
    Replicated Partitioned
    GEM
    Server
    GEM
    Server
    GEM
    Server
    GEM
    Server
    GEM
    Server
    GEM
    Server
    HASH

    View Slide

  19. Data Aware Clients
    Locator
    PCC
    Client
    Connection Pool
    Cache Server
    Cache Server
    Partitioned Region
    Cache Server
    0
    23
    4 11
    44 45 72
    18
    Total 113 Buckets (Configurable)
    2
    3
    Target Bucket = Hash(Key) % Number of Buckets
    1

    View Slide

  20. Failing data copies are
    replaced transparently
    Data is replicated to other
    clusters and sites (WAN)
    Split Brain Detection is enabled Client and cluster disconnections
    are handled gracefully
    Data is persisted on local
    disk for ultimate durability
    “split brain”

    View Slide

  21. View Slide

  22. Design Pattern: Look-Aside Caching
    22
    https://github.com/Pivotal-Field-Engineering/pad-pcc-demo

    View Slide

  23. Cross PCF Foundations Active-Active Replication
    23
    WAN Replication across
    Foundations, across Clouds

    View Slide

  24. Cross PCF Foundations Active-Active Replication
    Pivotal
    Application Service
    Pivotal
    Application Service
    East Foundation Central Foundation
    Services
    Network
    10.0.16.0/26
    ERT
    Network
    10.0.8.0/24
    Services
    Network
    10.0.24.0/26
    ERT
    Network
    10.0.8.0/24
    Google
    VPN
    Gateway
    Secure
    VPN Tunnel
    Pivotal
    Application Service
    Google
    VPN
    Gateway

    View Slide

  25. Demo:
    Active - Active Data Replication
    between PCF Foundations

    View Slide

  26. Client Side Eventing
    Client Subscriptions
    ○ Client interest in server side event changes
    ○ Filter events based on RegEx Interest
    Continuous Query(CQ)
    ○ Register interest through server side queries
    ○ Suitable for creating dashboards

    View Slide

  27. PCC Client Subscriptions

    View Slide

  28. Front-ending a RDBMS (In-line Caching)


    1000s of
    transactions per
    second
    100s of
    transactions per
    second

    View Slide

  29. POS Next Best Offer:
    Conceptual Demo

    View Slide

  30. High level Architecture
    30

    View Slide

  31. Transaction Flow

    View Slide

  32. Offers Flow
    32

    View Slide

  33. View Slide

  34. Cluster Scaling Out
    cf update-service pcc-prod-cluster -c '{"num_servers": 6}'

    View Slide

  35. Cluster Monitoring Dashboard

    View Slide

  36. Monitoring PCC Cluster: Node Failure

    View Slide

  37. View Slide