Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scalable Caching for Microservices

Scalable Caching for Microservices

As enterprises move towards hierarchical microservices architecture, Architects are often tasked to develop performant and scalable microservices which necessitates the need for data caching. A platform managed Caching solution inside Pivotal Cloud Foundry(PCF) makes it easy for these microservices to consume low latency caching while delivering self-provisioning for developers and automated Day 2 operations. In this talk, you will learn about how Pivotal Cloud Cache(PCC) helps enterprises boost microservices app performance and deliver the seamless user experiences to their customers!

AMEY BANARSE

April 29, 2019
Tweet

More Decks by AMEY BANARSE

Other Decks in Technology

Transcript

  1. Scalable Caching for
    Microservices

    View full-size slide

  2. Internet Scale Caching @ Netflix
    3
    “The Netflix customer experience relies
    heavily on caches to provide a
    high-volume, low latency, globally
    available data layer that backs our
    stateless services.
    Throughout a typical customer
    experience, from sitting down on the
    couch through playing a movie, caches
    play a role in every interaction.”
    #CachesEverywhere
    http://bit.ly/netflix-caching-2016

    View full-size slide

  3. Internet Scale Caching @ Netflix
    4

    View full-size slide

  4. Internet Scale Caching @ Netflix
    5

    View full-size slide







  5. View full-size slide

  6. Introducing Pivotal Cloud Cache (PCC)
    7







    View full-size slide

  7. 2 sides of a coin





    View full-size slide

  8. Create
    Service
    Plans
    Define
    Quotas
    Deploy
    PCC
    Broker
    Quotas
    - Max Cluster Size
    - Max # of Clusters
    On-Demand Service
    Broker
    - OpsMan Tile
    - Register with
    Marketplace
    Define Plans
    - Define VMs
    - Define Memory
    - Define CPU
    - Define Disk Size

    View full-size slide

  9. On-Demand Service
    ● Eliminate the “noisy neighbor” problem
    when one application hogs resources on a
    shared cluster
    ● No need to pre-provision a fixed quantity
    of IaaS resources
    ● Scale resource consumption linearly with
    need, without having to plan for
    pre-provisioning
    ● App developers get more control over
    resources, and do not have to acquire them
    through the operator

    View full-size slide

  10. Service-Service for Devs: Create PCC Cluster



    View full-size slide

  11. Binding Application to PCC Service

    View full-size slide

  12. Scalable Data and Session Caching




    1
    2
    Cache
    Server
    Cache
    Server
    Cache
    Server
    Locator
    1 2 3
    3 1
    Locator
    PCC
    Client
    Connection Pool
    54.209.251.110 54.209.251.111 54.209.251.112
    Locator
    Cache
    Server
    2
    54.209.251.113
    PCF
    AZ 1
    PCF
    AZ 2

    View full-size slide

  13. PCC Topology Members
    Cache Server

    In-Memory Storage for Data Regions

    Standard PCC Process with one Cache
    Server per JVM
    Cache
    Server
    Cache
    Server
    Cache
    Server
    Locator
    Locator

    Cluster Discovery & Config

    Load Balancing for Servers

    Cache
    Server
    Cache
    Server
    Cache
    Server
    Locator

    View full-size slide

  14. Regions
    17
    ● Synonymous to a Table in
    NoSQL terminology
    ● Stores Data in
    pairs with unique Keys
    ● Divided into buckets across
    Cache Members

    View full-size slide

  15. Regions
    18
    Replicated Partitioned
    GEM
    Server
    GEM
    Server
    GEM
    Server
    GEM
    Server
    GEM
    Server
    GEM
    Server
    HASH

    View full-size slide

  16. Data Aware Clients
    Locator
    PCC
    Client
    Connection Pool
    Cache Server
    Cache Server
    Partitioned Region
    Cache Server
    0
    23
    4 11
    44 45 72
    18
    Total 113 Buckets (Configurable)
    2
    3
    Target Bucket = Hash(Key) % Number of Buckets
    1

    View full-size slide

  17. Failing data copies are
    replaced transparently
    Data is replicated to other
    clusters and sites (WAN)
    Split Brain Detection is enabled Client and cluster disconnections
    are handled gracefully
    Data is persisted on local
    disk for ultimate durability
    “split brain”

    View full-size slide

  18. Design Pattern: Look-Aside Caching
    22
    https://github.com/Pivotal-Field-Engineering/pad-pcc-demo

    View full-size slide

  19. Cross PCF Foundations Active-Active Replication
    23
    WAN Replication across
    Foundations, across Clouds

    View full-size slide

  20. Cross PCF Foundations Active-Active Replication
    Pivotal
    Application Service
    Pivotal
    Application Service
    East Foundation Central Foundation
    Services
    Network
    10.0.16.0/26
    ERT
    Network
    10.0.8.0/24
    Services
    Network
    10.0.24.0/26
    ERT
    Network
    10.0.8.0/24
    Google
    VPN
    Gateway
    Secure
    VPN Tunnel
    Pivotal
    Application Service
    Google
    VPN
    Gateway

    View full-size slide

  21. Demo:
    Active - Active Data Replication
    between PCF Foundations

    View full-size slide

  22. Client Side Eventing
    Client Subscriptions
    ○ Client interest in server side event changes
    ○ Filter events based on RegEx Interest
    Continuous Query(CQ)
    ○ Register interest through server side queries
    ○ Suitable for creating dashboards

    View full-size slide

  23. PCC Client Subscriptions

    View full-size slide

  24. Front-ending a RDBMS (In-line Caching)


    1000s of
    transactions per
    second
    100s of
    transactions per
    second

    View full-size slide

  25. POS Next Best Offer:
    Conceptual Demo

    View full-size slide

  26. High level Architecture
    30

    View full-size slide

  27. Transaction Flow

    View full-size slide

  28. Offers Flow
    32

    View full-size slide

  29. Cluster Scaling Out
    cf update-service pcc-prod-cluster -c '{"num_servers": 6}'

    View full-size slide

  30. Cluster Monitoring Dashboard

    View full-size slide

  31. Monitoring PCC Cluster: Node Failure

    View full-size slide