Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Suuchi - FifthElephant - Talk Outline

Suuchi - FifthElephant - Talk Outline

Sriram

May 23, 2017
Tweet

More Decks by Sriram

Other Decks in Technology

Transcript

  1. Suuchi, toolkit to
    build distributed
    systems

    View Slide

  2. About me
    Sriram
    @ Indix

    View Slide

  3. About Indix
    - 840M Products
    - 1.5B Offers
    - 50K+ Brands
    - ~7.5K Categories
    and counting...

    View Slide

  4. Handling Scale

    View Slide

  5. Layered Scaling

    View Slide

  6. Buck stops with DB Scaling

    View Slide

  7. Essentially - Scaling data
    systems

    View Slide

  8. rise of KVs
    Distributed - Replicated -
    Fault-tolerant - Sorted*
    *optional

    View Slide

  9. Distributed Data Stores &
    Distributed Compute
    Platforms

    View Slide

  10. Unify them?

    View Slide

  11. Existing Solutions
    CoProc with HBase

    View Slide

  12. Applications should OWN
    everything

    View Slide

  13. What if we can “enable”
    building them?

    View Slide

  14. Distributed System Primitives
    - Communication
    - Membership
    - Routing / Sharding / Partitioning
    - Replication
    - Handling Failure

    View Slide

  15. Communication
    gRPC

    View Slide

  16. Membership
    Static | Dynamic
    Config-Driven | Atomix

    View Slide

  17. Sharding / Partitioning /
    Routing
    Consistent Hash Ring, Pluggable

    View Slide

  18. Handling Failures

    View Slide

  19. Replication
    Sync | Async

    View Slide

  20. Storage*
    RocksDB | Pluggable
    * optional

    View Slide

  21. Getting Started
    - gRPC Service using Protobuf2
    - Generate stubs & implement them
    - Connect using Suuchi “Server” abstraction

    View Slide

  22. Server Abstraction
    - Pluggable membership mechanism
    - Pluggable routing strategy
    - Pluggable replication method

    View Slide