Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Suuchi - FifthElephant - Talk Outline

Suuchi - FifthElephant - Talk Outline

3ad75e16884079ab99b47f7e0bc52577?s=128

Sriram

May 23, 2017
Tweet

More Decks by Sriram

Other Decks in Technology

Transcript

  1. Suuchi, toolkit to build distributed systems

  2. About me Sriram @ Indix

  3. About Indix - 840M Products - 1.5B Offers - 50K+

    Brands - ~7.5K Categories and counting...
  4. Handling Scale

  5. Layered Scaling

  6. Buck stops with DB Scaling

  7. Essentially - Scaling data systems

  8. rise of KVs Distributed - Replicated - Fault-tolerant - Sorted*

    *optional
  9. Distributed Data Stores & Distributed Compute Platforms

  10. Unify them?

  11. Existing Solutions CoProc with HBase

  12. Applications should OWN everything

  13. What if we can “enable” building them?

  14. Distributed System Primitives - Communication - Membership - Routing /

    Sharding / Partitioning - Replication - Handling Failure
  15. Communication gRPC

  16. Membership Static | Dynamic Config-Driven | Atomix

  17. Sharding / Partitioning / Routing Consistent Hash Ring, Pluggable

  18. Handling Failures

  19. Replication Sync | Async

  20. Storage* RocksDB | Pluggable * optional

  21. Getting Started - gRPC Service using Protobuf2 - Generate stubs

    & implement them - Connect using Suuchi “Server” abstraction
  22. Server Abstraction - Pluggable membership mechanism - Pluggable routing strategy

    - Pluggable replication method