Suuchi - FifthElephant - Talk Outline

Transcript

1. Suuchi, toolkit to
   build distributed
   systems
   

   View Slide

2. About me
   Sriram
   @ Indix
   

   View Slide

3. About Indix
   - 840M Products
   - 1.5B Offers
   - 50K+ Brands
   - ~7.5K Categories
   and counting...
   

   View Slide

4. Handling Scale
   

   View Slide

5. Layered Scaling
   

   View Slide

6. Buck stops with DB Scaling
   

   View Slide

7. Essentially - Scaling data
   systems
   

   View Slide

8. rise of KVs
   Distributed - Replicated -
   Fault-tolerant - Sorted*
   *optional
   

   View Slide

9. Distributed Data Stores &
   Distributed Compute
   Platforms
   

   View Slide

10. Unify them?
    

    View Slide

11. Existing Solutions
    CoProc with HBase
    

    View Slide

12. Applications should OWN
    everything
    

    View Slide

13. What if we can “enable”
    building them?
    

    View Slide

14. Distributed System Primitives
    - Communication
    - Membership
    - Routing / Sharding / Partitioning
    - Replication
    - Handling Failure
    

    View Slide

15. Communication
    gRPC
    

    View Slide

16. Membership
    Static | Dynamic
    Config-Driven | Atomix
    

    View Slide

17. Sharding / Partitioning /
    Routing
    Consistent Hash Ring, Pluggable
    

    View Slide

18. Handling Failures
    

    View Slide

19. Replication
    Sync | Async
    

    View Slide

20. Storage*
    RocksDB | Pluggable
    * optional
    

    View Slide

21. Getting Started
    - gRPC Service using Protobuf2
    - Generate stubs & implement them
    - Connect using Suuchi “Server” abstraction
    

    View Slide

22. Server Abstraction
    - Pluggable membership mechanism
    - Pluggable routing strategy
    - Pluggable replication method
    

    View Slide