Building a Reactive Database Driver on the JVM

This talk was given at Span London 2014, a great single track conference about distributed systems.

Recording will also be available through Skills Matter.

Transcript

1. Building a
   Reactive Database Driver
   on the JVM
   Michael Nitschinger
   Software Engineer, Couchbase Inc.
   

   View Slide

2. Let’s start with WHY
   

   View Slide

3. View Slide

4. View Slide

5. View Slide

6. View Slide

7. JSON
   JSON
   JSON
   JSON
   JSON
   PERFORMANCE
   Scalability
   PERFORMANCE
   Performance
   Always On
   JSON
   JSON
   JSON
   JSON
   JSON
   PERFORMANCE
   Flexibility
   

   View Slide

8. View Slide

9. Java SDK 1.*
   • Extends Spymemcached
   • Adds Config & View Capabilities
   • Stable & Mature
   • Older codebase, difficult to maintain and evolve
   • API inconsistencies over time
   

   View Slide

10. View Slide

11. Reworking the API
    

    View Slide

12. View Slide

13. single multiple
    sync T Iterable
    async Future Observable
    

    View Slide

14. Event Iterable Observable
    data retrieval T  next() onNext(T)
    discover error throws  Exception onError()
    complete returns onCompleted()
    

    View Slide

15. Creating & Consuming
    just
    

    View Slide

16. Creating & Consuming
    

    View Slide

17. Transforming
    

    View Slide

18. Transforming
    

    View Slide

19. Transforming
    

    View Slide

20. Filtering
    

    View Slide

21. Reactive Database Query
    

    View Slide

22. Error Handling
    

    View Slide

23. Error Handling
    

    View Slide

24. Error Handling
    

    View Slide

25. Connecting
    

    View Slide

26. Connecting
    

    View Slide

27. Connecting
    

    View Slide

28. Consistent API
    

    View Slide

29. Document!
    

    View Slide

30. Wrap, not mix
    

    View Slide

31. Lessons Learned
    • Huge benefit in exposing reactive APIs
    • Also expose a synchronous API
    • Consistency matters
    • Document observable failure scenarios
    • Expect a learning curve, documentation helps
    

    View Slide

32. A new Architecture
    

    View Slide

33. View Slide

34. View Slide

35. Smart Batching
    http://mechanical-sympathy.blogspot.co.at/2011/10/smart-batching.html
    

    View Slide

36. Backpressure
    

    View Slide

37. View Slide

38. View Slide

39. Netty Pipeline
    Encoder(
    Encoder(
    Ne*y(
    Writes(
    Ne*y(
    Reads(
    Decoder(
    Decoder(
    

    View Slide

40. View Slide

41. KeyValue Endpoint
    

    View Slide

42. Lessons Learned
    • Build for the JVM, not for Java
    • Benchmark early and often
    • Implicit batching amortizes slow consumers
    • Use explicit back pressure to avoid swamping
    • Use Netty for IO and be happy
    • Bundle your dependencies
    

    View Slide

43. Thanks! Questions?
    Couchbase is hiring :)
    http://www.couchbase.com/careers
    

    View Slide