Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a Reactive Database Driver on the JVM

Building a Reactive Database Driver on the JVM

This talk was given at Span London 2014, a great single track conference about distributed systems.

Recording will also be available through Skills Matter.

Michael Nitschinger

October 28, 2014
Tweet

More Decks by Michael Nitschinger

Other Decks in Programming

Transcript

  1. Building a
    Reactive Database Driver
    on the JVM
    Michael Nitschinger
    Software Engineer, Couchbase Inc.

    View Slide

  2. Let’s start with WHY

    View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. JSON
    JSON
    JSON
    JSON
    JSON
    PERFORMANCE
    Scalability
    PERFORMANCE
    Performance
    Always On
    JSON
    JSON
    JSON
    JSON
    JSON
    PERFORMANCE
    Flexibility

    View Slide

  8. View Slide

  9. Java SDK 1.*
    • Extends Spymemcached
    • Adds Config & View Capabilities
    • Stable & Mature
    • Older codebase, difficult to maintain and evolve
    • API inconsistencies over time

    View Slide

  10. View Slide

  11. Reworking the API

    View Slide

  12. View Slide

  13. single multiple
    sync T Iterable
    async Future Observable

    View Slide

  14. Event Iterable Observable
    data retrieval T  next() onNext(T)
    discover error throws  Exception onError()
    complete returns onCompleted()

    View Slide

  15. Creating & Consuming
    just

    View Slide

  16. Creating & Consuming

    View Slide

  17. Transforming

    View Slide

  18. Transforming

    View Slide

  19. Transforming

    View Slide

  20. Filtering

    View Slide

  21. Reactive Database Query

    View Slide

  22. Error Handling

    View Slide

  23. Error Handling

    View Slide

  24. Error Handling

    View Slide

  25. Connecting

    View Slide

  26. Connecting

    View Slide

  27. Connecting

    View Slide

  28. Consistent API

    View Slide

  29. Document!

    View Slide

  30. Wrap, not mix

    View Slide

  31. Lessons Learned
    • Huge benefit in exposing reactive APIs
    • Also expose a synchronous API
    • Consistency matters
    • Document observable failure scenarios
    • Expect a learning curve, documentation helps

    View Slide

  32. A new Architecture

    View Slide

  33. View Slide

  34. View Slide

  35. Smart Batching
    http://mechanical-sympathy.blogspot.co.at/2011/10/smart-batching.html

    View Slide

  36. Backpressure

    View Slide

  37. View Slide

  38. View Slide

  39. Netty Pipeline
    Encoder(
    Encoder(
    Ne*y(
    Writes(
    Ne*y(
    Reads(
    Decoder(
    Decoder(

    View Slide

  40. View Slide

  41. KeyValue Endpoint

    View Slide

  42. Lessons Learned
    • Build for the JVM, not for Java
    • Benchmark early and often
    • Implicit batching amortizes slow consumers
    • Use explicit back pressure to avoid swamping
    • Use Netty for IO and be happy
    • Bundle your dependencies

    View Slide

  43. Thanks! Questions?
    Couchbase is hiring :)
    http://www.couchbase.com/careers

    View Slide