Building a Reactive Database Driver on the JVM

Building a Reactive Database Driver on the JVM

This talk was given at Span London 2014, a great single track conference about distributed systems.

Recording will also be available through Skills Matter.

D839d9aa56849a71d8a9aa3d292a6ce6?s=128

Michael Nitschinger

October 28, 2014
Tweet

Transcript

  1. Building a Reactive Database Driver on the JVM Michael Nitschinger

    Software Engineer, Couchbase Inc.
  2. Let’s start with WHY

  3. None
  4. None
  5. None
  6. None
  7. JSON JSON JSON JSON JSON PERFORMANCE Scalability PERFORMANCE Performance Always

    On JSON JSON JSON JSON JSON PERFORMANCE Flexibility
  8. None
  9. Java SDK 1.* • Extends Spymemcached • Adds Config &

    View Capabilities • Stable & Mature • Older codebase, difficult to maintain and evolve • API inconsistencies over time
  10. None
  11. Reworking the API

  12. None
  13. single multiple sync T Iterable<T> async Future<T> Observable<T>

  14. Event Iterable<T> Observable<T> data retrieval T  next() onNext(T) discover error

    throws  Exception onError() complete returns onCompleted()
  15. Creating & Consuming just

  16. Creating & Consuming

  17. Transforming

  18. Transforming

  19. Transforming

  20. Filtering

  21. Reactive Database Query

  22. Error Handling

  23. Error Handling

  24. Error Handling

  25. Connecting

  26. Connecting

  27. Connecting

  28. Consistent API

  29. Document!

  30. Wrap, not mix

  31. Lessons Learned • Huge benefit in exposing reactive APIs •

    Also expose a synchronous API • Consistency matters • Document observable failure scenarios • Expect a learning curve, documentation helps
  32. A new Architecture

  33. None
  34. None
  35. Smart Batching http://mechanical-sympathy.blogspot.co.at/2011/10/smart-batching.html

  36. Backpressure

  37. None
  38. None
  39. Netty Pipeline Encoder( Encoder( Ne*y( Writes( Ne*y( Reads( Decoder( Decoder(

  40. None
  41. KeyValue Endpoint

  42. Lessons Learned • Build for the JVM, not for Java

    • Benchmark early and often • Implicit batching amortizes slow consumers • Use explicit back pressure to avoid swamping • Use Netty for IO and be happy • Bundle your dependencies
  43. Thanks! Questions? Couchbase is hiring :) http://www.couchbase.com/careers