Building a Reactive Database Driver on the JVM

This talk was given at Span London 2014, a great single track conference about distributed systems.

Recording will also be available through Skills Matter.

Transcript

1. Building a Reactive Database Driver on the JVM Michael Nitschinger Software Engineer, Couchbase Inc.
2. Let’s start with WHY
7. JSON JSON JSON JSON JSON PERFORMANCE Scalability PERFORMANCE Performance Always On JSON JSON JSON JSON JSON PERFORMANCE Flexibility
9. Java SDK 1.* • Extends Spymemcached • Adds Config & View Capabilities • Stable & Mature • Older codebase, difficult to maintain and evolve • API inconsistencies over time
11. Reworking the API
13. single multiple sync T Iterable<T> async Future<T> Observable<T>
14. Event Iterable<T> Observable<T> data retrieval T  next() onNext(T) discover error throws  Exception onError() complete returns onCompleted()
15. Creating & Consuming just
16. Creating & Consuming
17. Transforming
18. Transforming
19. Transforming
20. Filtering
21. Reactive Database Query
22. Error Handling
23. Error Handling
24. Error Handling
25. Connecting
26. Connecting
27. Connecting
28. Consistent API
29. Document!
30. Wrap, not mix
31. Lessons Learned • Huge benefit in exposing reactive APIs • Also expose a synchronous API • Consistency matters • Document observable failure scenarios • Expect a learning curve, documentation helps
32. A new Architecture
35. Smart Batching http://mechanical-sympathy.blogspot.co.at/2011/10/smart-batching.html
36. Backpressure
39. Netty Pipeline Encoder( Encoder( Ne*y( Writes( Ne*y( Reads( Decoder( Decoder(
41. KeyValue Endpoint
42. Lessons Learned • Build for the JVM, not for Java • Benchmark early and often • Implicit batching amortizes slow consumers • Use explicit back pressure to avoid swamping • Use Netty for IO and be happy • Bundle your dependencies
43. Thanks! Questions? Couchbase is hiring :) http://www.couchbase.com/careers