Building a Reactive Database Driver on the JVM

Slide 1

Slide 1 text

Building a Reactive Database Driver on the JVM Michael Nitschinger Software Engineer, Couchbase Inc.

Slide 2

Slide 2 text

Let’s start with WHY

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

JSON JSON JSON JSON JSON PERFORMANCE Scalability PERFORMANCE Performance Always On JSON JSON JSON JSON JSON PERFORMANCE Flexibility

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

Java SDK 1.* • Extends Spymemcached • Adds Conﬁg & View Capabilities • Stable & Mature • Older codebase, difﬁcult to maintain and evolve • API inconsistencies over time

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

Reworking the API

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

single multiple sync T Iterable async Future Observable

Slide 14

Slide 14 text

Event Iterable Observable data retrieval T next() onNext(T) discover error throws Exception onError() complete returns onCompleted()

Slide 15

Slide 15 text

Creating & Consuming just

Slide 16

Slide 16 text

Creating & Consuming

Slide 17

Slide 17 text

Transforming

Slide 18

Slide 18 text

Transforming

Slide 19

Slide 19 text

Transforming

Slide 20

Slide 20 text

Filtering

Slide 21

Slide 21 text

Reactive Database Query

Slide 22

Slide 22 text

Error Handling

Slide 23

Slide 23 text

Error Handling

Slide 24

Slide 24 text

Error Handling

Slide 25

Slide 25 text

Connecting

Slide 26

Slide 26 text

Connecting

Slide 27

Slide 27 text

Connecting

Slide 28

Slide 28 text

Consistent API

Slide 29

Slide 29 text

Document!

Slide 30

Slide 30 text

Wrap, not mix

Slide 31

Slide 31 text

Lessons Learned • Huge beneﬁt in exposing reactive APIs • Also expose a synchronous API • Consistency matters • Document observable failure scenarios • Expect a learning curve, documentation helps

Slide 32

Slide 32 text

A new Architecture

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

Smart Batching http://mechanical-sympathy.blogspot.co.at/2011/10/smart-batching.html

Slide 36

Slide 36 text

Backpressure

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

Netty Pipeline Encoder( Encoder( Ne*y( Writes( Ne*y( Reads( Decoder( Decoder(

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

KeyValue Endpoint

Slide 42

Slide 42 text

Lessons Learned • Build for the JVM, not for Java • Benchmark early and often • Implicit batching amortizes slow consumers • Use explicit back pressure to avoid swamping • Use Netty for IO and be happy • Bundle your dependencies

Slide 43

Slide 43 text

Thanks! Questions? Couchbase is hiring :) http://www.couchbase.com/careers