$30 off During Our Annual Pro Sale. View Details »

Under the Hood of Reactive Data Access

Under the Hood of Reactive Data Access

Slides to the presentation I gave at Spring I/O 2018

Conference page: https://2018.springio.net/sessions/under-the-hood-of-reactive-data-access
GitHub repository: https://github.com/mp911de/under-the-hood-reactive

Mark Paluch

May 24, 2018
Tweet

More Decks by Mark Paluch

Other Decks in Programming

Transcript

  1. Mark Paluch
    @mp911de
    UNDER THE HOOD OF
    REACTIVE DATA ACCESS

    View Slide

  2. Spring Data Modules
    JPA

    View Slide

  3. Reactive Spring Data Modules
    JPA

    View Slide

  4. What about…
    JDBC?
    JPA?

    View Slide

  5. Relational Databases
    •Not a good fit for reactive access

    •Limited scalability

    •Resource synchronization

    •ACID Transactions

    View Slide

  6. JDBC
    •Fully blocking API

    •Non-blocking API announced: ADBA 

    (JDK-8188051)
    http://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/ • JDK-8188051: http://hg.openjdk.java.net/jdk/sandbox/shortlog/JDK-8188051-branch

    View Slide

  7. Query Behavior

    View Slide

  8. Query by Time
    Transport

    View Slide

  9. Synchronous Data Access
    Data store
    App
    Data store
    App
    Data store

    View Slide

  10. Reactive Data Access
    Data store
    App
    Data store Data store Data store
    App

    View Slide

  11. Differences to imperative Data Access
    •Drivers require threading infrastructure

    •Timeout handling different

    View Slide

  12. Synchronous Timeouts
    Socket.setSoTimeout(…)

    View Slide

  13. Aynchronous Timeouts
    •Active timeout tracking required

    View Slide

  14. Reactive Drivers
    •End-to-end nonblocking

    •Asynchronous request-response processing

    •Non-blocking pooling

    •Non-blocking connect

    View Slide

  15. Application Driver Application
    Request
    Server selection
    Queueing
    Write Response decoding
    Response buffering
    Request completion Response

    View Slide

  16. Reactive NoSQL drivers

    View Slide

  17. MongoDB
    •Reactive-over-async driver

    •Cursor lifecycle

    •Pluggable I/O

    •Pooled connections
    Java.NIO

    View Slide

  18. Query by Time
    Request
    Response
    (chunk)
    Response
    (complete)
    Prefetch
    (cont.d)
    Query
    Emit
    elements
    Result
    processing

    View Slide

  19. MongoDB Driver Fetching
    Operator
    MongoDB Subscriber
    request(Long.MAX_VALUE)
    request(Long.MAX_VALUE)

    View Slide

  20. MongoDB Driver Fetching
    Filter
    MongoDB Subscriber
    request(Long.MAX_VALUE)
    request(Long.MAX_VALUE)

    View Slide

  21. MongoDB Driver Fetching
    Filter
    MongoDB Subscriber
    request(Long.MAX_VALUE)
    request(1)
    Dropped element

    View Slide

  22. Specific Features
    •Cursor size derived from demand

    •Tailable Cursors (@Tailable)

    •MongoDB 3.6: Change Streams

    View Slide

  23. Redis
    •Reactive driver (Lettuce 5)

    •Based on Netty and Project Reactor

    •Singleton connection

    •Optionally: Pooled connections

    View Slide

  24. Command by Time
    Request
    Command
    Command
    (subscription)
    Response
    (chunk)
    Response
    (complete)

    View Slide

  25. Specifics
    •Use ByteBuffer instead byte[]

    •null handling: Emission of empty buffers

    •Suspend reading if no demand

    View Slide

  26. Apache Cassandra
    •Reactive bridge over async driver

    •Netty-based driver (Datastax)

    •Pooled connections

    View Slide

  27. Query by Time
    Request
    Response
    (chunk)
    Response
    (complete)
    Query
    Emit
    elements
    Result processing
    Next page

    View Slide

  28. Outlook
    •Cassandra driver 4.0 in the works

    •Likely support for reactive execution model

    •Emit items as they are received
    https://github.com/datastax/java-driver/tree/4.x • https://datastax-oss.atlassian.net/browse/JAVA-1505

    View Slide

  29. Couchbase
    •Reactive driver driver

    •Based on Netty and RxJava 1

    •Multiple connection schemes

    View Slide

  30. Key-Value, DCP
    •Request

    •Response

    •Emit results

    Search, Query, View
    •Request

    •Response streaming

    View Slide

  31. Command by Time
    Request
    Command
    Command
    (subscription)
    Response
    (chunk)
    Response
    (complete)

    View Slide

  32. Outlook
    •Couchbase driver likely to move to RxJava 2

    •No tickets/decisions yet

    View Slide

  33. Imperative and Reactive Together

    View Slide

  34. Blocking with Imperative Programming

    View Slide

  35. Blocking with Reactive Programming

    View Slide

  36. Imperative and Reactive
    @Component
    public class ApplicationInitializer {
    ReactiveMongoOperations ops;
    @PostConstruct
    public void postConstruct() {
    ops.remove(Person.class).all().block();
    }
    }

    View Slide

  37. Imperative and Reactive
    @Component
    public class ApplicationInitializer {
    ReactiveMongoOperations ops;
    @PostConstruct
    public void postConstruct() {
    ops.remove(Person.class).all().block();
    }
    }

    View Slide

  38. Event Loop
    Event Loop
    (single Thread)

    View Slide

  39. Event Loop
    Event Loop
    (single Thread)

    View Slide

  40. Event Loop
    Process next item
    Callback
    Register Callback
    Return
    Work
    Database
    Computation
    File
    I/O
    Event Loop
    (single Thread)
    Work Queue
    Work
    Request Callback

    View Slide

  41. Event Loop
    Process next item
    Callback Work
    Event Loop
    (single Thread)
    Work Queue
    Request Callback

    View Slide

  42. Event Loop
    Event Loop
    (single Thread)
    Work Queue Process next item
    Work
    Callback
    Work
    Wait for
    Request completion
    Return
    Request
    Request Callback

    View Slide

  43. Event Loop
    Event Loop
    (single Thread)
    Work Queue Process next item
    Work
    Callback
    Work
    Wait for
    Request completion
    Return
    Request
    Request Callback

    View Slide

  44. Thou shalt not block

    View Slide

  45. Resources
    •Slides – mp911.de/uhd-springio18

    •Example Code – mp911.de/uhd-code

    •Spring Data Examples – Repository @ Github

    •Find us on Gitter!

    View Slide

  46. Mark Paluch
    @mp911de
    THANKS!

    View Slide