Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Under the Hood of Reactive Data Access

Mark Paluch
December 05, 2017

Under the Hood of Reactive Data Access

Slides of the talk I gave at SpringOne Platform 2017.

Links:
* Slides – https://mp911.de/s1p-hood-slides
* Example Code – https://mp911.de/s1p-hood-code
* Spring Data Examples – https://github.com/spring-projects/spring-data-examples
* Find us on Gitter: https://gitter.im/spring-projects/spring-data

Mark Paluch

December 05, 2017
Tweet

More Decks by Mark Paluch

Other Decks in Programming

Transcript

  1. Under the Hood of
    Reactive Data Access
    Mark Paluch
    @mp911de
    1

    View Slide

  2. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Spring Data Modules
    2
    JPA

    View Slide

  3. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Reactive Spring Data Modules
    3
    JPA

    View Slide

  4. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    JDBC
    Fully-blocking API
    Non-blocking API announced (JDK-8188051)
    4
    http://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/ • JDK-8188051: http://hg.openjdk.java.net/jdk10/sandbox/jdk/rev/a31057bda7c5

    View Slide

  5. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Relational Databases
    Limited scalability
    Resource synchronization
    Transactions
    5

    View Slide

  6. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Query by time
    6
    Transport

    View Slide

  7. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Synchronous Data Access
    7
    Data store
    App
    Data store
    App
    Data store

    View Slide

  8. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Reactive Data Access
    8
    Data store
    App
    Data store Data store Data store
    App

    View Slide

  9. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Differences to imperative Data Access
    Drivers require threading infrastructure
    Timeout handling different
    9

    View Slide

  10. Reactive NoSql drivers

    View Slide

  11. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Reactive drivers
    End-to-end nonblocking
    Asynchronous request-response processing
    Non-blocking pooling
    Non-blocking connect
    11

    View Slide

  12. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Application Driver Application
    12
    Request Server selection
    Queueing
    Write Response decoding
    Response buffering
    Request completion Response

    View Slide

  13. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    MongoDB
    Reactive-over-async driver
    Cursor lifecycle
    Pluggable I/O
    Pooled connections
    13

    View Slide

  14. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Query by time
    14
    Request
    Response
    (chunk)
    Response
    (complete)
    Prefetch
    (cont.d)
    Query
    Emit
    elements
    Result
    processing

    View Slide

  15. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    MongoDB Driver Fetching
    15
    Operator
    MongoDB Subscriber
    request(Long.MAX_VALUE)
    request(Long.MAX_VALUE)

    View Slide

  16. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    MongoDB Driver Fetching
    16
    Filter
    MongoDB Subscriber
    request(Long.MAX_VALUE)
    request(Long.MAX_VALUE)

    View Slide

  17. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    MongoDB Driver Fetching
    17
    Operator
    (filter)
    MongoDB Subscriber
    request(1)
    Dropped element
    request(Long.MAX_VALUE)

    View Slide

  18. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Specific Features
    Tailable Cursors (@Tailable)
    MongoDB 3.6: Change Streams
    18
    Cursor size derived from demand

    View Slide

  19. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Redis
    Reactive driver (Lettuce 5)
    Based on Netty and Project Reactor
    Singleton connection
    Pooled connections
    19

    View Slide

  20. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Command by time
    20
    Request
    Command
    Command
    (subscription)
    Response
    (chunk)
    Response
    (complete)

    View Slide

  21. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Specifics
    Use ByteBuffer instead byte[]
    @null handling: Emission of empty buffers
    Suspend reading if no demand
    21

    View Slide

  22. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Apache Cassandra
    Reactive bridge over async driver
    Netty-based driver (Datastax)
    Pooled connections
    22

    View Slide

  23. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Query by time
    23
    Request
    Response
    (chunk)
    Response
    (complete)
    Query
    Emit
    elements
    Result processing
    Next page

    View Slide

  24. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Outlook
    Cassandra driver 4.0 in the works
    Likely support for reactive execution model
    Emit items as they are received
    24
    https://github.com/datastax/java-driver/tree/4.x • https://datastax-oss.atlassian.net/browse/JAVA-1505

    View Slide

  25. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Couchbase
    Reactive driver driver
    Based on Netty and RxJava 1
    Multiple connection schemes
    25

    View Slide

  26. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Couchbase Fetching
    Key-Value, DCP
    1. Request
    2. Response
    3. Emit results

    Search, Query, View
    1. Request
    2. Response streaming
    26

    View Slide

  27. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Command by time
    27
    Request
    Command
    Command
    (subscription)
    Response
    (chunk)
    Response
    (complete)

    View Slide

  28. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Outlook
    Couchbase driver likely to move to RxJava 2
    • No tickets/decisions yet
    28

    View Slide

  29. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Resources
    Slides – mp911.de/s1p-hood-slides
    Example Code – mp911.de/s1p-hood-code
    Spring Data Examples – Repository @ Github
    Find us on Gitter!
    29

    View Slide

  30. Feedback welcome

    View Slide

  31. Learn More. Stay Connected.
    31
    #springone
    @s1p

    View Slide

  32. Disclaimer

    View Slide

  33. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Safe Harbor Statement
    The following is intended to outline the general direction of Pivotal's offerings. It is
    intended for information purposes only and may not be incorporated into any
    contract. Any information regarding pre-release of Pivotal offerings, future updates or
    other planned modifications is subject to ongoing evaluation by Pivotal and is subject
    to change. This information is provided without warranty or any kind, express or
    implied, and is not a commitment to deliver any material, code, or functionality, and
    should not be relied upon in making purchasing decisions regarding Pivotal's
    offerings. These purchasing decisions should only be based on features currently
    available. The development, release, and timing of any features or functionality
    described for Pivotal's offerings in this presentation remain at the sole discretion of
    Pivotal. Pivotal has no obligation to update forward looking information in this
    presentation.
    33

    View Slide