Reactive Transactions Masterclass

@mp911de and @rotnroll666
#Devoxx #ReactiveProgramming
Reactive transaction
masterclass
Mark Paluch, Pivotal.
Michael Simons, Neo4j, Inc.

View Slide

#Devoxx #ReactiveProgramming @mp911de and @rotnroll666
Mark Paluch
• Spring Data Project Lead
• Lettuce Redis Driver Project Lead
• R2DBC Contributor
• All things Reactive Data & Open Source
github.com/mp911de • @mp911de • paluch.biz

View Slide

Michael Simons
• Neo4j since July 2018
• Java Champion and Oracle Groundbreaker
• Co-Founder and current lead of
Java User Group EuregJUG
• Author

View Slide

More books: Life behind bars
• I run…
• I bike…
• …and take pictures
Revenue will go to
Médecins Sans Frontières msf.org
this year
https://leanpub.com/lifebehindbars

View Slide

Imperative
Transactions

View Slide

Regular transaction

View Slide


View Slide


[ ]

View Slide


View Slide

Phantom read

View Slide


View Slide

No rollback /
Duplication

View Slide


View Slide

Broken transaction

View Slide


View Slide

Transaction
• Atomically apply changes
• Isolation
• Consistency
• Durable
• Require a transactional resource

View Slide

Resource-local Transaction
• Allocate transactional resource
• Apply work
• Cleanup transaction

View Slide

Database Example
• Open connection
• Begin transaction
• Apply work
• Reuse same connection across operations
• Commit/rollback transaction

View Slide

Demo

View Slide

Transaction Attributes
• Isolation Level
• Explicit Commit Mode
• Read-Only Optimizations

View Slide

Transaction Management
• Ensures transactional behaviour
• Apply transaction deﬁnition (propagation, attributes)
• Implementations
• JTA
• Spring
• Driver-speciﬁc

View Slide

Transaction Managers
•DataSourceTransactionManager
•MongoTransactionManager
•Neo4jTransactionManager

View Slide

Transaction Walkthrough
• Allocate connection
• Bind connection to transaction (Thread*)
• Prepare connection for transaction (isolation level, …)
• Issue queries (Reuse bound connection)
• Exit transactional scope
• Cleanup, commit or rollback
• Run synchronizations
• Unbind resources

View Slide

Demo
@Transactional

View Slide

Bound connection
• Transactional state associated with transport connection
• Works since the ’70s like that
• Alternatively: Session
• Newer implementations: Portable session
• MongoDB

View Slide

Imperative Transactions
• Stay on a single thread*
• ThreadLocal storage
• Using another thread breaks transactions
* ManagedExecutorService, @Async

View Slide

Imperative
transactions are
fully synchronous

View Slide

Time-dependency in
transactions

View Slide

Time dependency
• Begin Tx before work
• Work after begin Tx
• Work before cleanup Tx
• Cleanup Tx after work

View Slide

Distributed
Transactions

View Slide


View Slide

Distributed Transactions
• XA speciﬁcation
• Released in the 90’s
• Transactions across distributed services
• Two phase commit
• Turned out that it sucks

View Slide

Still Distributed Transactions
• Redis + JDBC Transaction
• Kind-of works
• Leading transaction manager
• Additional transactional resources may join
• Independent commit/rollback
• Still sucks

View Slide

Reactive
Transactions

View Slide

Database
• Open connection
• Begin transaction
• Apply work
• Reuse same connection across operations
• Commit/rollback transaction

View Slide

Transaction Managers
•R2dbcTransactionManager
•Neo4jTransactionManager
•MongoTransactionManager

View Slide

Transaction Walkthrough
• Allocate connection
• Bind connection to transaction (Subscription)
• Prepare connection for transaction (isolation level, …)
• Issue queries (Reuse bound connection)
• Exit transactional scope
• Cleanup, commit or rollback
• Run synchronizations
• Unbind resources

View Slide

Reactive
transactions are
fully synchronous

View Slide

Reactive
transactions are
non-blocking

View Slide

Time-dependency in
transactions

View Slide

Time dependency
• Begin before work
• Work after begin
• Work before cleanup
• Cleanup after work

View Slide

Imperative vs. Reactive
• Single-threaded processing
required
• ThreadLocal transaction
status
• Data stays within
transactional bounds
• Synchronized begin and
cleanup
• Multi-threaded processing
possible
• Subscription-bound
transaction status
• Data escapes transactional
bounds
• Synchronized begin and
cleanup

View Slide

Cancelation
(Connection
Terminated while
Processing)

View Slide


View Slide


(Client) (Server)

View Slide


(Client)
POST /some/data
(Server)

View Slide


(Client)
POST /some/data
(Server)

✂

View Slide


(Client)
POST /some/data
(Server)

View Slide

More on cancelation

View Slide

Cancellation
• .first(), .take(n)
• Send cancellation signal upstream
• „Thank you I don’t need additional data“
• „Connection dead. Cancelling“

View Slide

Cancellation effects
• Cancellation considered success
• Cancelled subscription does not await completion
• Commit may be not yet visible to subsequent operations

View Slide

// Define a flow that creates 1000 nodes in a database
Flux createNodes = Flux.using(
driver::rxSession,
session -> session.run(
"UNWIND range (1,1000) AS i CREATE (s:SomeNode {pos: i}) return s").records(),
RxSession::close
).map(r -> r.get("s").get("pos").asInt());
„Cancelled subscription does not await completion“
What does that even mean?

View Slide

// Just take some of those
createNodes.take(5);

View Slide

// Or more explicit
createNodes
.as(StepVerifier::create)
.expectNext(1, 2, 3, 4, 5)
.thenCancel()
.verify();

View Slide

// Can we verify it?
Flux.using(
driver::rxSession,
session -> session.run("MATCH (s:SomeNode) RETURN count(s) as cnt").records(),
RxSession::close
)
.map(r -> r.get("cnt").asLong())
.single().as(StepVerifier::create).expectNext(1000L).verifyComplete();

View Slide

Depends

View Slide

Cancellation
• Happens when the browser goes away
• Probably not a good idea to return transactional ﬂows from
web endpoints
(The same has been said for @Transactional on web
methods for a long time!)
• Again: .first(), .take(n)
• Also limitRequest(n)

View Slide

R2DBC

View Slide

Reactive Relational Database Connectivity
• A speciﬁcation designed from the ground up for reactive
programming
• End to end reactive and non-blocking

View Slide

Dependencies
• Reactive Streams
• Java 8

View Slide

Design Principles
• Embrace Reactive Types and Patterns
• Non-blocking, all the way to the database
• Documented speciﬁcation
• Shrink the driver SPI
• Enable multiple "humane" APIs

View Slide

SPI
• ConnectionFactory
• Connection
• Statement
• Result
• Row

View Slide

Drivers
• H2
• Google Cloud Spanner
• MySQL
• Postgres
• SQL Server
• SAP Hana

View Slide

ConnectionFactory Transactions
• Publisher beginTransaction();
• Publisher commitTransaction();
• Publisher rollbackTransaction();

View Slide

Demo

View Slide

Spring Data R2DBC
•R2dbcTransactionManager

View Slide

Spring Framework 5.2
• Reactive Transaction Manager SPI
• Support for @Transactional
• TransactionalOperator

View Slide

@Transactional
• Method signature declares known reactive type
• Requires ReactiveTransactionManager
• Code path for reactive transaction management
• Otherwise
• Uses PlatformTransactionManager

View Slide

Spring Data R2DBC
• Demo @Transactional
• Demo TransactionalOperator

View Slide

Data escape
• Transactional data escapes before transaction completion
• Demo

View Slide

Neo4j

View Slide

What is Neo4j?
• A (property) graph database
• An ecosystem
• I work on Spring and Spring Data integration as part of the drivers
team

View Slide

A Property Graph
:Movie :Person
:Person
Nodes represents objects
(Nouns)
:DIRECTED
:ACTED_IN
role: someRole
name: someName
Relationships connect nodes
and represent actions (verbs)
Both nodes and relationships
can have properties

View Slide

The whiteboard model
IS the physical model
• Just ﬁll the abstract nodes with
content

View Slide

Demo

View Slide

Neo4j 4.0
• Reactive Cypher engine
• Reactive Driver available (Beta)
• Spring Data Neo4j RX (Public preview)
• Imperative and reactive transaction managers

View Slide

Building blocks of Neo4j driver
Driver
Top level object for all Neo4j interaction
Session
Logical context for sequence of transactions
Transaction
Unit of work
Statement result
Stream of records plus metadata

View Slide

Driver Session
Connection pool
Connection
Connection
Server A
Server B
spawns
borrows
owns
owns to

View Slide

• Session is a logical concept over the connection
• One ongoing transaction per session
• Thus, the connection is bound to the transaction

View Slide

Challenges
• Driver shall provide a similar surface for various languages
• Java
• JavaScript
• Python
• .NET

View Slide

More challenges
• Already two different programming models
• Imperative / Blocking
• Asynchronous (Pretty much the default in .NET, feels like Co-
Routines in Kotlin / Lightweight Threads in Loom)
• Supported for all languages
• Reactive coming up
• Java
• JavaScript
• .NET

View Slide

Imperative
Transactions

View Slide

@Test
void implicit() { // Aka "Auto-Commit"
try (Session session = driver.session()) {
long personId = session
.run(
"CREATE (a:Person {name: $name}) RETURN id(a) as id“,
parameters("name", NAME))
.single().get("id").asLong();
}
}
Implicit

View Slide

@Test
void txFunctions() {
long personId = session.writeTransaction(
tx -> tx.run(
.single().get("id").asLong());
}
}
Transactional functions

View Slide

@Test
void explicit() {
Transaction tx = session.beginTransaction(TransactionConfig.builder().build());
long personId =
tx.run(
.single().get("id").asLong();
tx.commit();
}
}
Explicit

View Slide

Reactive
Transactions

View Slide

RxSession session = driver.rxSession();
Publisher txPublisher =
session.beginTransaction();
RxTransaction tx;
RxStatementResult result = tx.run();
Publisher recordPublisher = result.records()
Publisher commit = tx.commit();
Publisher rollback = tx.rollback();
Reactive surface

View Slide

@Test
void implicit() {
Flux personCreation = Flux.using(
driver::rxSession,
session -> session.run(
parameters("name", NAME)
).records(),
RxSession::close
).map(r -> r.get("id").asLong());
StepVerifier.create(personCreation)
.expectNextCount(1)
.verifyComplete();
}
Implicit

View Slide

@Test
void txFunctions() {
RxTransactionWork> txFunction = tx ->
Flux.from(tx.run(
"CREATE (a:Person {name: $name}) RETURN id(a) as id",
parameters("name", NAME)).records()
)
.map(r -> r.get("id").asLong());
Flux personCreation = Flux.using(driver::rxSession,
session -> session.writeTransaction(txFunction),
RxSession::close);
StepVerifier.create(personCreation)
.expectNextCount(1)
.verifyComplete();
}
Transactional functions

View Slide

@Test
void explicit() {
Function> actualWork = session ->
Flux.usingWhen(session.beginTransaction(),
// Yes, this looks pretty much like the `txFunction` in the example before
tx -> tx.run(
"CREATE (a:Person {name: $name}) RETURN id(a) as id",
parameters("name", NAME)
).records(),
RxTransaction::commit, // Success case
(tx, e) -> tx.rollback(), // Error / exceptional case
RxTransaction::commit // Cancelation
).map(r -> r.get("id").asLong());
Flux personCreation =
Flux.using(driver::rxSession, actualWork, RxSession::close);
}
Explicit

View Slide

Spring Data Neo4j⚡RX
(SDN/RX)

View Slide

How does SDN/RX differ from previous SDN?
• Modular design
• Fundamental design differences (e.g. mutability, incremental
mapping)
• SDN/OGM layers are now combined
• Transport functionality moved entirely to Java driver
• Supports all "ﬁndBy*" methods
• Query by Example
• Integration with Spring 5.2’s Reactive Transaction Manager

View Slide

SDN/RX
Spring Data Repositories
Neo4j Template
Neo4j Client
Driver
Neo4j
„uses“
sdn-rx-spring-boot-starter
neo4j-java-driver-spring-boot-starter
provides
provides
Integration with
Spring
Transactions from
here on
SDN/RX components

View Slide

ReactiveNeo4jClient reactiveClient;
Flux> directorAndMovies = reactiveClient
.query(
"MATCH (p:Person) - [:DIRECTED] -> (m:Movie {title: $title})," +
" (p) - [:WROTE] -> (om:Movie) " +
"WHERE p.name =~ $name " +
" AND p.born < $someDate.year " +
"RETURN p, om"
)
.bind(LocalDate.of(1979, 9, 21)).to("someDate")
.bindAll(Map.of("title", "The Matrix", "name", "Li.*"))
.fetch()
.all();
SDN/RX how does it look?

View Slide

ReactiveNeo4jTemplate reactiveTemplate;
Flux movies = reactiveTemplate
.findAll(MovieEntity.class);

View Slide

public interface MovieRepository
extends ReactiveNeo4jRepository {
Mono findOneByTitle(String title);
}

View Slide

@Service
public class MovieService {
@Transactional
public Flux doSomething() {
// Return type indicating reactive or imperative
return Flux.empty();
}
void doSomethingElse() {
ReactiveNeo4jTemplate reactiveTemplate;
TransactionalOperator transactionalOperator;
Flux movies = transactionalOperator.transactional(
reactiveTemplate.findAll(MovieEntity.class)
);
}
}
And transactions?

View Slide

ReactiveNeo4jTransactionManager
• Uses explicit reactive transactions
• Uses the driver instance as lookup for the transactional
resource
• Stores both the session and transaction belonging to that
session into resource holder

View Slide

• Basically this, in a Spring way:
Function>
actualWork = session ->
Flux.usingWhen(
session.beginTransaction(),
tx -> Flux.just(1L),
RxTransaction::commit,
(tx, e) -> tx.rollback(),
RxTransaction::commit);
Flux completeUnitOfWork =
Flux.using(driver::rxSession,
actualWork, RxSession::close)

View Slide

• Session acquisition
Function>
Flux.usingWhen(

View Slide

• Starting transaction
Function>
Flux.usingWhen(

View Slide

• Executing stuff
Function>
Flux.usingWhen(

View Slide

• Executing stuff
• Commit or rollback
Function>
Flux.usingWhen(

View Slide

• Executing stuff
• Commit or rollback
• Closing session
Function>
Flux.usingWhen(

View Slide

Neo4j resources

View Slide

Neo4j
• https://neo4j.com/download/
• Neo4j Desktop (Analyst centric)
• Neo4j Server (Community and Enterprise Edition)
Community Edition: GPLv3
Enterprise Edition: Proprietary

View Slide

Neo4j Datasets
• Preconfigured instance with several different datasets
https://neo4j.com/sandbox-v2/
• Neo4j Graph Gists, Example Models and Cypher Queries
https://neo4j.com/graphgists/
• Panama papers and other offshore leaks
https://offshoreleaks.icij.org/

View Slide

Spring related
• Neo4j Driver Spring Boot Starter
https://github.com/neo4j/neo4j-java-driver-spring-boot-starter
• Spring Data Neo4j⚡RX
https://github.com/neo4j/sdn-rx/

View Slide

Reactive Transactions Masterclass

Reactive Transactions Masterclass

Michael Simons

More Decks by Michael Simons

Other Decks in Programming

Featured

Transcript