Oracle Webex 23 Nov 2018: Why Reactive & Introduction To Project Reactor

Donald Knuth "Premature optimization is the root of all evil
(or at least most of it) in programming"

SYNCHRONOUS SYNCHRONOUS vs ASYNCHRONOUS ASYNCHRONOUS

SYNCHRONOUS SYNCHRONOUS

SYNCHRONOUS SYNCHRONOUS Code executes in the caller thread

SYNCHRONOUS SYNCHRONOUS Code executes in the caller thread (or) Caller
thread waits till the other thread to signal

thread waits till the other thread to signal ADVANTAGES ADVANTAGES

thread waits till the other thread to signal ADVANTAGES ADVANTAGES Simpler & Easy to understand

thread waits till the other thread to signal ADVANTAGES ADVANTAGES Simpler & Easy to understand Easy to debug due to stack traces availability

ASYNCHRONOUS ASYNCHRONOUS

ASYNCHRONOUS ASYNCHRONOUS Caller thread does not block while the other
thread signals later

thread signals later DISADVANTAGES DISADVANTAGES

thread signals later DISADVANTAGES DISADVANTAGES Our code gets executed by other thread in future

thread signals later DISADVANTAGES DISADVANTAGES Our code gets executed by other thread in future Callback style

thread signals later DISADVANTAGES DISADVANTAGES Our code gets executed by other thread in future Callback style Not easy to reason about, stack traces gets wiped out

STILL WHY ASYNCHRONY? STILL WHY ASYNCHRONY?

BACK TO FUNDAMENTALS BACK TO FUNDAMENTALS

CPU CPU vs RAM RAM

SPEED MISMATCH SPEED MISMATCH CPU 60%/year. DRAM 7%/year. 1 10
100 1000 1980 1981 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 … 2002 2003 2004 2005 2006 DRAM CPU 1982 Processor-Memory Performance Gap: (grows 50% / year) Performance Year “Moore’s Law” Memory Access vs CPU Speed Source: M. Bahi and C. Eisenbeis, "High Performance by Exploiting Information Locality through Reverse Computing," doi:10.1109/SBAC-PAD.2011.10

ADDRESSING MISMATCH ISSUE ADDRESSING MISMATCH ISSUE

ADDRESSING MISMATCH ISSUE ADDRESSING MISMATCH ISSUE OBSERVATION OBSERVATION

ADDRESSING MISMATCH ISSUE ADDRESSING MISMATCH ISSUE OBSERVATION OBSERVATION More than
90% of code accesses small area of memory

90% of code accesses small area of memory SOLUTION SOLUTION

90% of code accesses small area of memory SOLUTION SOLUTION Lets add an indirection

90% of code accesses small area of memory SOLUTION SOLUTION Lets add an indirection So CPU caches are born

90% of code accesses small area of memory SOLUTION SOLUTION Lets add an indirection So CPU caches are born L1

90% of code accesses small area of memory SOLUTION SOLUTION Lets add an indirection So CPU caches are born L1 L2

90% of code accesses small area of memory SOLUTION SOLUTION Lets add an indirection So CPU caches are born L1 L2 L3

90% of code accesses small area of memory SOLUTION SOLUTION Lets add an indirection So CPU caches are born L1 L2 L3 L4 (mainframes)

COSTS INVOLVED IN CONTEXT SWITCH COSTS INVOLVED IN CONTEXT SWITCH
RAM ALU + Registers L2 L1 CPU

RAM ALU + Registers L2 L1 CPU Save register state

RAM ALU + Registers L2 L1 CPU Save register state Scheduler overhead

RAM ALU + Registers L2 L1 CPU Save register state Scheduler overhead Fetch memory lines for new Thread

COST OF FETCHING A CACHE LINE COST OF FETCHING A
CACHE LINE 0:00 / 1:58

MULTI CORE CPU MULTI CORE CPU L3 RAM CPU ALU
+ Registers L2 L1 CORE 2 ALU + Registers L2 L1 CORE 1 ALU + Registers L2 L1 CORE 3 ALU + Registers L2 L1 CORE 4

OPTIMIZATION POINTS OPTIMIZATION POINTS javac JIT CPU JAVA CLASS 10101

FEW COMPILER OPTIMIZATIONS FEW COMPILER OPTIMIZATIONS

FEW COMPILER OPTIMIZATIONS FEW COMPILER OPTIMIZATIONS Reordering

FEW COMPILER OPTIMIZATIONS FEW COMPILER OPTIMIZATIONS Reordering Removing unused code

FEW COMPILER OPTIMIZATIONS FEW COMPILER OPTIMIZATIONS Reordering Removing unused code
Inlining

compiler can inline to static int max(int first, int second)
{ return first > second ? first : second; } static int min(int first, int second) { return first < second ? first : second; } int maxWithCeiling(int first, int second, int ceiling) { return min(max(first, second), ceiling); } int maxWithCeiling(int first, int second, int ceiling) { int temp = first > second ? first : second; return temp < ceiling ? temp : ceiling; }

JIT OPTIMIZATION EXAMPLE JIT OPTIMIZATION EXAMPLE class Processor { final
boolean unix; public Processor() { this.unix = PlatformUtil.isUnix(); } void process() { if (unix) { doUnixSpecificProcessing(); } else { doOtherProcessing(); } } }

can be transformed into a er first run by JIT
on Unix machines void process() { if (unix) { doUnixSpecificProcessing(); } else { doOtherProcessing(); } } void process() { doUnixSpecificProcessing(); }

REORDERING INTUITIVE REORDERING INTUITIVE A = 2 r1 = B
B = 1 r2 = A Thread 1 Thread 2 A = 2 r1 = B t B = 1 A = 2 r2 = A r1 = B B 1 A 2 r2 0 r1 1 t B = 1 r2 = A B 1 A 2 r2 2 r1 0 A = 2 r1 = B t B = 1 r1 = B B 1 A 2 r2 0 r1 1 t r2 = A B 1 A 2 r2 2 r1 1 A = 2 r2 = A t B = 1 B 1 A 2 r2 2 r1 1 A = 2 r2 = A r1 = B B = 1 A = 2 t r2 = A B 1 A 2 r2 2 r1 1 B = 1 r1 = B

REORDERING COUNTER INTUITIVE REORDERING COUNTER INTUITIVE B = 1 r2
= A Thread 1 Thread 2 A = 2 r1 = B B = 1 Thread 1 Thread 2 A = 2 r2 = A r1 = B B 1 A 2 r2 0 r1 1 1 2 2 1 1 2 2 0 t B = 1 A = 2 B 1 A 2 r2 0 r1 0 r2 = A r1 = B JMM compatible transformation Previous Interleaved Execution Values Valid execution B 1 A 2 r2 0 r1 1 1 2 2 1 1 2 2 0 1 2 0 0 Final State Table

PERFORMANCE VS CORRECTNESS PERFORMANCE VS CORRECTNESS

PERFORMANCE VS CORRECTNESS PERFORMANCE VS CORRECTNESS What if two threads
needs to update same variable simultaneously?

needs to update same variable simultaneously? What if one thread updates & another thread reads almost immediately?

needs to update same variable simultaneously? What if one thread updates & another thread reads almost immediately? What if they just need to update a single integer without any race?

needs to update same variable simultaneously? What if one thread updates & another thread reads almost immediately? What if they just need to update a single integer without any race? What if there are multiple variables whose state has to be consistent together?

JAVA MEMORY MODEL JAVA MEMORY MODEL Optimizers can execute in
any order consistent with sequentially execution

QUITE COMPLEX QUITE COMPLEX

QUITE COMPLEX QUITE COMPLEX In a Nutshell

QUITE COMPLEX QUITE COMPLEX In a Nutshell volatile writes are
visible to all subsequent reads

visible to all subsequent reads Unlocked state will be visible to subsequent lock acquirer

visible to all subsequent reads Unlocked state will be visible to subsequent lock acquirer All final variables should be initialized by all threads that reference

visible to all subsequent reads Unlocked state will be visible to subsequent lock acquirer All final variables should be initialized by all threads that reference Thread will see the state at the time of start() of thread that spawned it

visible to all subsequent reads Unlocked state will be visible to subsequent lock acquirer All final variables should be initialized by all threads that reference Thread will see the state at the time of start() of thread that spawned it Thread that waited to join() another thread will state that other thread on join

JAVA CONCURRENCY PRIMITIVES JAVA CONCURRENCY PRIMITIVES

JAVA CONCURRENCY PRIMITIVES JAVA CONCURRENCY PRIMITIVES Atomic*

JAVA CONCURRENCY PRIMITIVES JAVA CONCURRENCY PRIMITIVES Atomic* volatile

JAVA CONCURRENCY PRIMITIVES JAVA CONCURRENCY PRIMITIVES Atomic* volatile VarHandle -
in java 9+

JAVA CONCURRENCY PRIMITIVES JAVA CONCURRENCY PRIMITIVES Atomic* volatile VarHandle -
in java 9+ Lock & unlock

INTERMEDIATE CONCLUSION INTERMEDIATE CONCLUSION

INTERMEDIATE CONCLUSION INTERMEDIATE CONCLUSION MULTI THREADING IS HARD MULTI THREADING
IS HARD

INTERMEDIATE CONCLUSION INTERMEDIATE CONCLUSION MULTI THREADING IS HARD MULTI THREADING
IS HARD STILL WHY ASYNCHRONY? STILL WHY ASYNCHRONY?

JAVA & THREADS JAVA & THREADS Every Java thread has
a corresponding OS level thread

JAVA & THREADS JAVA & THREADS Every Java thread has
a corresponding OS level thread So, Lets understand how OS allocates memory

MEMORY LAYOUT MEMORY LAYOUT

PROCESS PROCESS Guard page Stack Heap Program Text Fixed size

PROCESS WITH THREADS PROCESS WITH THREADS Guard page Thread1 Stack
Heap Program Text Guard page Thread2 Stack Fixed size Guard page Thread3 Stack

PROBLEMS WITH JAVA PROBLEMS WITH JAVA and many other languages

Lets understand by looking at

Lets understand by looking at How servers allocate resources

TCP SOCKETS TCP SOCKETS

TCP SOCKETS TCP SOCKETS TCP socket is defined by a
unique combination of

unique combination of (source ip, source port, dest ip, dest port)

unique combination of (source ip, source port, dest ip, dest port) Ports on a machine/ip = ~216

unique combination of (source ip, source port, dest ip, dest port) Ports on a machine/ip = ~216 232 IPs for IPv4 = ~4B

unique combination of (source ip, source port, dest ip, dest port) Ports on a machine/ip = ~216 232 IPs for IPv4 = ~4B 2128 IPs for IPv6 = VERY large number

THEORETICAL MAXIMUM THEORETICAL MAXIMUM

THEORETICAL MAXIMUM THEORETICAL MAXIMUM For a fixed destination IP &
PORT

PORT For IPv4, 2(32+16 = 48) sockets

PORT For IPv4, 2(32+16 = 48) sockets For IPv6, 2(128+16 = 144) sockets

MAXIMUM NUMBER OF THREADS MAXIMUM NUMBER OF THREADS

MAXIMUM NUMBER OF THREADS MAXIMUM NUMBER OF THREADS Limited by
system resources

system resources Atleast 1MB stack overhead/thread

system resources Atleast 1MB stack overhead/thread For 4000 concurrent requests, 4000 * 1MB = 4GB just for Thread Stack

system resources Atleast 1MB stack overhead/thread For 4000 concurrent requests, 4000 * 1MB = 4GB just for Thread Stack What about heap allocation?

system resources Atleast 1MB stack overhead/thread For 4000 concurrent requests, 4000 * 1MB = 4GB just for Thread Stack What about heap allocation? What if most of threads are blocked on IO?

SOCKETS SOCKETS vs THREADS THREADS

COMPARISON COMPARISON

COMPARISON COMPARISON OS can

COMPARISON COMPARISON OS can create only limited threads based on
available system memory (RAM)

available system memory (RAM) handle almost billions of sockets

available system memory (RAM) handle almost billions of sockets Scalability is dependent on eﬀicient use of OS Threads

REQUEST HANDLING REQUEST HANDLING Lets look at three most common
models

models Thread per Connection (TPC)

models Thread per Connection (TPC) Thread per Request (TPR)

models Thread per Connection (TPC) Thread per Request (TPR) Shared Threads (ST)

THREAD PER CONNECTION (TPC) THREAD PER CONNECTION (TPC)

THREAD PER CONNECTION (TPC) THREAD PER CONNECTION (TPC) Thread is
dedicated for whole connection cycle

dedicated for whole connection cycle ADVANTAGES ADVANTAGES

dedicated for whole connection cycle ADVANTAGES ADVANTAGES Extremely low conceptual overhead

dedicated for whole connection cycle ADVANTAGES ADVANTAGES Extremely low conceptual overhead Extremely low latency as only thread scheduling overhead exists

TPC CONT. TPC CONT.

TPC CONT. TPC CONT. LIMITATIONS LIMITATIONS

TPC CONT. TPC CONT. LIMITATIONS LIMITATIONS Thread blocked till connection
closed/timesout

closed/timesout In typical web setting, performs very badly as most of the connections would be idle

closed/timesout In typical web setting, performs very badly as most of the connections would be idle Wasted resources as threads are reserved for whole connection period

closed/timesout In typical web setting, performs very badly as most of the connections would be idle Wasted resources as threads are reserved for whole connection period USECASES USECASES

closed/timesout In typical web setting, performs very badly as most of the connections would be idle Wasted resources as threads are reserved for whole connection period USECASES USECASES Real time gaming

THREAD PER REQUEST (TPR) THREAD PER REQUEST (TPR)

THREAD PER REQUEST (TPR) THREAD PER REQUEST (TPR) Thread is
dedicated for request-response cycle

dedicated for request-response cycle ADVANTAGES ADVANTAGES

dedicated for request-response cycle ADVANTAGES ADVANTAGES Low conceptual overhead & Low scalability

dedicated for request-response cycle ADVANTAGES ADVANTAGES Low conceptual overhead & Low scalability Better than TPC for many web usecases

TPR CONT. TPR CONT.

TPR CONT. TPR CONT. LIMITATIONS LIMITATIONS

TPR CONT. TPR CONT. LIMITATIONS LIMITATIONS Thread gets block if
we attempt to get data from DB/make a external http request

we attempt to get data from DB/make a external http request Limited concurrency as each thread has a fixed 1MB overhead

we attempt to get data from DB/make a external http request Limited concurrency as each thread has a fixed 1MB overhead USECASES USECASES

we attempt to get data from DB/make a external http request Limited concurrency as each thread has a fixed 1MB overhead USECASES USECASES Low volume enterprise applications

TPC ANIMATION TPC ANIMATION 0:00 / 5:28

SHARED THREADS (ST) SHARED THREADS (ST)

SHARED THREADS (ST) SHARED THREADS (ST) No dedicated thread for
request response cycle

request response cycle ADVANTAGES ADVANTAGES

request response cycle ADVANTAGES ADVANTAGES Uses non blocking IO

request response cycle ADVANTAGES ADVANTAGES Uses non blocking IO Single request-response cycle can be handled by multiple threads

request response cycle ADVANTAGES ADVANTAGES Uses non blocking IO Single request-response cycle can be handled by multiple threads Very high concurrency as no thread is ever blocked

ST CONT. ST CONT.

ST CONT. ST CONT. LIMITATIONS LIMITATIONS

ST CONT. ST CONT. LIMITATIONS LIMITATIONS Quite complex due to
thread switching is assumed default

thread switching is assumed default Requires developers to know the consequences of thread switch

thread switching is assumed default Requires developers to know the consequences of thread switch USECASES USECASES

thread switching is assumed default Requires developers to know the consequences of thread switch USECASES USECASES High volume websites/applications

thread switching is assumed default Requires developers to know the consequences of thread switch USECASES USECASES High volume websites/applications Chat/push applications

REACTIVE STREAMS REACTIVE STREAMS Specification that defines interfaces, protocol &
TCK

INTERFACES INTERFACES

INTERFACES INTERFACES Publisher - Source of 1 (or) more events

Subscriber - Consumer of events

Subscriber - Consumer of events Subscription - Connection between Publisher & Subscriber

Subscriber - Consumer of events Subscription - Connection between Publisher & Subscriber Processor - Both Publisher & Subscriber

REACTIVE STREAMS: PROTOCOL REACTIVE STREAMS: PROTOCOL (SIMPLIFIED) (SIMPLIFIED)

REACTIVE STREAMS: PROTOCOL REACTIVE STREAMS: PROTOCOL (SIMPLIFIED) (SIMPLIFIED) FEW ASPECTS
FEW ASPECTS

FEW ASPECTS Backpressure

FEW ASPECTS Backpressure Guarantees on onNext, onComplete & onError are quite helpful for common scenarios

FEW ASPECTS Backpressure Guarantees on onNext, onComplete & onError are quite helpful for common scenarios Read the spec here

TYPES OF IMPLEMENTATIONS TYPES OF IMPLEMENTATIONS

TYPES OF IMPLEMENTATIONS TYPES OF IMPLEMENTATIONS Highly eﬀicient & Low
resource implementation

resource implementation RxJava 2 & Reactor

resource implementation RxJava 2 & Reactor Heavyweight abstractions

resource implementation RxJava 2 & Reactor Heavyweight abstractions Akka

PROJECT REACTOR PROJECT REACTOR Reactive Streams complaint implementation

BASICS BASICS

BASICS BASICS FLUX FLUX

BASICS BASICS FLUX FLUX Publisher of one or more events

MONO MONO

MONO MONO Publisher of zero or one events

MARBLE DIAGRAMS MARBLE DIAGRAMS

OPERATOR OPERATOR Transforms publisher into another

EXAMPLE 1 EXAMPLE 1

EXAMPLE 1 EXAMPLE 1 Flux.map

EXAMPLE 1 EXAMPLE 1 Flux.map Transforms each element of the
stream

EXAMPLE 1 EXAMPLE 1 Flux.map Transforms each element of the
stream map ( )

EXAMPLE 2 EXAMPLE 2

EXAMPLE 2 EXAMPLE 2 Flux.zip

EXAMPLE 2 EXAMPLE 2 Flux.zip Zip elements with another stream
elements

EXAMPLE 2 EXAMPLE 2 Flux.zip Zip elements with another stream
elements zipWith A B D 1 2 C 4 3 E 1 A 2 B 3 C 4 D

SUBSCRIPTION SUBSCRIPTION

BEFORE SUBSCRIPTION BEFORE SUBSCRIPTION

BEFORE SUBSCRIPTION BEFORE SUBSCRIPTION UPSTREAM DOWNSTREAM Backpressure Backpressure Subscriber Publisher
2 Publisher 1 Merge Operator

2 Publisher 1 Merge Operator Similar to fitting all pipes in a plumbing

2 Publisher 1 Merge Operator Similar to fitting all pipes in a plumbing Nothing happens until you subscribe

AFTER SUBSCRIPTION AFTER SUBSCRIPTION

AFTER SUBSCRIPTION AFTER SUBSCRIPTION UPSTREAM DOWNSTREAM Backpressure Backpressure Subscriber Publisher
2 Publisher 1 Merge Operator

2 Publisher 1 Merge Operator Elements flow only a er subscription

2 Publisher 1 Merge Operator Elements flow only a er subscription Analogous to switching on motor & opening tap

AFTER SUBSCRIPTION CONT. AFTER SUBSCRIPTION CONT. UPSTREAM DOWNSTREAM Backpressure Backpressure
Subscriber Publisher 2 Publisher 1 Merge Operator

AFTER SUBSCRIPTION CONT. AFTER SUBSCRIPTION CONT. UPSTREAM DOWNSTREAM Backpressure Backpressure
Subscriber Publisher 2 Publisher 1 Merge Operator Backpressure exerted thru Subscription, like valve

Oracle Webex 23 Nov 2018: Why Reactive & Introd...

Oracle Webex 23 Nov 2018: Why Reactive & Introduction To Project Reactor

Other Decks in Technology

Featured

Transcript