Slide 1

Slide 1 text

Don’t be Homer Simpson 
 with your Reactor! Sergei Egorov, Pivotal @bsideup https://simpsonswiki.com/wiki/File:Springfield_Nuclear_Power_Plant_6.png

Slide 2

Slide 2 text

About me • Staff Engineer at Pivotal’s Spring R&D, working on Project Reactor ⚛ • Berlin Spring User Group co-organizer • Testcontainers co-maintainer • Developer tools geek • … although spent most of my career building
 highly scalable backend services @bsideup

Slide 3

Slide 3 text

My first experience with
 reactive programming was like…

Slide 4

Slide 4 text

@bsideup

Slide 5

Slide 5 text

@bsideup

Slide 6

Slide 6 text

Mono.fromRunnable()

Slide 7

Slide 7 text

Mono.fromRunnable() @bsideup

Slide 8

Slide 8 text

Mono.fromRunnable() My code @bsideup

Slide 9

Slide 9 text

Mono.fromRunnable() My code My code @bsideup

Slide 10

Slide 10 text

Mono.fromRunnable() My code My code My code Work! @bsideup

Slide 11

Slide 11 text

Mono.fromRunnable().subscribe() @bsideup

Slide 12

Slide 12 text

1 week later…

Slide 13

Slide 13 text

I agree with Heinrich Apfelmus that the essence of functional reactive programming is to specify the dynamic behavior of a value completely at the time of declaration. “Reactive Sir” @bsideup

Slide 14

Slide 14 text

1 week of production later…

Slide 15

Slide 15 text

@bsideup

Slide 16

Slide 16 text

What could go wrong

Slide 17

Slide 17 text

Exceptions

Slide 18

Slide 18 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); @bsideup

Slide 19

Slide 19 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) @bsideup

Slide 20

Slide 20 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) @bsideup

Slide 21

Slide 21 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Which source? @bsideup

Slide 22

Slide 22 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Which source? Where is “com.example.demo” package? @bsideup

Slide 23

Slide 23 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Which source? Where is “com.example.demo” package? Why they closed Google Inbox? @bsideup

Slide 24

Slide 24 text

Assembly vs Subscription

Slide 25

Slide 25 text

Flux.range(0, 5) Assembly @bsideup

Slide 26

Slide 26 text

Flux.range(0, 5) .filter(it -> it > 3) Assembly @bsideup

Slide 27

Slide 27 text

Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) Assembly @bsideup

Slide 28

Slide 28 text

Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) .single() Assembly @bsideup

Slide 29

Slide 29 text

Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) .single() .subscribeOn(Schedulers.parallel()) Assembly @bsideup

Slide 30

Slide 30 text

Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) .single() .subscribeOn(Schedulers.parallel()) .subscribe() Assembly @bsideup

Slide 31

Slide 31 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 32

Slide 32 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 * = subscriber @bsideup

Slide 33

Slide 33 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 34

Slide 34 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 35

Slide 35 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 36

Slide 36 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 37

Slide 37 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .onSubscribe( ) * = subscription @bsideup

Slide 38

Slide 38 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 39

Slide 39 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 40

Slide 40 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 41

Slide 41 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 42

Slide 42 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 43

Slide 43 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 44

Slide 44 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 45

Slide 45 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 46

Slide 46 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 47

Slide 47 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 48

Slide 48 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup

Slide 49

Slide 49 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup

Slide 50

Slide 50 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup

Slide 51

Slide 51 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup

Slide 52

Slide 52 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup

Slide 53

Slide 53 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(1) @bsideup

Slide 54

Slide 54 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(1) @bsideup

Slide 55

Slide 55 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(1) java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(Mono at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(Flux at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxR at reactor.core.publisher.MonoSingle$SingleSubscriber.request(Mono at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber. at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber. at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(M at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber. @bsideup

Slide 56

Slide 56 text

Demo http://eskipaper.com/homer-jay-simpson-cartoon.html

Slide 57

Slide 57 text

How ReactorDebugAgent works? @bsideup

Slide 58

Slide 58 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); @bsideup

Slide 59

Slide 59 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); @bsideup

Slide 60

Slide 60 text

// com/example/demo/Example.java Flux.range(0, 5) .checkpoint("Example.java:16") .single() .checkpoint("Example.java:17") .subscribeOn(Schedulers.parallel()) .checkpoint(“Example.java:18") .subscribe(); @bsideup

Slide 61

Slide 61 text

Demo outcomes • Use .checkpoint(“something”) to “mark” reactive “milestones” • Read about Hooks.onOperatorDebug()… • … but use reactor-tools’ ReactorDebugAgent (works in prod too) • https://spring.io/blog/2019/03/06/flight-of-the-flux-1-assembly-vs- subscription - great article from Simon Basle about the internals @bsideup

Slide 62

Slide 62 text

Blocking calls! https://blog.tfd.co.uk/2010/10/15/jackrabbit-performance/ @bsideup

Slide 63

Slide 63 text

How Reactor schedules tasks @bsideup

Slide 64

Slide 64 text

Default thread pools Schedulers.parallel() - N threads, where N matches the CPUs count. Schedulers.single() - 1 thread handling all submitted tasks Schedulers.elastic() - dynamic, thread caching pool @bsideup

Slide 65

Slide 65 text

Mono.delay(ofSeconds(1)) @bsideup

Slide 66

Slide 66 text

Mono.delay(ofSeconds(1)) Mono.delay(ofSeconds(1), Schedulers.parallel()) @bsideup

Slide 67

Slide 67 text

Mono.delay(ofSeconds(1)) Mono.delay(ofSeconds(1), Schedulers.parallel()) Every non-instant operation 
 runs on the parallel scheduler
 by default! @bsideup

Slide 68

Slide 68 text

parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup

Slide 69

Slide 69 text

parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup

Slide 70

Slide 70 text

parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup

Slide 71

Slide 71 text

parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup

Slide 72

Slide 72 text

parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup

Slide 73

Slide 73 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 74

Slide 74 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 75

Slide 75 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 76

Slide 76 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 77

Slide 77 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 78

Slide 78 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 79

Slide 79 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 80

Slide 80 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 81

Slide 81 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 82

Slide 82 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution ⚠ no more tasks scheduled 
 until this task returns @bsideup

Slide 83

Slide 83 text

How to fix?

Slide 84

Slide 84 text

Project Loom

Slide 85

Slide 85 text

@bsideup

Slide 86

Slide 86 text

@bsideup

Slide 87

Slide 87 text

Use non-blocking APIs, or…

Slide 88

Slide 88 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 89

Slide 89 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 90

Slide 90 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 91

Slide 91 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 92

Slide 92 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 93

Slide 93 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 94

Slide 94 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 95

Slide 95 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 96

Slide 96 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 97

Slide 97 text

Demo http://eskipaper.com/homer-jay-simpson-cartoon.html

Slide 98

Slide 98 text

How BlockHound works? @bsideup

Slide 99

Slide 99 text

public class Thread implements Runnable { public static native void sleep(long millis); } @bsideup

Slide 100

Slide 100 text

public class Thread implements Runnable { private static native void $$BlockHound$$_sleep(long millis); public static void sleep(long millis) { $$BlockHound$$_sleep(millis); } } @bsideup

Slide 101

Slide 101 text

public class Thread implements Runnable { private static native void $$BlockHound$$_sleep(long millis); public static void sleep(long millis) { reactor.BlockHoundRuntime.checkBlocking( "java.lang.Thread", "sleep", /*method modifiers*/ ); $$BlockHound$$_sleep(millis); } } @bsideup

Slide 102

Slide 102 text

public class Thread implements Runnable { private static native void $$BlockHound$$_sleep(long millis); public static void sleep(long millis) { reactor.BlockHoundRuntime.checkBlocking( "java.lang.Thread", "sleep", /*method modifiers*/ ); $$BlockHound$$_sleep(millis); } } @bsideup

Slide 103

Slide 103 text

Blocking check static void checkBlocking(String className, String methodName, int modifiers) { if (Boolean.FALSE == IS_ALLOWED.get()) { // Report } } static final ThreadLocal IS_ALLOWED = ThreadLocal.withInitial(() -> { if (threadPredicate.test(Thread.currentThread())) { return false; } else { // Optimization: use Three-state (true, false, null) // where `null` is `not non-blocking` return null; } }); @bsideup

Slide 104

Slide 104 text

Blocking check static void checkBlocking(String className, String methodName, int modifiers) { if (Boolean.FALSE == IS_ALLOWED.get()) { // Report } } static final ThreadLocal IS_ALLOWED = ThreadLocal.withInitial(() -> { if (threadPredicate.test(Thread.currentThread())) { return false; } else { // Optimization: use Three-state (true, false, null) // where `null` is `not non-blocking` return null; } }); @bsideup

Slide 105

Slide 105 text

Allowed/disallowed @bsideup

Slide 106

Slide 106 text

Allowed/disallowed class ClassLoader { // ... public Class> loadClass(String name) { return loadClass(name, false); } } @bsideup

Slide 107

Slide 107 text

Allowed/disallowed class ClassLoader { // ... public Class> loadClass(String name) { return loadClass(name, false); } } @bsideup Blocking!

Slide 108

Slide 108 text

Allowed/disallowed class ClassLoader { // ... public Class> loadClass(String name) { Boolean previous = BlockHoundRuntime.IS_ALLOWED.get(); BlockHoundRuntime.IS_ALLOWED.set( previous != null ? true : null ); try { // Original call return loadClass(name, false); } finally { BlockHoundRuntime.IS_ALLOWED.set(previous); } } } @bsideup

Slide 109

Slide 109 text

Demo outcomes • Blocking calls are bad, m’kay? • They may sneak into your production system • Use https://github.com/reactor/BlockHound to detect them • Supports multiple frameworks (Reactor, RxJava, etc) • … and maybe even Kotlin: 
 https://github.com/Kotlin/kotlinx.coroutines/issues/1031 • Use a dedicated pool for the necessary blocking calls, or schedule them on the Schedulers.elastic() built-in pool if they happen rarely @bsideup

Slide 110

Slide 110 text

“I use async APIs and I am safe!”

Slide 111

Slide 111 text

Yeah… sure.

Slide 112

Slide 112 text

Or…

Slide 113

Slide 113 text

Are you sure?

Slide 114

Slide 114 text

KafkaProducer#send(ProducerRecord,Callback) @bsideup

Slide 115

Slide 115 text

KafkaProducer#send(ProducerRecord,Callback) “Asynchronously send a record to a topic and invoke the provided callback when the send has been acknowledged.” - Javadoc @bsideup

Slide 116

Slide 116 text

KafkaProducer#send(ProducerRecord,Callback) “Asynchronously send a record to a topic and invoke the provided callback when the send has been acknowledged.” - Javadoc java.lang.Error: Blocking call! java.lang.Object#wait at reactor.BlockHound$Builder.lambda$new$0(BlockHound.java:154) at reactor.BlockHound$Builder.lambda$install$8(BlockHound.java:254) at reactor.BlockHoundRuntime.checkBlocking(BlockHoundRuntime.java:43) at java.lang.Object.wait(Object.java) at org.apache.kafka.clients.Metadata.awaitUpdate(Metadata.java:181) at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:938) at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:823) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:803) @bsideup

Slide 117

Slide 117 text

KafkaProducer#send(ProducerRecord,Callback) “Asynchronously send a record to a topic and invoke the provided callback when the send has been acknowledged.” - Javadoc java.lang.Error: Blocking call! java.lang.Object#wait at reactor.BlockHound$Builder.lambda$new$0(BlockHound.java:154) at reactor.BlockHound$Builder.lambda$install$8(BlockHound.java:254) at reactor.BlockHoundRuntime.checkBlocking(BlockHoundRuntime.java:43) at java.lang.Object.wait(Object.java) at org.apache.kafka.clients.Metadata.awaitUpdate(Metadata.java:181) at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:938) at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:823) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:803) https://issues.apache.org/jira/browse/KAFKA-3539 @bsideup

Slide 118

Slide 118 text

long remainingWaitMs = maxWaitMs; long elapsed; // Issue metadata requests until we have metadata for the topic or maxWaitTimeMs is exceeded. // In case we already have cached metadata for the topic, but the requested partition is greater // than expected, issue an update request only once. This is necessary in case the metadata // is stale and the number of partitions for this topic has increased in the meantime. do { log.trace("Requesting metadata update for topic {}.", topic); metadata.add(topic); int version = metadata.requestUpdate(); sender.wakeup(); try { metadata.awaitUpdate(version, remainingWaitMs); } catch (TimeoutException ex) { // Rethrow with original maxWaitMs to prevent logging exception with remainingWaitMs throw new TimeoutException("Failed to update metadata after " + maxWaitMs + " ms."); } cluster = metadata.fetch(); elapsed = time.milliseconds() - begin; if (elapsed >= maxWaitMs) throw new TimeoutException("Failed to update metadata after " + maxWaitMs + " ms."); if (cluster.unauthorizedTopics().contains(topic)) throw new TopicAuthorizationException(topic); remainingWaitMs = maxWaitMs - elapsed; partitionsCount = cluster.partitionCountForTopic(topic); } while (partitionsCount == null); waitOnMetadata(record.topic(), record.partition(), maxBlockTimeMs); Default is “60 seconds” @bsideup

Slide 119

Slide 119 text

@bsideup

Slide 120

Slide 120 text

How to “instrument” Reactor

Slide 121

Slide 121 text

Schedulers.onScheduleHook(fn) @bsideup

Slide 122

Slide 122 text

Schedulers.onScheduleHook(fn) @bsideup Schedulers.onScheduleHook("myHook", runnable -> { println("Before every scheduled runnable"); return () -> { println("Before execution"); runnable.run(); println("After execution"); }; });

Slide 123

Slide 123 text

Schedulers.onScheduleHook(fn) @bsideup Schedulers.onScheduleHook("mdc", runnable -> { String userId = MDC.get("userId"); return () -> { MDC.put("userId", userId); try { runnable.run(); } finally { MDC.remove("userId"); } }; });

Slide 124

Slide 124 text

Hooks.onLastOperator(fn) @bsideup Flux.range(0, 5) .map(it -> it.toString()) .single() .subscribeOn(Schedulers.parallel()) .subscribe();

Slide 125

Slide 125 text

Hooks.onLastOperator(fn) @bsideup Flux.range(0, 5) .map(it -> it.toString()) .single() .subscribeOn(Schedulers.parallel()) .transform(fn) .subscribe();

Slide 126

Slide 126 text

Your Reactor should be like:

Slide 127

Slide 127 text

https://9gag.com/gag/aqKWPwY

Slide 128

Slide 128 text

Takeaways • io.projectreactor:reactor-tools:3.3.0.RELEASE • Flux#checkpoint() / Flux#log() • Hooks class • https://github.com/reactor/BlockHound • Async APIs - trust, but verify @bsideup

Slide 129

Slide 129 text

@bsideup bsideup