Tweet
Tweet

Slide 1

Slide 1 text

Don’t be Homer Simpson 
 with your Reactor! Sergei Egorov, Pivotal @bsideup https://simpsonswiki.com/wiki/File:Springfield_Nuclear_Power_Plant_6.png

Slide 2

Slide 2 text

Safe Harbor Statement The following is intended to outline the general direction of Pivotal's offerings. It is intended for information purposes only and may not be incorporated into any contract. Any information regarding pre-release of Pivotal offerings, future updates or other planned modifications is subject to ongoing evaluation by Pivotal and is subject to change. This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions regarding Pivotal's offerings. These purchasing decisions should only be based on features currently available. The development, release, and timing of any features or functionality described for Pivotal's offerings in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to update forward looking information in this presentation.

Slide 3

Slide 3 text

About me • Staff Engineer at Pivotal’s Spring R&D, working on Project Reactor ⚛ • Testcontainers co-maintainer • Developer tools geek • … although spent most of my career building
 highly scalable backend services @bsideup

Slide 4

Slide 4 text

My first experience with
 reactive programming was like…

Slide 5

Slide 5 text

@bsideup

Slide 6

Slide 6 text

alright… alright…

Slide 7

Slide 7 text

@bsideup

Slide 8

Slide 8 text

Mono.fromRunnable()

Slide 9

Slide 9 text

Mono.fromRunnable() @bsideup

Slide 10

Slide 10 text

Mono.fromRunnable() My code @bsideup

Slide 11

Slide 11 text

Mono.fromRunnable() My code My code @bsideup

Slide 12

Slide 12 text

Mono.fromRunnable() My code My code My code Work! @bsideup

Slide 13

Slide 13 text

Mono.fromRunnable().subscribe() @bsideup

Slide 14

Slide 14 text

1 week later…

Slide 15

Slide 15 text

I agree with Heinrich Apfelmus that the essence of functional reactive programming is to specify the dynamic behavior of a value completely at the time of declaration. “Reactive Sir” @bsideup

Slide 16

Slide 16 text

1 week of production later…

Slide 17

Slide 17 text

@bsideup

Slide 18

Slide 18 text

What could go wrong

Slide 19

Slide 19 text

Whoa #1: exceptions

Slide 20

Slide 20 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); @bsideup

Slide 21

Slide 21 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) @bsideup

Slide 22

Slide 22 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) @bsideup

Slide 23

Slide 23 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Which source? @bsideup

Slide 24

Slide 24 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Which source? Where is “com.example.demo” package? @bsideup

Slide 25

Slide 25 text

// com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Which source? Where is “com.example.demo” package? Does anyone rent an apartment in Prenzlauer Berg? @bsideup

Slide 26

Slide 26 text

Assembly vs Subscription

Slide 27

Slide 27 text

Flux.range(0, 5) Assembly @bsideup

Slide 28

Slide 28 text

Flux.range(0, 5) .filter(it -> it > 3) Assembly @bsideup

Slide 29

Slide 29 text

Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) Assembly @bsideup

Slide 30

Slide 30 text

Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) .single() Assembly @bsideup

Slide 31

Slide 31 text

Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) .single() .subscribeOn(Schedulers.parallel()) Assembly @bsideup

Slide 32

Slide 32 text

Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) .single() .subscribeOn(Schedulers.parallel()) .subscribe() Assembly @bsideup

Slide 33

Slide 33 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 34

Slide 34 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 * = subscriber @bsideup

Slide 35

Slide 35 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 36

Slide 36 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 37

Slide 37 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 38

Slide 38 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 39

Slide 39 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .onSubscribe( ) * = subscription @bsideup

Slide 40

Slide 40 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 41

Slide 41 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 42

Slide 42 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 43

Slide 43 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup

Slide 44

Slide 44 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 45

Slide 45 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 46

Slide 46 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 47

Slide 47 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 48

Slide 48 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 49

Slide 49 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup

Slide 50

Slide 50 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup

Slide 51

Slide 51 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup

Slide 52

Slide 52 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup

Slide 53

Slide 53 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup

Slide 54

Slide 54 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup

Slide 55

Slide 55 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(1) @bsideup

Slide 56

Slide 56 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(1) @bsideup

Slide 57

Slide 57 text

FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(1) java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(Mono at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(Flux at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxR at reactor.core.publisher.MonoSingle$SingleSubscriber.request(Mono at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber. at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber. at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(M at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber. @bsideup

Slide 58

Slide 58 text

Demo http://eskipaper.com/homer-jay-simpson-cartoon.html

Slide 59

Slide 59 text

Demo outcomes • Use .checkpoint(“something”) to “mark” reactive “milestones” • Read about Hooks.onOperatorDebug()… • … but use reactor-tools’ ReactorDebugAgent (works in prod too) • https://spring.io/blog/2019/03/06/flight-of-the-flux-1-assembly-vs- subscription - great article from Simon Basle about the internals @bsideup

Slide 60

Slide 60 text

Whoa #2: Blocking calls! https://blog.tfd.co.uk/2010/10/15/jackrabbit-performance/ @bsideup

Slide 61

Slide 61 text

How Reactor works @bsideup

Slide 62

Slide 62 text

Default thread pools Schedulers.parallel() - N threads, where N matches the CPUs count. Schedulers.single() - 1 thread handling all submitted tasks Schedulers.elastic() - dynamic, thread caching pool @bsideup

Slide 63

Slide 63 text

Mono.delay(ofSeconds(1)) @bsideup

Slide 64

Slide 64 text

Mono.delay(ofSeconds(1)) Mono.delay(ofSeconds(1), Schedulers.parallel()) @bsideup

Slide 65

Slide 65 text

Mono.delay(ofSeconds(1)) Mono.delay(ofSeconds(1), Schedulers.parallel()) Every non-instant operation 
 runs on the parallel scheduler
 by default! @bsideup

Slide 66

Slide 66 text

parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup

Slide 67

Slide 67 text

parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup

Slide 68

Slide 68 text

parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup

Slide 69

Slide 69 text

parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup

Slide 70

Slide 70 text

parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup

Slide 71

Slide 71 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 72

Slide 72 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 73

Slide 73 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 74

Slide 74 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 75

Slide 75 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 76

Slide 76 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 77

Slide 77 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 78

Slide 78 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 79

Slide 79 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup

Slide 80

Slide 80 text

parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution ⚠ no more tasks scheduled 
 until this task returns @bsideup

Slide 81

Slide 81 text

How to fix?

Slide 82

Slide 82 text

Use non-blocking APIs, or…

Slide 83

Slide 83 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 84

Slide 84 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 85

Slide 85 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 86

Slide 86 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 87

Slide 87 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 88

Slide 88 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 89

Slide 89 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 90

Slide 90 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 91

Slide 91 text

parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup

Slide 92

Slide 92 text

Demo http://eskipaper.com/homer-jay-simpson-cartoon.html

Slide 93

Slide 93 text

Demo outcomes • Blocking calls are bad, m’kay? • They may sneak into your production system • Use https://github.com/reactor/BlockHound to detect them • Use a dedicated pool for the necessary blocking calls, or schedule them on the Schedulers.elastic() built-in pool if they happen rarely @bsideup

Slide 94

Slide 94 text

“I use async APIs and I am safe!”

Slide 95

Slide 95 text

Yeah… sure.

Slide 96

Slide 96 text

Or…

Slide 97

Slide 97 text

Are you sure?

Slide 98

Slide 98 text

KafkaProducer#send(ProducerRecord,Callback) @bsideup

Slide 99

Slide 99 text

KafkaProducer#send(ProducerRecord,Callback) “Asynchronously send a record to a topic and invoke the provided callback when the send has been acknowledged.” - Javadoc @bsideup

Slide 100

Slide 100 text

KafkaProducer#send(ProducerRecord,Callback) “Asynchronously send a record to a topic and invoke the provided callback when the send has been acknowledged.” - Javadoc java.lang.Error: Blocking call! java.lang.Object#wait at reactor.BlockHound$Builder.lambda$new$0(BlockHound.java:154) at reactor.BlockHound$Builder.lambda$install$8(BlockHound.java:254) at reactor.BlockHoundRuntime.checkBlocking(BlockHoundRuntime.java:43) at java.lang.Object.wait(Object.java) at org.apache.kafka.clients.Metadata.awaitUpdate(Metadata.java:181) at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:938) at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:823) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:803) @bsideup

Slide 101

Slide 101 text

KafkaProducer#send(ProducerRecord,Callback) “Asynchronously send a record to a topic and invoke the provided callback when the send has been acknowledged.” - Javadoc java.lang.Error: Blocking call! java.lang.Object#wait at reactor.BlockHound$Builder.lambda$new$0(BlockHound.java:154) at reactor.BlockHound$Builder.lambda$install$8(BlockHound.java:254) at reactor.BlockHoundRuntime.checkBlocking(BlockHoundRuntime.java:43) at java.lang.Object.wait(Object.java) at org.apache.kafka.clients.Metadata.awaitUpdate(Metadata.java:181) at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:938) at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:823) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:803) @bsideup

Slide 102

Slide 102 text

KafkaProducer#send(ProducerRecord,Callback) “Asynchronously send a record to a topic and invoke the provided callback when the send has been acknowledged.” - Javadoc java.lang.Error: Blocking call! java.lang.Object#wait at reactor.BlockHound$Builder.lambda$new$0(BlockHound.java:154) at reactor.BlockHound$Builder.lambda$install$8(BlockHound.java:254) at reactor.BlockHoundRuntime.checkBlocking(BlockHoundRuntime.java:43) at java.lang.Object.wait(Object.java) at org.apache.kafka.clients.Metadata.awaitUpdate(Metadata.java:181) at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:938) at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:823) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:803) https://issues.apache.org/jira/browse/KAFKA-3539 @bsideup

Slide 103

Slide 103 text

long remainingWaitMs = maxWaitMs; long elapsed; // Issue metadata requests until we have metadata for the topic or maxWaitTimeMs is exceeded. // In case we already have cached metadata for the topic, but the requested partition is greater // than expected, issue an update request only once. This is necessary in case the metadata // is stale and the number of partitions for this topic has increased in the meantime. do { log.trace("Requesting metadata update for topic {}.", topic); metadata.add(topic); int version = metadata.requestUpdate(); sender.wakeup(); try { metadata.awaitUpdate(version, remainingWaitMs); } catch (TimeoutException ex) { // Rethrow with original maxWaitMs to prevent logging exception with remainingWaitMs throw new TimeoutException("Failed to update metadata after " + maxWaitMs + " ms."); } cluster = metadata.fetch(); elapsed = time.milliseconds() - begin; if (elapsed >= maxWaitMs) throw new TimeoutException("Failed to update metadata after " + maxWaitMs + " ms."); if (cluster.unauthorizedTopics().contains(topic)) throw new TopicAuthorizationException(topic); remainingWaitMs = maxWaitMs - elapsed; partitionsCount = cluster.partitionCountForTopic(topic); } while (partitionsCount == null); waitOnMetadata(record.topic(), record.partition(), maxBlockTimeMs); Default is “60 seconds” @bsideup

Slide 104

Slide 104 text

@bsideup

Slide 105

Slide 105 text

Tips & tricks http://eskipaper.com/homer-jay-simpson-cartoon.html @bsideup

Slide 106

Slide 106 text

Your Reactor should be like:

Slide 107

Slide 107 text

https://9gag.com/gag/aqKWPwY

Slide 108

Slide 108 text

Takeaways • https://github.com/reactor/reactor-tools • Flux#checkpoint() / Flux#log() • https://github.com/reactor/BlockHound • Async APIs - trust, but verify @bsideup

Slide 109

Slide 109 text

@bsideup bsideup