GeekOut 2019: Don’t be Homer Simpson with your Reactor!

https://2019.geekout.ee/sergei-egorov#talk

Transcript

1. Don’t be Homer Simpson 
 with your Reactor! Sergei Egorov, Pivotal @bsideup https://simpsonswiki.com/wiki/File:Springfield_Nuclear_Power_Plant_6.png
2. About me • Staff Engineer at Pivotal’s Spring R&D, working on Project Reactor ⚛ • Berlin Spring User Group co-organizer • Testcontainers co-maintainer • Developer tools geek • … although spent most of my career building
 highly scalable backend services @bsideup
3. My first experience with
 reactive programming was like…
4. @bsideup
5. @bsideup
6. Mono.fromRunnable()
7. Mono.fromRunnable() @bsideup
8. Mono.fromRunnable() My code @bsideup
9. Mono.fromRunnable() My code My code @bsideup
10. Mono.fromRunnable() My code My code My code Work! @bsideup
11. Mono.fromRunnable().subscribe() @bsideup
12. 1 week later…
13. I agree with Heinrich Apfelmus that the essence of functional reactive programming is to specify the dynamic behavior of a value completely at the time of declaration. “Reactive Sir” @bsideup
14. 1 week of production later…
15. @bsideup
16. What could go wrong
17. Exceptions
18. // com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); @bsideup
19. // com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) @bsideup
20. // com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) @bsideup
21. // com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Which source? @bsideup
22. // com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Which source? Where is “com.example.demo” package? @bsideup
23. // com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(MonoSingle.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(FluxRange.java:129) at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:107) at reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.trySchedule(MonoSubscribeOn.java:186) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.onSubscribe(MonoSubscribeOn.java:131) at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114) at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:123) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Which source? Where is “com.example.demo” package? Why they closed Google Inbox? @bsideup
24. Assembly vs Subscription
25. Flux.range(0, 5) Assembly @bsideup
26. Flux.range(0, 5) .filter(it -> it > 3) Assembly @bsideup
27. Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) Assembly @bsideup
28. Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) .single() Assembly @bsideup
29. Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) .single() .subscribeOn(Schedulers.parallel()) Assembly @bsideup
30. Flux.range(0, 5) .filter(it -> it > 3) .map(it -> it.toString()) .single() .subscribeOn(Schedulers.parallel()) .subscribe() Assembly @bsideup
31. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup
32. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 * = subscriber @bsideup
33. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup
34. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup
35. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup
36. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup
37. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .onSubscribe( ) * = subscription @bsideup
38. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup
39. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup
40. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup
41. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 @bsideup
42. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup
43. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup
44. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup
45. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup
46. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup
47. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Subscription current thread parallel-1 .request(max) @bsideup
48. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup
49. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup
50. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup
51. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup
52. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(0) @bsideup
53. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(1) @bsideup
54. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(1) @bsideup
55. FluxRange FluxFilter FluxMap MonoSingle MonoSubscribeOn Execution current thread parallel-1 .onNext(1) java.lang.IndexOutOfBoundsException: Source emitted more than one item at reactor.core.publisher.MonoSingle$SingleSubscriber.onNext(Mono at reactor.core.publisher.FluxRange$RangeSubscription.fastPath(Flux at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxR at reactor.core.publisher.MonoSingle$SingleSubscriber.request(Mono at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber. at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber. at reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(M at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:68) at reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58) at reactor.core.publisher.Mono.subscribe(Mono.java:3711) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber. @bsideup
56. Demo http://eskipaper.com/homer-jay-simpson-cartoon.html
57. How ReactorDebugAgent works? @bsideup
58. // com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); @bsideup
59. // com/example/demo/Example.java Flux.range(0, 5) .single() .subscribeOn(Schedulers.parallel()) .subscribe(); @bsideup
60. // com/example/demo/Example.java Flux.range(0, 5) .checkpoint("Example.java:16") .single() .checkpoint("Example.java:17") .subscribeOn(Schedulers.parallel()) .checkpoint(“Example.java:18") .subscribe(); @bsideup
61. Demo outcomes • Use .checkpoint(“something”) to “mark” reactive “milestones” • Read about Hooks.onOperatorDebug()… • … but use reactor-tools’ ReactorDebugAgent (works in prod too) • https://spring.io/blog/2019/03/06/flight-of-the-flux-1-assembly-vs- subscription - great article from Simon Basle about the internals @bsideup
62. Blocking calls! https://blog.tfd.co.uk/2010/10/15/jackrabbit-performance/ @bsideup
63. How Reactor schedules tasks @bsideup
64. Default thread pools Schedulers.parallel() - N threads, where N matches the CPUs count. Schedulers.single() - 1 thread handling all submitted tasks Schedulers.elastic() - dynamic, thread caching pool @bsideup
65. Mono.delay(ofSeconds(1)) @bsideup
66. Mono.delay(ofSeconds(1)) Mono.delay(ofSeconds(1), Schedulers.parallel()) @bsideup
67. Mono.delay(ofSeconds(1)) Mono.delay(ofSeconds(1), Schedulers.parallel()) Every non-instant operation 
 runs on the parallel scheduler
 by default! @bsideup
68. parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup
69. parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup
70. parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup
71. parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup
72. parallel-1 parallel-2 parallel-3 parallel-4 Non-blocking execution @bsideup
73. parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup
74. parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup
75. parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup
76. parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup
77. parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup
78. parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup
79. parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup
80. parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup
81. parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution @bsideup
82. parallel-1 parallel-2 parallel-3 parallel-4 Blocking execution ⚠ no more tasks scheduled 
 until this task returns @bsideup
83. How to fix?
84. Project Loom
85. @bsideup
86. @bsideup
87. Use non-blocking APIs, or…
88. parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup
89. parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup
90. parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup
91. parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup
92. parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup
93. parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup
94. parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup
95. parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup
96. parallel-1 parallel-2 parallel-3 parallel-4 Custom thread pool! custom-1 custom-2 @bsideup
97. Demo http://eskipaper.com/homer-jay-simpson-cartoon.html
98. How BlockHound works? @bsideup
99. public class Thread implements Runnable { public static native void sleep(long millis); } @bsideup
100. public class Thread implements Runnable { private static native void $$BlockHound$$_sleep(long millis); public static void sleep(long millis) { $$BlockHound$$_sleep(millis); } } @bsideup
101. public class Thread implements Runnable { private static native void $$BlockHound$$_sleep(long millis); public static void sleep(long millis) { reactor.BlockHoundRuntime.checkBlocking( "java.lang.Thread", "sleep", /*method modifiers*/ ); $$BlockHound$$_sleep(millis); } } @bsideup
102. public class Thread implements Runnable { private static native void $$BlockHound$$_sleep(long millis); public static void sleep(long millis) { reactor.BlockHoundRuntime.checkBlocking( "java.lang.Thread", "sleep", /*method modifiers*/ ); $$BlockHound$$_sleep(millis); } } @bsideup
103. Blocking check static void checkBlocking(String className, String methodName, int modifiers) { if (Boolean.FALSE == IS_ALLOWED.get()) { // Report } } static final ThreadLocal<Boolean> IS_ALLOWED = ThreadLocal.withInitial(() -> { if (threadPredicate.test(Thread.currentThread())) { return false; } else { // Optimization: use Three-state (true, false, null) // where `null` is `not non-blocking` return null; } }); @bsideup
104. Blocking check static void checkBlocking(String className, String methodName, int modifiers) { if (Boolean.FALSE == IS_ALLOWED.get()) { // Report } } static final ThreadLocal<Boolean> IS_ALLOWED = ThreadLocal.withInitial(() -> { if (threadPredicate.test(Thread.currentThread())) { return false; } else { // Optimization: use Three-state (true, false, null) // where `null` is `not non-blocking` return null; } }); @bsideup
105. Allowed/disallowed @bsideup
106. Allowed/disallowed class ClassLoader { // ... public Class<?> loadClass(String name) { return loadClass(name, false); } } @bsideup
107. Allowed/disallowed class ClassLoader { // ... public Class<?> loadClass(String name) { return loadClass(name, false); } } @bsideup Blocking!
108. Allowed/disallowed class ClassLoader { // ... public Class<?> loadClass(String name) { Boolean previous = BlockHoundRuntime.IS_ALLOWED.get(); BlockHoundRuntime.IS_ALLOWED.set( previous != null ? true : null ); try { // Original call return loadClass(name, false); } finally { BlockHoundRuntime.IS_ALLOWED.set(previous); } } } @bsideup
109. Demo outcomes • Blocking calls are bad, m’kay? • They may sneak into your production system • Use https://github.com/reactor/BlockHound to detect them • Supports multiple frameworks (Reactor, RxJava, etc) • … and maybe even Kotlin: 
 https://github.com/Kotlin/kotlinx.coroutines/issues/1031 • Use a dedicated pool for the necessary blocking calls, or schedule them on the Schedulers.elastic() built-in pool if they happen rarely @bsideup
110. “I use async APIs and I am safe!”
111. Yeah… sure.
112. Or…
113. Are you sure?
114. KafkaProducer#send(ProducerRecord,Callback) @bsideup
115. KafkaProducer#send(ProducerRecord,Callback) “Asynchronously send a record to a topic and invoke the provided callback when the send has been acknowledged.” - Javadoc @bsideup
116. KafkaProducer#send(ProducerRecord,Callback) “Asynchronously send a record to a topic and invoke the provided callback when the send has been acknowledged.” - Javadoc java.lang.Error: Blocking call! java.lang.Object#wait at reactor.BlockHound$Builder.lambda$new$0(BlockHound.java:154) at reactor.BlockHound$Builder.lambda$install$8(BlockHound.java:254) at reactor.BlockHoundRuntime.checkBlocking(BlockHoundRuntime.java:43) at java.lang.Object.wait(Object.java) at org.apache.kafka.clients.Metadata.awaitUpdate(Metadata.java:181) at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:938) at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:823) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:803) @bsideup
117. KafkaProducer#send(ProducerRecord,Callback) “Asynchronously send a record to a topic and invoke the provided callback when the send has been acknowledged.” - Javadoc java.lang.Error: Blocking call! java.lang.Object#wait at reactor.BlockHound$Builder.lambda$new$0(BlockHound.java:154) at reactor.BlockHound$Builder.lambda$install$8(BlockHound.java:254) at reactor.BlockHoundRuntime.checkBlocking(BlockHoundRuntime.java:43) at java.lang.Object.wait(Object.java) at org.apache.kafka.clients.Metadata.awaitUpdate(Metadata.java:181) at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:938) at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:823) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:803) https://issues.apache.org/jira/browse/KAFKA-3539 @bsideup
118. long remainingWaitMs = maxWaitMs; long elapsed; // Issue metadata requests until we have metadata for the topic or maxWaitTimeMs is exceeded. // In case we already have cached metadata for the topic, but the requested partition is greater // than expected, issue an update request only once. This is necessary in case the metadata // is stale and the number of partitions for this topic has increased in the meantime. do { log.trace("Requesting metadata update for topic {}.", topic); metadata.add(topic); int version = metadata.requestUpdate(); sender.wakeup(); try { metadata.awaitUpdate(version, remainingWaitMs); } catch (TimeoutException ex) { // Rethrow with original maxWaitMs to prevent logging exception with remainingWaitMs throw new TimeoutException("Failed to update metadata after " + maxWaitMs + " ms."); } cluster = metadata.fetch(); elapsed = time.milliseconds() - begin; if (elapsed >= maxWaitMs) throw new TimeoutException("Failed to update metadata after " + maxWaitMs + " ms."); if (cluster.unauthorizedTopics().contains(topic)) throw new TopicAuthorizationException(topic); remainingWaitMs = maxWaitMs - elapsed; partitionsCount = cluster.partitionCountForTopic(topic); } while (partitionsCount == null); waitOnMetadata(record.topic(), record.partition(), maxBlockTimeMs); Default is “60 seconds” @bsideup
119. @bsideup
120. How to “instrument” Reactor
121. Schedulers.onScheduleHook(fn) @bsideup
122. Schedulers.onScheduleHook(fn) @bsideup Schedulers.onScheduleHook("myHook", runnable -> { println("Before every scheduled runnable"); return () -> { println("Before execution"); runnable.run(); println("After execution"); }; });
123. Schedulers.onScheduleHook(fn) @bsideup Schedulers.onScheduleHook("mdc", runnable -> { String userId = MDC.get("userId"); return () -> { MDC.put("userId", userId); try { runnable.run(); } finally { MDC.remove("userId"); } }; });
124. Hooks.onLastOperator(fn) @bsideup Flux.range(0, 5) .map(it -> it.toString()) .single() .subscribeOn(Schedulers.parallel()) .subscribe();
125. Hooks.onLastOperator(fn) @bsideup Flux.range(0, 5) .map(it -> it.toString()) .single() .subscribeOn(Schedulers.parallel()) .transform(fn) .subscribe();
126. Your Reactor should be like:
127. https://9gag.com/gag/aqKWPwY
128. Takeaways • io.projectreactor:reactor-tools:3.3.0.RELEASE • Flux#checkpoint() / Flux#log() • Hooks class • https://github.com/reactor/BlockHound • Async APIs - trust, but verify @bsideup
129. @bsideup bsideup