Flight of the Flux

A look at Reactor's execution model

Transcript

1. Flight of the Flux
   a look at Reactor's execution model
   

   View Slide

2. Flight of the Flux
   a look at Reactor's execution model
   @simonbasle
   

   View Slide

3. What we'll learn
   

   View Slide

4. View Slide

5. Nothing happens
   until you subscribe,
   Assembly time vs
   Execution time
   

   View Slide

6. Nothing happens
   until you subscribe,
   Assembly time vs Execution time
   Cold or Hot?
   

   View Slide

7. Nothing happens
   until you subscribe,
   Assembly time vs Execution time
   Cold or Hot?
   Concurrent
   agnostic
   

   View Slide

8. Cold or Hot?
   Concurrent
   agnostic
   Schedulers,
   PublishOn vs
   SubscribeOn
   Nothing happens
   until you subscribe,
   Assembly time vs Execution time
   

   View Slide

9. Cold or Hot?
   Concurrent
   agnostic
   Schedulers,
   PublishOn vs
   SubscribeOn
   Work Stealing
   Nothing happens
   until you subscribe,
   Assembly time vs Execution time
   

   View Slide

10. Cold or Hot?
    Concurrent
    agnostic
    Schedulers,
    PublishOn vs
    SubscribeOn
    Work Stealing
    Operator Fusion
    Nothing happens
    until you subscribe,
    Assembly time vs Execution time
    

    View Slide

11. Cold or Hot?
    Concurrent
    agnostic
    Schedulers,
    PublishOn vs
    SubscribeOn
    Work Stealing
    Operator Fusion
    Nothing happens
    until you subscribe,
    Assembly time vs Execution time
    

    View Slide

12. View Slide

13. Publisher
    Subscriber
    

    View Slide

14. View Slide

15. ...
    

    View Slide

16. ...
    

    View Slide

17. ...
    

    View Slide

18. Subscription
    

    View Slide

19. Nothing Happens until you subscribe
    Assembly time
    

    View Slide

20. Nothing happen until you
    subscribe
    

    View Slide

21. Assembly time
    

    View Slide

22. this.myFlux =
    Flux.just("foo")
    .map(String::length);
    MyTest#init()
    

    View Slide

23. Execution time
    

    View Slide

24. Execution time
    Subscription
    

    View Slide

25. this.myFlux
    .subscribe();
    MyTest#test()
    

    View Slide

26. what would I see in a stack trace?
    

    View Slide

27. java.lang.ArithmeticException: / by zero
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:115)
    at reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:99)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.BlockingSingleSubscriber.onSubscribe(BlockingSingleSubscriber.java:49)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxJust.subscribe(FluxJust.java:70)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.Flux.subscribe(Flux.java:6873)
    at mycode.test(MyTest.java:xx)
    

    View Slide

28. java.lang.ArithmeticException: / by zero
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:115)
    at reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:99)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.BlockingSingleSubscriber.onSubscribe(BlockingSingleSubscriber.java:49)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxJust.subscribe(FluxJust.java:70)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.Flux.subscribe(Flux.java:6873)
    at mycode.test(MyTest.java:xx)
    ACCIO
    INTERESTING
    PARTS!
    

    View Slide

29. java.lang.ArithmeticException: / by zero
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:115)
    at reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:99)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.BlockingSingleSubscriber.onSubscribe(BlockingSingleSubscriber.java:49)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxJust.subscribe(FluxJust.java:70)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.Flux.subscribe(Flux.java:6873)
    at mycode.test(MyTest.java:xx)
    ACCIO
    INTERESTING
    PARTS!
    

    View Slide

30. java.lang.ArithmeticException: / by zero
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:115)
    at reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:99)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.BlockingSingleSubscriber.onSubscribe(BlockingSingleSubscriber.java:49)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxJust.subscribe(FluxJust.java:70)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.Flux.subscribe(Flux.java:6873)
    at mycode.test(MyTest.java:xx)
    

    View Slide

31. java.lang.ArithmeticException: / by zero
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:115)
    at reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:99)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.BlockingSingleSubscriber.onSubscribe(BlockingSingleSubscriber.java:49)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxJust.subscribe(FluxJust.java:70)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.Flux.subscribe(Flux.java:6873)
    at mycode.test(MyTest.java:xx)
    

    View Slide

32. java.lang.ArithmeticException: / by zero
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:115)
    at reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:99)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
    at reactor.core.publisher.BlockingSingleSubscriber.onSubscribe(BlockingSingleSubscriber.java:49)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    at reactor.core.publisher.FluxJust.subscribe(FluxJust.java:70)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    at reactor.core.publisher.Flux.subscribe(Flux.java:6873)
    at mycode.test(MyTest.java:xx)
    

    View Slide

33. checkpoint()
    

    View Slide

34. Hot or Cold ?
    

    View Slide

35. Cold publisher
    

    View Slide

36. View Slide

37. View Slide

38. View Slide

39. View Slide

40. View Slide

41. View Slide

42. View Slide

43. always true... until it's false
    " Nothing happen until you subscribe "
    

    View Slide

44. Hot publisher
    

    View Slide

45. View Slide

46. View Slide

47. Concurrent
    Agnostic
    

    View Slide

48. do I have to accept a particular
    execution model?
    

    View Slide

49. No, we provide
    tools to fine tune
    

    View Slide

50. But, frameworks & libraries built on top
    might be opinionated
    

    View Slide

51. what about time operators?
    

    View Slide

52. ...they do need to change threads
    

    View Slide

53. Schedulers
    & how to switch threads
    

    View Slide

54. Scheduler abstraction
    

    View Slide

55. , part
    part
    

    View Slide

56. publishOn
    

    View Slide

57. Flux.op1.op2.publishOn.op3.op4 .subscribe
    

    View Slide

58. 1
    2
    Flux.op1.op2.publishOn.op3.op4 .subscribe
    

    View Slide

59. Thread calling subscribe()
    is where data flows initially
    1
    

    View Slide

60. After publishOn
    data flows in Thread 2
    2
    

    View Slide

61. subscribeOn
    

    View Slide

62. Flux.op1.op2 .subscribeOn .op3.op4 .subscribe
    

    View Slide

63. 2
    Flux.op1.op2 .subscribeOn .op3.op4 .subscribe
    1
    

    View Slide

64. subscribeOn() changes where
    sequence subscription happens...
    1
    

    View Slide

65. ... which is also
    where data flows initially
    2
    

    View Slide

66. and the two together?
    

    View Slide

67. Flux .op1 .subscribeOn .op2 .op4 .subscribe(
    v -> doSomething(v))
    .publishOn
    

    View Slide

68. Flux .op1 .subscribeOn .op2 .op4 .subscribe(
    v -> doSomething(v))
    .publishOn
    

    View Slide

69. Flux .op1 .subscribeOn .op2 .op4 .subscribe(
    v -> doSomething(v))
    .publishOn
    1
    2 3 4
    

    View Slide

70. Work
    Stealing
    

    View Slide

71. so where do (most) operators run, again?
    

    View Slide

72. publishOn
    

    View Slide

73. what is work stealing?
    

    View Slide

74. 1 operator that
    combines data coming
    from 2 threads
    

    View Slide

75. shared
    queue,
    drain loop
    

    View Slide

76. offer
    

    View Slide

77. offer
    work
    

    View Slide

78. offer
    offer
    work
    

    View Slide

79. offer
    offer
    work
    steal
    exit
    

    View Slide

80. offer
    offer
    work
    steal
    exit
    

    View Slide

81. offer
    offer
    work
    steal
    exit
    work
    

    View Slide

82. offer
    offer
    work
    steal
    exit
    work
    exit
    

    View Slide

83. Operator
    Fusion
    

    View Slide

84. macro fusion
    

    View Slide

85. a.then(b).then(c).then(d)
    

    View Slide

86. a.then(b,c,d)
    

    View Slide

87. micro fusion
    

    View Slide

88. a.map(->).map(->).filter(->)
    

    View Slide

89. a.map(->).map(->).filter(->)
    

    View Slide

90. a.map(->).map(->).filter(->)
    bounded queues (for backpressure)
    

    View Slide

91. a.map(->).map(->).filter(->)
    

    View Slide

92. a.map(->).map(->).filter(->)
    single transverse bounded queue
    

    View Slide

93. View Slide

94. Questions?
    

    View Slide

95. Thanks!
    

    View Slide

96. • geese flight: CC0 (via pxhere)
    
    • tools: CC0 via (pxhere)
    
    • harry potter: © Warner Bros (fair use)
    
    • badly drawn ron: CC0 (via pixabay)
    • fire and ice cubes: CC0 (via pxhere)
    
    • blank canvas: CC0 (via publicdomainpictures)
    
    • rail switch: CC0 (via pxhere)
    
    • pickpocket: CC-By by Wellcome Library, London (via wikimedia commons)
    
    • mask icon: CC-BySa by Angelus (via wikimedia commons)
    • metal fusion: CC-BySa by P Sakthy (via wikimedia commons)
    
    • The End sand: CC0 by ElektroPlan (via pixabay)
    Image Credits
    

    View Slide