An intro to Reactive Extensions

An not so short introduction to Reactive Extensions

https://www.youtube.com/watch?v=YnL1dPyFyhg

Transcript

1. Reactive Extensions 入門勉強会 #漫才ではない Benoît Quenaudon 金子 良太
2. Agenda • Why Reactive? ◦ Problem with Android ◦ Problem with Back-end • What is Reactive Extensions? ◦ Observables ◦ Operators ◦ Schedulers • Use Cases ◦ API gateway pattern ◦ Data writing across distributed systems
3. Why Reactive? #Android
4. Why Reactive? #android interface UserManager { User getUser(); void setName(String name); void setAge(int age); }
5. Why Reactive? #android interface UserManager { User getUser(); void setName(String name); void setAge(int age); } UserManager um = new UserManager(); System.out.println(um.getUser()); um.setName("Bo Jackson"); System.out.println(um.getUser());
6. interface UserManager { User getUser(); void setName(String name); // <-- now async void setAge(int age); // <-- now async }
7. interface UserManager { User getUser(); void setName(String name, Runnable callback); void setAge(int age, Runnable callback); }
8. interface UserManager { User getUser(); void setName(String name, Runnable callback); void setAge(int age, Runnable callback); } UserManager um = new UserManager(); System.out.println(um.getUser()); um.setName("Bo Jackson", new Runnable() { @Override public void run() { System.out.println(um.getUser()); } });
9. interface UserManager { User getUser(); void setName(String name, Listener listener); void setAge(int age, Listener listener); interface Listener { void success(User user); void failure(IOException exception); } }
10. UserManager um = new UserManager(); System.out.println(um.getUser()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { System.out.println(um.getUser()); } @Override public void failure(IOException exception) { // TODO show the error... } });
11. UserManager um = new UserManager(); System.out.println(um.getUser()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { System.out.println(um.getUser()); } @Override public void failure(IOException exception) { // TODO show the error… } }); um.setAge(54, new UserManager.Listener() { @Override public void success() { System.out.println(um.getUser()); } @Override public void failure(IOException exception) { // TODO show the error… } });
12. UserManager um = new UserManager(); System.out.println(um.getUser()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { System.out.println(um.getUser()); um.setAge(54, new UserManager.Listener() { @Override public void success() { System.out.println(um.getUser()); } @Override public void failure(IOException exception) { // TODO show the error… } }); } @Override public void failure(IOException exception) { // TODO show the error… } });
13. public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { tv.setText(um.getUser().toString()); } @Override public void failure(IOException exception) { // TODO show the error… } }); } }
14. public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { tv.setText(um.getUser().toString()); } @Override public void failure(IOException exception) { // TODO show the error… } }); } }
15. public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { if (isDestroyed()) { tv.setText(um.getUser().toString()); } } @Override public void failure(IOException exception) { // TODO show the error… } }); } }
16. public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { if (isDestroyed()) { tv.setText(um.getUser().toString()); } } @Override public void failure(IOException exception) { // TODO show the error… } }); } }
17. public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { if (isDestroyed()) { tv.setText(um.getUser().toString()); } } @Override public void failure(IOException exception) { // TODO show the error… } }); } }
18. public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { runOnUiThread(new Runnable() { @Override public void run() { if (isDestroyed()) { tv.setText(um.getUser().toString()); } } }); } @Override public void failure(IOException exception) { // TODO show the error… } }); }
19. public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { runOnUiThread(new Runnable() { @Override public void run() { if (isDestroyed()) { tv.setText(um.getUser().toString()); } } }); } @Override public void failure(IOException exception) { // TODO show the error… } }); }
20. ぶっちゃけ・・・
21. 理想
22. Why Reactive? #BackEnd
23. Netflix Story
24. Client Server Old Netflix Model API
25. Problems • Low spec devices • Why not leverage the servers? ⇒ Client = one request only
26. Client Server New Netflix Model API
27. Async + Callbacks
28. Challenge • Service Composition • Error handling • Developer Productivity ⇒ Callbacks hell ⇒ Too hard
29. Async + Callbacks
30. Try • java.util.concurrent.Future ◦ Future.get() is blocking… ◦ Future<List<Future<T>>> • How about Reactive Programming? ◦ Observable<T> ⇒ Reactive Programming for the win
31. Reactive Programming ?
32. Reactive programming is programming with asynchronous data streams.
33. Reactive Extensions?
34. “ReactiveX is a library for composing asynchronous and event-based programs by using observable sequences”
35. ReactiveX • Extension of Observable Pattern • Declarative Composition of Streams • Abstraction of ◦ low-level threading, ◦ Synchronization, ◦ Thread-safety, ◦ concurrent data structures, ◦ and non-blocking I/O.
36. What is the Observable Pattern?
37. 例：Stream of Click time Click event Stream has completed Error
38. Observable Pattern • Listen to Stream = Subscribing • Only async ◦ event ⇒ funcA() ◦ error ⇒ funcB() ◦ complete ⇒ funcC() • funcA, B, C = Observers = Consumer • Stream = Subject = Observable
39. Iterator vs Observable デモンストレーション
40. event Iterable (pull) Observable (push) retrieve data T next() onNext(T) discover error throws Exception onError(Exception) complete !hasNext() onCompleted()
41. Reactive Extensions • Observables • Operators • Schedulers
42. Observables
43. Observables • Usually do work when start/stop listening • One event, many events or empty • Terminates with an error or completion • May never terminate
44. Observables interface UserManager { User getUser(); void setName(String name); void setAge(int age); }
45. Observables interface UserManager { Observable<User> getUser(); void setName(String name); void setAge(int age); }
46. Observables interface UserManager { Observable<User> getUser(); Completable setName(String name); Completable setAge(int age); }
47. Creating Observables • RxJava ◦ Observable.just("Bo Jacks"); ◦ Observable.fromArray(array); ◦ Observable.fromIterable(list); ◦ Observable.create(...); • RxJS ◦ Rx.Observable.just(42) ◦ Rx.Observable.from(iterable) ◦ Rx.Observable.create(subscribe)
48. Operators
49. Operators • Manipulate or combine data • Manipulate threading • Manipulate emissions
50. Operators String greeting = "Hello"; String yelling = greeting.toUpperCase();
51. Operators Observable<String> greeting = Observable.just("Hello"); String yelling = greeting.toUpperCase();
52. Operators Observable<String> greeting = Observable.just("Hello"); Observable<String> yelling = greeting.map(s -> s.toUpperCase());
53. Operators Observable<String> greeting = Observable.just("Hello"); Observable<String> yelling = greeting.map(s -> s.toUpperCase());
54. map as a marble diagram
55. Transforming Operators • Buffer • FlatMap • GroupBy • Map • Scan • Window
56. Filtering Operators • Debounce • Distinct • ElementAt • Filter • First • IgnoreElements • Last • Sample • Skip • SkipLast • Take
57. filter as a marble diagram
58. Combining Operators • And/Then/When • CombineLatest • Join • Merge • StartWith • Switch • Zip
59. So on... • Error Handling Operators ◦ catch, retry • Observable Utility Operators ◦ delay, timeout, observeOn, subscribeOn, etc • Conditional and Boolean Operators ◦ all, skipUntil, contains, etc • Mathematical and Aggregate Operators ◦ average, count, max, etc • etc
60. Schedulers
61. Schedulers getDataFromNetwork() // Observable<String> .skip(10) .take(5) .map(s -> s + " transformed") .subscribe(s -> println s)
62. Schedulers getDataFromNetwork() .skip(10) .take(5) .map(s -> s + " transformed") .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread())) .subscribe(s -> println s)
63. Schedulers • subscribeOn: ◦ Everything from top to the next observeOn run on my thread. • observeOn: ◦ Everything below me run on my thread.
64. Schedulers getDataFromNetwork() .skip(10) .take(5) .map(s -> s + " transformed") .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread())) .subscribe(s -> println s) IO Main
65. Examples
66. RxJS: 事前 [1, 2, 3].map(x => x + 1) // [2, 3, 4] [ [1], [2, 3], [], [4] ].concatAll() // [1, 2, 3, 4]
67. RxJS getElementDrags = function(elmt) { return elmt.mouseDowns. map(mouseDown => document.mouseMoves. takeUntil(document.mouseUps)). concatAll(); } getElementDrags(image). forEach(pos => image.position = pos);
68. D D D map(mouseDown => document.mouseMoves .takeUntil(document.mouseUps)) U U U concatAll() forEach(pos => image.position = pos) Operator Operator Observer M M M M M M M M M M
69. RxJS getElementDrags = function(elmt) { return elmt.mouseDowns. map(mouseDown => document.mouseMoves. takeUntil(document.mouseUps)). concatAll(); } getElementDrags(image). forEach(pos => image.position = pos);
70. RxJava
71. RxJava Observable<String> editText; editText .filter(text -> text.length > 2) .debounce(250, MILLISECONDS) .map(text -> api.search(text)) .subscribe(result -> showResult(result));
72. filter(text -> text.length > 2) map(text -> api.search(text)) result -> showResult(result) Operator Operator Observer k debounce(250, MILLISECONDS) Operator ka kai kaiz kaize kaizen kai kaiz kaize kaizen kaiz kaizen 250ms 250ms
73. RxJava Observable<String> editText; editText .filter(text -> text.length > 2) .debounce(250, MILLISECONDS) .map(text -> api.search(text)) .subscribe(result -> showResult(result));
74. ご注意
75. Back Pressure デモンストレーション
76. Hot vs Cold Observable • Cold ◦ Start to work on subscription • Hot ◦ Start to work on creation
77. How about Functional Reactive Programming? A different Animal
78. Use Cases
79. Background • Moore’s Law Is Dead. Now What? • We need the power. ◦ multi-devices, IoT, Big Data ◦ For organizations' scalability
80. Background • Microservices ◦ ≒ distributed systems • Need API-based collaboration • Need combination of multi tasks
81. Complexity • How do we combine multi data? • How do we resolve dependencies between tasks? • Concurrency / multi-threading / blocking • Network • Retry strategy / Message delivery reliability • Eventual consistency • Anti-fragile / Fault tolerant
82. Use case 1 Service Composition
83. Pattern: API Gateway http://microservices.io/patterns/apigateway.html
84. Scenario • API-based collaboration ◦ Authenticate ◦ Get video contents ◦ Get recommendations
85. API as a Stream
86. API as a Stream public interface AccountApi { @POST("api/account/login") Observable<LoginOutputForm> login(@Body LoginInputForm inputForm); } • Retrofit & RxJava2 Adapter • If you don't use these adapters, you can wrap API responses by yourself
87. Service as a Stream
88. Service as a Stream @Service @RequiredArgsConstructor(onConstructor = @__(@Autowired)) public class AccountService { private final ApiRegistry apiRegistry; public Observable<LoginOutputForm> login(String account, String password) { AccountApi accountApi = apiRegistry.of(AccountApi.class); return accountApi.login(new LoginInputForm(account, password)); } }
89. Service Composition
90. @RequestMapping(path = "/api/example1", method = RequestMethod.POST) public ReadMergeOutputForm run(@Validated @RequestBody ReadMergeInputForm inputForm) { return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) ) .zipWith( videoService.get() , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle(); }
91. @RequestMapping(path = "/api/example1", method = RequestMethod.POST) public ReadMergeOutputForm run(@Validated @RequestBody ReadMergeInputForm inputForm) { return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) ) .zipWith( videoService.get() , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle(); }
92. @RequestMapping(path = "/api/example1", method = RequestMethod.POST) public ReadMergeOutputForm run(@Validated @RequestBody ReadMergeInputForm inputForm) { return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) ) .zipWith( videoService.get() , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle(); }
93. @RequestMapping(path = "/api/example1", method = RequestMethod.POST) public ReadMergeOutputForm run(@Validated @RequestBody ReadMergeInputForm inputForm) { return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) ) .zipWith( videoService.get() , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle(); }
94. @RequestMapping(path = "/api/example1", method = RequestMethod.POST) public ReadMergeOutputForm run(@Validated @RequestBody ReadMergeInputForm inputForm) { return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) ) .zipWith( videoService.get() , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle(); }
95. @RequestMapping(path = "/api/example1", method = RequestMethod.POST) public ReadMergeOutputForm run(@Validated @RequestBody ReadMergeInputForm inputForm) { return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) ) .zipWith( videoService.get() , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle(); }
96. Error Handling
97. return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .map(loginOutputForm -> { if (!loginOutputForm.isSuccess()) { throw new RuntimeException(); } return loginOutputForm; }) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) ) .zipWith( videoService.get() , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle();
98. Fallback to default Or cache
99. return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .map(loginOutputForm -> { if (!loginOutputForm.isSuccess()) { throw new RuntimeException(); } return loginOutputForm; }) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) .onErrorReturnItem(RECOMMENDATIONS_ON_ERROR) ) .zipWith( videoService.get() , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle();
100. Multi-Threading
101. return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .map(loginOutputForm -> { if (!loginOutputForm.isSuccess()) { throw new RuntimeException(); } return loginOutputForm; }) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) .onErrorReturnItem(RECOMMENDATIONS_ON_ERROR) ) .zipWith( videoService.get().subscribeOn(Schedulers.computation()) , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle();
102. Logging
103. return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .doOnError(Throwable::printStackTrace) .doOnNext(loginOutputForm -> log.info(Thread.currentThread().getName()) ) .map(loginOutputForm -> { if (!loginOutputForm.isSuccess()) { throw new RuntimeException(); } return loginOutputForm; });
104. Use case 2 Write data across distributed systems
105. Consistency in a single system • Can use the transaction • e.g. ◦ When a corporation data is created, an address data is also created ◦ The corporation object manages the address object's life cycle
106. Consistency in a single system @Transactional(rollbackFor = {Exception.class}) public Corporation create(Corporation anonymousCorporation, Address anonymousAddress) { Corporation corporation = corporationRepository.save(anonymousCorporation); Address unidentifiedAddress = Address.builder() .corporationId(corporation.getId()) .state(anonymousAddress.getState()) .phone(anonymousAddress.getPhone()) .build(); Address address = addressRepository.save(unidentifiedAddress); return Corporation.aggregate(corporation, address); }
107. Eventual Consistency for multi-services • We couldn't use the transaction for the data consistency in HTTP protocol based on multi-services collaboration • Instead of the transaction, we often use the eventual consistency • We need take care of some things: ◦ Idempotency ◦ Message delivery reliability
108. Scenario • End user inputs corporation and address data • Service A enqueues with end users' input data • Worker processes the queue ◦ Worker is responsible for guaranteeing for message delivery reliability • Save corporation data in the service A with unique key message • Service A requests Service B to save address data via Web API ◦ Retry till successful response from Service B ⇒ Has to be Idempotent
119. Service down
120. Network busy
121. Guarantee the Message delivery reliability
124. Objects life cycle • Requirements ◦ When a corporation data is created, an address data should also be created.
125. Repository for creating a corporation
126. Repository public Corporation save(Corporation corporation) { Corporation saved = findByMessageUniqueKey(corporation.getMessageUniqueKey()); if (saved == null) { return insert(corporation); } Corporation adjusted = Corporation.of(saved.getId(), corporation.getName(), corporation.getMessageUniqueKey()); return update(adjusted); }
127. Wrap Repository using Observables @Service @RequiredArgsConstructor(onConstructor = @__(@Autowired)) public class CorporationService { private final CorporationRepostory corporationRepository; public Observable<Corporation> save(Corporation corporation) { return Observable.fromCallable(() -> corporationRepository.save(corporation)); } }
128. Web API request/response
129. Web APIs public interface AddressApi { @GET("api/service-b/address/{key}") Observable<Response<GetAddressOutputForm>> get(@Path("key") String key); @POST("api/service-b/address/create") Observable<Response<CreateAddressOutputForm>> create(@Body CreateAddressInputForm inputForm); }
130. Wrap Web API for idempotency public Observable<Address> save(Address anonymousAddress) { AddressApi addressApi = apiRegistry.of(AddressApi.class); Observable<Response<GetAddressOutputForm>> getAddressResp$ = addressApi.get(anonymousAddress.getMessageUniqueKey()); return getAddressResp$.flatMap(getAddressResp -> { if (getAddressResp.isSuccessful()) { return Observable.just(getAddressResp.body().getAddress()); } if (getAddressResp.code() != 404) { new UnsupportedOperationException(getAddressResp.errorBody().string()); } return addressApi.create( new CreateAddressInputForm(anonymousAddress.getCorporationId(), anonymousAddress.getState(), anonymousAddress.getMessageUniqueKey())) .map(createAddressOutputFormResp -> { Long addressId = createAddressOutputFormResp.body().getAddressId(); return Address.of(addressId, anonymousAddress); }); }); }
131. Compose idempotent APIs public Observable<Corporation> execute(CreateCorporationCommand aCommand) { Corporation anonymousCorporation = Corporation.anonymous( aCommand.getCorporationName() , aCommand.getMessageUniqueKey() ); return corporationService.save(anonymousCorporation).flatMap( corporation -> { Address anonymousAddress = Address.anonymous( corporation.getId() , aCommand.getState(), aCommand.getMessageUniqueKey() ); return addressService.save(anonymousAddress) .map(address -> Corporation.aggregate(corporation, address)); }); }
132. Conclusion • Rx’s APIs provide us high level abstraction • We write how we process data • We can modify our code for: ◦ error handling, ◦ logging, ◦ multi-threading and so on • The more we break into appropriate components, the easier the testing
133. Fin Slides: https://goo.gl/ygaIgW Benoît Quenaudon 金子 良太
134. Resources • Rx official website ◦ http://reactivex.io/ • The introduction to Reactive Programming you've been missing ◦ https://gist.github.com/staltz/868e7e9bc2a7b8c1f754 • RxJava ◦ https://github.com/ReactiveX/RxJava • RxJS ◦ https://github.com/ReactiveX/RxJS • Reactive Fault Tolerant Programming with Hystrix and RxJava ◦ https://goo.gl/2BrwPd • Functional Reactive Programming with RxJava • Ben Christensen ◦ https://www.youtube.com/watch?v=_t06LRX0DV0 • Exploring RxJava 2 for Android • Jake Wharton ◦ https://www.youtube.com/watch?v=htIXKI5gOQU