An intro to Reactive Extensions

Slide 1

Slide 1 text

Reactive Extensions 入門勉強会 #漫才ではない Benoît Quenaudon 金子良太

Slide 2

Slide 2 text

Agenda ● Why Reactive? ○ Problem with Android ○ Problem with Back-end ● What is Reactive Extensions? ○ Observables ○ Operators ○ Schedulers ● Use Cases ○ API gateway pattern ○ Data writing across distributed systems

Slide 3

Slide 3 text

Why Reactive? #Android

Slide 4

Slide 4 text

Why Reactive? #android interface UserManager { User getUser(); void setName(String name); void setAge(int age); }

Slide 5

Slide 5 text

Why Reactive? #android interface UserManager { User getUser(); void setName(String name); void setAge(int age); } UserManager um = new UserManager(); System.out.println(um.getUser()); um.setName("Bo Jackson"); System.out.println(um.getUser());

Slide 6

Slide 6 text

interface UserManager { User getUser(); void setName(String name); // <-- now async void setAge(int age); // <-- now async }

Slide 7

Slide 7 text

interface UserManager { User getUser(); void setName(String name, Runnable callback); void setAge(int age, Runnable callback); }

Slide 8

Slide 8 text

interface UserManager { User getUser(); void setName(String name, Runnable callback); void setAge(int age, Runnable callback); } UserManager um = new UserManager(); System.out.println(um.getUser()); um.setName("Bo Jackson", new Runnable() { @Override public void run() { System.out.println(um.getUser()); } });

Slide 9

Slide 9 text

interface UserManager { User getUser(); void setName(String name, Listener listener); void setAge(int age, Listener listener); interface Listener { void success(User user); void failure(IOException exception); } }

Slide 10

Slide 10 text

Slide 11

Slide 11 text

UserManager um = new UserManager(); System.out.println(um.getUser()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { System.out.println(um.getUser()); } @Override public void failure(IOException exception) { // TODO show the error… } }); um.setAge(54, new UserManager.Listener() { @Override public void success() { System.out.println(um.getUser()); } @Override public void failure(IOException exception) { // TODO show the error… } });

Slide 12

Slide 12 text

UserManager um = new UserManager(); System.out.println(um.getUser()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { System.out.println(um.getUser()); um.setAge(54, new UserManager.Listener() { @Override public void success() { System.out.println(um.getUser()); } @Override public void failure(IOException exception) { // TODO show the error… } }); } @Override public void failure(IOException exception) { // TODO show the error… } });

Slide 13

Slide 13 text

public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { tv.setText(um.getUser().toString()); } @Override public void failure(IOException exception) { // TODO show the error… } }); } }

Slide 14

Slide 14 text

Slide 15

Slide 15 text

public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { if (isDestroyed()) { tv.setText(um.getUser().toString()); } } @Override public void failure(IOException exception) { // TODO show the error… } }); } }

Slide 16

Slide 16 text

public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { if (isDestroyed()) { tv.setText(um.getUser().toString()); } } @Override public void failure(IOException exception) { // TODO show the error… } }); } }

Slide 17

Slide 17 text

public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { if (isDestroyed()) { tv.setText(um.getUser().toString()); } } @Override public void failure(IOException exception) { // TODO show the error… } }); } }

Slide 18

Slide 18 text

public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { runOnUiThread(new Runnable() { @Override public void run() { if (isDestroyed()) { tv.setText(um.getUser().toString()); } } }); } @Override public void failure(IOException exception) { // TODO show the error… } }); }

Slide 19

Slide 19 text

public final class UserActivity extends Activity { private final UserManager um = new UserManager(); @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.user); TextView tv = (TextView) findViewById(R.id.username); tv.setText(um.getUser().toString()); um.setName("Bo Jackson", new UserManager.Listener() { @Override public void success() { runOnUiThread(new Runnable() { @Override public void run() { if (isDestroyed()) { tv.setText(um.getUser().toString()); } } }); } @Override public void failure(IOException exception) { // TODO show the error… } }); }

Slide 20

Slide 20 text

ぶっちゃけ・・・

Slide 21

Slide 21 text

理想

Slide 22

Slide 22 text

Why Reactive? #BackEnd

Slide 23

Slide 23 text

Netflix Story

Slide 24

Slide 24 text

Client Server Old Netflix Model API

Slide 25

Slide 25 text

Problems ● Low spec devices ● Why not leverage the servers? ⇒ Client = one request only

Slide 26

Slide 26 text

Client Server New Netflix Model API

Slide 27

Slide 27 text

Async + Callbacks

Slide 28

Slide 28 text

Challenge ● Service Composition ● Error handling ● Developer Productivity ⇒ Callbacks hell ⇒ Too hard

Slide 29

Slide 29 text

Async + Callbacks

Slide 30

Slide 30 text

Try ● java.util.concurrent.Future ○ Future.get() is blocking… ○ Future>> ● How about Reactive Programming? ○ Observable ⇒ Reactive Programming for the win

Slide 31

Slide 31 text

Reactive Programming ?

Slide 32

Slide 32 text

Reactive programming is programming with asynchronous data streams.

Slide 33

Slide 33 text

Reactive Extensions?

Slide 34

Slide 34 text

“ReactiveX is a library for composing asynchronous and event-based programs by using observable sequences”

Slide 35

Slide 35 text

ReactiveX ● Extension of Observable Pattern ● Declarative Composition of Streams ● Abstraction of ○ low-level threading, ○ Synchronization, ○ Thread-safety, ○ concurrent data structures, ○ and non-blocking I/O.

Slide 36

Slide 36 text

What is the Observable Pattern?

Slide 37

Slide 37 text

例：Stream of Click time Click event Stream has completed Error

Slide 38

Slide 38 text

Observable Pattern ● Listen to Stream = Subscribing ● Only async ○ event ⇒ funcA() ○ error ⇒ funcB() ○ complete ⇒ funcC() ● funcA, B, C = Observers = Consumer ● Stream = Subject = Observable

Slide 39

Slide 39 text

Iterator vs Observable デモンストレーション

Slide 40

Slide 40 text

event Iterable (pull) Observable (push) retrieve data T next() onNext(T) discover error throws Exception onError(Exception) complete !hasNext() onCompleted()

Slide 41

Slide 41 text

Reactive Extensions ● Observables ● Operators ● Schedulers

Slide 42

Slide 42 text

Observables

Slide 43

Slide 43 text

Observables ● Usually do work when start/stop listening ● One event, many events or empty ● Terminates with an error or completion ● May never terminate

Slide 44

Slide 44 text

Observables interface UserManager { User getUser(); void setName(String name); void setAge(int age); }

Slide 45

Slide 45 text

Observables interface UserManager { Observable getUser(); void setName(String name); void setAge(int age); }

Slide 46

Slide 46 text

Observables interface UserManager { Observable getUser(); Completable setName(String name); Completable setAge(int age); }

Slide 47

Slide 47 text

Creating Observables ● RxJava ○ Observable.just("Bo Jacks"); ○ Observable.fromArray(array); ○ Observable.fromIterable(list); ○ Observable.create(...); ● RxJS ○ Rx.Observable.just(42) ○ Rx.Observable.from(iterable) ○ Rx.Observable.create(subscribe)

Slide 48

Slide 48 text

Operators

Slide 49

Slide 49 text

Operators ● Manipulate or combine data ● Manipulate threading ● Manipulate emissions

Slide 50

Slide 50 text

Operators String greeting = "Hello"; String yelling = greeting.toUpperCase();

Slide 51

Slide 51 text

Operators Observable greeting = Observable.just("Hello"); String yelling = greeting.toUpperCase();

Slide 52

Slide 52 text

Operators Observable greeting = Observable.just("Hello"); Observable yelling = greeting.map(s -> s.toUpperCase());

Slide 53

Slide 53 text

Operators Observable greeting = Observable.just("Hello"); Observable yelling = greeting.map(s -> s.toUpperCase());

Slide 54

Slide 54 text

map as a marble diagram

Slide 55

Slide 55 text

Transforming Operators ● Buffer ● FlatMap ● GroupBy ● Map ● Scan ● Window

Slide 56

Slide 56 text

Filtering Operators ● Debounce ● Distinct ● ElementAt ● Filter ● First ● IgnoreElements ● Last ● Sample ● Skip ● SkipLast ● Take

Slide 57

Slide 57 text

filter as a marble diagram

Slide 58

Slide 58 text

Combining Operators ● And/Then/When ● CombineLatest ● Join ● Merge ● StartWith ● Switch ● Zip

Slide 59

Slide 59 text

So on... ● Error Handling Operators ○ catch, retry ● Observable Utility Operators ○ delay, timeout, observeOn, subscribeOn, etc ● Conditional and Boolean Operators ○ all, skipUntil, contains, etc ● Mathematical and Aggregate Operators ○ average, count, max, etc ● etc

Slide 60

Slide 60 text

Schedulers

Slide 61

Slide 61 text

Schedulers getDataFromNetwork() // Observable .skip(10) .take(5) .map(s -> s + " transformed") .subscribe(s -> println s)

Slide 62

Slide 62 text

Schedulers getDataFromNetwork() .skip(10) .take(5) .map(s -> s + " transformed") .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread())) .subscribe(s -> println s)

Slide 63

Slide 63 text

Schedulers ● subscribeOn: ○ Everything from top to the next observeOn run on my thread. ● observeOn: ○ Everything below me run on my thread.

Slide 64

Slide 64 text

Schedulers getDataFromNetwork() .skip(10) .take(5) .map(s -> s + " transformed") .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread())) .subscribe(s -> println s) IO Main

Slide 65

Slide 65 text

Examples

Slide 66

Slide 66 text

RxJS: 事前 [1, 2, 3].map(x => x + 1) // [2, 3, 4] [ [1], [2, 3], [], [4] ].concatAll() // [1, 2, 3, 4]

Slide 67

Slide 67 text

RxJS getElementDrags = function(elmt) { return elmt.mouseDowns. map(mouseDown => document.mouseMoves. takeUntil(document.mouseUps)). concatAll(); } getElementDrags(image). forEach(pos => image.position = pos);

Slide 68

Slide 68 text

D D D map(mouseDown => document.mouseMoves .takeUntil(document.mouseUps)) U U U concatAll() forEach(pos => image.position = pos) Operator Operator Observer M M M M M M M M M M

Slide 69

Slide 69 text

Slide 70

Slide 70 text

RxJava

Slide 71

Slide 71 text

RxJava Observable editText; editText .filter(text -> text.length > 2) .debounce(250, MILLISECONDS) .map(text -> api.search(text)) .subscribe(result -> showResult(result));

Slide 72

Slide 72 text

filter(text -> text.length > 2) map(text -> api.search(text)) result -> showResult(result) Operator Operator Observer k debounce(250, MILLISECONDS) Operator ka kai kaiz kaize kaizen kai kaiz kaize kaizen kaiz kaizen 250ms 250ms

Slide 73

Slide 73 text

RxJava Observable editText; editText .filter(text -> text.length > 2) .debounce(250, MILLISECONDS) .map(text -> api.search(text)) .subscribe(result -> showResult(result));

Slide 74

Slide 74 text

ご注意

Slide 75

Slide 75 text

Back Pressure デモンストレーション

Slide 76

Slide 76 text

Hot vs Cold Observable ● Cold ○ Start to work on subscription ● Hot ○ Start to work on creation

Slide 77

Slide 77 text

How about Functional Reactive Programming? A different Animal

Slide 78

Slide 78 text

Use Cases

Slide 79

Slide 79 text

Background ● Moore’s Law Is Dead. Now What? ● We need the power. ○ multi-devices, IoT, Big Data ○ For organizations' scalability

Slide 80

Slide 80 text

Background ● Microservices ○ ≒ distributed systems ● Need API-based collaboration ● Need combination of multi tasks

Slide 81

Slide 81 text

Complexity ● How do we combine multi data? ● How do we resolve dependencies between tasks? ● Concurrency / multi-threading / blocking ● Network ● Retry strategy / Message delivery reliability ● Eventual consistency ● Anti-fragile / Fault tolerant

Slide 82

Slide 82 text

Use case 1 Service Composition

Slide 83

Slide 83 text

Pattern: API Gateway http://microservices.io/patterns/apigateway.html

Slide 84

Slide 84 text

Scenario ● API-based collaboration ○ Authenticate ○ Get video contents ○ Get recommendations

Slide 85

Slide 85 text

API as a Stream

Slide 86

Slide 86 text

API as a Stream public interface AccountApi { @POST("api/account/login") Observable login(@Body LoginInputForm inputForm); } ● Retrofit & RxJava2 Adapter ● If you don't use these adapters, you can wrap API responses by yourself

Slide 87

Slide 87 text

Service as a Stream

Slide 88

Slide 88 text

Service as a Stream @Service @RequiredArgsConstructor(onConstructor = @__(@Autowired)) public class AccountService { private final ApiRegistry apiRegistry; public Observable login(String account, String password) { AccountApi accountApi = apiRegistry.of(AccountApi.class); return accountApi.login(new LoginInputForm(account, password)); } }

Slide 89

Slide 89 text

Service Composition

Slide 90

Slide 90 text

@RequestMapping(path = "/api/example1", method = RequestMethod.POST) public ReadMergeOutputForm run(@Validated @RequestBody ReadMergeInputForm inputForm) { return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) ) .zipWith( videoService.get() , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle(); }

Slide 91

Slide 91 text

Slide 92

Slide 92 text

Slide 93

Slide 93 text

Slide 94

Slide 94 text

Slide 95

Slide 95 text

Slide 96

Slide 96 text

Error Handling

Slide 97

Slide 97 text

return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .map(loginOutputForm -> { if (!loginOutputForm.isSuccess()) { throw new RuntimeException(); } return loginOutputForm; }) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) ) .zipWith( videoService.get() , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle();

Slide 98

Slide 98 text

Fallback to default Or cache

Slide 99

Slide 99 text

return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .map(loginOutputForm -> { if (!loginOutputForm.isSuccess()) { throw new RuntimeException(); } return loginOutputForm; }) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) .onErrorReturnItem(RECOMMENDATIONS_ON_ERROR) ) .zipWith( videoService.get() , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle();

Slide 100

Slide 100 text

Multi-Threading

Slide 101

Slide 101 text

return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .map(loginOutputForm -> { if (!loginOutputForm.isSuccess()) { throw new RuntimeException(); } return loginOutputForm; }) .flatMap( loginOutputForm -> recommendationService .get(loginOutputForm.accountId()) .onErrorReturnItem(RECOMMENDATIONS_ON_ERROR) ) .zipWith( videoService.get().subscribeOn(Schedulers.computation()) , (recommendationOutputForm, videoOutputForm) -> new ReadMergeOutputForm( videoOutputForm.getVideos() , recommendationOutputForm.getRecommendations() ) ) .blockingSingle();

Slide 102

Slide 102 text

Logging

Slide 103

Slide 103 text

return accountService .login(inputForm.getAccount(), inputForm.getPassword()) .doOnError(Throwable::printStackTrace) .doOnNext(loginOutputForm -> log.info(Thread.currentThread().getName()) ) .map(loginOutputForm -> { if (!loginOutputForm.isSuccess()) { throw new RuntimeException(); } return loginOutputForm; });

Slide 104

Slide 104 text

Use case 2 Write data across distributed systems

Slide 105

Slide 105 text

Consistency in a single system ● Can use the transaction ● e.g. ○ When a corporation data is created, an address data is also created ○ The corporation object manages the address object's life cycle

Slide 106

Slide 106 text

Consistency in a single system @Transactional(rollbackFor = {Exception.class}) public Corporation create(Corporation anonymousCorporation, Address anonymousAddress) { Corporation corporation = corporationRepository.save(anonymousCorporation); Address unidentifiedAddress = Address.builder() .corporationId(corporation.getId()) .state(anonymousAddress.getState()) .phone(anonymousAddress.getPhone()) .build(); Address address = addressRepository.save(unidentifiedAddress); return Corporation.aggregate(corporation, address); }

Slide 107

Slide 107 text

Eventual Consistency for multi-services ● We couldn't use the transaction for the data consistency in HTTP protocol based on multi-services collaboration ● Instead of the transaction, we often use the eventual consistency ● We need take care of some things: ○ Idempotency ○ Message delivery reliability

Slide 108

Slide 108 text

Scenario ● End user inputs corporation and address data ● Service A enqueues with end users' input data ● Worker processes the queue ○ Worker is responsible for guaranteeing for message delivery reliability ● Save corporation data in the service A with unique key message ● Service A requests Service B to save address data via Web API ○ Retry till successful response from Service B ⇒ Has to be Idempotent

Slide 109

Slide 109 text

No content

Slide 110

Slide 110 text

No content

Slide 111

Slide 111 text

No content

Slide 112

Slide 112 text

No content

Slide 113

Slide 113 text

No content

Slide 114

Slide 114 text

No content

Slide 115

Slide 115 text

No content

Slide 116

Slide 116 text

No content

Slide 117

Slide 117 text

No content

Slide 118

Slide 118 text

No content

Slide 119

Slide 119 text

Service down

Slide 120

Slide 120 text

Network busy

Slide 121

Slide 121 text

Guarantee the Message delivery reliability

Slide 122

Slide 122 text

No content

Slide 123

Slide 123 text

No content

Slide 124

Slide 124 text

Objects life cycle ● Requirements ○ When a corporation data is created, an address data should also be created.

Slide 125

Slide 125 text

Repository for creating a corporation

Slide 126

Slide 126 text

Repository public Corporation save(Corporation corporation) { Corporation saved = findByMessageUniqueKey(corporation.getMessageUniqueKey()); if (saved == null) { return insert(corporation); } Corporation adjusted = Corporation.of(saved.getId(), corporation.getName(), corporation.getMessageUniqueKey()); return update(adjusted); }

Slide 127

Slide 127 text

Wrap Repository using Observables @Service @RequiredArgsConstructor(onConstructor = @__(@Autowired)) public class CorporationService { private final CorporationRepostory corporationRepository; public Observable save(Corporation corporation) { return Observable.fromCallable(() -> corporationRepository.save(corporation)); } }

Slide 128

Slide 128 text

Web API request/response

Slide 129

Slide 129 text

Web APIs public interface AddressApi { @GET("api/service-b/address/{key}") Observable> get(@Path("key") String key); @POST("api/service-b/address/create") Observable> create(@Body CreateAddressInputForm inputForm); }

Slide 130

Slide 130 text

Wrap Web API for idempotency public Observable

save(Address anonymousAddress) { AddressApi addressApi = apiRegistry.of(AddressApi.class); Observable> getAddressResp$ = addressApi.get(anonymousAddress.getMessageUniqueKey()); return getAddressResp$.flatMap(getAddressResp -> { if (getAddressResp.isSuccessful()) { return Observable.just(getAddressResp.body().getAddress()); } if (getAddressResp.code() != 404) { new UnsupportedOperationException(getAddressResp.errorBody().string()); } return addressApi.create( new CreateAddressInputForm(anonymousAddress.getCorporationId(), anonymousAddress.getState(), anonymousAddress.getMessageUniqueKey())) .map(createAddressOutputFormResp -> { Long addressId = createAddressOutputFormResp.body().getAddressId(); return Address.of(addressId, anonymousAddress); }); }); }

Slide 131

Slide 131 text

Compose idempotent APIs public Observable execute(CreateCorporationCommand aCommand) { Corporation anonymousCorporation = Corporation.anonymous( aCommand.getCorporationName() , aCommand.getMessageUniqueKey() ); return corporationService.save(anonymousCorporation).flatMap( corporation -> { Address anonymousAddress = Address.anonymous( corporation.getId() , aCommand.getState(), aCommand.getMessageUniqueKey() ); return addressService.save(anonymousAddress) .map(address -> Corporation.aggregate(corporation, address)); }); }

Slide 132

Slide 132 text

Conclusion ● Rx’s APIs provide us high level abstraction ● We write how we process data ● We can modify our code for: ○ error handling, ○ logging, ○ multi-threading and so on ● The more we break into appropriate components, the easier the testing

Slide 133

Slide 133 text

Fin Slides: https://goo.gl/ygaIgW Benoît Quenaudon 金子良太

Slide 134

Slide 134 text

Resources ● Rx official website ○ http://reactivex.io/ ● The introduction to Reactive Programming you've been missing ○ https://gist.github.com/staltz/868e7e9bc2a7b8c1f754 ● RxJava ○ https://github.com/ReactiveX/RxJava ● RxJS ○ https://github.com/ReactiveX/RxJS ● Reactive Fault Tolerant Programming with Hystrix and RxJava ○ https://goo.gl/2BrwPd ● Functional Reactive Programming with RxJava • Ben Christensen ○ https://www.youtube.com/watch?v=_t06LRX0DV0 ● Exploring RxJava 2 for Android • Jake Wharton ○ https://www.youtube.com/watch?v=htIXKI5gOQU