Learning Rx by example (2)

Given at Oredev 2016: https://vimeo.com/190922794

(For a beginner's version of this talk check here: https://www.youtube.com/watch?v=k3D0cWyNno4)

Rx (Reactive extensions) is a powerful API for Asynchronous programming.

It has a steep learning curve. Surprisingly though, the easiest way to grasp the concepts is by examples. So in this talk, we'll look at just 3 examples.

These are marginally complex requirements that most developers would run into these days. We dissect the problem using Rx and try to come up with elegant and simple solutions to an otherwise complicated problem.

Transcript

1. Learning Rx

   by example
   

   View Slide

2. 2 mt intro to Rx
   3 (intermediate) examples
   1

   2
   

   View Slide

3. 2 mt
   Intro to Rx
   

   View Slide

4. API for
   Asynchronous programming
   

   View Slide

5. Observer pattern done right
   Best ideas from:
   Observer pattern
   Iterator pattern
   Functional programming
   

   View Slide

6. Observer pattern done right
   Observable.just(1)
   .subscribeOn(Schedulers.io())
   .observeOn(AndroidSchedulers.mainThread())
   .subscribe(getSubscriber());
   .useSomeFunkyOperators()
   

   View Slide

7. Example 1
   Loading from
   Disk Cache + Network Call
   

   View Slide

8. Cached data

   Fast
   Example 1
   Database
   Fresh data

   Slow
   Server
   Requirement:
   

   View Slide

9. .concatEager(
   .merge(
   Example 1
   Code:
   Observable
   .subscribeOn(Schedulers.io())

   .observeOn(AndroidSchedulers.mainThrd())

   .subscribe();
   .concatEager
   getDiskResults(),
   getNetworkResults())
   .merge
   

   View Slide

10. .concatEager(
    .merge(
    Example 1
    Code:
    Observable
    .subscribeOn(Schedulers.io())

    .observeOn(AndroidSchedulers.mainThrd())

    .subscribe();
    .concatEager
    getDiskResults(),
    getNetworkResults())
    .merge
    Both of these methods
    return
    Observable
    

    View Slide

11. Example 1
    Database Network request
    Problem:
    .merge
    

    View Slide

12. Example 1
    Database Network request
    Problem:
    .merge
    

    View Slide

13. Example 1
    Code:
    .merge
    Observable
    .subscribeOn(Schedulers.io())

    .observeOn(AndroidSchedulers.mainThrd())

    .subscribe();
    .merge(
    getDiskResults(),
    getNetworkResults())
    .subscribe(new Subscriber() {

    @Override

    public void onCompleted() { //... }

    

    @Override

    public void onError(Throwable e) { //... }

    

    @Override

    public void onNext(Result result) {

    if ( list.contains(result) &&
    isExistingResultFromNetwork(result))

    return;

    // usual "add result to list" logic

    list.add(result);
    list.refresh();

    }

    });
    

    View Slide

14. Example 1
    Code:
    .merge
    Observable
    .subscribeOn(Schedulers.io())

    .observeOn(AndroidSchedulers.mainThrd())

    .subscribe();
    .merge(
    getDiskResults(),
    getNetworkResults())
    .subscribe(new Subscriber() {

    @Override

    public void onCompleted() { //... }

    

    @Override

    public void onError(Throwable e) { //... }

    

    @Override

    public void onNext(Result result) {

    if ( list.contains(result) &&
    isResultFromNetwork(result))

    return;

    // usual "add result to list" logic

    list.add(result);
    list.refresh();

    }

    });
    NO!

    The Rx is not strong
    with this code
    https://twitter.com/JakeWharton/status/786363146990649345
    

    View Slide

15. Example 1
    Code:
    getNetworkData()
    .publish(
    network ->
    )
    getDiskData()
    .takeUntil(network)
    Observable
    .merge(
    network,
    )
    

    View Slide

16. Example 1
    Code:
    Get database results
    but stop after network results
    getDiskResults()
    .takeUntil(getNetworkResults())
    getDiskButStopAfterNetwork()
    =
    

    View Slide

17. Example 1
    Code:
    Get disk results that occur before Network starts
    +
    Network results
    getDiskResults()
    .takeUntil(getNetworkResults())
    getDiskButStopAfterNetwork()
    =
    getDiskButStopAfterNetwork()
    )
    getNetworkResults()
    Observable
    .merge(
    

    View Slide

18. Example 1
    Code:
    Get disk results that occur before Network starts
    +
    Network results
    )
    .publish(
    getNetworkResults()
    getDiskResults()
    .takeUntil(getNetworkResults())
    Observable
    .merge(
    

    View Slide

19. Example 1
    Code:
    Get disk results that occur before Network starts
    +
    Network results
    Observable
    .merge(
    ,
    )
    .publish(
    getNetworkResults()
    network ->
    getNetworkResults()
    getDiskResults()
    .takeUntil(getNetworkResults())
    

    View Slide

20. Example 1
    Code:
    Get disk results that occur before Network starts
    +
    Network results
    Observable
    .merge(
    ,
    .publish(
    getNetworkResults()
    network ->
    network,
    )
    getDiskResults()
    .takeUntil(getNetworkResults())
    

    View Slide

21. Example 1
    Code:
    Get disk results that occur before Network starts
    +
    Network results
    Observable
    .merge(
    ,
    .publish(
    getNetworkResults()
    network ->
    )
    getDiskResults()
    .takeUntil(getNetworkResults())
    network
    network,
    

    View Slide

22. Example 1
    Code:
    Observable
    .subscribeOn(Schedulers.io())

    .observeOn(AndroidSchedulers.mainThrd())

    .subscribe();
    .concat(
    getSlowCachedDiskData(),
    getFreshNetworkData())
    .publish
    getNetworkData()

    .publish(
    network ->

    Observable
    .merge(
    network,
    getDiskData()
    .takeUntil(network)
    )
    )
    

    View Slide

23. Example 2
    Pagination

    (using Subjects)
    

    View Slide

24. Example 2b
    Setup starting seq. once

    (network call for results)
    Observable
    Observable

    .just(1)

    .flatMap(pageNo -> getNetworkResults(pageNo))
    

    View Slide

25. compose
    Observable Observable
    map
    R T
    flatmap
    R Observable
    Holy trinity of RxConversion
    

    View Slide

26. Example 2b
    Setup starting seq. once

    (network call for results)
    Subscribe
    here
    Observable
    Observable

    .just(1)

    .flatMap(pageNo -> getNetworkResults(pageNo))
    .subscribe(new Subscriber>() {

    @Override

    public void onCompleted() {

    // all items downloaded

    }

    

    @Override

    public void onError(Throwable e) {

    // handle error

    }

    

    @Override

    public void onNext(List items) {

    

    }

    });
    private Observable> getNetworkResults(int pageNo) {

    // make network request
    // get List of Items for page number

    }
    addToList(items);
    

    View Slide

27. Example 2b
    Setup starting seq. once

    (network call for results)
    Subscribe
    here
    Observable
    Observable

    .just(1)

    .flatMap(pageNo -> getNetworkResults(pageNo))
    .subscribe(new Subscriber>() {

    @Override

    public void onCompleted() {

    // all items downloaded

    }

    

    @Override

    public void onError(Throwable e) {

    // handle error

    }

    

    @Override

    public void onNext(List items) {

    

    }

    });
    .concatMap(pageNo -> getNetworkResults(pageNo))
    addToList(items);
    

    View Slide

28. Example 2b
    Setup starting seq. once

    (network call for results)
    Subscribe here
    like page numbers!
    Observable
    What if I could keep feeding
    inputs to get the
    next set of results?
    

    View Slide

29. Network call 

    to get results for page n

    Subject
    Code:
    PublishSubject paginator =
    PublishSubject.create();
    Example 2b
    Observable

    .just(1)

    .concatMap(pgNo -> getNetworkResults(pgNo))
    paginator
    .observeOn(AndroidSchedulers.mainThread())

    .subscribe(new Subscriber>() {

    @Override

    public void onCompleted() {

    // all items downloaded

    }

    

    @Override

    public void onError(Throwable e) {

    // handle error

    }

    

    @Override

    public void onNext(List items) {

    

    }

    });
    addToList(items);
    

    View Slide

30. Network call 

    to get results for page n

    Subscribe here
    Example 2a
    Code:
    Subject
    void onReachedEndOfList() {

    //...

    

    }
    paginator.onNext(nextPage);
    .concatMap(pgNo -> getNetworkResults(pgNo))
    paginator
    .observeOn(AndroidSchedulers.mainThread())

    .subscribe(new Subscriber>() {

    @Override

    public void onCompleted() {

    // all items downloaded

    }

    

    @Override

    public void onError(Throwable e) {

    // handle error

    }

    

    @Override

    public void onNext(List items) {

    

    }

    });
    addToList(items);
    

    View Slide

31. Example 3
    TrueTime: Implementing NTP with Rx
    github.com/instacart/truetime-android
    

    View Slide

32. Example 3
    TrueTime
    NTP

    

    Pool
    DNS

    Resolve
    SNTP Rq 2
    SNTP Request 1
    SNTP Rq 3
    SNTP Rq 4
    SNTP Rq 5
    Least

    roundtrip
    delay
    5
    IP1
    IP2
    IP3
    IP4
    Sort by
    clock offset
    +
    Pick
    Median
    

    View Slide

33. Example 3
    TrueTime
    NTP

    

    Pool
    DNS

    Resolve
    SNTP Rq 2
    SNTP Request 1
    SNTP Rq 3
    SNTP Rq 4
    SNTP Rq 5
    Least

    roundtrip
    delay
    5
    IP1
    IP2
    IP3
    IP4
    TrueTime
    Sort by
    clock offset
    +
    Pick
    Median
    

    View Slide

34. Observable

    .just(ntpPool)

    .compose(resolveNtpPool())

    NTP
    DNS

    Resolve
    time.apple.com
    Code:
    compose
    Observable Observable
    IP1
    IP2
    IP3
    IP4
    

    View Slide

35. Observable

    .just(ntpPool)

    .compose(resolveNtpPool())

    NTP
    DNS

    Resolve
    time.apple.com
    Code:
    private Transformer resolveNtpPool(){

    return ntpPool -> {

    try {

    return Observable.from(
    InetAddress.getAllByName(ntpPool));
    } catch (UnknownHostException e) {

    return Observable.error(e);

    }

    })

    }

    

    View Slide

36. Observable

    .just(ntpPool)

    .compose(resolveNtpPool())

    NTP
    DNS

    Resolve
    time.apple.com
    Code:
    IP1
    

    View Slide

37. Code:
    Observable

    .just(ntpPool)

    .compose(resolveNtpPool())

    .flatMap(bestResponseAgainstSingleIp())

    IP1
    T (IP1) Observable (SNTP response)
    Request 1
    SNTP
    Request 2
    SNTP
    Request 3
    SNTP
    Request 4
    SNTP
    Request 5
    SNTP
    

    View Slide

38. compose
    Observable Observable
    map
    R T
    flatmap
    T Observable
    Holy trinity of RxConversion
    

    View Slide

39. Code:
    IP1
    Observable

    .just(ntpPool)

    .compose(resolveNtpPool())

    .flatMap(bestResponseAgainstSingleIp())

    bestResponseAgainstSingleIp()
    Func1 >
    return Observable
    .just(singleIp)

    .repeat(5)
    {
    Request 1
    SNTP
    Request 2
    SNTP
    Request 3
    SNTP
    Request 4
    SNTP
    Request 5
    SNTP
    

    View Slide

40. Code:
    .flatMap(ipAddress ->
    sntpNtwrkReq(ipAddress))
    Observable

    .just(ntpPool)

    .compose(resolveNtpPool())

    .flatMap(bestResponseAgainstSingleIp())

    bestResponseAgainstSingleIp()
    return Observable
    .just(singleIp)

    .repeat(5)
    {
    IP1
    Request 1
    SNTP
    Request 2
    SNTP
    Request 3
    SNTP
    Request 4
    SNTP
    Request 5
    SNTP
    Func1 >
    

    View Slide

41. Code:
    IP1
    Observable

    .just(ntpPool)

    .compose(resolveNtpPool())

    .flatMap(bestResponseAgainstSingleIp())

    bestResponseAgainstSingleIp()
    return Observable
    .just(singleIp)

    .repeat(5)
    {
    5
    .flatMap(ipAddress ->
    sntpNtwrkReq(ipAddress))
    Request 1
    SNTP
    Request 2
    SNTP
    Request 3
    SNTP
    Request 4
    SNTP
    Request 5
    SNTP 

    .doOnError(throwable -> {

    // this request alone failed
    // retry this req alone

    })

    .retry(5))
    }
    Func1 >
    

    View Slide

42. Code:
    IP1
    Observable

    .just(ntpPool)

    .compose(resolveNtpPool())

    .flatMap(bestResponseAgainstSingleIp())

    bestResponseAgainstSingleIp()
    return Observable
    .just(singleIp)

    .repeat(5)
    {
    5
    .flatMap(ipAddress ->
    sntpNtwrkReq(ipAddress))
    Request 1
    SNTP
    Request 2
    SNTP
    Request 3
    SNTP
    Request 4
    SNTP
    Request 5
    SNTP
    }
    Func1 >
    

    .doOnError(throwable -> {

    // this request alone failed
    // retry this req alone

    })

    .retry(5))
    )
    

    .toList()
    

    View Slide

43. 

    .map(responseList -> {

    Collections.sort(responseList,
    comparator);
    

    return responseList.get(0);
    })
    Least

    roundtrip
    delay
    Code:
    

    .toList()
    Observable

    .just(ntpPool)

    .compose(resolveNtpPool())

    .flatMap(bestResponseAgainstSingleIp())

    bestResponseAgainstSingleIp()
    Func1 >
    {
    }
    

    .doOnError(throwable -> {

    // this request alone failed
    // retry this req alone

    })

    .retry(5))
    )
    

    View Slide

44. Example 3
    TrueTime
    NTP

    

    Pool
    DNS

    Resolve
    SNTP Rq 2
    SNTP Request 1
    SNTP Rq 3
    SNTP Rq 4
    SNTP Rq 5
    Least

    roundtrip
    delay
    5
    IP1
    IP2
    IP3
    IP4
    Sort by
    clock offset
    +
    Pick
    Median
    

    View Slide

45. Example 3
    TrueTime
    NTP

    

    Pool
    DNS

    Resolve
    SNTP Rq 2
    SNTP Request 1
    SNTP Rq 3
    SNTP Rq 4
    SNTP Rq 5
    5
    IP1
    IP2
    IP3
    IP4
    Sort by
    clock offset
    +
    Pick
    Median
    

    View Slide

46. .map(filterMedianResponse())
    Code: Observable

    .just(ntpPool)

    .compose(resolveNtpPool())

    .flatMap(bestResponseAgainstSingleIp())

    .toList()
    filterMedianResponse()
    Func1 , Response>
    return responseList -> {
    }
    Collections.sort(responses, comparator);

    Sort by
    clock offset
    return bestResponses
    .get(bestResponses.size() / 2);
    Pick
    Median
    Sort by
    clock offset
    +
    Pick
    Median
    

    View Slide

47. Observable

    .just(ntpPool)

    .compose(resolveNtpPool())

    .flatMap(bestResponseAgainstSingleIp())

    .doOnNext(response -> convertToTime(response));
    .map(filterMedianResponse())
    .toList()
    Code:
    TrueTime
    Sort by
    clock offset
    +
    Pick
    Median
    

    View Slide

48. Example 3
    TrueTime
    NTP

    

    Pool
    DNS

    Resolve
    SNTP Rq 2
    SNTP Request 1
    SNTP Rq 3
    SNTP Rq 4
    SNTP Rq 5
    Least

    roundtrip
    delay
    5
    IP1
    IP2
    IP3
    IP4
    TrueTime
    Sort by
    clock offset
    +
    Pick
    Median
    

    View Slide

49. ✌
    fragmentedpodcast.com
    tech.instacart.com
    @kaushikgopal
    kaush.co
    My thanks to @cyrilmotier
    who graciously allowed me to rip-off his slide deck theme
    

    View Slide