Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CodeMentor April 2015

CodeMentor April 2015

This was presented as a hangout session on codementor:

https://www.codementor.io/officehours/3965269104/codementor-office-hours-revolutionizing-ipc-with-nitesh-kant-of-netflix

This presentation introduces reactive IPC, it's core tenants and what value adds it brings vis-a-vis traditional IPC technologies.

A3668e66eb7b8980ac91daaa4e9fe691?s=128

Nitesh Kant

April 20, 2015
Tweet

More Decks by Nitesh Kant

Other Decks in Technology

Transcript

  1. Nitesh Kant - @NiteshKant - Software Engineer, Cloud Platform, Netflix

    Reactive IPC Revolutionize your Microservices
  2. Nitesh Kant Who Am I? ❖ Engineer, Cloud Platform, Netflix.

    ❖ Leading Reactive IPC @Netflix ❖ Core contributor, RxNetty* ❖ Contributor, Eureka** ❖ Suffers with OCD => API design * https://github.com/ReactiveX/RxNetty ** https://github.com/Netflix/eureka @NiteshKant
  3. Inter Process Communication (IPC)

  4. Delivering Netflix What goes behind the scenes?

  5. Calling Home One of many HTTP calls to deliver Netflix.

    http://www.netflix.com/WiHome
  6. Pipe Dream Always available, infinitely scalable machine.

  7. Reality Thousands of machines with hundreds of microservices

  8. Microservices A typical relationship between different microservices.

  9. None
  10. None
  11. None
  12. IPC Any call made outside of your process to a

    remote machine or local, is an IPC call.
  13. Reactive IPC

  14. Instances variations We auto-scale up/down all the time.

  15. Failures Instances fail in various ways. X X X X

  16. Fallacies of distributed computing http://en.wikipedia.org/ wiki/ Fallacies_of_distributed_comp uting#The_fallacies

  17. None
  18. Traditional IPC

  19. Synchronous systems Consuming a resource (thread) per user request does

    bad things to you. http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
  20. Synchronous systems Consuming a resource (thread) per user request does

    bad things to you. http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html You isolate yourself by introducing a queuing point.
  21. Synchronous systems Consuming a resource (thread) per user request does

    bad things to you. http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html You isolate yourself by introducing a queuing point. But then you are limited by the number of threads in the system.
  22. Reactive IPC

  23. Asynchronous systems Multiple connections/requests multiplexed on a single thread (Eventloop).

    Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection Application Code
  24. Asynchronous systems Multiple connections/requests multiplexed on a single thread (Eventloop).

    An Eventloop only does CPU work and does not block. Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection Application Code
  25. Asynchronous systems Multiple connections/requests multiplexed on a single thread (Eventloop).

    An Eventloop only does CPU work and does not block. A single bad dependency/connection does not bring down the system. Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection Application Code
  26. Asynchronous systems Multiple connections/requests multiplexed on a single thread (Eventloop).

    An Eventloop only does CPU work and does not block. As many Eventloops as CPUs. A single bad dependency/connection does not bring down the system. Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection Application Code Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection
  27. Let’s look at the bigger picture

  28. Service discovery IPC Reactive IPC Pull based discovery

  29. Movie Service User Service Service discovery Which are the instances

    of User Service?
  30. Movie Service User Service Service discovery Which are the instances

    of User Service? Instances I1, I2, I3
  31. Movie Service User Service Service discovery Which are the instances

    of User Service? Instances I1, I2, I3 Any changes? Nope
  32. Movie Service User Service Service discovery Which are the instances

    of User Service? Instances I1, I2, I3 Any changes? Nope Any changes? I2 is dead
  33. Movie Service User Service Service discovery Which are the instances

    of User Service? Instances I1, I2, I3 Any changes? Nope Any changes? I2 is dead Somewhere here I2 died
  34. Movie Service User Service Service discovery Which are the instances

    of User Service? Instances I1, I2, I3 Any changes? Nope Any changes? I2 is dead Somewhere here I2 died
  35. Service discovery IPC Reactive IPC Pull based discovery Push based

    discovery
  36. Movie Service User Service Service discovery I am interested in

    User Service instances.
  37. Movie Service User Service Service discovery I am interested in

    User Service instances. Instances I1, I2, I3
  38. Movie Service User Service Service discovery I am interested in

    User Service instances. Instances I1, I2, I3 Any changes? Nope X
  39. Movie Service User Service Service discovery I am interested in

    User Service instances. Instances I1, I2, I3 I2 died
  40. Service discovery IPC Reactive IPC Pull based discovery Push based

    discovery Implicit multi-stage delays
  41. Movie Service User Service Service discovery Which are the instances

    of User Service? Instances I1, I2, I3 Any changes? Nope Any changes? I2 is dead Somewhere here I2 died
  42. Service discovery nodes

  43. Service discovery nodes

  44. Movie service node Service discovery client User service client

  45. Service discovery IPC Reactive IPC Pull based discovery Push based

    discovery Implicit multi-stage delays Instantaneous updates
  46. Service discovery nodes X X X X

  47. Movie service node Service discovery client User service client X

  48. As soon as an instance is shutdown, it is removed

    from the clients using them.
  49. Control IPC Reactive IPC No optimal cancellations

  50. Delivering Netflix Edge Service Personalized catalog service Ratings Service Video

    Metadata Service Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing
  51. Edge Service Personalized catalog service Ratings Service Video Metadata Service

    Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing
  52. Edge Service Personalized catalog service Ratings Service Video Metadata Service

    Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing X User cancellation
  53. Control IPC Reactive IPC No optimal cancellations Built-in Cancellations

  54. Edge Service Personalized catalog service Ratings Service Video Metadata Service

    Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing X User cancellation X X X X
  55. Control IPC Reactive IPC No optimal cancellations Built-in Cancellations Transport

    level flow-control
  56. Delivering Netflix Edge Service Video Metadata Service Disclaimer: This is

    an example and not an exact representation of the processing
  57. Edge Service Video Metadata Service Disclaimer: This is an example

    and not an exact representation of the processing Get 100 videos details (Batch API) Send Video details
  58. Edge Service Video Metadata Service Disclaimer: This is an example

    and not an exact representation of the processing Get 100 videos details (Batch API) Send Video details }X n
  59. Edge Service Video Metadata Service Disclaimer: This is an example

    and not an exact representation of the processing Get 100 videos details (Batch API) Send Video details }X n That’s too much of data!
  60. Control IPC Reactive IPC No optimal cancellations Built-in Cancellations Transport

    level flow-control Application flow-control
  61. Edge Service Video Metadata Service Disclaimer: This is an example

    and not an exact representation of the processing Get 100 videos details (Batch API)
  62. Edge Service Video Metadata Service Disclaimer: This is an example

    and not an exact representation of the processing Get 100 videos details (Batch API) Request 5
  63. Edge Service Video Metadata Service Disclaimer: This is an example

    and not an exact representation of the processing Get 100 videos details (Batch API) Request 5 Send 5
  64. Edge Service Video Metadata Service Disclaimer: This is an example

    and not an exact representation of the processing Get 100 videos details (Batch API) Request 5 Send 5 Request 5 Send 5
  65. Edge Service Video Metadata Service Disclaimer: This is an example

    and not an exact representation of the processing Get 100 videos details (Batch API) Request 5 Send 5 Request 5 Send 5
  66. Failure detection IPC Reactive IPC Polling

  67. Client lib Failure detector Is Instance I1 healthy? Any client

    library
  68. Client lib Failure detector Is Instance I1 healthy? Yep Any

    client library
  69. Client lib Failure detector Is Instance I1 healthy? Yep Still

    healthy? Yep Any client library
  70. Client lib Failure detector Is Instance I1 healthy? Yep Still

    healthy? Nope Any client library Still healthy? Yep
  71. Client lib Failure detector Is Instance I1 healthy? Yep Still

    healthy? Nope Any client library Still healthy? Yep Somewhere here I1 died
  72. Client lib Failure detector Is Instance I1 healthy? Yep Still

    healthy? Nope Any client library Still healthy? Yep Somewhere here I1 died Errors & Retries
  73. Failure detection IPC Reactive IPC Polling Push based

  74. Client lib Failure detector I am using I1, tell me

    when it dies Any client library
  75. Client lib Failure detector I am using I1, tell me

    when it dies I1 died Any client library
  76. Client lib Failure detector I am using I1, tell me

    when it dies I1 died Any client library Minimal Errors & Retries
  77. Reactive IPC

  78. http://reactivex.io/

  79. RxJava A library for composing asynchronous and event-based programs using

    observable sequences for the Java VM. http://github.com/ReactiveX/RxJava
  80. Single Multiple Sync T getData() Iterable<T> getData() Stream<T> getData() Async

    Future<T> getData() Observable<T> getData()
  81. Observable Publisher of Data

  82. Observable Subscriber Publisher of Data Consumer of Data

  83. Observable Publisher of Data Subscription Cancellation control Subscriber Consumer of

    Data
  84. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));

  85. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));

  86. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));

  87. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));

  88. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));

  89. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));

  90. Subscription sub = Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));

    . . . // Later based on some other event sub.unsubscribe()
  91. Observable.range(1, 100) .takeUntil(Observable.timer(1, TimeUnit.Minutes) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done")); Idiomatic

    unsubscribe
  92. Observable.interval(1, TimeUnit.Seconds) .takeUntil(Observable.timer(1, TimeUnit.Minutes) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));

  93. HttpClient.newClient() .createGet("http://myserver.com:8080/hello") .flatMap(HttpClientResponse::getContent) .takeUntil(Observable.timer(1, TimeUnit.Minutes) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));

  94. HttpClient.newClient() .createGet("http://www.netflix.com/WiHome") .flatMap(HttpClientResponse::getContent) .takeUntil(Observable.timer(1, TimeUnit.Minutes) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));

  95. Your Service as an Observable.* * Courtesy: “Your Server as

    a function” - Marius Eriksen
  96. HttpServer.newServer(8080) .start((req, resp) -> resp.sendHeaders() .write(Observable.just("Hello World!!"))); Hello World!

  97. Delivering Netflix Edge Service Personalized catalog service Ratings Service Video

    Metadata Service Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing
  98. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog

    -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
  99. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog

    -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
  100. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog

    -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
  101. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog

    -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
  102. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog

    -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
  103. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog

    -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
  104. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog

    -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
  105. Delivering Netflix Edge Service Observable Personalized catalog Service Ratings Service

    Video Metadata Service Observable Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing Observable Observable Observable
  106. Extending an Observable beyond Network Boundaries

  107. Client Server Unidirectional protocol (eg: HTTP/1) Observable Write Request Subscribe

  108. Client Server Unidirectional protocol (eg: HTTP/1) Observable Write Request Subscribe

    Send response onNext
  109. Client Server Unidirectional protocol (eg: HTTP/1) Observable Send response onNext

    Close connection unsubscribe Write Request Subscribe
  110. Client Server Bidirectional protocol (eg: HTTP/2) Observable Write Request Subscribe

  111. Client Server Write Request Subscribe Send response onNext Bidirectional protocol

    (eg: HTTP/2) Observable
  112. Client Server Send response onNext Send unsubscribe unsubscribe Write Request

    Subscribe Bidirectional protocol (eg: HTTP/2) Observable
  113. Your Service Services as an Observable.* * Courtesy: “Your Server

    as a function” - Marius Eriksen
  114. Your Services as an Observable.* * Courtesy: “Your Server as

    a function” - Marius Eriksen Reactive IPC
  115. Thank you! RxJava: https://github.com/ReactiveX/RxJava Reactive IPC: https://github.com/ReactiveX/RxNetty Service Discovery: https://github.com/Netflix/eureka/tree/2.x

    Load balancing: https://github.com/Netflix/ocelli Server as a function: http://monkey.org/~marius/funsrv.pdf Fallacies of distributed computing: en.wikipedia.org/wiki/ Fallacies_of_distributed_computing#The_fallacies Nitesh Kant @NiteshKant