CodeMentor April 2015

This was presented as a hangout session on codementor:

https://www.codementor.io/officehours/3965269104/codementor-office-hours-revolutionizing-ipc-with-nitesh-kant-of-netflix

This presentation introduces reactive IPC, it's core tenants and what value adds it brings vis-a-vis traditional IPC technologies.

1. Nitesh Kant - @NiteshKant - Software Engineer, Cloud Platform, Netflix Reactive IPC Revolutionize your Microservices
2. Nitesh Kant Who Am I? ❖ Engineer, Cloud Platform, Netflix. ❖ Leading Reactive IPC @Netflix ❖ Core contributor, RxNetty* ❖ Contributor, Eureka** ❖ Suffers with OCD => API design * https://github.com/ReactiveX/RxNetty ** https://github.com/Netflix/eureka @NiteshKant
3. Inter Process Communication (IPC)
4. Delivering Netflix What goes behind the scenes?
5. Calling Home One of many HTTP calls to deliver Netflix. http://www.netflix.com/WiHome
6. Pipe Dream Always available, infinitely scalable machine.
7. Reality Thousands of machines with hundreds of microservices
8. Microservices A typical relationship between different microservices.
12. IPC Any call made outside of your process to a remote machine or local, is an IPC call.
13. Reactive IPC
14. Instances variations We auto-scale up/down all the time.
15. Failures Instances fail in various ways. X X X X
16. Fallacies of distributed computing http://en.wikipedia.org/ wiki/ Fallacies_of_distributed_comp uting#The_fallacies
18. Traditional IPC
19. Synchronous systems Consuming a resource (thread) per user request does bad things to you. http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
20. Synchronous systems Consuming a resource (thread) per user request does bad things to you. http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html You isolate yourself by introducing a queuing point.
21. Synchronous systems Consuming a resource (thread) per user request does bad things to you. http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html You isolate yourself by introducing a queuing point. But then you are limited by the number of threads in the system.
22. Reactive IPC
23. Asynchronous systems Multiple connections/requests multiplexed on a single thread (Eventloop). Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection Application Code
24. Asynchronous systems Multiple connections/requests multiplexed on a single thread (Eventloop). An Eventloop only does CPU work and does not block. Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection Application Code
25. Asynchronous systems Multiple connections/requests multiplexed on a single thread (Eventloop). An Eventloop only does CPU work and does not block. A single bad dependency/connection does not bring down the system. Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection Application Code
26. Asynchronous systems Multiple connections/requests multiplexed on a single thread (Eventloop). An Eventloop only does CPU work and does not block. As many Eventloops as CPUs. A single bad dependency/connection does not bring down the system. Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection Application Code Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloop (Outbound) Connection Connection Connection Connection Connection
27. Let’s look at the bigger picture
28. Service discovery IPC Reactive IPC Pull based discovery
29. Movie Service User Service Service discovery Which are the instances of User Service?
30. Movie Service User Service Service discovery Which are the instances of User Service? Instances I1, I2, I3
31. Movie Service User Service Service discovery Which are the instances of User Service? Instances I1, I2, I3 Any changes? Nope
32. Movie Service User Service Service discovery Which are the instances of User Service? Instances I1, I2, I3 Any changes? Nope Any changes? I2 is dead
33. Movie Service User Service Service discovery Which are the instances of User Service? Instances I1, I2, I3 Any changes? Nope Any changes? I2 is dead Somewhere here I2 died
34. Movie Service User Service Service discovery Which are the instances of User Service? Instances I1, I2, I3 Any changes? Nope Any changes? I2 is dead Somewhere here I2 died
35. Service discovery IPC Reactive IPC Pull based discovery Push based discovery
36. Movie Service User Service Service discovery I am interested in User Service instances.
37. Movie Service User Service Service discovery I am interested in User Service instances. Instances I1, I2, I3
38. Movie Service User Service Service discovery I am interested in User Service instances. Instances I1, I2, I3 Any changes? Nope X
39. Movie Service User Service Service discovery I am interested in User Service instances. Instances I1, I2, I3 I2 died
40. Service discovery IPC Reactive IPC Pull based discovery Push based discovery Implicit multi-stage delays
41. Movie Service User Service Service discovery Which are the instances of User Service? Instances I1, I2, I3 Any changes? Nope Any changes? I2 is dead Somewhere here I2 died
42. Service discovery nodes
43. Service discovery nodes
44. Movie service node Service discovery client User service client
45. Service discovery IPC Reactive IPC Pull based discovery Push based discovery Implicit multi-stage delays Instantaneous updates
46. Service discovery nodes X X X X
47. Movie service node Service discovery client User service client X
48. As soon as an instance is shutdown, it is removed from the clients using them.
49. Control IPC Reactive IPC No optimal cancellations
50. Delivering Netflix Edge Service Personalized catalog service Ratings Service Video Metadata Service Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing
51. Edge Service Personalized catalog service Ratings Service Video Metadata Service Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing
52. Edge Service Personalized catalog service Ratings Service Video Metadata Service Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing X User cancellation
53. Control IPC Reactive IPC No optimal cancellations Built-in Cancellations
54. Edge Service Personalized catalog service Ratings Service Video Metadata Service Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing X User cancellation X X X X
55. Control IPC Reactive IPC No optimal cancellations Built-in Cancellations Transport level flow-control
56. Delivering Netflix Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing
57. Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Get 100 videos details (Batch API) Send Video details
58. Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Get 100 videos details (Batch API) Send Video details }X n
59. Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Get 100 videos details (Batch API) Send Video details }X n That’s too much of data!
60. Control IPC Reactive IPC No optimal cancellations Built-in Cancellations Transport level flow-control Application flow-control
61. Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Get 100 videos details (Batch API)
62. Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Get 100 videos details (Batch API) Request 5
63. Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Get 100 videos details (Batch API) Request 5 Send 5
64. Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Get 100 videos details (Batch API) Request 5 Send 5 Request 5 Send 5
65. Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Get 100 videos details (Batch API) Request 5 Send 5 Request 5 Send 5
66. Failure detection IPC Reactive IPC Polling
67. Client lib Failure detector Is Instance I1 healthy? Any client library
68. Client lib Failure detector Is Instance I1 healthy? Yep Any client library
69. Client lib Failure detector Is Instance I1 healthy? Yep Still healthy? Yep Any client library
70. Client lib Failure detector Is Instance I1 healthy? Yep Still healthy? Nope Any client library Still healthy? Yep
71. Client lib Failure detector Is Instance I1 healthy? Yep Still healthy? Nope Any client library Still healthy? Yep Somewhere here I1 died
72. Client lib Failure detector Is Instance I1 healthy? Yep Still healthy? Nope Any client library Still healthy? Yep Somewhere here I1 died Errors & Retries
73. Failure detection IPC Reactive IPC Polling Push based
74. Client lib Failure detector I am using I1, tell me when it dies Any client library
75. Client lib Failure detector I am using I1, tell me when it dies I1 died Any client library
76. Client lib Failure detector I am using I1, tell me when it dies I1 died Any client library Minimal Errors & Retries
77. Reactive IPC
78. http://reactivex.io/
79. RxJava A library for composing asynchronous and event-based programs using observable sequences for the Java VM. http://github.com/ReactiveX/RxJava
80. Single Multiple Sync T getData() Iterable<T> getData() Stream<T> getData() Async Future<T> getData() Observable<T> getData()
81. Observable Publisher of Data
82. Observable Subscriber Publisher of Data Consumer of Data
83. Observable Publisher of Data Subscription Cancellation control Subscriber Consumer of Data
84. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));
85. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));
86. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));
87. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));
88. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));
89. Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));
90. Subscription sub = Observable.range(1, 100) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done")); . . . // Later based on some other event sub.unsubscribe()
91. Observable.range(1, 100) .takeUntil(Observable.timer(1, TimeUnit.Minutes) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done")); Idiomatic unsubscribe
92. Observable.interval(1, TimeUnit.Seconds) .takeUntil(Observable.timer(1, TimeUnit.Minutes) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));
93. HttpClient.newClient() .createGet("http://myserver.com:8080/hello") .flatMap(HttpClientResponse::getContent) .takeUntil(Observable.timer(1, TimeUnit.Minutes) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));
94. HttpClient.newClient() .createGet("http://www.netflix.com/WiHome") .flatMap(HttpClientResponse::getContent) .takeUntil(Observable.timer(1, TimeUnit.Minutes) .subscribe(System.out::println, Throwable::printStackTrace, () -> System.out.println("Done"));
95. Your Service as an Observable.* * Courtesy: “Your Server as a function” - Marius Eriksen
96. HttpServer.newServer(8080) .start((req, resp) -> resp.sendHeaders() .write(Observable.just("Hello World!!"))); Hello World!
97. Delivering Netflix Edge Service Personalized catalog service Ratings Service Video Metadata Service Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing
98. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
99. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
100. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
101. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
102. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
103. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
104. return HttpServer.newServer(port) .start((req, resp) -> catalogClient.createGet("http://catalogservice/catalog") .switchMap(catalogResp -> resp.sendHeaders() .write(catalogResp.flatMap(catalog -> Catalog::videos) .flatMap(video -> Observable.zip(bookmarkClient.createGet("http://bookmarkservice/" + video.id()) .<Bookmark>flatMap(HttpClientResponse::getContent), ratingClient.createGet("http://ratingsservice/" + video.id()) .<Rating>flatMap(HttpClientResponse::getContent), metadataClient.createGet("http://metadataservice/" + video.id()) .<VideoMetadata>flatMap(HttpClientResponse::getContent), (b, r, m) -> combineVideoData(video, b, r, m))) ))); Delivering Netflix! Disclaimer: This is an example and not an exact representation of the processing
105. Delivering Netflix Edge Service Observable Personalized catalog Service Ratings Service Video Metadata Service Observable Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing Observable Observable Observable
106. Extending an Observable beyond Network Boundaries
107. Client Server Unidirectional protocol (eg: HTTP/1) Observable Write Request Subscribe
108. Client Server Unidirectional protocol (eg: HTTP/1) Observable Write Request Subscribe Send response onNext
109. Client Server Unidirectional protocol (eg: HTTP/1) Observable Send response onNext Close connection unsubscribe Write Request Subscribe
110. Client Server Bidirectional protocol (eg: HTTP/2) Observable Write Request Subscribe
111. Client Server Write Request Subscribe Send response onNext Bidirectional protocol (eg: HTTP/2) Observable
112. Client Server Send response onNext Send unsubscribe unsubscribe Write Request Subscribe Bidirectional protocol (eg: HTTP/2) Observable
113. Your Service Services as an Observable.* * Courtesy: “Your Server as a function” - Marius Eriksen
114. Your Services as an Observable.* * Courtesy: “Your Server as a function” - Marius Eriksen Reactive IPC
115. Thank you! RxJava: https://github.com/ReactiveX/RxJava Reactive IPC: https://github.com/ReactiveX/RxNetty Service Discovery: https://github.com/Netflix/eureka/tree/2.x Load balancing: https://github.com/Netflix/ocelli Server as a function: http://monkey.org/~marius/funsrv.pdf Fallacies of distributed computing: en.wikipedia.org/wiki/ Fallacies_of_distributed_computing#The_fallacies Nitesh Kant @NiteshKant