Embracing Streams - Everywhere

“I don’t need stream processing because I don’t have streaming data.” - Anonymous

In general people relate to streams as data models which are naturally streaming (infinite asynchronous messages) or relate to realtime big data processing. In this talk Nitesh Kant, will try to break this myth by emphasizing the fact that streams exists everywhere, be it data read from sockets, protocols like HTTP or microservice composition. He will explain how extending this ubiquitous interaction model into applications can result in simpler, resilient and maintainable systems.
You will learn, how to start thinking “streaming first” through concrete examples and how adopting this mental model makes writing application easier.

Presented at ReactiveSummit2016: https://reactivesummit2016.sched.org/event/7geB/embracing-streamseverywhere

Video: https://www.youtube.com/watch?v=5FE6xnH5Lak

Transcript

1. Embracing Streams … Everywhere Nitesh Kant, Software Engineer, Netflix Edge Engineering. @NiteshKant
2. Nitesh Kant Who Am I? ❖ Engineer, Edge Engineering, Netflix. ❖ Core contributor, RxNetty* ❖ Contributor, ReactiveSocket** * https://github.com/ReactiveX/RxNetty ** https://github.com/ReactiveSocket @NiteshKant https://speakerdeck.com/niteshkant
9. This is also true for machines!
10. A simple example. Showing a movie on Netflix.
11. Video Metadata
12. Video Bookmark
13. Video Rating
14. public Movie getMovie(String movieId) { Metadata metadata = getMovieMetadata(movieId); Bookmark bookmark = getBookmark(movieId, userId); Rating rating = getRatings(movieId); return new Movie(metadata, bookmark, rating); } Disclaimer: This is an example and not an exact representation of the processing
15. In a microservices world Edge Service Ratings Service Video Metadata Service Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing
16. Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing GET /movie?id=3 HTTP/1.1
17. Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service
18. Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service
19. Disclaimer: This is an example and not an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service
20. Disclaimer: This is an example and not an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service Meanwhile
21. Disclaimer: This is an example and not an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service Request pileup
24. Two-way street Communication should be in both ways at all times.
25. Disclaimer: This is an example and not an exact representation of the processing public Movie getMovie(String movieId) { Metadata metadata = getMovieMetadata(movieId); Bookmark bookmark = getBookmark(movieId, userId); Rating rating = getRatings(movieId); return new Movie(metadata, bookmark, rating); } Edge Service
26. One-way street No way to provide feedback. Disclaimer: This is an example and not an exact representation of the processing public Movie getMovie(String movieId) { Metadata metadata = getMovieMetadata(movieId); Bookmark bookmark = getBookmark(movieId, userId); Rating rating = getRatings(movieId); return new Movie(metadata, bookmark, rating); }
27. Can we do better?
28. Streams
29. Lazy sources of data.
30. Lazy sources of data. Emits zero or more items.
31. Lazy sources of data. Emits zero or more items. Emissions are controlled by the consumer.
32. Streams exist everywhere.
33. Text Files Stream of lines.
34. TCP Bi-directional streams of bytes.
35. HTTP chunking Uni-directional stream of chunks.
36. Friendly neighborhood stream :) Stream of friendly people!
37. Modelling a Stream? ReactiveStreams https://github.com/reactive-streams/reactive-streams-jvm
38. Publisher Source of the stream. public interface Publisher<T> { public void subscribe(Subscriber<? super T> s); }
39. public interface Publisher<T> { public void subscribe(Subscriber<? super T> s); } Lazy
40. Subscriber Sink for the stream. public interface Subscriber<T> { public void onSubscribe(Subscription s); public void onNext(T t); public void onError(Throwable t); public void onComplete(); }
41. public interface Subscriber<T> { public void onSubscribe(Subscription s); public void onNext(T t); public void onError(Throwable t); public void onComplete(); } Zero or more items.
42. public interface Subscriber<T> { public void onSubscribe(Subscription s); public void onNext(T t); public void onError(Throwable t); public void onComplete(); } At most one terminal event.
43. public interface Subscriber<T> { public void onSubscribe(Subscription s); public void onNext(T t); public void onError(Throwable t); public void onComplete(); } Publisher supplied control handle.
44. Subscription Publisher supplied control handle. public interface Subscription { public void request(long n); public void cancel(); }
45. public interface Subscription { public void request(long n); public void cancel(); } Flow control
46. public interface Subscription { public void request(long n); public void cancel(); } Cancellation
47. In a distributed system…
48. Request Processing Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Bookmarks Service Ratings Service
49. Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Rating service C* store C* store /movie?id=123
50. Streams over network boundaries
51. Network Protocol
52. Network Protocol Application Level Flow Control
53. Network Protocol Application Level Flow Control Cancellation
54. Network Protocol Application Level Flow Control Item Emissions (onNext) Cancellation
55. Network Protocol Application Level Flow Control Terminal Events (onError/Complete) Cancellation Item Emissions (onNext)
56. Network Protocol Application Level Flow Control Cancellation Item Emissions (onNext) Terminal Events (onError/Complete)
57. Network Protocol Application Level Flow Control Cancellation Item Emissions (onNext) Terminal Events (onError/Complete) }Consumer => Producer
58. Network Protocol Application Level Flow Control Cancellation Item Emissions (onNext) Terminal Events (onError/Complete) { Producer => Consumer
59. Bi-directional Network Protocol
60. Bi-directional Network Protocol HTTP/1.1 ?
61. HTTP/1.1 GET /movie?id=1 HTTP/1.1
62. HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1
63. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1
64. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK ID: 1 …
65. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK ID: 1 … HTTP/1.1 200 OK ID: 2 …
66. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK ID: 1 … HTTP/1.1 200 OK ID: 2 … HTTP/1.1 200 OK ID: 3 …
67. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK ID: 1 … HTTP/1.1 200 OK ID: 2 … HTTP/1.1 200 OK ID: 3 … Uni-directional
68. Network Protocol HTTP/2.0 ?
69. HTTP/2.0 Bi-directional
70. HTTP/2.0 Bi-directional Application Level Flow Control
71. Network Protocol ReactiveSocket https://github.com/ReactiveSocket/reactivesocket/
72. ReactiveStreams over Network Boundaries
73. ReactiveStreams over Network Boundaries First class flow controlled, cancellable, bi-directional streams support
74. ReactiveSocket Bi-directional Application Level Flow Control
75. Embracing Streams Streams as a building block.
84. Typical Streams Existing as polling sources in all websites.
85. Atypical Streams Static lists of zero or one items.
86. Big Deal? Sure, but what benefit would it have?
87. Varying form factors. Different screen sizes means different viewing area.
89. Common entity Flow controlled stream User (device) controlled stream is a natural way to paginate, hence, reducing network data transfer.
90. Streams imply State
91. Streams imply State State over a connection.
92. Full duplex protocol Ephemeral state management provides newer avenues.
93. Eliminate repetitive information exchange. a.k.a HTTP Cookies
94. curl -v "http://www.netflix.com"
95. curl -v "http://www.netflix.com"\ > GET / HTTP/1.1\ > \ < HTTP/1.1 302 Found\ < Date: Tue, 04 Oct 2016 02:37:51 GMT\ < set-cookie: nflx-rgn=uw2| 1475548671570; Max-Age=-1; Expires=Tue, 04 Oct 2016 02:37:50 GMT; Path=/; Domain=.netflix.com\ < Connection: keep-alive\ < \
96. curl -v "http://www.netflix.com"\ > GET / HTTP/1.1\ > Cookie:nflx-rgn=uw2|1475548671570 > \
97. Stateless protocols exchange repetitive information.
98. Stateful protocols may exchange information once per connection.
99. Handshake user_id=1
100. Handshake user_id=1 user_id=1 geo=cal . .
101. Requests user_id=1 geo=cal . . getMovie?movie?id=1
102. Requests user_id=1 geo=cal . . getMovie?movie?id=2 getMovie?movie?id=1
103. Request Caching
104. Request Caching HTTP ETags
105. Request Caching HTTP ETags Hystrix* Request caching * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching
106. Request Caching HTTP ETags Hystrix* Request caching * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching }Time based caches.
107. Request Caching * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching Time based caches. Expiration?
108. Request Caching * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching Time based caches. Expiration?
109. Live Request Cache
110. Cache Streams getBookmark?id=1
111. Cache Streams getBookmark?id=1 position=x
112. Cache Streams getBookmark?id=1 position=x
113. Cache Streams getBookmark?id=1 position=x position=y
114. Cache Streams getBookmark?id=1 position=y
115. Cache Streams Time based caches. Expiration Server push X
116. Request Cancellations Eliminate unnecessary work.
117. Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 Disclaimer: This is an example and not an exact representation of the processing
118. Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 X Disclaimer: This is an example and not an exact representation of the processing
119. Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 X X X X X Disclaimer: This is an example and not an exact representation of the processing
120. These are usually built as special features!
121. Resiliency How does this model help to increase resiliency?
122. Resiliency Managing server overload.
123. Typical throttling Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service
124. Typical throttling Disclaimer: This is an example and not an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service HTTP/1.1 503 Service Unavailable
125. Typical throttling Disclaimer: This is an example and not an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service HTTP/1.1 503 Service Unavailable Backoff
126. Typical throttling Disclaimer: This is an example and not an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service HTTP/1.1 503 Service Unavailable Backoff When will this server be ok?
127. Eliminating guesswork? Disclaimer: This is an example and not an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service HTTP/1.1 503 Service Unavailable Backoff When will this server be ok?
128. Request-leasing http://reactivesocket.io/
129. Request-leasing Peer 1 Peer 2 Network connection
130. Request-leasing Peer 1 Peer 2 “Lease” 5 requests for 1 second. Network connection
131. Request-leasing Peer 1 Peer 2 “Lease” 5 requests for 1 second. GET /movie?id=1 HTTP/1.1 GET /movie?id=2 HTTP/1.1 Network connection
132. Server Client 1 Client 2 Client 8
133. Server Capacity: 100 RPS Client 1 Client 2 Client 8 “Lease” 10 requests for 1 second.
134. Server Capacity: 100 RPS Client 1 Client 2 Client 8 “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second.
135. Server Capacity: 100 RPS Client 1 Client 2 Client 8 “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. Reserve Capacity: 20 RPS
136. Time bound lease. “Lease” 10 requests for 1 second.
137. Time bound lease. No extra work for cancelling leases. “Lease” 10 requests for 1 second.
138. Time bound lease. No extra work for cancelling leases. Receiver controls the flow of requests “Lease” 10 requests for 1 second.
139. When things go south
140. Server Capacity: 20 RPS Client 1 Client 2 Client 8 “Lease” 5 requests for 1 second. “Lease” 2 requests for 1 second. X No more “Lease”
141. Server Capacity: 20 RPS Client 1 Client 2 Client 8 “Lease” 5 requests for 1 second. “Lease” 2 requests for 1 second. X No more “Lease” Prioritization
142. Server Capacity: 100 RPS Client 1 Client 2 Client 8 “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. Reserve Capacity: 20 RPS
143. Server Capacity: 100 RPS Client 1 Client 2 Client 8 “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. Reserve Capacity: 20 RPS Self Healing!
144. Self Healing! Under provisioning => Catastrophic Failures /
145. Self Healing! Under provisioning => Catastrophic Failures / Graceful degradation
146. Edge Service Video Metadata Service Rating service C* store C* store Disclaimer: This is an example and not an exact representation of the processing /movie?id=123
147. Edge Service Video Metadata Service Rating service C* store C* store Disclaimer: This is an example and not an exact representation of the processing /movie?id=123 Publisher<Movie>
148. Request processing as a stream! Publisher<Movie>
149. Powerful Abstractions Generic processing models provide features naturally, hence, reducing feature clutter in the applications.
150. public interface Publisher<T> { public void subscribe(Subscriber<? super T> s); } public interface Subscriber<T> { public void onSubscribe(Subscription s); public void onNext(T t); public void onError(Throwable t); public void onComplete(); } public interface Subscription { public void request(long n); public void cancel(); } Stream s
151. Nitesh Kant, Netflix Edge Engineering @NiteshKant https://speakerdeck.com/niteshkant