Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Embracing Streams - Everywhere

A3668e66eb7b8980ac91daaa4e9fe691?s=47 Nitesh Kant
October 10, 2016

Embracing Streams - Everywhere

“I don’t need stream processing because I don’t have streaming data.” - Anonymous

In general people relate to streams as data models which are naturally streaming (infinite asynchronous messages) or relate to realtime big data processing. In this talk Nitesh Kant, will try to break this myth by emphasizing the fact that streams exists everywhere, be it data read from sockets, protocols like HTTP or microservice composition. He will explain how extending this ubiquitous interaction model into applications can result in simpler, resilient and maintainable systems.
You will learn, how to start thinking “streaming first” through concrete examples and how adopting this mental model makes writing application easier.

Presented at ReactiveSummit2016: https://reactivesummit2016.sched.org/event/7geB/embracing-streamseverywhere

Video: https://www.youtube.com/watch?v=5FE6xnH5Lak

A3668e66eb7b8980ac91daaa4e9fe691?s=128

Nitesh Kant

October 10, 2016
Tweet

Transcript

  1. Embracing Streams … Everywhere Nitesh Kant, Software Engineer, Netflix Edge

    Engineering. @NiteshKant
  2. Nitesh Kant Who Am I? ❖ Engineer, Edge Engineering, Netflix.

    ❖ Core contributor, RxNetty* ❖ Contributor, ReactiveSocket** * https://github.com/ReactiveX/RxNetty ** https://github.com/ReactiveSocket @NiteshKant https://speakerdeck.com/niteshkant
  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. This is also true for machines!

  10. A simple example. Showing a movie on Netflix.

  11. Video Metadata

  12. Video Bookmark

  13. Video Rating

  14. public Movie getMovie(String movieId) { Metadata metadata = getMovieMetadata(movieId); Bookmark

    bookmark = getBookmark(movieId, userId); Rating rating = getRatings(movieId); return new Movie(metadata, bookmark, rating); } Disclaimer: This is an example and not an exact representation of the processing
  15. In a microservices world Edge Service Ratings Service Video Metadata

    Service Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing
  16. Edge Service Video Metadata Service Disclaimer: This is an example

    and not an exact representation of the processing GET /movie?id=3 HTTP/1.1
  17. Video Metadata Service Disclaimer: This is an example and not

    an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service
  18. Video Metadata Service Disclaimer: This is an example and not

    an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service
  19. Disclaimer: This is an example and not an exact representation

    of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service
  20. Disclaimer: This is an example and not an exact representation

    of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service Meanwhile
  21. Disclaimer: This is an example and not an exact representation

    of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service Request pileup
  22. None
  23. None
  24. Two-way street Communication should be in both ways at all

    times.
  25. Disclaimer: This is an example and not an exact representation

    of the processing public Movie getMovie(String movieId) { Metadata metadata = getMovieMetadata(movieId); Bookmark bookmark = getBookmark(movieId, userId); Rating rating = getRatings(movieId); return new Movie(metadata, bookmark, rating); } Edge Service
  26. One-way street No way to provide feedback. Disclaimer: This is

    an example and not an exact representation of the processing public Movie getMovie(String movieId) { Metadata metadata = getMovieMetadata(movieId); Bookmark bookmark = getBookmark(movieId, userId); Rating rating = getRatings(movieId); return new Movie(metadata, bookmark, rating); }
  27. Can we do better?

  28. Streams

  29. Lazy sources of data.

  30. Lazy sources of data. Emits zero or more items.

  31. Lazy sources of data. Emits zero or more items. Emissions

    are controlled by the consumer.
  32. Streams exist everywhere.

  33. Text Files Stream of lines.

  34. TCP Bi-directional streams of bytes.

  35. HTTP chunking Uni-directional stream of chunks.

  36. Friendly neighborhood stream :) Stream of friendly people!

  37. Modelling a Stream? ReactiveStreams https://github.com/reactive-streams/reactive-streams-jvm

  38. Publisher Source of the stream. public interface Publisher<T> { public

    void subscribe(Subscriber<? super T> s); }
  39. public interface Publisher<T> { public void subscribe(Subscriber<? super T> s);

    } Lazy
  40. Subscriber Sink for the stream. public interface Subscriber<T> { public

    void onSubscribe(Subscription s); public void onNext(T t); public void onError(Throwable t); public void onComplete(); }
  41. public interface Subscriber<T> { public void onSubscribe(Subscription s); public void

    onNext(T t); public void onError(Throwable t); public void onComplete(); } Zero or more items.
  42. public interface Subscriber<T> { public void onSubscribe(Subscription s); public void

    onNext(T t); public void onError(Throwable t); public void onComplete(); } At most one terminal event.
  43. public interface Subscriber<T> { public void onSubscribe(Subscription s); public void

    onNext(T t); public void onError(Throwable t); public void onComplete(); } Publisher supplied control handle.
  44. Subscription Publisher supplied control handle. public interface Subscription { public

    void request(long n); public void cancel(); }
  45. public interface Subscription { public void request(long n); public void

    cancel(); } Flow control
  46. public interface Subscription { public void request(long n); public void

    cancel(); } Cancellation
  47. In a distributed system…

  48. Request Processing Edge Service Video Metadata Service Disclaimer: This is

    an example and not an exact representation of the processing Bookmarks Service Ratings Service
  49. Edge Service Video Metadata Service Disclaimer: This is an example

    and not an exact representation of the processing Rating service C* store C* store /movie?id=123
  50. Streams over network boundaries

  51. Network Protocol

  52. Network Protocol Application Level Flow Control

  53. Network Protocol Application Level Flow Control Cancellation

  54. Network Protocol Application Level Flow Control Item Emissions (onNext) Cancellation

  55. Network Protocol Application Level Flow Control Terminal Events (onError/Complete) Cancellation

    Item Emissions (onNext)
  56. Network Protocol Application Level Flow Control Cancellation Item Emissions (onNext)

    Terminal Events (onError/Complete)
  57. Network Protocol Application Level Flow Control Cancellation Item Emissions (onNext)

    Terminal Events (onError/Complete) }Consumer => Producer
  58. Network Protocol Application Level Flow Control Cancellation Item Emissions (onNext)

    Terminal Events (onError/Complete) { Producer => Consumer
  59. Bi-directional Network Protocol

  60. Bi-directional Network Protocol HTTP/1.1 ?

  61. HTTP/1.1 GET /movie?id=1 HTTP/1.1

  62. HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1

  63. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1

  64. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1

    HTTP/1.1 200 OK ID: 1 …
  65. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1

    HTTP/1.1 200 OK ID: 1 … HTTP/1.1 200 OK ID: 2 …
  66. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1

    HTTP/1.1 200 OK ID: 1 … HTTP/1.1 200 OK ID: 2 … HTTP/1.1 200 OK ID: 3 …
  67. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1

    HTTP/1.1 200 OK ID: 1 … HTTP/1.1 200 OK ID: 2 … HTTP/1.1 200 OK ID: 3 … Uni-directional
  68. Network Protocol HTTP/2.0 ?

  69. HTTP/2.0 Bi-directional

  70. HTTP/2.0 Bi-directional Application Level Flow Control

  71. Network Protocol ReactiveSocket https://github.com/ReactiveSocket/reactivesocket/

  72. ReactiveStreams over Network Boundaries

  73. ReactiveStreams over Network Boundaries First class flow controlled, cancellable, bi-directional

    streams support
  74. ReactiveSocket Bi-directional Application Level Flow Control

  75. Embracing Streams Streams as a building block.

  76. None
  77. None
  78. None
  79. None
  80. None
  81. None
  82. None
  83. None
  84. Typical Streams Existing as polling sources in all websites.

  85. Atypical Streams Static lists of zero or one items.

  86. Big Deal? Sure, but what benefit would it have?

  87. Varying form factors. Different screen sizes means different viewing area.

  88. None
  89. Common entity Flow controlled stream User (device) controlled stream is

    a natural way to paginate, hence, reducing network data transfer.
  90. Streams imply State

  91. Streams imply State State over a connection.

  92. Full duplex protocol Ephemeral state management provides newer avenues.

  93. Eliminate repetitive information exchange. a.k.a HTTP Cookies

  94. curl -v "http://www.netflix.com"

  95. curl -v "http://www.netflix.com"\ > GET / HTTP/1.1\ > \ <

    HTTP/1.1 302 Found\ < Date: Tue, 04 Oct 2016 02:37:51 GMT\ < set-cookie: nflx-rgn=uw2| 1475548671570; Max-Age=-1; Expires=Tue, 04 Oct 2016 02:37:50 GMT; Path=/; Domain=.netflix.com\ < Connection: keep-alive\ < \
  96. curl -v "http://www.netflix.com"\ > GET / HTTP/1.1\ > Cookie:nflx-rgn=uw2|1475548671570 >

    \
  97. Stateless protocols exchange repetitive information.

  98. Stateful protocols may exchange information once per connection.

  99. Handshake user_id=1

  100. Handshake user_id=1 user_id=1 geo=cal . .

  101. Requests user_id=1 geo=cal . . getMovie?movie?id=1

  102. Requests user_id=1 geo=cal . . getMovie?movie?id=2 getMovie?movie?id=1

  103. Request Caching

  104. Request Caching HTTP ETags

  105. Request Caching HTTP ETags Hystrix* Request caching * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching

  106. Request Caching HTTP ETags Hystrix* Request caching * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching }Time

    based caches.
  107. Request Caching * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching Time based caches. Expiration?

  108. Request Caching * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching Time based caches. Expiration?

  109. Live Request Cache

  110. Cache Streams getBookmark?id=1

  111. Cache Streams getBookmark?id=1 position=x

  112. Cache Streams getBookmark?id=1 position=x

  113. Cache Streams getBookmark?id=1 position=x position=y

  114. Cache Streams getBookmark?id=1 position=y

  115. Cache Streams Time based caches. Expiration Server push X

  116. Request Cancellations Eliminate unnecessary work.

  117. Edge Service Video Metadata Service Rating service C* store C*

    store /movie?id=123 Disclaimer: This is an example and not an exact representation of the processing
  118. Edge Service Video Metadata Service Rating service C* store C*

    store /movie?id=123 X Disclaimer: This is an example and not an exact representation of the processing
  119. Edge Service Video Metadata Service Rating service C* store C*

    store /movie?id=123 X X X X X Disclaimer: This is an example and not an exact representation of the processing
  120. These are usually built as special features!

  121. Resiliency How does this model help to increase resiliency?

  122. Resiliency Managing server overload.

  123. Typical throttling Video Metadata Service Disclaimer: This is an example

    and not an exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service
  124. Typical throttling Disclaimer: This is an example and not an

    exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service HTTP/1.1 503 Service Unavailable
  125. Typical throttling Disclaimer: This is an example and not an

    exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service HTTP/1.1 503 Service Unavailable Backoff
  126. Typical throttling Disclaimer: This is an example and not an

    exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service HTTP/1.1 503 Service Unavailable Backoff When will this server be ok?
  127. Eliminating guesswork? Disclaimer: This is an example and not an

    exact representation of the processing GET /movie?id=3 HTTP/1.1 HTTP/1.1 200 OK Edge Service HTTP/1.1 503 Service Unavailable Backoff When will this server be ok?
  128. Request-leasing http://reactivesocket.io/

  129. Request-leasing Peer 1 Peer 2 Network connection

  130. Request-leasing Peer 1 Peer 2 “Lease” 5 requests for 1

    second. Network connection
  131. Request-leasing Peer 1 Peer 2 “Lease” 5 requests for 1

    second. GET /movie?id=1 HTTP/1.1 GET /movie?id=2 HTTP/1.1 Network connection
  132. Server Client 1 Client 2 Client 8

  133. Server Capacity: 100 RPS Client 1 Client 2 Client 8

    “Lease” 10 requests for 1 second.
  134. Server Capacity: 100 RPS Client 1 Client 2 Client 8

    “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second.
  135. Server Capacity: 100 RPS Client 1 Client 2 Client 8

    “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. Reserve Capacity: 20 RPS
  136. Time bound lease. “Lease” 10 requests for 1 second.

  137. Time bound lease. No extra work for cancelling leases. “Lease”

    10 requests for 1 second.
  138. Time bound lease. No extra work for cancelling leases. Receiver

    controls the flow of requests “Lease” 10 requests for 1 second.
  139. When things go south

  140. Server Capacity: 20 RPS Client 1 Client 2 Client 8

    “Lease” 5 requests for 1 second. “Lease” 2 requests for 1 second. X No more “Lease”
  141. Server Capacity: 20 RPS Client 1 Client 2 Client 8

    “Lease” 5 requests for 1 second. “Lease” 2 requests for 1 second. X No more “Lease” Prioritization
  142. Server Capacity: 100 RPS Client 1 Client 2 Client 8

    “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. Reserve Capacity: 20 RPS
  143. Server Capacity: 100 RPS Client 1 Client 2 Client 8

    “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. “Lease” 10 requests for 1 second. Reserve Capacity: 20 RPS Self Healing!
  144. Self Healing! Under provisioning => Catastrophic Failures /

  145. Self Healing! Under provisioning => Catastrophic Failures / Graceful degradation

  146. Edge Service Video Metadata Service Rating service C* store C*

    store Disclaimer: This is an example and not an exact representation of the processing /movie?id=123
  147. Edge Service Video Metadata Service Rating service C* store C*

    store Disclaimer: This is an example and not an exact representation of the processing /movie?id=123 Publisher<Movie>
  148. Request processing as a stream! Publisher<Movie>

  149. Powerful Abstractions Generic processing models provide features naturally, hence, reducing

    feature clutter in the applications.
  150. public interface Publisher<T> { public void subscribe(Subscriber<? super T> s);

    } public interface Subscriber<T> { public void onSubscribe(Subscription s); public void onNext(T t); public void onError(Throwable t); public void onComplete(); } public interface Subscription { public void request(long n); public void cancel(); } Stream s
  151. Nitesh Kant, Netflix Edge Engineering @NiteshKant https://speakerdeck.com/niteshkant