Embracing Streams - Everywhere

“I don’t need stream processing because I don’t have streaming data.” - Anonymous

In general people relate to streams as data models which are naturally streaming (infinite asynchronous messages) or relate to realtime big data processing. In this talk Nitesh Kant, will try to break this myth by emphasizing the fact that streams exists everywhere, be it data read from sockets, protocols like HTTP or microservice composition. He will explain how extending this ubiquitous interaction model into applications can result in simpler, resilient and maintainable systems.
You will learn, how to start thinking “streaming first” through concrete examples and how adopting this mental model makes writing application easier.

Presented at ReactiveSummit2016: https://reactivesummit2016.sched.org/event/7geB/embracing-streamseverywhere

Video: https://www.youtube.com/watch?v=5FE6xnH5Lak

Transcript

1. Embracing Streams
   … Everywhere
   Nitesh Kant,
   Software Engineer,
   Netflix Edge Engineering.
   @NiteshKant
   

   View Slide

2. Nitesh Kant
   Who Am I?
   ❖ Engineer, Edge Engineering, Netflix.
   ❖ Core contributor, RxNetty*
   ❖ Contributor, ReactiveSocket**
   * https://github.com/ReactiveX/RxNetty
   ** https://github.com/ReactiveSocket
   @NiteshKant
   https://speakerdeck.com/niteshkant
   

   View Slide

3. View Slide

4. View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. This is also true for
   machines!
   

   View Slide

10. A simple example. Showing a movie on Netflix.
    

    View Slide

11. Video Metadata
    

    View Slide

12. Video Bookmark
    

    View Slide

13. Video Rating
    

    View Slide

14. public Movie getMovie(String movieId) {
    Metadata metadata = getMovieMetadata(movieId);
    Bookmark bookmark = getBookmark(movieId, userId);
    Rating rating = getRatings(movieId);
    return new Movie(metadata, bookmark, rating);
    }
    Disclaimer: This is an example and not an exact representation of the processing
    

    View Slide

15. In a microservices world
    Edge Service
    Ratings Service
    Video Metadata Service
    Bookmarks Service
    Disclaimer: This is an example and not an exact representation of the processing
    

    View Slide

16. Edge Service
    Video Metadata Service
    Disclaimer: This is an example and not an exact representation of the processing
    GET /movie?id=3 HTTP/1.1
    

    View Slide

17. Video Metadata Service
    Disclaimer: This is an example and not an exact representation of the processing
    GET /movie?id=3 HTTP/1.1
    HTTP/1.1 200 OK
    Edge Service
    

    View Slide

18. Video Metadata Service
    Disclaimer: This is an example and not an exact representation of the processing
    GET /movie?id=3 HTTP/1.1
    HTTP/1.1 200 OK
    Edge Service
    

    View Slide

19. Disclaimer: This is an example and not an exact representation of the processing
    GET /movie?id=3 HTTP/1.1
    HTTP/1.1 200 OK
    Edge Service
    

    View Slide

20. Disclaimer: This is an example and not an exact representation of the processing
    GET /movie?id=3 HTTP/1.1
    HTTP/1.1 200 OK
    Edge Service
    Meanwhile
    

    View Slide

21. Disclaimer: This is an example and not an exact representation of the processing
    GET /movie?id=3 HTTP/1.1
    HTTP/1.1 200 OK
    Edge Service Request pileup
    

    View Slide

22. View Slide

23. View Slide

24. Two-way street Communication should be in both ways
    at all times.
    

    View Slide

25. Disclaimer: This is an example and not an exact representation of the processing
    public Movie getMovie(String movieId) {
    Metadata metadata = getMovieMetadata(movieId);
    Bookmark bookmark = getBookmark(movieId, userId);
    Rating rating = getRatings(movieId);
    return new Movie(metadata, bookmark, rating);
    }
    Edge Service
    

    View Slide

26. One-way street No way to provide feedback.
    Disclaimer: This is an example and not an exact representation of the processing
    public Movie getMovie(String movieId) {
    Metadata metadata = getMovieMetadata(movieId);
    Bookmark bookmark = getBookmark(movieId, userId);
    Rating rating = getRatings(movieId);
    return new Movie(metadata, bookmark, rating);
    }
    

    View Slide

27. Can we do better?
    

    View Slide

28. Streams
    

    View Slide

29. Lazy sources of data.
    

    View Slide

30. Lazy sources of data.
    Emits zero or more items.
    

    View Slide

31. Lazy sources of data.
    Emits zero or more items.
    Emissions are controlled by the consumer.
    

    View Slide

32. Streams exist everywhere.
    

    View Slide

33. Text Files
    Stream of lines.
    

    View Slide

34. TCP
    Bi-directional streams of bytes.
    

    View Slide

35. HTTP chunking
    Uni-directional stream of chunks.
    

    View Slide

36. Friendly neighborhood
    stream :)
    Stream of friendly people!
    

    View Slide

37. Modelling a Stream?
    ReactiveStreams
    https://github.com/reactive-streams/reactive-streams-jvm
    

    View Slide

38. Publisher Source of the stream.
    public interface Publisher {
    public void subscribe(Subscriber super T> s);
    }
    

    View Slide

39. public interface Publisher {
    public void subscribe(Subscriber super T> s);
    }
    Lazy
    

    View Slide

40. Subscriber Sink for the stream.
    public interface Subscriber {
    public void onSubscribe(Subscription s);
    public void onNext(T t);
    public void onError(Throwable t);
    public void onComplete();
    }
    

    View Slide

41. public interface Subscriber {
    public void onSubscribe(Subscription s);
    public void onNext(T t);
    public void onError(Throwable t);
    public void onComplete();
    }
    Zero or more items.
    

    View Slide

42. public interface Subscriber {
    public void onSubscribe(Subscription s);
    public void onNext(T t);
    public void onError(Throwable t);
    public void onComplete();
    }
    At most one terminal event.
    

    View Slide

43. public interface Subscriber {
    public void onSubscribe(Subscription s);
    public void onNext(T t);
    public void onError(Throwable t);
    public void onComplete();
    }
    Publisher supplied control handle.
    

    View Slide

44. Subscription Publisher supplied control
    handle.
    public interface Subscription {
    public void request(long n);
    public void cancel();
    }
    

    View Slide

45. public interface Subscription {
    public void request(long n);
    public void cancel();
    }
    Flow control
    

    View Slide

46. public interface Subscription {
    public void request(long n);
    public void cancel();
    }
    Cancellation
    

    View Slide

47. In a distributed system…
    

    View Slide

48. Request Processing
    Edge Service
    Video Metadata Service
    Disclaimer: This is an example and not an exact representation of the processing
    Bookmarks Service Ratings Service
    

    View Slide

49. Edge Service
    Video Metadata Service
    Disclaimer: This is an example and not an exact representation of the processing
    Rating service C* store
    C* store
    /movie?id=123
    

    View Slide

50. Streams over
    network boundaries
    

    View Slide

51. Network Protocol
    

    View Slide

52. Network Protocol
    Application Level Flow Control
    

    View Slide

53. Network Protocol
    Application Level Flow
    Control
    Cancellation
    

    View Slide

54. Network Protocol
    Application Level Flow
    Control
    Item Emissions (onNext)
    Cancellation
    

    View Slide

55. Network Protocol
    Application Level Flow
    Control
    Terminal Events (onError/Complete)
    Cancellation
    Item Emissions (onNext)
    

    View Slide

56. Network Protocol
    Application Level Flow
    Control
    Cancellation
    Item Emissions (onNext)
    Terminal Events
    (onError/Complete)
    

    View Slide

57. Network Protocol
    Application Level Flow
    Control
    Cancellation
    Item Emissions (onNext)
    Terminal Events
    (onError/Complete)
    }Consumer => Producer
    

    View Slide

58. Network Protocol
    Application Level Flow
    Control
    Cancellation
    Item Emissions (onNext)
    Terminal Events
    (onError/Complete)
    {
    Producer => Consumer
    

    View Slide

59. Bi-directional Network Protocol
    

    View Slide

60. Bi-directional Network Protocol
    HTTP/1.1 ?
    

    View Slide

61. HTTP/1.1
    GET /movie?id=1 HTTP/1.1
    

    View Slide

62. HTTP/1.1
    GET /movie?id=2 HTTP/1.1
    GET /movie?id=1 HTTP/1.1
    

    View Slide

63. HTTP/1.1
    GET /movie?id=3 HTTP/1.1
    GET /movie?id=2 HTTP/1.1
    GET /movie?id=1 HTTP/1.1
    

    View Slide

64. HTTP/1.1
    GET /movie?id=3 HTTP/1.1
    GET /movie?id=2 HTTP/1.1
    GET /movie?id=1 HTTP/1.1
    HTTP/1.1 200 OK
    ID: 1
    …
    

    View Slide

65. HTTP/1.1
    GET /movie?id=3 HTTP/1.1
    GET /movie?id=2 HTTP/1.1
    GET /movie?id=1 HTTP/1.1
    HTTP/1.1 200 OK
    ID: 1
    … HTTP/1.1 200 OK
    ID: 2
    …
    

    View Slide

66. HTTP/1.1
    GET /movie?id=3 HTTP/1.1
    GET /movie?id=2 HTTP/1.1
    GET /movie?id=1 HTTP/1.1
    HTTP/1.1 200 OK
    ID: 1
    … HTTP/1.1 200 OK
    ID: 2
    …
    HTTP/1.1 200 OK
    ID: 3
    …
    

    View Slide

67. HTTP/1.1
    GET /movie?id=3 HTTP/1.1
    GET /movie?id=2 HTTP/1.1
    GET /movie?id=1 HTTP/1.1
    HTTP/1.1 200 OK
    ID: 1
    … HTTP/1.1 200 OK
    ID: 2
    …
    HTTP/1.1 200 OK
    ID: 3
    …
    Uni-directional
    

    View Slide

68. Network Protocol
    HTTP/2.0 ?
    

    View Slide

69. HTTP/2.0
    Bi-directional
    

    View Slide

70. HTTP/2.0
    Bi-directional
    Application Level Flow Control
    

    View Slide

71. Network Protocol
    ReactiveSocket
    https://github.com/ReactiveSocket/reactivesocket/
    

    View Slide

72. ReactiveStreams over Network
    Boundaries
    

    View Slide

73. ReactiveStreams over Network
    Boundaries
    First class flow controlled, cancellable,
    bi-directional streams support
    

    View Slide

74. ReactiveSocket
    Bi-directional
    Application Level Flow Control
    

    View Slide

75. Embracing Streams
    Streams as a building block.
    

    View Slide

76. View Slide

77. View Slide

78. View Slide

79. View Slide

80. View Slide

81. View Slide

82. View Slide

83. View Slide

84. Typical Streams Existing as polling sources in all
    websites.
    

    View Slide

85. Atypical Streams Static lists of zero or one items.
    

    View Slide

86. Big Deal? Sure, but what benefit would it
    have?
    

    View Slide

87. Varying form factors. Different screen sizes means
    different viewing area.
    

    View Slide

88. View Slide

89. Common entity
    Flow controlled stream User (device) controlled stream is a natural
    way to paginate, hence, reducing network
    data transfer.
    

    View Slide

90. Streams imply State
    

    View Slide

91. Streams imply State
    State over a connection.
    

    View Slide

92. Full duplex protocol Ephemeral state management
    provides newer avenues.
    

    View Slide

93. Eliminate repetitive information
    exchange.
    a.k.a HTTP Cookies
    

    View Slide

94. curl -v "http://www.netflix.com"
    

    View Slide

95. curl -v "http://www.netflix.com"\
    > GET / HTTP/1.1\
    > \
    < HTTP/1.1 302 Found\
    < Date: Tue, 04 Oct 2016 02:37:51 GMT\
    < set-cookie: nflx-rgn=uw2|
    1475548671570; Max-Age=-1; Expires=Tue,
    04 Oct 2016 02:37:50 GMT; Path=/;
    Domain=.netflix.com\
    < Connection: keep-alive\
    < \
    

    View Slide

96. curl -v "http://www.netflix.com"\
    > GET / HTTP/1.1\
    > Cookie:nflx-rgn=uw2|1475548671570
    > \
    

    View Slide

97. Stateless protocols exchange
    repetitive information.
    

    View Slide

98. Stateful protocols may exchange
    information once per connection.
    

    View Slide

99. Handshake
    user_id=1
    

    View Slide

100. Handshake
     user_id=1
     user_id=1
     geo=cal
     .
     .
     

     View Slide

101. Requests
     user_id=1
     geo=cal
     .
     .
     getMovie?movie?id=1
     

     View Slide

102. Requests
     user_id=1
     geo=cal
     .
     .
     getMovie?movie?id=2
     getMovie?movie?id=1
     

     View Slide

103. Request Caching
     

     View Slide

104. Request Caching
     HTTP ETags
     

     View Slide

105. Request Caching
     HTTP ETags
     Hystrix* Request caching
     * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching
     

     View Slide

106. Request Caching
     HTTP ETags
     Hystrix* Request caching
     * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching
     }Time based caches.
     

     View Slide

107. Request Caching
     * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching
     Time based caches. Expiration?
     

     View Slide

108. Request Caching
     * https://github.com/Netflix/Hystrix/wiki/How-To-Use#Caching
     Time based caches. Expiration?
     

     View Slide

109. Live Request Cache
     

     View Slide

110. Cache Streams
     getBookmark?id=1
     

     View Slide

111. Cache Streams
     getBookmark?id=1
     position=x
     

     View Slide

112. Cache Streams
     getBookmark?id=1
     position=x
     

     View Slide

113. Cache Streams
     getBookmark?id=1
     position=x
     position=y
     

     View Slide

114. Cache Streams
     getBookmark?id=1
     position=y
     

     View Slide

115. Cache Streams
     Time based caches.
     Expiration
     Server push
     X
     

     View Slide

116. Request
     Cancellations Eliminate unnecessary work.
     

     View Slide

117. Edge Service
     Video Metadata Service
     Rating service C* store
     C* store
     /movie?id=123
     Disclaimer: This is an example and not an exact representation of the processing
     

     View Slide

118. Edge Service
     Video Metadata Service
     Rating service C* store
     C* store
     /movie?id=123
     X
     Disclaimer: This is an example and not an exact representation of the processing
     

     View Slide

119. Edge Service
     Video Metadata Service
     Rating service C* store
     C* store
     /movie?id=123
     X
     X
     X
     X
     X
     Disclaimer: This is an example and not an exact representation of the processing
     

     View Slide

120. These are usually built as special
     features!
     

     View Slide

121. Resiliency How does this model help to
     increase resiliency?
     

     View Slide

122. Resiliency
     Managing server overload.
     

     View Slide

123. Typical throttling
     Video Metadata Service
     Disclaimer: This is an example and not an exact representation of the processing
     GET /movie?id=3 HTTP/1.1
     HTTP/1.1 200 OK
     Edge Service
     

     View Slide

124. Typical throttling
     Disclaimer: This is an example and not an exact representation of the processing
     GET /movie?id=3 HTTP/1.1
     HTTP/1.1 200 OK
     Edge Service
     HTTP/1.1 503 Service Unavailable
     

     View Slide

125. Typical throttling
     Disclaimer: This is an example and not an exact representation of the processing
     GET /movie?id=3 HTTP/1.1
     HTTP/1.1 200 OK
     Edge Service
     HTTP/1.1 503 Service Unavailable
     Backoff
     

     View Slide

126. Typical throttling
     Disclaimer: This is an example and not an exact representation of the processing
     GET /movie?id=3 HTTP/1.1
     HTTP/1.1 200 OK
     Edge Service
     HTTP/1.1 503 Service Unavailable
     Backoff When will this
     server be ok?
     

     View Slide

127. Eliminating guesswork?
     Disclaimer: This is an example and not an exact representation of the processing
     GET /movie?id=3 HTTP/1.1
     HTTP/1.1 200 OK
     Edge Service
     HTTP/1.1 503 Service Unavailable
     Backoff
     When will this
     server be ok?
     

     View Slide

128. Request-leasing
     http://reactivesocket.io/
     

     View Slide

129. Request-leasing
     Peer 1
     Peer 2
     Network connection
     

     View Slide

130. Request-leasing
     Peer 1
     Peer 2
     “Lease” 5 requests for 1 second.
     Network connection
     

     View Slide

131. Request-leasing
     Peer 1
     Peer 2
     “Lease” 5 requests for 1 second.
     GET /movie?id=1 HTTP/1.1
     GET /movie?id=2 HTTP/1.1
     Network connection
     

     View Slide

132. Server
     Client 1 Client 2 Client 8
     

     View Slide

133. Server
     Capacity: 100 RPS
     Client 1 Client 2 Client 8
     “Lease” 10 requests for 1 second.
     

     View Slide

134. Server
     Capacity: 100 RPS
     Client 1 Client 2 Client 8
     “Lease” 10 requests for 1 second.
     “Lease” 10 requests for 1 second.
     “Lease” 10 requests for 1 second.
     

     View Slide

135. Server
     Capacity: 100 RPS
     Client 1 Client 2 Client 8
     “Lease” 10 requests for 1 second.
     “Lease” 10 requests for 1 second.
     “Lease” 10 requests for 1 second.
     Reserve Capacity:
     20 RPS
     

     View Slide

136. Time bound lease.
     “Lease” 10 requests for 1 second.
     

     View Slide

137. Time bound lease.
     No extra work for cancelling leases.
     “Lease” 10 requests for 1 second.
     

     View Slide

138. Time bound lease.
     No extra work for cancelling leases.
     Receiver controls the flow of requests
     “Lease” 10 requests for 1 second.
     

     View Slide

139. When things go south
     

     View Slide

140. Server
     Capacity: 20 RPS
     Client 1 Client 2 Client 8
     “Lease” 5 requests for 1 second.
     “Lease” 2 requests for 1 second.
     X No more “Lease”
     

     View Slide

141. Server
     Capacity: 20 RPS
     Client 1 Client 2 Client 8
     “Lease” 5 requests for 1 second.
     “Lease” 2 requests for 1 second.
     X No more “Lease”
     Prioritization
     

     View Slide

142. Server
     Capacity: 100 RPS
     Client 1 Client 2 Client 8
     “Lease” 10 requests for 1 second.
     “Lease” 10 requests for 1 second.
     “Lease” 10 requests for 1 second.
     Reserve Capacity:
     20 RPS
     

     View Slide

143. Server
     Capacity: 100 RPS
     Client 1 Client 2 Client 8
     “Lease” 10 requests for 1 second.
     “Lease” 10 requests for 1 second.
     “Lease” 10 requests for 1 second.
     Reserve Capacity:
     20 RPS
     Self Healing!
     

     View Slide

144. Self Healing!
     Under provisioning => Catastrophic Failures
     /
     

     View Slide

145. Self Healing!
     Under provisioning => Catastrophic Failures
     /
     Graceful degradation
     

     View Slide

146. Edge Service
     Video Metadata Service
     Rating service C* store
     C* store
     Disclaimer: This is an example and not an exact representation of the processing
     /movie?id=123
     

     View Slide

147. Edge Service
     Video Metadata Service
     Rating service C* store
     C* store
     Disclaimer: This is an example and not an exact representation of the processing
     /movie?id=123 Publisher
     

     View Slide

148. Request processing as a stream!
     Publisher
     

     View Slide

149. Powerful
     Abstractions
     Generic processing models provide
     features naturally, hence, reducing feature
     clutter in the applications.
     

     View Slide

150. public interface Publisher {
     public void subscribe(Subscriber super T> s);
     }
     public interface Subscriber {
     public void onSubscribe(Subscription s);
     public void onNext(T t);
     public void onError(Throwable t);
     public void onComplete();
     }
     public interface Subscription {
     public void request(long n);
     public void cancel();
     }
     Stream
     s
     

     View Slide

151. Nitesh Kant, Netflix Edge Engineering @NiteshKant
     https://speakerdeck.com/niteshkant
     

     View Slide