An introduction to fibers

Transcript

1. Fibers
2. An
 introduction
 to Denis Defreyne
 Ruby Wine
 April 13th, 2019 Fibers
3. 3 1.
4. 3 1. 2.
5. Basics 4
6. 5 f = lambda do puts "giraffe" end f.call
7. 5 f = lambda do puts "giraffe" end f.call giraffe
8. 6 f = Fiber.new do puts "giraffe" end f.resume
9. 6 f = Fiber.new do puts "giraffe" end f.resume giraffe
10. Photo by Harshil Gudka on Unsplash
11. 8 f = lambda do puts "zebra" end f.call zebra
12. 9 f = lambda do puts "zebra" return puts "invisible zebra" end f.call
13. 9 f = lambda do puts "zebra" return puts "invisible zebra" end f.call zebra
14. 10 f = Fiber.new do puts "zebra" return puts "invisible zebra" end f.resume
15. 10 f = Fiber.new do puts "zebra" return puts "invisible zebra" end f.resume zebra
16. 10 f = Fiber.new do puts "zebra" return puts "invisible zebra" end f.resume zebra sample.rb:3:
 in `block in <main>':
 unexpected return
 (LocalJumpError)
17. 11 f = Fiber.new do puts "zebra" Fiber.yield puts "invisible zebra" end f.resume
18. 11 f = Fiber.new do puts "zebra" Fiber.yield puts "invisible zebra" end f.resume zebra
19. Photo by Arleen wiese on Unsplash
20. 13 f = lambda do puts "donkey" end f.call
 donkey
21. 14 f = lambda do puts "donkey" end f.call f.call
22. 14 f = lambda do puts "donkey" end f.call f.call donkey donkey
23. 15 f = Fiber.new do puts "donkey" end f.resume f.resume
24. 15 f = Fiber.new do puts "donkey" end f.resume f.resume donkey
25. 15 f = Fiber.new do puts "donkey" end f.resume f.resume donkey a.rb:6:
 in `resume':
 dead fiber called
 (FiberError)
26. Photo by Spencer Watson on Unsplash
27. 17 f = lambda do puts "kangaroo" return puts "wallaby" end f.call
 kangaroo

28. 18 f = lambda do puts "kangaroo" return puts "wallaby" end f.call f.call
29. 18 f = lambda do puts "kangaroo" return puts "wallaby" end f.call f.call kangaroo kangaroo
30. 19 f = Fiber.new do puts "kangaroo" Fiber.yield puts "wallaby" end f.resume f.resume
31. 19 f = Fiber.new do puts "kangaroo" Fiber.yield puts "wallaby" end f.resume f.resume kangaroo wallaby
32. Photo by Austin Elder on Unsplash
33. Image by sandid from Pixabay
34. 22 f = Fiber.new { … } Create a fiber
35. 22 f = Fiber.new { … } Create a fiber f.resume Start or resume a fiber
36. 22 f = Fiber.new { … } Create a fiber f.resume Start or resume a fiber Fiber.yield Return from a fiber
37. 23 f = Fiber.new do puts 'quokka' end f.resume quokka
38. 24 f = Fiber.new do |a| puts a end f.resume('quokka')
39. 24 f = Fiber.new do |a| puts a end f.resume('quokka') quokka
40. Image by Tracey Wong from Pixabay
41. 26 f = Fiber.new do |a| puts a
 
 end f.resume('samoyed')
 samoyed
42. 27 f = Fiber.new do |a| puts a Fiber.yield puts 'arctic fox' end f.resume('samoyed') f.resume samoyed
 arctic fox
43. 28 f = Fiber.new do |a| puts a b = Fiber.yield puts b end f.resume('samoyed') f.resume('arctic fox')
44. 28 f = Fiber.new do |a| puts a b = Fiber.yield puts b end f.resume('samoyed') f.resume('arctic fox') samoyed arctic fox
45. Image by coolcoolleah from Pixabay
46. Photo by Jonatan Pie on Unsplash
47. 31 f = Fiber.new do puts 'red panda' end f.resume red panda
48. 32 f = Fiber.new do 'red panda' end puts f.resume
49. 32 f = Fiber.new do 'red panda' end puts f.resume red panda
50. Photo by Michael Payne on Unsplash
51. 34 f = Fiber.new do puts 'maine coon' end f.resume maine coon

52. 35 f = Fiber.new do puts 'maine coon' Fiber.yield 'raccoon' end f.resume puts f.resume maine coon
 raccoon
53. 36 f = Fiber.new do
 Fiber.yield('maine coon')
 'raccoon' end puts f.resume puts f.resume
54. 36 f = Fiber.new do
 Fiber.yield('maine coon')
 'raccoon' end puts f.resume puts f.resume maine coon raccoon
55. Photo by Bee Felten-Leidel on Unsplash
56. Photo by Gary Bendig on Unsplash
57. 39 f = Fiber.new { |…| … } Create a fiber
58. 39 f = Fiber.new { |…| … } Create a fiber r = f.resume(…) Start or resume a fiber
59. 39 f = Fiber.new { |…| … } Create a fiber r = f.resume(…) Start or resume a fiber a = Fiber.yield(…) Return from a fiber
60. Three Practical Applications 40
61. 1. Generators 41
62. 42 [7, 2, 4].each do |num| puts num end 7 2 4
63. 43 class PeopleDirectory end
64. 44 class PeopleDirectory RADIA = 'Dr. Radia Perlman' KATIE = 'Dr. Katie Bouman' ANITA = 'Dr. Anita Borg' end
65. 45 class PeopleDirectory RADIA = 'Dr. Radia Perlman' KATIE = 'Dr. Katie Bouman' ANITA = 'Dr. Anita Borg' def each end end
66. 46 class PeopleDirectory RADIA = 'Dr. Radia Perlman' KATIE = 'Dr. Katie Bouman' ANITA = 'Dr. Anita Borg' def each yield RADIA yield KATIE yield ANITA end end
67. 47 class PeopleDirectory RADIA = 'Dr. Radia Perlman' KATIE = 'Dr. Katie Bouman' ANITA = 'Dr. Anita Borg' def each yield RADIA yield KATIE yield ANITA end end PeopleDirectory.new
68. 48 class PeopleDirectory RADIA = 'Dr. Radia Perlman' KATIE = 'Dr. Katie Bouman' ANITA = 'Dr. Anita Borg' def each yield RADIA yield KATIE yield ANITA end end PeopleDirectory.new.each do |name| puts name end
69. 49 class PeopleDirectory RADIA = 'Dr. Radia Perlman' KATIE = 'Dr. Katie Bouman' ANITA = 'Dr. Anita Borg' def each yield RADIA yield KATIE yield ANITA end end PeopleDirectory.new.each do |name| puts name end Dr. Radia Perlman Dr. Katie Bouman Dr. Anita Borg
70. 50 class LinearCongruentialGenerator 
 
 

71. 51 class LinearCongruentialGenerator M = 0x7FFFFFF A = 1103515245 C = 12345
72. 52 class LinearCongruentialGenerator M = 0x7FFFFFF A = 1103515245 C = 12345 def initialize(seed) @value = seed end
73. 53 class LinearCongruentialGenerator M = 0x7FFFFFF A = 1103515245 C = 12345 def initialize(seed) @value = seed end def next @value = (A * @value + C) % M end end
74. 54 class LinearCongruentialGenerator M = 0x7FFFFFF A = 1103515245 C = 12345 def initialize(seed) @value = seed end def next @value = (A * @value + C) % M end end rng = LinearCongruentialGenerator.new(123)
75. 55 class LinearCongruentialGenerator M = 0x7FFFFFF A = 1103515245 C = 12345 def initialize(seed) @value = seed end def next @value = (A * @value + C) % M end end rng = LinearCongruentialGenerator.new(123) p rng.next p rng.next p rng.next
76. 56 class LinearCongruentialGenerator M = 0x7FFFFFF A = 1103515245 C = 12345 def initialize(seed) @value = seed end def next @value = (A * @value + C) % M end end rng = LinearCongruentialGenerator.new(123) p rng.next p rng.next p rng.next 440917719 357222964 2107447239
77. 57 class LinearCongruentialGenerator # … 
 
 def each end end
 
 

78. 58 class LinearCongruentialGenerator # … 
 
 def each loop do end end end
 
 

79. 59 class LinearCongruentialGenerator # … 
 
 def each loop do self.next end end end
 
 

80. 60 class LinearCongruentialGenerator # … 
 
 def each loop do self.next yield(@value) end end end
 
 

81. 61 class LinearCongruentialGenerator # … include Enumerable def each loop do self.next yield(@value) end end end
 
 

82. 62 class LinearCongruentialGenerator # … include Enumerable def each loop do self.next yield(@value) end end end
 
 rng = LinearCongruentialGenerator.new(123)

83. 63 class LinearCongruentialGenerator # … include Enumerable def each loop do self.next yield(@value) end end end
 
 rng = LinearCongruentialGenerator.new(123)
 p rng.take(3)
84. 64 class LinearCongruentialGenerator # … include Enumerable def each loop do self.next yield(@value) end end end
 
 rng = LinearCongruentialGenerator.new(123)
 p rng.take(3) [440917719, 357222964, 2107447239]
85. 65 M = 0x7FFFFFFF A = 1103515245 C = 12345 
 
 
 
 

86. 66 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) end
 
 
 
 

87. 67 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) Fiber.new do end end
 
 
 
 

88. 68 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) Fiber.new do loop do end end end
 
 
 
 

89. 69 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) Fiber.new do loop do value = (A * value + C) % M end end end
 
 
 
 

90. 70 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) Fiber.new do loop do value = (A * value + C) % M Fiber.yield(value) end end end
 
 
 
 

91. 71 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) Fiber.new do loop do value = (A * value + C) % M Fiber.yield(value) end end end
 
 rng = make_rng(123) 
 

92. 72 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) Fiber.new do loop do value = (A * value + C) % M Fiber.yield(value) end end end
 
 rng = make_rng(123) p rng.resume
 p rng.resume
 p rng.resume
93. 73 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) Fiber.new do loop do value = (A * value + C) % M Fiber.yield(value) end end end
 
 rng = make_rng(123) p rng.resume
 p rng.resume
 p rng.resume 440917719 357222964 2107447239
94. 74 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) Enumerator.new do |enum| loop do value = (A * value + C) % M enum.yield(value) end end end
 
 rng = make_rng(123) p rng.next
 p rng.next
 p rng.next 440917719 357222964 2107447239
95. 75 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) Enumerator.new do |enum| loop do value = (A * value + C) % M enum.yield(value) end end end
 
 rng = make_rng(123) p rng.next
 p rng.next
 p rng.next 440917719 357222964 2107447239
96. 76 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) Enumerator.new do |enum| loop do value = (A * value + C) % M enum.yield(value) end end end
 
 rng = make_rng(123) p rng.take(2)
 

97. 77 M = 0x7FFFFFFF A = 1103515245 C = 12345 def make_rng(value) Enumerator.new do |enum| loop do value = (A * value + C) % M enum.yield(value) end end end
 
 rng = make_rng(123) p rng.take(2)
 
 [440917719, 357222964]
98. 1. Generators ✓ 78
99. 2. Resumable computation 79
101. 81 def compile end
102. 82 def compile queue = PriorityQueue.new(pages) end
103. 83 def compile queue = PriorityQueue.new(pages) until queue.empty? end end
104. 84 def compile queue = PriorityQueue.new(pages) until queue.empty? page = queue.next end end
105. 85 def compile queue = PriorityQueue.new(pages) until queue.empty? page = queue.next compile_page(page) end end
109. 87 def compiled_content_of(page) @compiled_content[page] end

110. 88 def compiled_content_of(page) unless @compiled_content.key?(page) raise MissingDependencyError.new(page) end @compiled_content[page] end
111. 89 def compile queue = PriorityQueue.new(pages) until queue.empty? page = queue.next
 
 compile_page(page)
 
 
 
 end end
112. 90 def compile queue = PriorityQueue.new(pages) until queue.empty? page = queue.next begin compile_page(page) rescue MissingDependencyError => error end end end
113. 91 def compile queue = PriorityQueue.new(pages) until queue.empty? page = queue.next begin compile_page(page) rescue MissingDependencyError => error queue.add(page) end end end
114. 92 def compile queue = PriorityQueue.new(pages) until queue.empty? page = queue.next begin compile_page(page) rescue MissingDependencyError => error queue.add(page) queue.prioritize(error.missing_page) end end end
129. 95 def fiber_for(page) @fibers[page] ||= Fiber.new { compile_page(page) } end
130. 96 def compile queue = PriorityQueue.new(pages) until queue.empty? page = queue.next fiber_for(page).resume end end
131. 97 def compiled_content_of(page) unless @compiled_content.key?(page) Fiber.yield(MissingDependency.new(page)) end @compiled_content[page] end
132. 98 def compile queue = PriorityQueue.new(pages) until queue.empty? page = queue.next result = fiber_for(page).resume end end
133. 99 def compile queue = PriorityQueue.new(pages) until queue.empty? page = queue.next result = fiber_for(page).resume if result.is_a?(MissingDependency) end end end
134. 100 def compile queue = PriorityQueue.new(pages) until queue.empty? page = queue.next result = fiber_for(page).resume if result.is_a?(MissingDependency) queue.add(page) end end end
135. 101 def compile queue = PriorityQueue.new(pages) until queue.empty? page = queue.next result = fiber_for(page).resume if result.is_a?(MissingDependency) queue.add(page) queue.prioritize(result.missing_page) end end end
136. 102 def compiled_content_of(page) unless @compiled_content.key?(page) Fiber.yield(MissingDependency.new(page)) end @compiled_content[page] end
137. 2. Resumable computation ✓ 103
138. 3. Non-blocking I/O 104
140. 106 urls = [ 'https://denis.ws', 'https://denis.ws/cv', 'https://denis.ws/software', 'https://denis.ws/talks', 'https://denis.ws/toolbox', 'https://nanoc.ws', … ]
141. 107 Version 1 (sequential)
142. 108 urls.each do |url| end
143. 109 urls.each do |url| response = Net::HTTP.get_response(URI.parse(url)) end
144. 110 urls.each do |url| response = Net::HTTP.get_response(URI.parse(url)) if response.code != "200" end end
145. 111 urls.each do |url| response = Net::HTTP.get_response(URI.parse(url)) if response.code != "200" puts "Bad URL: #{url}" end end
146. 112 Version 2 (threaded)
147. 113 urls.each do |url| response = Net::HTTP.get_response(URI.parse(url)) if response.code != 200 puts "Bad URL: #{url}" end end
148. 114 urls.each do |url| Thread.new do response = Net::HTTP.get_response(URI.parse(url)) if response.code != 200 puts "Bad URL: #{url}" end end end
149. 115 threads = urls.map do |url| Thread.new do response = Net::HTTP.get_response(URI.parse(url)) if response.code != 200 puts "Bad URL: #{url}" end end end
150. 116 threads = urls.map do |url| Thread.new do response = Net::HTTP.get_response(URI.parse(url)) if response.code != 200 puts "Bad URL: #{url}" end end end threads.each(&:join)
151. 117 Version 3 (non-blocking IO)
152. This is a low-level implementation. 118 CAUTION I
153. Sockets must never block. 119 CAUTION II
154. 120 make_request_fiber('denis.ws', '/cv/').resume
155. 121 def make_request_fiber(host, path) end make_request_fiber('denis.ws', '/cv/').resume
156. 122 def make_request_fiber(host, path) Fiber.new do end end make_request_fiber('denis.ws', '/cv/').resume
157. 123 def make_request_fiber(host, path) Fiber.new do socket = NonBlockingSSLSocket.new(host, 443) end end make_request_fiber('denis.ws', '/cv/').resume
158. 124 def make_request_fiber(host, path) Fiber.new do socket = NonBlockingSSLSocket.new(host, 443) connect_tcp(socket) end end make_request_fiber('denis.ws', '/cv/').resume
159. 125 def make_request_fiber(host, path) Fiber.new do socket = NonBlockingSSLSocket.new(host, 443) connect_tcp(socket) connect_ssl(socket) end end make_request_fiber('denis.ws', '/cv/').resume
160. 126 def make_request_fiber(host, path) Fiber.new do socket = NonBlockingSSLSocket.new(host, 443) connect_tcp(socket) connect_ssl(socket) write(socket, build_http_request(host, path)) end end make_request_fiber('denis.ws', '/cv/').resume
161. 127 def make_request_fiber(host, path) Fiber.new do socket = NonBlockingSSLSocket.new(host, 443) connect_tcp(socket) connect_ssl(socket) write(socket, build_http_request(host, path)) puts read(socket, 1024) end end make_request_fiber('denis.ws', '/cv/').resume
162. 128 def make_request_fiber(host, path) Fiber.new do socket = NonBlockingSSLSocket.new(host, 443) connect_tcp(socket) connect_ssl(socket) write(socket, build_http_request(host, path)) puts read(socket, 1024) end end make_request_fiber('denis.ws', '/cv/').resume
163. 129 def connect_tcp(socket) end
164. 130 def connect_tcp(socket) socket.connect_tcp end
165. 131 def connect_tcp(socket) case socket.connect_tcp when :wait_readable when :wait_writable else end end
166. 132 def connect_tcp(socket) case socket.connect_tcp when :wait_readable Fiber.yield when :wait_writable else end end
167. 133 def connect_tcp(socket) case socket.connect_tcp when :wait_readable Fiber.current Fiber.yield when :wait_writable else end end
168. 134 def connect_tcp(socket) case socket.connect_tcp when :wait_readable $awaiting_readable[socket] = Fiber.current Fiber.yield when :wait_writable else end end
169. 135 make_request_fiber('denis.ws', '/cv/').resume
170. 136 make_request_fiber('denis.ws', '/cv/').resume # … fiber got suspended; now what?
171. 137 IO.select( )
172. 138 IO.select( $awaiting_readable.keys, $awaiting_writable.keys, )
173. 139 readable, writable = IO.select( $awaiting_readable.keys, $awaiting_writable.keys, )
174. 140 readable, writable = IO.select( $awaiting_readable.keys, $awaiting_writable.keys, ) readable.each do |socket| end
175. 141 readable, writable = IO.select( $awaiting_readable.keys, $awaiting_writable.keys, ) readable.each do |socket| fiber = $awaiting_readable.fetch(socket) end
176. 142 readable, writable = IO.select( $awaiting_readable.keys, $awaiting_writable.keys, ) readable.each do |socket| fiber = $awaiting_readable.fetch(socket) fiber.resume end
177. 143 readable, writable = IO.select( $awaiting_readable.keys, $awaiting_writable.keys, ) readable.each do |socket| fiber = $awaiting_readable.fetch(socket) fiber.resume end writable.each do |socket| fiber = $awaiting_writable.fetch(socket) fiber.resume end
178. 144 loop do readable, writable = IO.select( $awaiting_readable.keys, $awaiting_writable.keys, ) readable.each do |socket| fiber = $awaiting_readable.fetch(socket) fiber.resume end writable.each do |socket| fiber = $awaiting_writable.fetch(socket) fiber.resume end end
179. 145 def connect_tcp(socket) case socket.connect_tcp when :wait_readable $awaiting_readable[socket] = Fiber.current Fiber.yield when :wait_writable else end end
180. 146 def connect_tcp(socket) case socket.connect_tcp when :wait_readable $awaiting_readable[socket] = Fiber.current Fiber.yield $awaiting_readable.delete(socket) when :wait_writable else end end
181. 147 def connect_tcp(socket) case socket.connect_tcp when :wait_readable $awaiting_readable[socket] = Fiber.current Fiber.yield $awaiting_readable.delete(socket) when :wait_writable Fiber.yield else end end
182. 148 def connect_tcp(socket) case socket.connect_tcp when :wait_readable $awaiting_readable[socket] = Fiber.current Fiber.yield $awaiting_readable.delete(socket) when :wait_writable $awaiting_writable[socket] = Fiber.current Fiber.yield else end end
183. 149 def connect_tcp(socket) case socket.connect_tcp when :wait_readable $awaiting_readable[socket] = Fiber.current Fiber.yield $awaiting_readable.delete(socket) when :wait_writable $awaiting_writable[socket] = Fiber.current Fiber.yield $awaiting_writable.delete(socket) else end end
184. 150 def connect_tcp(socket) loop do case socket.connect_tcp when :wait_readable $awaiting_readable[socket] = Fiber.current Fiber.yield $awaiting_readable.delete(socket) when :wait_writable $awaiting_writable[socket] = Fiber.current Fiber.yield $awaiting_writable.delete(socket) else end end end
185. 151 def connect_tcp(socket) loop do case socket.connect_tcp when :wait_readable $awaiting_readable[socket] = Fiber.current Fiber.yield $awaiting_readable.delete(socket) when :wait_writable $awaiting_writable[socket] = Fiber.current Fiber.yield $awaiting_writable.delete(socket) else break end end end
186. 152 def make_request_fiber(host, path) Fiber.new do socket = NonBlockingSSLSocket.new(host, 443) connect_tcp(socket) connect_ssl(socket) write(socket, build_http_request(host, path)) puts read(socket, 1024) end end make_request_fiber('denis.ws', '/cv/').resume
187. 153 def make_request_fiber(host, path) Fiber.new do socket = NonBlockingSSLSocket.new(host, 443) connect_tcp(socket) connect_ssl(socket) write(socket, build_http_request(host, path)) puts read(socket, 1024) end end make_request_fiber('denis.ws', '/cv/').resume
188. 154 def connect_ssl(socket) end
189. 155 def connect_ssl(socket) socket.connect_ssl end
190. 156 def connect_ssl(socket) case socket.connect_ssl end
191. 157 def connect_ssl(socket) case socket.connect_ssl when :wait_readable when :wait_writable else end end
192. 157 def connect_ssl(socket) case socket.connect_ssl when :wait_readable when :wait_writable else end end
193. 158 require 'async' require 'async/http'
194. 159 require 'async' require 'async/http' endpoint = Async::HTTP::URLEndpoint.parse('https://denis.ws')
195. 160 require 'async' require 'async/http' endpoint = Async::HTTP::URLEndpoint.parse('https://denis.ws') client = Async::HTTP::Client.new(endpoint)
196. 161 require 'async' require 'async/http' endpoint = Async::HTTP::URLEndpoint.parse('https://denis.ws') client = Async::HTTP::Client.new(endpoint) Async do |task| end
197. 162 require 'async' require 'async/http' endpoint = Async::HTTP::URLEndpoint.parse('https://denis.ws') client = Async::HTTP::Client.new(endpoint) Async do |task| task.async { p client.get("/").status } task.async { p client.get("/cv/").status } task.async { p client.get("/software/").status } task.async { p client.get("/talks/").status } task.async { p client.get("/toolbox/").status } end
198. 162 require 'async' require 'async/http' endpoint = Async::HTTP::URLEndpoint.parse('https://denis.ws') client = Async::HTTP::Client.new(endpoint) Async do |task| task.async { p client.get("/").status } task.async { p client.get("/cv/").status } task.async { p client.get("/software/").status } task.async { p client.get("/talks/").status } task.async { p client.get("/toolbox/").status } end https://github.com/socketry/async
199. 3. Non-blocking I/O ✓ 163
200. 164 1.
201. 164 1. 2.
202. 164 1. 2. 3.
203. 165 f = Fiber.new { |…| … } Create a fiber
204. 165 f = Fiber.new { |…| … } Create a fiber r = f.resume(…) Start or resume a fiber
205. 165 f = Fiber.new { |…| … } Create a fiber r = f.resume(…) Start or resume a fiber a = Fiber.yield(…) Return from a fiber
206. Now you know everything about Fibers.
207. Q&A Find me at @ddfreyne on Twitter, or at denis.ws!