Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to write synchronization mechanisms for Fiber

How to write synchronization mechanisms for Fiber

RubyKaigi 2017

seki at druby.org

September 20, 2017
Tweet

More Decks by seki at druby.org

Other Decks in Programming

Transcript

  1. slide sponsors ְאַせ〢㾊3VCZ⠓陽ח⿫⸇׃׋ְ㖑⯋孖זךד!JXBEPO 䎢䃊ד׮044(BUF׃׋ְ!LUPV E3VCZח״׷ⴓ侔٥8FCفؚٗٓىؚٝ ؔ٦ي爡F#PPL4UPSF IUUQ FTUPSFPINTIBDPKQUJUMFT1!IJTBTIJN !N@TFLJךSF⿾䗁Ꟛ涪׾⿾䗁㷕统׃זָ׵⿾䗁Ꟛ涪חزٓ؎׃גְת ׅ!BXB[FLJ

    ⚅歲剑㣐.BTUPEPOչ1BXPPպכ3VCZך⸂ד麊欽׃גתׅ!QJYJW 〴괏׾⛦׶馉ִ䎢䃊ח⛦׶鴥׿׌涺圫ծ➙䎃׮N@TFLJ眍׾ֶ嚂׃׫ֻ׌ׁ ְ剑鵚չ،صًך鋅׵׸倯պהְֲず➂钞׾⠓爡ד⡲׏גתׅךד׉ ׍׵׮״׹׃ֻדׅկ!UTVCPJ ה׍ֺךٖه٦زֶ嚂׃׫חTNBMSVCZֹֹ׋ַ׏׋!WFTUJHF@
  2. Process ↔ Thread ↔ Fiber 1SPDFTT ًٌٔכⰟ剣׃׋ְ׿׌ 5ISFBE ؝ٝذؗأزأ؎حثכ荈ⴓדװ׷ 'JCFS

    - Process -- Want to share memories! --- Thread ---- Do context switch by myself! ----- Fiber
  3. Before Process (90s) ⥂隊ׁ׸זְًٌٔ ⼿锃㘗وٕثةأؙ ظٝفٔؒٝفذ؍ـ ֿ׸'JCFSׄׯ׿ - no memory

    protection - Cooperative multitasking -- non preemptive -- Is this a ... Fiber?
  4. ࣮ࡍͷγεςϜ 醱侧ךفٗإأה׉׸׵ָⵃ欽ׅ׷项彁 ر٦ةד圓䧭ׁ׸׷ ꥫꨄׁ׸׋ًٌٔ瑞꟦׾馄ִ׷䗳銲ָ֮׷ فٗإأ׌ֽꥫꨄ׃ג׮穠㽷כ The Actual System - Assembled

    by multiple process and the resources that are used by them - Needs to jump out of the isolated memory space - Isolating processes is one thing, but for the others...
  5. ProcessͰॻ͘ͱ͖ͷؾ࣋ͪ קַךفٗإأהוֲחַ׃גⰟ剣׃״ֲ ׭׿וְֻֽׁו How you feel when you are using

    process - Try to share with other processes -- Even though it's a pain in the neck
  6. Thread ًٌٔ瑞꟦׾Ⱏ剣ׅ׷⸋⻉晛فٗإأ فٔؒٝفذ؍ـז؝ٝذؗأزأ؎حث ⸋⻉ה䒷ֹ䳔ִח鯪ꆀ 㢳ֻך➂חכفٗإأך倯ְְָה䙼ֲ degraded version of process that

    shares memory space - The system automatically switches to Thread - Bad copy, but lightweight To most people, processes are probably better
  7. ThreadͰॻ͘ͱ͖ͷؾ࣋ͪ קַךأٖحسךًٌٔ׾鍗׵זְ״ֲח ׮ךׅ׀ֻ孡׾אֽ׷ 孡׾䫙ֻה鍗׏גְ׷ 鍗׏ג׮㣐♱㣗זהֿ׹׌ֽ׾鍗׷ What it feels like to

    write Threads Try best to be very careful not to change memories of other threads - Tend to change it whenever you lose focus - Only change the places that are okay to change
  8. Threadʹඞཁͳၗຫ͞ չ⥯ֻ׵ְחז׷ה04ח걾׵זֻג׮㸜Ⰻ ח⸬桦葺ֻ剅ֽ׷״պ ה鎉ִ׷➂ך׋׭ך娀㐻 04ךؔ٦غ٦قحسָ孡חז׷➂פ Attitude when using Threads "Secure

    and efficient coding without OS? No sweat." - threads will be a great advantage for you - for coders who have concerns about OS overheads
  9. FiberͰॻ͘ͱ͖ͷؾ࣋ͪ ֲתְ؝ٝذؗأزأ؎حثך⽃⡘׾鋅א ֽ״ֲ ـٗحׇׁؙזְ״ֲחׅ׷חכ ׬׃׹'JCFS׾⢪ֲֿהָ湡涸 Attitude when using Threads "Secure

    and efficient coding without OS? No sweat." - threads will be a great advantage for you - for coders who have concerns about OS overheads
  10. Process ↔ Thread ↔ Fiber 1SPDFTT ًٌٔכⰟ剣׃׋ְ׿׌ 5ISFBE ؝ٝذؗأزأ؎حثכ荈ⴓדװ׷ 'JCFS

    - Process -- Want to share memory! --- Thread ---- Do context switch by myself! ----- Fiber
  11. Process ↔ Thread ↔ Fiber 'JCFS 荈⹛涸חأ؎حث׃ג״ 5ISFBE ًٌٔ⥂隊׃גֻ׸זְה㔭׷׿׌ 1SPDFTT

    ----- Fiber ---- Why won't it switch automatically? --- Thread -- Can't code if it doesn't protect memory! - Process
  12. ͳͥFiberͷ࿩Λ͢Δͷ͔ ٙةءع9ד(6*הTPDLFUفؚٗٓىؚٝ ָثّحزرٕؗ ךדծ荈ⴓד׮剅ְג׫׋ֻז׏׋ أٖحسד醱侧ךفٗإأה鸐⥋ׅ ׷׮ך I can socket program

    with x11 and GUI a little. - So, I decided that I'd like to use it myself -- Something that communicates with multiple processes with one Thread
  13. Α͋͘ΔFibonacci ٕ٦فד剅ֻה知⽃ [0, 1] [1, 1] [2, 2] [3, 3]

    [4, 5] [5, 8] [6, 13] [7, 21] [8, 34] [9, 55] Plain Fibonacci - Easy to write with Loop def fibonacci(n) a, b = 1, 1 n.times do |i| p [i, a] a, b = b, a + b end end fibonacci(10)
  14. FiberͰॻ͖௚͢ ֿ׸׾⢪׏ג"1*׾铡僇׃תׅ fib = Fiber.new do a, b = 1,

    1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end def fibonacci(n) a, b = 1, 1 n.times do |i| p [i, a] a, b = b, a + b end end fibonacci(10) Rewrite with Fiber - Explain API using this
  15. Fiber.new ⴽ'JCFSדװ׶׋ְⳢ椚׾ـٗحؙד床ׅ ת׌ـٗحؙך؝٦سח鹌׫תׇ׿ fib = Fiber.new do a, b =

    1, 1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end Fiber.new - Pass value with another Fiber to do a different job - The block code will not run yet.
  16. Fiber#resume 'JCFSח؝ٝذؗأز׾ⴖ׶剏ִ׷ fib = Fiber.new do a, b = 1,

    1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end Switch context to the Fiber
  17. Fiber.yield SFTVNF׃׋'JCFSחⴖ׶剏ִ׷ 䒷侧ָSFTVNFך⦼חז׷ fib = Fiber.new do a, b =

    1, 1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end Switch to fiber.resume The argument becomes the value of the fiber
  18. resume, again ✳䏝湡⟃꣬ךSFTVNFכZJFMEַ׵ⱄꟚׅ׷ fib = Fiber.new do a, b =

    1, 1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end resume, again From the second time on, resume restarts from yield
  19. resumeͷߦ͖ઌ 'JCFS׾OFX׃׋׌ֽדכ㹋遤ׁ׸זְ SFTVNFד'JCFSח؝ٝذؗأز׾獳ׅ ♧䏝湡כ⯓걧פ ✳䏝湡⟃꣬כ'JCFSZJFME׃׋הֿ׹פ ֿ׸כ⡦䏝湡ךSFTVNFַז Where resume jumps to

    Would not execute just by Fiber new Pass the value to Fiber by resume -- Back to the top on the first round -- Back to Fiber.yield from the second round on --- Which round am I on...?
  20. APIΛઆ໌ͨ͠ SFTVNFהZJFMEך嚊䙀ָה׏אֹחְֻ 荈歋ֺׅגוֲ׃גְְַ׻ַ׵זְ Explained the API -The concept of resume

    and yield is hard to grep -Don't know what to do, there are so many things we can do with it
  21. EnumeratorͰॻ͖௚͢ ׉׏ֻ׶ fib = Fiber.new do a, b = 1,

    1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end def fibonacci a, b = 1, 1 while true yield(a) a, b = b, a + b end end fib = to_enum(:fibonacci) 10.times do |n| p [n, fib.next] end Rewrite with Enumerator - Nearly identical
  22. Enumerator 単׾⡲׷ֿהד罋ִװֻׅז׷ ֶׅׅ׭ ❌ 'JCFSךꨇ׃ׁח䩧׍⯘אהְֲ䩛䘔ִ ָזְ Enumerator - Easier to

    understand when put in a frame - Recommended ❌ Can't really get the feel of overcoming the difficulties of Fiber
  23. QueueΛհͨ͠ڠௐ 2VFVF׾➜׃ג醱侧ך5ISFBEָ鸬䵿ׅ׷ ׉׸׊׸ך5ISFBEכֶ✼ְ׾濼׵זְ 欰欵罏ծ嶊顤罏כ醱侧ְג׮⹛ֻ q = SizedQueue.new(1) Thread.new do 10.times

    do |n| sleep(rand) q.push(n) p [:push, n] end end Thread.new do 10.times do |n| sleep(rand) p [:pop, q.pop] end end.join Sync with Queue - Multiple Threads synchronize through Queue - Each Thread does not know anything about other Threads - Works with multiple producers and consumers
  24. ͜ΕͷFiber൛ 3EW׾➜׃ג醱侧ך'JCFSָ鸬䵿ׅ׷ SFOEF[WPVT 䖉׍さ׻ׇ rdv = Rdv.new Fiber.new do 10.times

    do |n| rdv.push(n) p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume Fiber version - Multiple Fibers work together through Rdv - rendez-vous
  25. Rdvͷ࣮૷ JOJUJBMJ[F QVTI QPQ def initialize @reader = [] @writer

    = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end Implementation of Rdv
  26. ͱΓ͋͑ͣresume 'JCFSOFX׃׋׵ּׅSFTVNFׅ׷ 5ISFBEOFXך״ֲחּׅ⹛ֹ⳿ׅ 剑ⴱךSFTVNFך暴ⴽז⹛ֹ׾罋ִזֻ ג״ְ Just use resume for now

    - Resume right after Fiber.new -- Works right away, just like Thread.new --- Don't have to think about the special behavior on the first round
  27. ߦ͖٧·ͬͨΒyield װ׸׷ֿהָ֮׷꟦כדֹ׷׌ֽװ׷ װ׸׷ֿהָזֻז׏׋׵ZJFME 䖉א湱䩛ח㼎׃גծ荈ⴓ 'JCFSDVSSFOU ׾鋙ִג׮׵ֲ Yield when you're stuck

    - Do as much as it can while it can do it - Yield when it has nothing to do -- Let the partner know about yourself (alias fiber.current)
  28. initialize !SFBEFSכQPQ䖉׍ծ!XSJUFSכQVTI䖉׍ ך'JCFS׾ًٌׅ׷ def initialize @reader = [] @writer =

    [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @reader holds fibers waiting for pop @writer holds fibers waiting for push
  29. push !SFBEFSְָזְהֹծ荈ⴓ׾ًٌ׃ג䖉 א ZJFME  !SFBEFSְָ׷הֹծ饯ֿׅ SFTVNF def initialize @reader

    = [] @writer = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end - When there is no @reader, it records itself and wait (yield) - Resume when @reader is present (resume)
  30. @reader͕͍ͳ͍ ر٦ةה'JCFSDVSSFOU׾ًٌ׃גZJFME def initialize @reader = [] @writer = []

    end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end When @reader is not there - Records the data and Fiber.current, then yield
  31. @reader͕͍Δ !SFBEFS׾♧אSFTVNFׅ׷ QPQד䖉׏גְ׋'JCFSָⱄꟚׅ׷ def initialize @reader = [] @writer =

    [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @reader is present - Resume one @reader -- Fiber that was waiting for the pop resumes
  32. pop !XSJUFSְָזְהֹծ荈ⴓ׾ًٌ׃ג䖉 א ZJFME  !XSJUFSְָ׷הֹծ饯ֿׅ SFTVNF  ׉׃גر٦ة׾鵤ׅ def

    initialize @reader = [] @writer = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end When there is no @writer, it records itself and waits (yield) - When @writer is present, it resumes (resume) -- and returns data Skip!
  33. @writer͕͍ͳ͍ 'JCFSDVSSFOU׾ًٌ׃גZJFME def initialize @reader = [] @writer = []

    end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @writer is gone Record Fiber.current and yield Skip!
  34. @writer͕͍Δ !XSJUFS׾♧אSFTVNFׅ׷ QVTIד䖉׏גְ׋'JCFSָⱄꟚׅ׷ ַًٌ׵ر٦ة׾鵤ׅ def initialize @reader = [] @writer

    = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @writer is present - Resume one @writer -- a Fiber that was waiting for push resumes - Returns data value from the note Skip!
  35. ݕࠪதͷഉଞ੍ޚ 'JCFSדכ勴⟝嗚叨ך䱖➭ⵖ䖴ָ♶銲 5ISFBEדכ⽬ꤹ def initialize @reader = [] @writer =

    [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end Exclusive control while inspecting - With Fiber, there is no need for exclusive control - Very risky on Threads!!
  36. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume How it works. Fiber.new kara-matz is summoned! Fiber.newした
 kara-matzを召喚
  37. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume kara-matz is resumed It's kara-matz's turn! kara-matzをresume
 kara-matzのターン

  38. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume kara-matz does push() There are no opponents.. Record data and himself, then yield. kara-matzのpush!
 相手がいない。
 自分とデータをメモしyield。ターン終了
  39. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume Main's turn! Fiber.new ichi-matz is summoned! メインのターン
 Fiber.new
 ichi-matzを召喚
  40. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matz resumed It's ichi-matz's turn! ichi-matzをresume
 ichi-matzのターン

  41. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matz does pop! ichi-matz reads the memo takes data. kara-matz resumes the finished. ichi-matzのpop!
 ichi-matzはメモを読んだ
 データを手にいれた。kara-matzをresumeしターン終了
  42. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume kara-matz does push() There are no opponents. Record data and himself, then yield. kara-matzのpush!
 相手がいない。
 自分とデータをメモしyield。ターン終了
  43. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matzのpop!
 ichi-matzはメモを読んだ
 データを手にいれた。kara-matzをresumeしターン終了 ichi-matz does pop! ichi-matz reads the memo takes data. kara-matz resumes the finished. Skip!
  44. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matz dies. It's main's turn Main dies. ichi-matzはしんでしまった
 メインのターン
 メインはしんでしまった Skip!
  45. ͜͜ͷ·ͱΊ 'JCFSָず劍ׅ׷׋׭ך➬穈׫׾⡲׏׋ 猘ך㥨ֹז؎ر؍ؔي׾荈䢧׃׋ ה׶ִ֮׆SFTVNF 遤ֹ鑥ת׏׋׵ZJFME Wrapping up Made a structure

    of synchronizing fibers Proudly presented my favorite idiom - just use resume - yield when you're stuck
  46. ͜͜ͷ·ͱΊ ず劍ך勴⟝嗚叨ח䱖➭ⵖ䖴ְ׵זְ 农럀涸ח5ISFBEח㢌䳔ׁ׸׷ה㔭׷ 5ISFBE׾⢪׏גְ׋א׮׶ָ㹋כ'JCFS ׌׏׋ך倯ְְָה䙼ֲ Wrapping up We don't need

    exclusive control when sync-ing - Don't want it to be implicitly converted into Threads -- It's better off that it was actually Fiber when you thought you were using Threads
  47. ͢͢Ί͔ͨ E3VCZ؟٦غ٦ך毟⡂؝٦سד铡僇 稆劰׌ֽו⚛遤ח⹛ַזְ׮ך 5ISFBE׾欽ְג⚛遤ח⹛ֻ׮ך 5ISFBE晛׾'JCFS晛ח׃׋׮ך؎و؎ ث ׉ך何葺晛 Table of Contents:

    explain with dRuby server pseudo code - Coding is simple, but doesn't move in parallel - Works in parallel with Threads - Code above rewritten to use Fiber (bit awkward) -- Better version using Fiber
  48. ૉ๿͚ͩͲҰ్ ⽃秪ח♧湫简ח剅ְ׋׮ך *0כـٗحؙׅ׷ֽו孡ח׃זְ ؙٓ؎،ٝز׃ַ⹛ַזְ def main_loop loop do select fds.each

    do |fd| call_callback end end end def on_accept c = accept loop do read_request(c) do_it write_response(c) end end Simple but not parallel - Written straight forward - Ignore I/O blocks - Only works with one client def read_n(fd, sz) ... ... end
  49. Thread൛ GE׀הח5ISFBE׾饯⹛ׅ׷ ֿ׸׌ֽד⚛遤ח⹛ֻ def main_loop loop do select fds.each do

    |fd| call_callback end end end def on_accept c = accept Thread.new(c) do |fd| loop do read_request(fd) do_it write_response(fd) end end end Thread ver - Start a new thread on every fd - Add 2 columns and it works in parallel def read_n(fd, sz) ... ... end ௥Ճ͸͚ͩ͜͜ ௥Ճ͸͚ͩ͜͜
  50. Fiber൛ ظٝفٔؒٝفذ؍ـ ׉ךתתדכ⚛遤ח⹛ַזְ def main_loop loop do select fds.each do

    |fd| call_callback end end end def on_accept c = accept Fiber.new do loop do read_request(c) do_it write_response(c) end end.resume end Fiber ver - non-preemptive - Not quite parallel, yet def read_n(fd, sz) ... ... end Swapped with Fiber
  51. Fiber൛վ *0ָـٗحؙ׃׉ֲזהֹծֿ׏׉׶؝ٝ ذؗأز׾ⴖ׶剏ִ׷⡲䨌 ـٗحؙ괏ظٝـٗحؙ*0 def main_loop loop do select fds.each

    do |fd| call_callback end end end def on_accept c = accept Fiber.new do loop do read_request(c) do_it write_response(c) end end.resume end Fiber ver Revisioned - Switch context before it I/O blocks -- Blocking-style non-blocking I/O def read_n(fd, sz) ... fd.read_nonblock(sz) rescue WaitReadable Fiber.yield retry end
  52. Block-ish non-block I/O ـٗحؙ׃׉ֲזהֹծ'JCFSZJFME׃ג NBJO@MPPQח׮ו׷ ؝٦ٕغحؙדSFTVNFׅ׷ def main_loop loop do

    select fds.each do |fd| call_callback end end end def on_accept c = accept Fiber.new do loop do read_request(c) do_it write_response(c) end end.resume end It Fiber.yields back to main_loop before it I/O blocks Resumes on call back def read_n(fd, sz) ... fd.read_nonblock(sz) rescue WaitReadable Fiber.yield retry end
  53. Bartender::Context SFBEBCMF XSJUBCMFחז׏׋ֿהח孡בְ׋ ׵؝٦ٕغحؙ ֿ׸׾⢪׏גCMPDLJTIOPOCMPDL*0׾ Context#[]=(event, fd, callback) Πϕϯτछʢ:read, :writeʣͱfdͷ૊ʹίʔϧόοΫΛొ࿥

    Bartender::Context Callbacks when readable or writeable Use this to implement block-ish non-block I/O register callbacks for the combination of event types (:read, :write) and fd
  54. _read() SFBE@OPOCMPDLׅ׷կ⢽㢩ך㜥さכ SFBEBCMFחז׷ך׾䖉׏גַ׵SFUSZׅ׷ 䖉׏ג ׉ֿד䖉׏׍ׯتً ⴽך'JCFSח؝ٝذؗأزأ؎حثׅ׷ def _read(fd, sz) return

    fd.read_nonblock(sz) rescue IO::WaitReadable wait_readable(fd) retry end def wait_readable(fd); wait_io(:read, fd); end def wait_io(event, fd) self[event, fd] = Fiber.current.method(:resume) Fiber.yield ensure delete(event, fd) end - read_nonblock. On exception, it waits to be readable before it retries. - Waits to be what? -- We need to context switch to another fiber
  55. read_nonblock ּׅ铣׭׷ر٦ةָ֮׸ל铣׿ד鵤ׅ T[劢弫ך؛٦أ׮״ֻ֮׷ ּׅח铣׭׷ر٦ةָזְ㜥さחכ⢽㢩 *08BJU3FBEBCMF def _read(fd, sz) return fd.read_nonblock(sz)

    rescue IO::WaitReadable wait_readable(fd) retry end def wait_readable(fd); wait_io(:read, fd); end def wait_io(event, fd) self[event, fd] = Fiber.current.method(:resume) Fiber.yield ensure delete(event, fd) end If there are any readable data, it reads and returns - There are cases when it's below sz Excepts when there is no readable data - IO::WaitReadable
  56. wait_io() ؝٦ٕغحؙה׃גSFTVNF׾涫ꐮ ׉׃ג'JCFSZJFME 剑穄涸חً؎ٕٝ٦فח䨱׶TFMFDUׅ׷ SFTVNFׁ׸׋הֹSFBEBCMFחז׏ג׷ def _read(fd, sz) return fd.read_nonblock(sz)

    rescue IO::WaitReadable wait_readable(fd) retry end def wait_readable(fd); wait_io(:read, fd); end def wait_io(event, fd) self[event, fd] = Fiber.current.method(:resume) Fiber.yield ensure delete(event, fd) end Register resume as callback Fiber.yield - Goes back to mainloop and selects at the end readable when resumed
  57. tiny_drbͷൈਮ ًحإ٦آ铣׿ד穈׫甧ג׷鿇ⴓדׅ ـٗحؙ׾孡חׇ׆⽃秪זSFBE׾⚛ץג׷ ـٗحؙׅ׷㜥さחכ农럀涸ח؝ٝذؗأ ز׾أ؎حث׃תׅ def req_drb(reader) ref = load(reader,

    false) msg = load(reader) argc = load(reader) argv = argc.times.collect {load(reader)} block = load(reader, false) [ref, msg, argv] end def load(reader, marshal=true) sz = reader.read(4) sz = sz.unpack('N')[0] data = reader.read(sz) return data unless marshal begin Marshal.load(data) rescue DRb::DRbUnknown.new($!, data) end end extracted from tiny_drb - Where it constructs with the read messages - Just lining up reads, blocks are ignored - When blocked, it context switches implicitly.
  58. ੲΑ͘ݟͨόά TFMFDU ך֮הծًحإ٦آ׾铣׿ד穈׫甧 ג׷ꟼ侧׾؝٦ٕ 鹼䒀ծꥺ㹱ָ֮׷הةأؙأ؎حثדֹ ׆ח㔿ת׷ Bugs in the good

    Ol' days (in C) - Call functions that read & construct messages after select() -- Cannot switch tasks when there is latency and/or incident, and freezes
  59. Threadͱͷ଴ͪ߹Θͤ 5ISFBEWBMVFׅ׷ה姺ת׏ג׃תֲ *0QJQFךDMPTF׾⢪׏ג穄✪׾鸐濼ׅ׷ class ThreadTask def initialize(*args, &block) left, @right

    = IO.pipe @task = Thread.new do begin block.call(*args) ensure left.close end end end def value if @right Bartender.wait_readable(@right) @right.close end @task.value ensure @right = nil end end Sync Threads with FD - Stops on Thread.value - Notifies stop with IO.pipe close Skip!
  60. ࠓճͷྫ͸ 农鎸דֹ׷ֻ׵ְ㼭ְׁ➬穈׫׌׏׋ ➬穈׫חさ׻ׇג،فٔ؛٦ءّٝ׾剅 ֽ׷ 如ך״ֲז縐׾鼘ֽ׵׸׋ This example - Was pretty

    small, small enough to memorize it all -- Can code & adapt according to a particular application - Didn't get stuck in the following traps:
  61. ᠘ 'JCFSך⚥ד鹼ְⳢ椚׾׃גـٗحؙ ㄎ׿׌⯓ד5ISFBE⡲׏גְג'JCFS&SSPS ׌׸ַךHFNךזַדְֲֲֿֿהװ׏ג ׷ַ׮ Traps: - Get blocked inside

    Fiber by processing something slow in it - Read some place that used a thread and FiberError - Somebody might have already done this in gem somewhere
  62. ͭ·Γ ،فٔ؛٦ءّٝך⢪ֲٓ؎ـٓٔךׅ׫ ׆׫תד㹋鄲׾椚鍑׃גזְהعو׷ عو׷׏ג鋙ִגֶֽל㣐♱㣗 㹋כ'JCFSׄׯזֻג׮عو׷ The point is - that

    you need to understand everything about the library or you'll get stuck -- If you know that you'll get stuck, you're okay --- (Actually, it doesn't need to be Fiber to get stuck)
  63. ϋϚΔྫ 'JCFS晛ך5XJUUFSךة؎يٓ؎ٝך铣׫鴥׫ ⡲׹ֲ +40/QBSTFכأزٔ٦ي׾Ⰻ鿇铣׿ד 4USJOHח׃׍ׯֲ̔تً أزٔ٦يַ׵铣׬+40/QBSTFזְ Examples of getting stuck

    - Making a Fiber version of Twitter timeline reading -- JSON.parse reads and puts everything into a string →Won't work - Is there JSON.parse that reads from streams?
  64. ·ͱΊ 5ISFBEכꨇ׃ְָ'JCFSכ׮׏הꨇ׃ְ 'JCFSך"1*הず劍ծ*0ך䘔欽⢽׾爙׃׋ 剑鵚㥨ֹז؎ر؍ؔي׾稱➜׃׋ Summary - Threads are hard, but

    Fibers are even harder - Explained Fiber API, synchronization, I/O Multiplexer application - Introduced my favorite idiom
  65. Process ↔ Thread ↔ Fiber 'JCFS 荈⹛涸חأ؎حث׃ג״ 5ISFBE ًٌٔ⥂隊׃גֻ׸זְה㔭׷׿׌ 1SPDFTT

    ----- Fiber ---- Why won't it switch automatically? --- Thread -- Can't code if it doesn't protect memory! - Process