Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to write synchronization mechanisms for Fiber

How to write synchronization mechanisms for Fiber

RubyKaigi 2017

seki at druby.org

September 20, 2017
Tweet

More Decks by seki at druby.org

Other Decks in Programming

Transcript

  1. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ ͝ཡͷεϙϯαʔͷఏڙ ʹΑΓ͓ૹΓ͠·͢

  2. slide sponsors ְאַせ〢㾊3VCZ⠓陽ח⿫⸇׃׋ְ㖑⯋孖זךד!JXBEPO 䎢䃊ד׮044(BUF׃׋ְ!LUPV E3VCZח״׷ⴓ侔٥8FCفؚٗٓىؚٝ ؔ٦ي爡F#PPL4UPSF IUUQ FTUPSFPINTIBDPKQUJUMFT1!IJTBTIJN [email protected]ךSF⿾䗁Ꟛ涪׾⿾䗁㷕统׃זָ׵⿾䗁Ꟛ涪חزٓ؎׃גְת ׅ!BXB[FLJ

    ⚅歲剑㣐.BTUPEPOչ1BXPPպכ3VCZך⸂ד麊欽׃גתׅ!QJYJW 〴괏׾⛦׶馉ִ䎢䃊ח⛦׶鴥׿׌涺圫ծ➙䎃׮[email protected]眍׾ֶ嚂׃׫ֻ׌ׁ ְ剑鵚չ،صًך鋅׵׸倯պהְֲず➂钞׾⠓爡ד⡲׏גתׅךד׉ ׍׵׮״׹׃ֻדׅկ!UTVCPJ ה׍ֺךٖه٦زֶ嚂׃׫חTNBMSVCZֹֹ׋ַ׏׋[email protected]
  3. How to write synchronization mechanisms for Fiber [email protected] Cookpad, Thank

    you for paying the travel expenses
  4. How to write synchronization mechanisms for Fiber [email protected] Special thanks

    to @_sasn for translation
  5. @m_seki 衼⡲ 3VCZ؝ىحة &3# E3VCZ 3JOEB - Ruby committer -

    Author of ERB, dRuby, Rinda
  6. 2005 - ·ͩॳ࡮Γങ͑·͢ dRuby ʹΑΔ ؔকढ़ஶ ෼ࢄ ɾ Web ϓϩάϥϛϯά

    First printing from 2005 (Still available)
  7. 2012 (-2014 ઈ൛) Publication from 2012 (Become out of print

    in 2014)
  8. 2017 re-published on the web Re-published it on the web.

    CC by SA
  9. Agenda 'JCFSךֶׁ׵ְ 'JCFSⰅꟌ &OVNFSBUPS 4ZODISPOJ[F .VMUJQMFYFS Agenda - Recap: Fiber

    - Introduction of Fiber
  10. Fiberͷ͓͞Β͍ 1SPDFTT 5ISFBE 'JCFS Recap: Fiber

  11. Process ↔ Thread ↔ Fiber 1SPDFTT ًٌٔכⰟ剣׃׋ְ׿׌ 5ISFBE ؝ٝذؗأزأ؎حثכ荈ⴓדװ׷ 'JCFS

    - Process -- Want to share memories! --- Thread ---- Do context switch by myself! ----- Fiber
  12. Process ⥂隊ׁ׸׋ًٌٔ瑞꟦ فٔؒٝفذ؍ـז؝ٝذؗأزأ؎حث 猘חה׏גך䤫׸ Protected memory space preemptive task switch

    my dreams
  13. Before Process فٗإأז׿גזַ׏׋ ,FZ׾《׶ח遤ֻ ⯓遤Ⰵ⸂זח׉׸ https://www.youtube.com/watch?v=TOnS97C-RGQ

  14. Before Process (80s) 04ז׿גזַ׏׋ 1$♧〴ָفٗإأ♧א Before Process (80s) - There

    weren't any OS's - 1 process, 1 computer
  15. ϓϩηε΁ͷಌΕ ״ֻ׻ַ׵זְֽו铣׿ד׋ ֿ׿ז⚅歲ָ֮׷ךַ A Craving for Processes - Didn't get

    it much, but I used to read it - Didn't know such a world
  16. Before Process (90s) ⥂隊ׁ׸זְًٌٔ ⼿锃㘗وٕثةأؙ ظٝفٔؒٝفذ؍ـ ֿ׸'JCFSׄׯ׿ - no memory

    protection - Cooperative multitasking -- non preemptive -- Is this a ... Fiber?
  17. ࣮ࡍͷγεςϜ 醱侧ךفٗإأה׉׸׵ָⵃ欽ׅ׷项彁 ر٦ةד圓䧭ׁ׸׷ ꥫꨄׁ׸׋ًٌٔ瑞꟦׾馄ִ׷䗳銲ָ֮׷ فٗإأ׌ֽꥫꨄ׃ג׮穠㽷כ The Actual System - Assembled

    by multiple process and the resources that are used by them - Needs to jump out of the isolated memory space - Isolating processes is one thing, but for the others...
  18. Processes فٗإأ׌ֽד㸣穠׃זְ 㢩鿇הךꟼ׻׶ ワ鴟鄲縧 ؿ؋؎ٕ ➭ךفٗإأ - They don't work

    alone - Works with -- devices, storage, other processes
  19. ProcessͰॻ͘ͱ͖ͷؾ࣋ͪ קַךفٗإأהוֲחַ׃גⰟ剣׃״ֲ ׭׿וְֻֽׁו How you feel when you are using

    process - Try to share with other processes -- Even though it's a pain in the neck
  20. ProcessͰॻ͘ͱ͖ͷؾ࣋ͪ Skip!

  21. IPC͸ΊΜͲ͍͘͞ QJQF 4ZTUFN7*1$ TPDLFU ׭׿וֲ׌٦ IPCs are a pain in

    the neck too - Such a drag
  22. Thread ًٌٔ瑞꟦׾Ⱏ剣ׅ׷⸋⻉晛فٗإأ فٔؒٝفذ؍ـז؝ٝذؗأزأ؎حث ⸋⻉ה䒷ֹ䳔ִח鯪ꆀ 㢳ֻך➂חכفٗإأך倯ְְָה䙼ֲ degraded version of process that

    shares memory space - The system automatically switches to Thread - Bad copy, but lightweight To most people, processes are probably better
  23. ThreadͰॻ͘ͱ͖ͷؾ࣋ͪ קַךأٖحسךًٌٔ׾鍗׵זְ״ֲח ׮ךׅ׀ֻ孡׾אֽ׷ 孡׾䫙ֻה鍗׏גְ׷ 鍗׏ג׮㣐♱㣗זהֿ׹׌ֽ׾鍗׷ What it feels like to

    write Threads Try best to be very careful not to change memories of other threads - Tend to change it whenever you lose focus - Only change the places that are okay to change
  24. Threadʹඞཁͳၗຫ͞ չ⥯ֻ׵ְחז׷ה04ח걾׵זֻג׮㸜Ⰻ ח⸬桦葺ֻ剅ֽ׷״պ ה鎉ִ׷➂ך׋׭ך娀㐻 04ךؔ٦غ٦قحسָ孡חז׷➂פ Attitude when using Threads "Secure

    and efficient coding without OS? No sweat." - threads will be a great advantage for you - for coders who have concerns about OS overheads
  25. Fiber ظٝفٔؒٝفذ؍ـז⸋⻉晛5ISFBE و؎ؙٗأٖحس 5ISFBEⰻח醱侧ך؝ٝذؗأزָ֮׷ ⴽךⳢ椚ך鷿⚥פأ؎حثׅ׷ קה׿וך➂חכ5ISFBEך倯ְְָה䙼ֲ A non-preemptive degraded version

    of Thread - Micro thread - Has multiple context in one Thread - Switches during other jobs To most people, Threads are probably better
  26. Fiberʹඞཁͳࣗ৴ չ⥯ֻ׵ְחז׷ה3VCZ⚛׫חֲתְأ؛ آُ٦ָٓ剅ֽ׷״պ Attitude when using Fibers "Coding a scheduler

    that's as good as Ruby? Piece of cake."
  27. FiberͰॻ͘ͱ͖ͷؾ࣋ͪ ֲתְ؝ٝذؗأزأ؎حثך⽃⡘׾鋅א ֽ״ֲ ـٗحׇׁؙזְ״ֲחׅ׷חכ ׬׃׹'JCFS׾⢪ֲֿהָ湡涸 Attitude when using Threads "Secure

    and efficient coding without OS? No sweat." - threads will be a great advantage for you - for coders who have concerns about OS overheads
  28. Process ↔ Thread ↔ Fiber 1SPDFTT ًٌٔכⰟ剣׃׋ְ׿׌ 5ISFBE ؝ٝذؗأزأ؎حثכ荈ⴓדװ׷ 'JCFS

    - Process -- Want to share memory! --- Thread ---- Do context switch by myself! ----- Fiber
  29. Process ↔ Thread ↔ Fiber 'JCFS 荈⹛涸חأ؎حث׃ג״ 5ISFBE ًٌٔ⥂隊׃גֻ׸זְה㔭׷׿׌ 1SPDFTT

    ----- Fiber ---- Why won't it switch automatically? --- Thread -- Can't code if it doesn't protect memory! - Process
  30. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ ෼͘Β͍͔ͳ ✓ 1SPDFTT 5ISFBE 'JCFS ✓ ࣍͸'JCFSΛקΊΔΑʂ please

    wait for a moment - Done: Process, Thread, and Fiber
  31. ͳͥFiberͷ࿩Λ͢Δͷ͔ 'JCFS⢪׏׋ؿٖ٦يٙ٦ָؙׅ׀ְ׵׃ְ ה耀ְ׋ ׅ׀ְ׋ְפ׿ ׅ׀ְְֻ Why Fiber? Fiber used frameworks

    are cool - from what I heard -- is it hard to do? -- is it really good?
  32. ͳͥFiberͷ࿩Λ͢Δͷ͔ ٙةءع9ד(6*הTPDLFUفؚٗٓىؚٝ ָثّحزرٕؗ ךדծ荈ⴓד׮剅ְג׫׋ֻז׏׋ أٖحسד醱侧ךفٗإأה鸐⥋ׅ ׷׮ך I can socket program

    with x11 and GUI a little. - So, I decided that I'd like to use it myself -- Something that communicates with multiple processes with one Thread
  33. ͳͥFiberͷ࿩Λ͢Δͷ͔ ✮䟝דכ5ISFBE״׶׮ꨇ׃ְה䙼׏׋׿׌ ֽו׉ך鸐׶ד׃׋ךדծ׫זׁ׿ח׮ֶ ׅ׉ⴓֽ׃׋ְ I suspected that it would be

    harder than Threads It was. Let me share my experiences with you.
  34. Fiberೖ໳ 'JCPOBDDJ&OVNFSBUPS 3EW4J[FE2VFVF DIBOOFM .VMUJQMFYFS Fiber 101

  35. ࠷ॳͷςʔϚ &OVNFSBUPSך״ֲז圓鸡ךفؚٗٓي׾ 铣׫זָ׵ծ'JCFSך"1*׾㷕ןתׅ 'JCFSד剅ֹ湫ׅ &OVNFSUPSד剅ֹ湫ׅ First topic We will use

    an Enumerator like structure to learn Fiber API - Rewrite with Fiber - Rewrite with Enumerator
  36. Α͋͘ΔFibonacci ٕ٦فד剅ֻה知⽃ [0, 1] [1, 1] [2, 2] [3, 3]

    [4, 5] [5, 8] [6, 13] [7, 21] [8, 34] [9, 55] Plain Fibonacci - Easy to write with Loop def fibonacci(n) a, b = 1, 1 n.times do |i| p [i, a] a, b = b, a + b end end fibonacci(10)
  37. FiberͰॻ͖௚͢ ֿ׸׾⢪׏ג"1*׾铡僇׃תׅ fib = Fiber.new do a, b = 1,

    1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end def fibonacci(n) a, b = 1, 1 n.times do |i| p [i, a] a, b = b, a + b end end fibonacci(10) Rewrite with Fiber - Explain API using this
  38. API 'JCFSOFX 'JCFSSFTVNF 'JCFSZJFME

  39. Fiber.new ⴽ'JCFSדװ׶׋ְⳢ椚׾ـٗحؙד床ׅ ת׌ـٗحؙך؝٦سח鹌׫תׇ׿ fib = Fiber.new do a, b =

    1, 1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end Fiber.new - Pass value with another Fiber to do a different job - The block code will not run yet.
  40. Fiber#resume 'JCFSח؝ٝذؗأز׾ⴖ׶剏ִ׷ fib = Fiber.new do a, b = 1,

    1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end Switch context to the Fiber
  41. Fiber.yield SFTVNF׃׋'JCFSחⴖ׶剏ִ׷ 䒷侧ָSFTVNFך⦼חז׷ fib = Fiber.new do a, b =

    1, 1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end Switch to fiber.resume The argument becomes the value of the fiber
  42. resume, again ✳䏝湡⟃꣬ךSFTVNFכZJFMEַ׵ⱄꟚׅ׷ fib = Fiber.new do a, b =

    1, 1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end resume, again From the second time on, resume restarts from yield
  43. resumeͷߦ͖ઌ 'JCFS׾OFX׃׋׌ֽדכ㹋遤ׁ׸זְ SFTVNFד'JCFSח؝ٝذؗأز׾獳ׅ ♧䏝湡כ⯓걧פ ✳䏝湡⟃꣬כ'JCFSZJFME׃׋הֿ׹פ ֿ׸כ⡦䏝湡ךSFTVNFַז Where resume jumps to

    Would not execute just by Fiber new Pass the value to Fiber by resume -- Back to the top on the first round -- Back to Fiber.yield from the second round on --- Which round am I on...?
  44. APIΛઆ໌ͨ͠ SFTVNFהZJFMEך嚊䙀ָה׏אֹחְֻ 荈歋ֺׅגוֲ׃גְְַ׻ַ׵זְ Explained the API -The concept of resume

    and yield is hard to grep -Don't know what to do, there are so many things we can do with it
  45. EnumeratorͰॻ͖௚͢ ׉׏ֻ׶ fib = Fiber.new do a, b = 1,

    1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end def fibonacci a, b = 1, 1 while true yield(a) a, b = b, a + b end end fib = to_enum(:fibonacci) 10.times do |n| p [n, fib.next] end Rewrite with Enumerator - Nearly identical
  46. Enumerator 単׾⡲׷ֿהד罋ִװֻׅז׷ ֶׅׅ׭ ❌ 'JCFSךꨇ׃ׁח䩧׍⯘אהְֲ䩛䘔ִ ָזְ Enumerator - Easier to

    understand when put in a frame - Recommended ❌ Can't really get the feel of overcoming the difficulties of Fiber
  47. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ &OVNFSBUPSͰ"1*Λઆ໌ ✓ ೉ָ͕͠͞͠Ίͳ͍ ✓ ࣍͸3EW please wait for

    a moment - Done: Explained the API
 - Next: Synchronizaition
  48. ࣍ͷςʔϚ ず劍䖉׍さ׻ׇ Next topic - Synchronization

  49. QueueΛհͨ͠ڠௐ 2VFVF׾➜׃ג醱侧ך5ISFBEָ鸬䵿ׅ׷ ׉׸׊׸ך5ISFBEכֶ✼ְ׾濼׵זְ 欰欵罏ծ嶊顤罏כ醱侧ְג׮⹛ֻ q = SizedQueue.new(1) Thread.new do 10.times

    do |n| sleep(rand) q.push(n) p [:push, n] end end Thread.new do 10.times do |n| sleep(rand) p [:pop, q.pop] end end.join Sync with Queue - Multiple Threads synchronize through Queue - Each Thread does not know anything about other Threads - Works with multiple producers and consumers
  50. ͜ΕͷFiber൛ 3EW׾➜׃ג醱侧ך'JCFSָ鸬䵿ׅ׷ SFOEF[WPVT 䖉׍さ׻ׇ rdv = Rdv.new Fiber.new do 10.times

    do |n| rdv.push(n) p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume Fiber version - Multiple Fibers work together through Rdv - rendez-vous
  51. rendez-vous ⚓ rendez-vous

  52. ड͚औΔଆ͕଴ͭ৔߹ ⚓ rendez-vous When the consumer waits for the producer

    rendezvous
  53. ͜͏͍͏৔߹΋͋Δ ⚓ rendez-vous This might happen

  54. Rdvͷ࣮૷ JOJUJBMJ[F QVTI QPQ def initialize @reader = [] @writer

    = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end Implementation of Rdv
  55. ࠷ۙ޷͖ͳΠσΟΦϜ וֲװ׵ֿ׸׌ֽדֲתֻ剅ֽ׉ֲ ה׶ִ֮׆SFTVNF 遤ֹ鑥ת׏׋׵ZJFME My recent favorite idiom Feeling that

    it will go smoothly with these - just use resume - yield when you're stuck
  56. ͱΓ͋͑ͣresume 'JCFSOFX׃׋׵ּׅSFTVNFׅ׷ 5ISFBEOFXך״ֲחּׅ⹛ֹ⳿ׅ 剑ⴱךSFTVNFך暴ⴽז⹛ֹ׾罋ִזֻ ג״ְ Just use resume for now

    - Resume right after Fiber.new -- Works right away, just like Thread.new --- Don't have to think about the special behavior on the first round
  57. ߦ͖٧·ͬͨΒyield װ׸׷ֿהָ֮׷꟦כדֹ׷׌ֽװ׷ װ׸׷ֿהָזֻז׏׋׵ZJFME 䖉א湱䩛ח㼎׃גծ荈ⴓ 'JCFSDVSSFOU ׾鋙ִג׮׵ֲ Yield when you're stuck

    - Do as much as it can while it can do it - Yield when it has nothing to do -- Let the partner know about yourself (alias fiber.current)
  58. initialize !SFBEFSכQPQ䖉׍ծ!XSJUFSכQVTI䖉׍ ך'JCFS׾ًٌׅ׷ def initialize @reader = [] @writer =

    [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @reader holds fibers waiting for pop @writer holds fibers waiting for push
  59. push !SFBEFSְָזְהֹծ荈ⴓ׾ًٌ׃ג䖉 א ZJFME  !SFBEFSְָ׷הֹծ饯ֿׅ SFTVNF def initialize @reader

    = [] @writer = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end - When there is no @reader, it records itself and wait (yield) - Resume when @reader is present (resume)
  60. @reader͕͍ͳ͍ ر٦ةה'JCFSDVSSFOU׾ًٌ׃גZJFME def initialize @reader = [] @writer = []

    end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end When @reader is not there - Records the data and Fiber.current, then yield
  61. @reader͕͍Δ !SFBEFS׾♧אSFTVNFׅ׷ QPQד䖉׏גְ׋'JCFSָⱄꟚׅ׷ def initialize @reader = [] @writer =

    [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @reader is present - Resume one @reader -- Fiber that was waiting for the pop resumes
  62. pop !XSJUFSְָזְהֹծ荈ⴓ׾ًٌ׃ג䖉 א ZJFME  !XSJUFSְָ׷הֹծ饯ֿׅ SFTVNF  ׉׃גر٦ة׾鵤ׅ def

    initialize @reader = [] @writer = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end When there is no @writer, it records itself and waits (yield) - When @writer is present, it resumes (resume) -- and returns data Skip!
  63. @writer͕͍ͳ͍ 'JCFSDVSSFOU׾ًٌ׃גZJFME def initialize @reader = [] @writer = []

    end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @writer is gone Record Fiber.current and yield Skip!
  64. @writer͕͍Δ !XSJUFS׾♧אSFTVNFׅ׷ QVTIד䖉׏גְ׋'JCFSָⱄꟚׅ׷ ַًٌ׵ر٦ة׾鵤ׅ def initialize @reader = [] @writer

    = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @writer is present - Resume one @writer -- a Fiber that was waiting for push resumes - Returns data value from the note Skip!
  65. ݕࠪதͷഉଞ੍ޚ 'JCFSדכ勴⟝嗚叨ך䱖➭ⵖ䖴ָ♶銲 5ISFBEדכ⽬ꤹ def initialize @reader = [] @writer =

    [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end Exclusive control while inspecting - With Fiber, there is no need for exclusive control - Very risky on Threads!!
  66. Threadͱͷҧ͍ 5ISFBEדכ嗚叨׃גְ׷꟦ח׮؝ٝذؗأ زأ؎حثׅ׷ .POJUPSהַ䗳銲 䙳ֻגسؗسׅؗ׷ Difference from Threads - Thread

    context switches during the inspection -- needs monitoring -- nerve-racking
  67. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume How it works. Fiber.new kara-matz is summoned! Fiber.newした
 kara-matzを召喚
  68. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume kara-matz is resumed It's kara-matz's turn! kara-matzをresume
 kara-matzのターン

  69. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume kara-matz does push() There are no opponents.. Record data and himself, then yield. kara-matzのpush!
 相手がいない。
 自分とデータをメモしyield。ターン終了
  70. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume Main's turn! Fiber.new ichi-matz is summoned! メインのターン
 Fiber.new
 ichi-matzを召喚
  71. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matz resumed It's ichi-matz's turn! ichi-matzをresume
 ichi-matzのターン

  72. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matz does pop! ichi-matz reads the memo takes data. kara-matz resumes the finished. ichi-matzのpop!
 ichi-matzはメモを読んだ
 データを手にいれた。kara-matzをresumeしターン終了
  73. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume kara-matz does push() There are no opponents. Record data and himself, then yield. kara-matzのpush!
 相手がいない。
 自分とデータをメモしyield。ターン終了
  74. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matzのpop!
 ichi-matzはメモを読んだ
 データを手にいれた。kara-matzをresumeしターン終了 ichi-matz does pop! ichi-matz reads the memo takes data. kara-matz resumes the finished. Skip!
  75. 㔐װ׶תׅ Repeat this 10 times Skip!

  76. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matz dies. It's main's turn Main dies. ichi-matzはしんでしまった
 メインのターン
 メインはしんでしまった Skip!
  77. ͏͖ͦ͝͏ʂ ֲ׀ֹתׅ It might work! - and it does

  78. ͜͜ͷ·ͱΊ 'JCFSָず劍ׅ׷׋׭ך➬穈׫׾⡲׏׋ 猘ך㥨ֹז؎ر؍ؔي׾荈䢧׃׋ ה׶ִ֮׆SFTVNF 遤ֹ鑥ת׏׋׵ZJFME Wrapping up Made a structure

    of synchronizing fibers Proudly presented my favorite idiom - just use resume - yield when you're stuck
  79. ͜͜ͷ·ͱΊ ず劍ך勴⟝嗚叨ח䱖➭ⵖ䖴ְ׵זְ 农럀涸ח5ISFBEח㢌䳔ׁ׸׷ה㔭׷ 5ISFBE׾⢪׏גְ׋א׮׶ָ㹋כ'JCFS ׌׏׋ך倯ְְָה䙼ֲ Wrapping up We don't need

    exclusive control when sync-ing - Don't want it to be implicitly converted into Threads -- It's better off that it was actually Fiber when you thought you were using Threads
  80. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ 'JCFSͷಉظͷ࣮૷ ✓ 3EW ✓ ࣍͸ιέοτϓϩάϥϛ ϯά please wait

    for a moment - Done: Synchronization 
 - Next: Sockets
  81. ࣍ͷςʔϚ TPDLFUך؟٦غ٦فؚٗٓىؚٝ فٗز؝ٕךⳢ椚׾荈搫ח剅ֹ׋ְ׃ծ醱 侧ך䱸竲׮䪔ְ׋ְ׿׌ֽו٦ Next Topic - Socket server programming

    - Want to code protocol processes smartly and support multiple connections..
  82. ͢͢Ί͔ͨ E3VCZ؟٦غ٦ך毟⡂؝٦سד铡僇 稆劰׌ֽו⚛遤ח⹛ַזְ׮ך 5ISFBE׾欽ְג⚛遤ח⹛ֻ׮ך 5ISFBE晛׾'JCFS晛ח׃׋׮ך؎و؎ ث ׉ך何葺晛 Table of Contents:

    explain with dRuby server pseudo code - Coding is simple, but doesn't move in parallel - Works in parallel with Threads - Code above rewritten to use Fiber (bit awkward) -- Better version using Fiber
  83. ૉ๿͚ͩͲҰ్ ⽃秪ח♧湫简ח剅ְ׋׮ך *0כـٗحؙׅ׷ֽו孡ח׃זְ ؙٓ؎،ٝز׃ַ⹛ַזְ def main_loop loop do select fds.each

    do |fd| call_callback end end end def on_accept c = accept loop do read_request(c) do_it write_response(c) end end Simple but not parallel - Written straight forward - Ignore I/O blocks - Only works with one client def read_n(fd, sz) ... ... end
  84. Thread൛ GE׀הח5ISFBE׾饯⹛ׅ׷ ֿ׸׌ֽד⚛遤ח⹛ֻ def main_loop loop do select fds.each do

    |fd| call_callback end end end def on_accept c = accept Thread.new(c) do |fd| loop do read_request(fd) do_it write_response(fd) end end end Thread ver - Start a new thread on every fd - Add 2 columns and it works in parallel def read_n(fd, sz) ... ... end ௥Ճ͸͚ͩ͜͜ ௥Ճ͸͚ͩ͜͜
  85. Fiber൛ ظٝفٔؒٝفذ؍ـ ׉ךתתדכ⚛遤ח⹛ַזְ def main_loop loop do select fds.each do

    |fd| call_callback end end end def on_accept c = accept Fiber.new do loop do read_request(c) do_it write_response(c) end end.resume end Fiber ver - non-preemptive - Not quite parallel, yet def read_n(fd, sz) ... ... end Swapped with Fiber
  86. Fiber൛վ *0ָـٗحؙ׃׉ֲזהֹծֿ׏׉׶؝ٝ ذؗأز׾ⴖ׶剏ִ׷⡲䨌 ـٗحؙ괏ظٝـٗحؙ*0 def main_loop loop do select fds.each

    do |fd| call_callback end end end def on_accept c = accept Fiber.new do loop do read_request(c) do_it write_response(c) end end.resume end Fiber ver Revisioned - Switch context before it I/O blocks -- Blocking-style non-blocking I/O def read_n(fd, sz) ... fd.read_nonblock(sz) rescue WaitReadable Fiber.yield retry end
  87. Block-ish non-block I/O ـٗحؙ׃׉ֲזהֹծ'JCFSZJFME׃ג [email protected]ח׮ו׷ ؝٦ٕغحؙדSFTVNFׅ׷ def main_loop loop do

    select fds.each do |fd| call_callback end end end def on_accept c = accept Fiber.new do loop do read_request(c) do_it write_response(c) end end.resume end It Fiber.yields back to main_loop before it I/O blocks Resumes on call back def read_n(fd, sz) ... fd.read_nonblock(sz) rescue WaitReadable Fiber.yield retry end
  88. ίʔϧόοΫͰresume ֿך׋׭ח㼭ׁזٓ؎ـٓٔ׾剅ֹת׃׋ Resumes on call back - I made a

    library to do this
  89. Bartender GEךSFBEBCMF XSJUBCMFח㼎䘔׃׋؝٦ٕ غحָؙ剅ֽ׷׌ֽך噰㼭ךٓ؎ـٓٔ https://github.com/seki/bartender A tiny callback library that

    corresponds to fd readable & writable
  90. Bartender::Context SFBEBCMF XSJUBCMFחז׏׋ֿהח孡בְ׋ ׵؝٦ٕغحؙ ֿ׸׾⢪׏גCMPDLJTIOPOCMPDL*0׾ Context#[]=(event, fd, callback) Πϕϯτछʢ:read, :writeʣͱfdͷ૊ʹίʔϧόοΫΛొ࿥

    Bartender::Context Callbacks when readable or writeable Use this to implement block-ish non-block I/O register callbacks for the combination of event types (:read, :write) and fd
  91. _read() [email protected]ׅ׷կ⢽㢩ך㜥さכ SFBEBCMFחז׷ך׾䖉׏גַ׵SFUSZׅ׷ 䖉׏ג ׉ֿד䖉׏׍ׯتً ⴽך'JCFSח؝ٝذؗأزأ؎حثׅ׷ def _read(fd, sz) return

    fd.read_nonblock(sz) rescue IO::WaitReadable wait_readable(fd) retry end def wait_readable(fd); wait_io(:read, fd); end def wait_io(event, fd) self[event, fd] = Fiber.current.method(:resume) Fiber.yield ensure delete(event, fd) end - read_nonblock. On exception, it waits to be readable before it retries. - Waits to be what? -- We need to context switch to another fiber
  92. read_nonblock ּׅ铣׭׷ر٦ةָ֮׸ל铣׿ד鵤ׅ T[劢弫ך؛٦أ׮״ֻ֮׷ ּׅח铣׭׷ر٦ةָזְ㜥さחכ⢽㢩 *08BJU3FBEBCMF def _read(fd, sz) return fd.read_nonblock(sz)

    rescue IO::WaitReadable wait_readable(fd) retry end def wait_readable(fd); wait_io(:read, fd); end def wait_io(event, fd) self[event, fd] = Fiber.current.method(:resume) Fiber.yield ensure delete(event, fd) end If there are any readable data, it reads and returns - There are cases when it's below sz Excepts when there is no readable data - IO::WaitReadable
  93. wait_io() ؝٦ٕغحؙה׃גSFTVNF׾涫ꐮ ׉׃ג'JCFSZJFME 剑穄涸חً؎ٕٝ٦فח䨱׶TFMFDUׅ׷ SFTVNFׁ׸׋הֹSFBEBCMFחז׏ג׷ def _read(fd, sz) return fd.read_nonblock(sz)

    rescue IO::WaitReadable wait_readable(fd) retry end def wait_readable(fd); wait_io(:read, fd); end def wait_io(event, fd) self[event, fd] = Fiber.current.method(:resume) Fiber.yield ensure delete(event, fd) end Register resume as callback Fiber.yield - Goes back to mainloop and selects at the end readable when resumed
  94. 㹋ꥷך،فٔ؛٦ءّٝד铡僇׃תׅ I'll show this to you on an actual application

  95. tiny_drbͷൈਮ ًحإ٦آ铣׿ד穈׫甧ג׷鿇ⴓדׅ ـٗحؙ׾孡חׇ׆⽃秪זSFBE׾⚛ץג׷ ـٗحؙׅ׷㜥さחכ农럀涸ח؝ٝذؗأ ز׾أ؎حث׃תׅ def req_drb(reader) ref = load(reader,

    false) msg = load(reader) argc = load(reader) argv = argc.times.collect {load(reader)} block = load(reader, false) [ref, msg, argv] end def load(reader, marshal=true) sz = reader.read(4) sz = sz.unpack('N')[0] data = reader.read(sz) return data unless marshal begin Marshal.load(data) rescue DRb::DRbUnknown.new($!, data) end end extracted from tiny_drb - Where it constructs with the read messages - Just lining up reads, blocks are ignored - When blocked, it context switches implicitly.
  96. צҧ͍ ❌ SFBEBCMFז׵ًحإ٦آ׾ּׅחⰋ鿇铣׭׷ ❌ XSJUBCMFז׵ًحإ٦آ׾ּׅחⰋ鿇剅ֽ׷ misinterpretation Can read all messages

    when it's readable Can write all messages when its writeable
  97. ੲΑ͘ݟͨόά TFMFDU ך֮הծًحإ٦آ׾铣׿ד穈׫甧 ג׷ꟼ侧׾؝٦ٕ 鹼䒀ծꥺ㹱ָ֮׷הةأؙأ؎حثדֹ ׆ח㔿ת׷ Bugs in the good

    Ol' days (in C) - Call functions that read & construct messages after select() -- Cannot switch tasks when there is latency and/or incident, and freezes
  98. σϞʢ͸͋Γ·ͤΜʣ 醱侧ךؙٓ؎،ٝزָ⹛ֻ圫㶨ָTBNQMFח 剅ְג֮׶תָׅ ⹛ַ׃ג׮鋅ג׻ַ׵זַ׏׋ךד (There is no) DEMO - You

    can see how multiple clients work in the sample.. -- But you couldn't really *see* it moving
  99. tiny_drb.rb irb -r drb irb -r drb tiny_drb.rb Rdv https://github.com/seki/bartender/tree/master/sample

    Skip!
  100. tiny_drbͷ͓΋͠Ζ͍ͱ͜Ζ CMPDLJTIOPOCMPDL*0 僇爙涸זأ؛آُ٦ָٓזְ ♧אך'JCFSָ醱侧ך➬穈׫TFMFDU겲ה 3EWהַהֲתֻ⹛ְגְ׷ Interesting stuff about tiny_drb -

    block-ish nonblock I/O - No written scheduler -- One Fiber is working around well with multiple nodes (select, Rdv, etc.)
  101. ଞʹࢼ࡞ͨ͠΋ͷ 'JCFS晛3JOEB5VQMF4QBDF 'JCFS晛8&#SJDL '%׾ⵃ欽׃׋5ISFBEהך䖉׍さ׻ׇ Other prototypes - Rinda::TupleSpace - Fiber

    edition - WEBrick - Fiber edition - Synchronize Threads with FD Skip!
  102. Threadͱͷ଴ͪ߹Θͤ 5ISFBEWBMVFׅ׷ה姺ת׏ג׃תֲ *0QJQFךDMPTF׾⢪׏ג穄✪׾鸐濼ׅ׷ class ThreadTask def initialize(*args, &block) left, @right

    = IO.pipe @task = Thread.new do begin block.call(*args) ensure left.close end end end def value if @right Bartender.wait_readable(@right) @right.close end @task.value ensure @right = nil end end Sync Threads with FD - Stops on Thread.value - Notifies stop with IO.pipe close Skip!
  103. Multiplexerͷ·ͱΊ ⽃秪ח♧湫简ח剅ֹ׋ְWT⚛遤Ⳣ椚 5ISFBE⤑ⵃ 'JCFSד⚛遤䏝♳־׷ךꨇ׃ְ CMPDLJTIOPOCMPDL*0׾⡲׏׋ Multiplexer Wrap-up - Simple coding

    vs parallelism - Threads are useful - Hard to implement perfect parallel with Fiber -- made block-ish non-block I/O
  104. Fiberೖ໳ͷ·ͱΊ 'JCPOBDDJ&OVNFSBUPS 3EW4J[FE2VFVF DIBOOFM .VMUJQMFYFS ꨇ׃ְ״י Fiber 101 Wrap-up -

    Difficult, ain't it?
  105. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ ࣌ؒ͋Δ͔ͳʁ ✓ ͓·͚ please wait for a moment

  106. ͓·͚ 'JCFSכꨇ׃ְהֿ׹ָ꬗涯ְ ➙㔐כֲתְֻ׏׋ֽו㣟侁׃׋鑧 Digression - Fiber is hard, and fun

    - It went well this time, but a story about it *not* working Skip?
  107. ࠓճͷྫ͸ 农鎸דֹ׷ֻ׵ְ㼭ְׁ➬穈׫׌׏׋ ➬穈׫חさ׻ׇג،فٔ؛٦ءّٝ׾剅 ֽ׷ 如ך״ֲז縐׾鼘ֽ׵׸׋ This example - Was pretty

    small, small enough to memorize it all -- Can code & adapt according to a particular application - Didn't get stuck in the following traps:
  108. ᠘ 'JCFSך⚥ד鹼ְⳢ椚׾׃גـٗحؙ ㄎ׿׌⯓ד5ISFBE⡲׏גְג'JCFS&SSPS ׌׸ַךHFNךזַדְֲֲֿֿהװ׏ג ׷ַ׮ Traps: - Get blocked inside

    Fiber by processing something slow in it - Read some place that used a thread and FiberError - Somebody might have already done this in gem somewhere
  109. ͭ·Γ ،فٔ؛٦ءّٝך⢪ֲٓ؎ـٓٔךׅ׫ ׆׫תד㹋鄲׾椚鍑׃גזְהعو׷ عو׷׏ג鋙ִגֶֽל㣐♱㣗 㹋כ'JCFSׄׯזֻג׮عو׷ The point is - that

    you need to understand everything about the library or you'll get stuck -- If you know that you'll get stuck, you're okay --- (Actually, it doesn't need to be Fiber to get stuck)
  110. ϋϚΔྫ 'JCFS晛ך5XJUUFSךة؎يٓ؎ٝך铣׫鴥׫ ⡲׹ֲ +40/QBSTFכأزٔ٦ي׾Ⰻ鿇铣׿ד 4USJOHח׃׍ׯֲ̔تً أزٔ٦يַ׵铣׬+40/QBSTFזְ Examples of getting stuck

    - Making a Fiber version of Twitter timeline reading -- JSON.parse reads and puts everything into a string →Won't work - Is there JSON.parse that reads from streams?
  111. ϋϚΓͦ͏ͳྫ 3%#.4ךسٓ؎غהַ 㔿ת׵זְך More examples of places to get stuck

    - RDBMS drivers...? -- Do they freeze?
  112. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ ͓ΘΓ please wait for a moment - Almost

    the end
  113. ·ͱΊ 5ISFBEכꨇ׃ְָ'JCFSכ׮׏הꨇ׃ְ 'JCFSך"1*הず劍ծ*0ך䘔欽⢽׾爙׃׋ 剑鵚㥨ֹז؎ر؍ؔي׾稱➜׃׋ Summary - Threads are hard, but

    Fibers are even harder - Explained Fiber API, synchronization, I/O Multiplexer application - Introduced my favorite idiom
  114. Process ↔ Thread ↔ Fiber 'JCFS 荈⹛涸חأ؎حث׃ג״ 5ISFBE ًٌٔ⥂隊׃גֻ׸זְה㔭׷׿׌ 1SPDFTT

    ----- Fiber ---- Why won't it switch automatically? --- Thread -- Can't code if it doesn't protect memory! - Process