Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to write synchronization mechanisms for Fiber

How to write synchronization mechanisms for Fiber

RubyKaigi 2017

40f4d1f2e77078955bd01e9fb4a503ba?s=128

seki at druby.org

September 20, 2017
Tweet

Transcript

  1. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ ͝ཡͷεϙϯαʔͷఏڙ ʹΑΓ͓ૹΓ͠·͢

  2. slide sponsors ְאַせ〢㾊3VCZ⠓陽ח⿫⸇׃׋ְ㖑⯋孖זךד!JXBEPO 䎢䃊ד׮044(BUF׃׋ְ!LUPV E3VCZח״׷ⴓ侔٥8FCفؚٗٓىؚٝ ؔ٦ي爡F#PPL4UPSF IUUQ FTUPSFPINTIBDPKQUJUMFT1!IJTBTIJN !N@TFLJךSF⿾䗁Ꟛ涪׾⿾䗁㷕统׃זָ׵⿾䗁Ꟛ涪חزٓ؎׃גְת ׅ!BXB[FLJ

    ⚅歲剑㣐.BTUPEPOչ1BXPPպכ3VCZך⸂ד麊欽׃גתׅ!QJYJW 〴괏׾⛦׶馉ִ䎢䃊ח⛦׶鴥׿׌涺圫ծ➙䎃׮N@TFLJ眍׾ֶ嚂׃׫ֻ׌ׁ ְ剑鵚չ،صًך鋅׵׸倯պהְֲず➂钞׾⠓爡ד⡲׏גתׅךד׉ ׍׵׮״׹׃ֻדׅկ!UTVCPJ ה׍ֺךٖه٦زֶ嚂׃׫חTNBMSVCZֹֹ׋ַ׏׋!WFTUJHF@
  3. How to write synchronization mechanisms for Fiber seki@ruby-lang.org Cookpad, Thank

    you for paying the travel expenses
  4. How to write synchronization mechanisms for Fiber seki@ruby-lang.org Special thanks

    to @_sasn for translation
  5. @m_seki 衼⡲ 3VCZ؝ىحة &3# E3VCZ 3JOEB - Ruby committer -

    Author of ERB, dRuby, Rinda
  6. 2005 - ·ͩॳ࡮Γങ͑·͢ dRuby ʹΑΔ ؔকढ़ஶ ෼ࢄ ɾ Web ϓϩάϥϛϯά

    First printing from 2005 (Still available)
  7. 2012 (-2014 ઈ൛) Publication from 2012 (Become out of print

    in 2014)
  8. 2017 re-published on the web Re-published it on the web.

    CC by SA
  9. Agenda 'JCFSךֶׁ׵ְ 'JCFSⰅꟌ &OVNFSBUPS 4ZODISPOJ[F .VMUJQMFYFS Agenda - Recap: Fiber

    - Introduction of Fiber
  10. Fiberͷ͓͞Β͍ 1SPDFTT 5ISFBE 'JCFS Recap: Fiber

  11. Process ↔ Thread ↔ Fiber 1SPDFTT ًٌٔכⰟ剣׃׋ְ׿׌ 5ISFBE ؝ٝذؗأزأ؎حثכ荈ⴓדװ׷ 'JCFS

    - Process -- Want to share memories! --- Thread ---- Do context switch by myself! ----- Fiber
  12. Process ⥂隊ׁ׸׋ًٌٔ瑞꟦ فٔؒٝفذ؍ـז؝ٝذؗأزأ؎حث 猘חה׏גך䤫׸ Protected memory space preemptive task switch

    my dreams
  13. Before Process فٗإأז׿גזַ׏׋ ,FZ׾《׶ח遤ֻ ⯓遤Ⰵ⸂זח׉׸ https://www.youtube.com/watch?v=TOnS97C-RGQ

  14. Before Process (80s) 04ז׿גזַ׏׋ 1$♧〴ָفٗإأ♧א Before Process (80s) - There

    weren't any OS's - 1 process, 1 computer
  15. ϓϩηε΁ͷಌΕ ״ֻ׻ַ׵זְֽו铣׿ד׋ ֿ׿ז⚅歲ָ֮׷ךַ A Craving for Processes - Didn't get

    it much, but I used to read it - Didn't know such a world
  16. Before Process (90s) ⥂隊ׁ׸זְًٌٔ ⼿锃㘗وٕثةأؙ ظٝفٔؒٝفذ؍ـ ֿ׸'JCFSׄׯ׿ - no memory

    protection - Cooperative multitasking -- non preemptive -- Is this a ... Fiber?
  17. ࣮ࡍͷγεςϜ 醱侧ךفٗإأה׉׸׵ָⵃ欽ׅ׷项彁 ر٦ةד圓䧭ׁ׸׷ ꥫꨄׁ׸׋ًٌٔ瑞꟦׾馄ִ׷䗳銲ָ֮׷ فٗإأ׌ֽꥫꨄ׃ג׮穠㽷כ The Actual System - Assembled

    by multiple process and the resources that are used by them - Needs to jump out of the isolated memory space - Isolating processes is one thing, but for the others...
  18. Processes فٗإأ׌ֽד㸣穠׃זְ 㢩鿇הךꟼ׻׶ ワ鴟鄲縧 ؿ؋؎ٕ ➭ךفٗإأ - They don't work

    alone - Works with -- devices, storage, other processes
  19. ProcessͰॻ͘ͱ͖ͷؾ࣋ͪ קַךفٗإأהוֲחַ׃גⰟ剣׃״ֲ ׭׿וְֻֽׁו How you feel when you are using

    process - Try to share with other processes -- Even though it's a pain in the neck
  20. ProcessͰॻ͘ͱ͖ͷؾ࣋ͪ Skip!

  21. IPC͸ΊΜͲ͍͘͞ QJQF 4ZTUFN7*1$ TPDLFU ׭׿וֲ׌٦ IPCs are a pain in

    the neck too - Such a drag
  22. Thread ًٌٔ瑞꟦׾Ⱏ剣ׅ׷⸋⻉晛فٗإأ فٔؒٝفذ؍ـז؝ٝذؗأزأ؎حث ⸋⻉ה䒷ֹ䳔ִח鯪ꆀ 㢳ֻך➂חכفٗإأך倯ְְָה䙼ֲ degraded version of process that

    shares memory space - The system automatically switches to Thread - Bad copy, but lightweight To most people, processes are probably better
  23. ThreadͰॻ͘ͱ͖ͷؾ࣋ͪ קַךأٖحسךًٌٔ׾鍗׵זְ״ֲח ׮ךׅ׀ֻ孡׾אֽ׷ 孡׾䫙ֻה鍗׏גְ׷ 鍗׏ג׮㣐♱㣗זהֿ׹׌ֽ׾鍗׷ What it feels like to

    write Threads Try best to be very careful not to change memories of other threads - Tend to change it whenever you lose focus - Only change the places that are okay to change
  24. Threadʹඞཁͳၗຫ͞ չ⥯ֻ׵ְחז׷ה04ח걾׵זֻג׮㸜Ⰻ ח⸬桦葺ֻ剅ֽ׷״պ ה鎉ִ׷➂ך׋׭ך娀㐻 04ךؔ٦غ٦قحسָ孡חז׷➂פ Attitude when using Threads "Secure

    and efficient coding without OS? No sweat." - threads will be a great advantage for you - for coders who have concerns about OS overheads
  25. Fiber ظٝفٔؒٝفذ؍ـז⸋⻉晛5ISFBE و؎ؙٗأٖحس 5ISFBEⰻח醱侧ך؝ٝذؗأزָ֮׷ ⴽךⳢ椚ך鷿⚥פأ؎حثׅ׷ קה׿וך➂חכ5ISFBEך倯ְְָה䙼ֲ A non-preemptive degraded version

    of Thread - Micro thread - Has multiple context in one Thread - Switches during other jobs To most people, Threads are probably better
  26. Fiberʹඞཁͳࣗ৴ չ⥯ֻ׵ְחז׷ה3VCZ⚛׫חֲתְأ؛ آُ٦ָٓ剅ֽ׷״պ Attitude when using Fibers "Coding a scheduler

    that's as good as Ruby? Piece of cake."
  27. FiberͰॻ͘ͱ͖ͷؾ࣋ͪ ֲתְ؝ٝذؗأزأ؎حثך⽃⡘׾鋅א ֽ״ֲ ـٗحׇׁؙזְ״ֲחׅ׷חכ ׬׃׹'JCFS׾⢪ֲֿהָ湡涸 Attitude when using Threads "Secure

    and efficient coding without OS? No sweat." - threads will be a great advantage for you - for coders who have concerns about OS overheads
  28. Process ↔ Thread ↔ Fiber 1SPDFTT ًٌٔכⰟ剣׃׋ְ׿׌ 5ISFBE ؝ٝذؗأزأ؎حثכ荈ⴓדװ׷ 'JCFS

    - Process -- Want to share memory! --- Thread ---- Do context switch by myself! ----- Fiber
  29. Process ↔ Thread ↔ Fiber 'JCFS 荈⹛涸חأ؎حث׃ג״ 5ISFBE ًٌٔ⥂隊׃גֻ׸זְה㔭׷׿׌ 1SPDFTT

    ----- Fiber ---- Why won't it switch automatically? --- Thread -- Can't code if it doesn't protect memory! - Process
  30. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ ෼͘Β͍͔ͳ ✓ 1SPDFTT 5ISFBE 'JCFS ✓ ࣍͸'JCFSΛקΊΔΑʂ please

    wait for a moment - Done: Process, Thread, and Fiber
  31. ͳͥFiberͷ࿩Λ͢Δͷ͔ 'JCFS⢪׏׋ؿٖ٦يٙ٦ָؙׅ׀ְ׵׃ְ ה耀ְ׋ ׅ׀ְ׋ְפ׿ ׅ׀ְְֻ Why Fiber? Fiber used frameworks

    are cool - from what I heard -- is it hard to do? -- is it really good?
  32. ͳͥFiberͷ࿩Λ͢Δͷ͔ ٙةءع9ד(6*הTPDLFUفؚٗٓىؚٝ ָثّحزرٕؗ ךדծ荈ⴓד׮剅ְג׫׋ֻז׏׋ أٖحسד醱侧ךفٗإأה鸐⥋ׅ ׷׮ך I can socket program

    with x11 and GUI a little. - So, I decided that I'd like to use it myself -- Something that communicates with multiple processes with one Thread
  33. ͳͥFiberͷ࿩Λ͢Δͷ͔ ✮䟝דכ5ISFBE״׶׮ꨇ׃ְה䙼׏׋׿׌ ֽו׉ך鸐׶ד׃׋ךדծ׫זׁ׿ח׮ֶ ׅ׉ⴓֽ׃׋ְ I suspected that it would be

    harder than Threads It was. Let me share my experiences with you.
  34. Fiberೖ໳ 'JCPOBDDJ&OVNFSBUPS 3EW4J[FE2VFVF DIBOOFM .VMUJQMFYFS Fiber 101

  35. ࠷ॳͷςʔϚ &OVNFSBUPSך״ֲז圓鸡ךفؚٗٓي׾ 铣׫זָ׵ծ'JCFSך"1*׾㷕ןתׅ 'JCFSד剅ֹ湫ׅ &OVNFSUPSד剅ֹ湫ׅ First topic We will use

    an Enumerator like structure to learn Fiber API - Rewrite with Fiber - Rewrite with Enumerator
  36. Α͋͘ΔFibonacci ٕ٦فד剅ֻה知⽃ [0, 1] [1, 1] [2, 2] [3, 3]

    [4, 5] [5, 8] [6, 13] [7, 21] [8, 34] [9, 55] Plain Fibonacci - Easy to write with Loop def fibonacci(n) a, b = 1, 1 n.times do |i| p [i, a] a, b = b, a + b end end fibonacci(10)
  37. FiberͰॻ͖௚͢ ֿ׸׾⢪׏ג"1*׾铡僇׃תׅ fib = Fiber.new do a, b = 1,

    1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end def fibonacci(n) a, b = 1, 1 n.times do |i| p [i, a] a, b = b, a + b end end fibonacci(10) Rewrite with Fiber - Explain API using this
  38. API 'JCFSOFX 'JCFSSFTVNF 'JCFSZJFME

  39. Fiber.new ⴽ'JCFSדװ׶׋ְⳢ椚׾ـٗحؙד床ׅ ת׌ـٗحؙך؝٦سח鹌׫תׇ׿ fib = Fiber.new do a, b =

    1, 1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end Fiber.new - Pass value with another Fiber to do a different job - The block code will not run yet.
  40. Fiber#resume 'JCFSח؝ٝذؗأز׾ⴖ׶剏ִ׷ fib = Fiber.new do a, b = 1,

    1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end Switch context to the Fiber
  41. Fiber.yield SFTVNF׃׋'JCFSחⴖ׶剏ִ׷ 䒷侧ָSFTVNFך⦼חז׷ fib = Fiber.new do a, b =

    1, 1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end Switch to fiber.resume The argument becomes the value of the fiber
  42. resume, again ✳䏝湡⟃꣬ךSFTVNFכZJFMEַ׵ⱄꟚׅ׷ fib = Fiber.new do a, b =

    1, 1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end resume, again From the second time on, resume restarts from yield
  43. resumeͷߦ͖ઌ 'JCFS׾OFX׃׋׌ֽדכ㹋遤ׁ׸זְ SFTVNFד'JCFSח؝ٝذؗأز׾獳ׅ ♧䏝湡כ⯓걧פ ✳䏝湡⟃꣬כ'JCFSZJFME׃׋הֿ׹פ ֿ׸כ⡦䏝湡ךSFTVNFַז Where resume jumps to

    Would not execute just by Fiber new Pass the value to Fiber by resume -- Back to the top on the first round -- Back to Fiber.yield from the second round on --- Which round am I on...?
  44. APIΛઆ໌ͨ͠ SFTVNFהZJFMEך嚊䙀ָה׏אֹחְֻ 荈歋ֺׅגוֲ׃גְְַ׻ַ׵זְ Explained the API -The concept of resume

    and yield is hard to grep -Don't know what to do, there are so many things we can do with it
  45. EnumeratorͰॻ͖௚͢ ׉׏ֻ׶ fib = Fiber.new do a, b = 1,

    1 while true Fiber.yield(a) a, b = b, a + b end end 10.times do |n| p [n, fib.resume] end def fibonacci a, b = 1, 1 while true yield(a) a, b = b, a + b end end fib = to_enum(:fibonacci) 10.times do |n| p [n, fib.next] end Rewrite with Enumerator - Nearly identical
  46. Enumerator 単׾⡲׷ֿהד罋ִװֻׅז׷ ֶׅׅ׭ ❌ 'JCFSךꨇ׃ׁח䩧׍⯘אהְֲ䩛䘔ִ ָזְ Enumerator - Easier to

    understand when put in a frame - Recommended ❌ Can't really get the feel of overcoming the difficulties of Fiber
  47. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ &OVNFSBUPSͰ"1*Λઆ໌ ✓ ೉ָ͕͠͞͠Ίͳ͍ ✓ ࣍͸3EW please wait for

    a moment - Done: Explained the API
 - Next: Synchronizaition
  48. ࣍ͷςʔϚ ず劍䖉׍さ׻ׇ Next topic - Synchronization

  49. QueueΛհͨ͠ڠௐ 2VFVF׾➜׃ג醱侧ך5ISFBEָ鸬䵿ׅ׷ ׉׸׊׸ך5ISFBEכֶ✼ְ׾濼׵זְ 欰欵罏ծ嶊顤罏כ醱侧ְג׮⹛ֻ q = SizedQueue.new(1) Thread.new do 10.times

    do |n| sleep(rand) q.push(n) p [:push, n] end end Thread.new do 10.times do |n| sleep(rand) p [:pop, q.pop] end end.join Sync with Queue - Multiple Threads synchronize through Queue - Each Thread does not know anything about other Threads - Works with multiple producers and consumers
  50. ͜ΕͷFiber൛ 3EW׾➜׃ג醱侧ך'JCFSָ鸬䵿ׅ׷ SFOEF[WPVT 䖉׍さ׻ׇ rdv = Rdv.new Fiber.new do 10.times

    do |n| rdv.push(n) p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume Fiber version - Multiple Fibers work together through Rdv - rendez-vous
  51. rendez-vous ⚓ rendez-vous

  52. ड͚औΔଆ͕଴ͭ৔߹ ⚓ rendez-vous When the consumer waits for the producer

    rendezvous
  53. ͜͏͍͏৔߹΋͋Δ ⚓ rendez-vous This might happen

  54. Rdvͷ࣮૷ JOJUJBMJ[F QVTI QPQ def initialize @reader = [] @writer

    = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end Implementation of Rdv
  55. ࠷ۙ޷͖ͳΠσΟΦϜ וֲװ׵ֿ׸׌ֽדֲתֻ剅ֽ׉ֲ ה׶ִ֮׆SFTVNF 遤ֹ鑥ת׏׋׵ZJFME My recent favorite idiom Feeling that

    it will go smoothly with these - just use resume - yield when you're stuck
  56. ͱΓ͋͑ͣresume 'JCFSOFX׃׋׵ּׅSFTVNFׅ׷ 5ISFBEOFXך״ֲחּׅ⹛ֹ⳿ׅ 剑ⴱךSFTVNFך暴ⴽז⹛ֹ׾罋ִזֻ ג״ְ Just use resume for now

    - Resume right after Fiber.new -- Works right away, just like Thread.new --- Don't have to think about the special behavior on the first round
  57. ߦ͖٧·ͬͨΒyield װ׸׷ֿהָ֮׷꟦כדֹ׷׌ֽװ׷ װ׸׷ֿהָזֻז׏׋׵ZJFME 䖉א湱䩛ח㼎׃גծ荈ⴓ 'JCFSDVSSFOU ׾鋙ִג׮׵ֲ Yield when you're stuck

    - Do as much as it can while it can do it - Yield when it has nothing to do -- Let the partner know about yourself (alias fiber.current)
  58. initialize !SFBEFSכQPQ䖉׍ծ!XSJUFSכQVTI䖉׍ ך'JCFS׾ًٌׅ׷ def initialize @reader = [] @writer =

    [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @reader holds fibers waiting for pop @writer holds fibers waiting for push
  59. push !SFBEFSְָזְהֹծ荈ⴓ׾ًٌ׃ג䖉 א ZJFME  !SFBEFSְָ׷הֹծ饯ֿׅ SFTVNF def initialize @reader

    = [] @writer = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end - When there is no @reader, it records itself and wait (yield) - Resume when @reader is present (resume)
  60. @reader͕͍ͳ͍ ر٦ةה'JCFSDVSSFOU׾ًٌ׃גZJFME def initialize @reader = [] @writer = []

    end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end When @reader is not there - Records the data and Fiber.current, then yield
  61. @reader͕͍Δ !SFBEFS׾♧אSFTVNFׅ׷ QPQד䖉׏גְ׋'JCFSָⱄꟚׅ׷ def initialize @reader = [] @writer =

    [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @reader is present - Resume one @reader -- Fiber that was waiting for the pop resumes
  62. pop !XSJUFSְָזְהֹծ荈ⴓ׾ًٌ׃ג䖉 א ZJFME  !XSJUFSְָ׷הֹծ饯ֿׅ SFTVNF  ׉׃גر٦ة׾鵤ׅ def

    initialize @reader = [] @writer = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end When there is no @writer, it records itself and waits (yield) - When @writer is present, it resumes (resume) -- and returns data Skip!
  63. @writer͕͍ͳ͍ 'JCFSDVSSFOU׾ًٌ׃גZJFME def initialize @reader = [] @writer = []

    end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @writer is gone Record Fiber.current and yield Skip!
  64. @writer͕͍Δ !XSJUFS׾♧אSFTVNFׅ׷ QVTIד䖉׏גְ׋'JCFSָⱄꟚׅ׷ ַًٌ׵ر٦ة׾鵤ׅ def initialize @reader = [] @writer

    = [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end @writer is present - Resume one @writer -- a Fiber that was waiting for push resumes - Returns data value from the note Skip!
  65. ݕࠪதͷഉଞ੍ޚ 'JCFSדכ勴⟝嗚叨ך䱖➭ⵖ䖴ָ♶銲 5ISFBEדכ⽬ꤹ def initialize @reader = [] @writer =

    [] end def push(it) if @reader.empty? @writer << [it, Fiber.current] return Fiber.yield end @reader.shift.resume(it) end def pop if @writer.empty? @reader << Fiber.current return Fiber.yield end value, fiber = @writer.shift fiber.resume return value end Exclusive control while inspecting - With Fiber, there is no need for exclusive control - Very risky on Threads!!
  66. Threadͱͷҧ͍ 5ISFBEדכ嗚叨׃גְ׷꟦ח׮؝ٝذؗأ زأ؎حثׅ׷ .POJUPSהַ䗳銲 䙳ֻגسؗسׅؗ׷ Difference from Threads - Thread

    context switches during the inspection -- needs monitoring -- nerve-racking
  67. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume How it works. Fiber.new kara-matz is summoned! Fiber.newした
 kara-matzを召喚
  68. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume kara-matz is resumed It's kara-matz's turn! kara-matzをresume
 kara-matzのターン

  69. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume kara-matz does push() There are no opponents.. Record data and himself, then yield. kara-matzのpush!
 相手がいない。
 自分とデータをメモしyield。ターン終了
  70. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume Main's turn! Fiber.new ichi-matz is summoned! メインのターン
 Fiber.new
 ichi-matzを召喚
  71. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matz resumed It's ichi-matz's turn! ichi-matzをresume
 ichi-matzのターン

  72. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matz does pop! ichi-matz reads the memo takes data. kara-matz resumes the finished. ichi-matzのpop!
 ichi-matzはメモを読んだ
 データを手にいれた。kara-matzをresumeしターン終了
  73. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume kara-matz does push() There are no opponents. Record data and himself, then yield. kara-matzのpush!
 相手がいない。
 自分とデータをメモしyield。ターン終了
  74. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matzのpop!
 ichi-matzはメモを読んだ
 データを手にいれた。kara-matzをresumeしターン終了 ichi-matz does pop! ichi-matz reads the memo takes data. kara-matz resumes the finished. Skip!
  75. 㔐װ׶תׅ Repeat this 10 times Skip!

  76. ͏༷͘͝ࢠ rdv = Rdv.new Fiber.new do 10.times do |n| rdv.push(n)

    p [:push, n] end end.resume Fiber.new do 10.times do |n| p [:pop, rdv.pop] end end.resume ichi-matz dies. It's main's turn Main dies. ichi-matzはしんでしまった
 メインのターン
 メインはしんでしまった Skip!
  77. ͏͖ͦ͝͏ʂ ֲ׀ֹתׅ It might work! - and it does

  78. ͜͜ͷ·ͱΊ 'JCFSָず劍ׅ׷׋׭ך➬穈׫׾⡲׏׋ 猘ך㥨ֹז؎ر؍ؔي׾荈䢧׃׋ ה׶ִ֮׆SFTVNF 遤ֹ鑥ת׏׋׵ZJFME Wrapping up Made a structure

    of synchronizing fibers Proudly presented my favorite idiom - just use resume - yield when you're stuck
  79. ͜͜ͷ·ͱΊ ず劍ך勴⟝嗚叨ח䱖➭ⵖ䖴ְ׵זְ 农럀涸ח5ISFBEח㢌䳔ׁ׸׷ה㔭׷ 5ISFBE׾⢪׏גְ׋א׮׶ָ㹋כ'JCFS ׌׏׋ך倯ְְָה䙼ֲ Wrapping up We don't need

    exclusive control when sync-ing - Don't want it to be implicitly converted into Threads -- It's better off that it was actually Fiber when you thought you were using Threads
  80. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ 'JCFSͷಉظͷ࣮૷ ✓ 3EW ✓ ࣍͸ιέοτϓϩάϥϛ ϯά please wait

    for a moment - Done: Synchronization 
 - Next: Sockets
  81. ࣍ͷςʔϚ TPDLFUך؟٦غ٦فؚٗٓىؚٝ فٗز؝ٕךⳢ椚׾荈搫ח剅ֹ׋ְ׃ծ醱 侧ך䱸竲׮䪔ְ׋ְ׿׌ֽו٦ Next Topic - Socket server programming

    - Want to code protocol processes smartly and support multiple connections..
  82. ͢͢Ί͔ͨ E3VCZ؟٦غ٦ך毟⡂؝٦سד铡僇 稆劰׌ֽו⚛遤ח⹛ַזְ׮ך 5ISFBE׾欽ְג⚛遤ח⹛ֻ׮ך 5ISFBE晛׾'JCFS晛ח׃׋׮ך؎و؎ ث ׉ך何葺晛 Table of Contents:

    explain with dRuby server pseudo code - Coding is simple, but doesn't move in parallel - Works in parallel with Threads - Code above rewritten to use Fiber (bit awkward) -- Better version using Fiber
  83. ૉ๿͚ͩͲҰ్ ⽃秪ח♧湫简ח剅ְ׋׮ך *0כـٗحؙׅ׷ֽו孡ח׃זְ ؙٓ؎،ٝز׃ַ⹛ַזְ def main_loop loop do select fds.each

    do |fd| call_callback end end end def on_accept c = accept loop do read_request(c) do_it write_response(c) end end Simple but not parallel - Written straight forward - Ignore I/O blocks - Only works with one client def read_n(fd, sz) ... ... end
  84. Thread൛ GE׀הח5ISFBE׾饯⹛ׅ׷ ֿ׸׌ֽד⚛遤ח⹛ֻ def main_loop loop do select fds.each do

    |fd| call_callback end end end def on_accept c = accept Thread.new(c) do |fd| loop do read_request(fd) do_it write_response(fd) end end end Thread ver - Start a new thread on every fd - Add 2 columns and it works in parallel def read_n(fd, sz) ... ... end ௥Ճ͸͚ͩ͜͜ ௥Ճ͸͚ͩ͜͜
  85. Fiber൛ ظٝفٔؒٝفذ؍ـ ׉ךתתדכ⚛遤ח⹛ַזְ def main_loop loop do select fds.each do

    |fd| call_callback end end end def on_accept c = accept Fiber.new do loop do read_request(c) do_it write_response(c) end end.resume end Fiber ver - non-preemptive - Not quite parallel, yet def read_n(fd, sz) ... ... end Swapped with Fiber
  86. Fiber൛վ *0ָـٗحؙ׃׉ֲזהֹծֿ׏׉׶؝ٝ ذؗأز׾ⴖ׶剏ִ׷⡲䨌 ـٗحؙ괏ظٝـٗحؙ*0 def main_loop loop do select fds.each

    do |fd| call_callback end end end def on_accept c = accept Fiber.new do loop do read_request(c) do_it write_response(c) end end.resume end Fiber ver Revisioned - Switch context before it I/O blocks -- Blocking-style non-blocking I/O def read_n(fd, sz) ... fd.read_nonblock(sz) rescue WaitReadable Fiber.yield retry end
  87. Block-ish non-block I/O ـٗحؙ׃׉ֲזהֹծ'JCFSZJFME׃ג NBJO@MPPQח׮ו׷ ؝٦ٕغحؙדSFTVNFׅ׷ def main_loop loop do

    select fds.each do |fd| call_callback end end end def on_accept c = accept Fiber.new do loop do read_request(c) do_it write_response(c) end end.resume end It Fiber.yields back to main_loop before it I/O blocks Resumes on call back def read_n(fd, sz) ... fd.read_nonblock(sz) rescue WaitReadable Fiber.yield retry end
  88. ίʔϧόοΫͰresume ֿך׋׭ח㼭ׁזٓ؎ـٓٔ׾剅ֹת׃׋ Resumes on call back - I made a

    library to do this
  89. Bartender GEךSFBEBCMF XSJUBCMFח㼎䘔׃׋؝٦ٕ غحָؙ剅ֽ׷׌ֽך噰㼭ךٓ؎ـٓٔ https://github.com/seki/bartender A tiny callback library that

    corresponds to fd readable & writable
  90. Bartender::Context SFBEBCMF XSJUBCMFחז׏׋ֿהח孡בְ׋ ׵؝٦ٕغحؙ ֿ׸׾⢪׏גCMPDLJTIOPOCMPDL*0׾ Context#[]=(event, fd, callback) Πϕϯτछʢ:read, :writeʣͱfdͷ૊ʹίʔϧόοΫΛొ࿥

    Bartender::Context Callbacks when readable or writeable Use this to implement block-ish non-block I/O register callbacks for the combination of event types (:read, :write) and fd
  91. _read() SFBE@OPOCMPDLׅ׷կ⢽㢩ך㜥さכ SFBEBCMFחז׷ך׾䖉׏גַ׵SFUSZׅ׷ 䖉׏ג ׉ֿד䖉׏׍ׯتً ⴽך'JCFSח؝ٝذؗأزأ؎حثׅ׷ def _read(fd, sz) return

    fd.read_nonblock(sz) rescue IO::WaitReadable wait_readable(fd) retry end def wait_readable(fd); wait_io(:read, fd); end def wait_io(event, fd) self[event, fd] = Fiber.current.method(:resume) Fiber.yield ensure delete(event, fd) end - read_nonblock. On exception, it waits to be readable before it retries. - Waits to be what? -- We need to context switch to another fiber
  92. read_nonblock ּׅ铣׭׷ر٦ةָ֮׸ל铣׿ד鵤ׅ T[劢弫ך؛٦أ׮״ֻ֮׷ ּׅח铣׭׷ر٦ةָזְ㜥さחכ⢽㢩 *08BJU3FBEBCMF def _read(fd, sz) return fd.read_nonblock(sz)

    rescue IO::WaitReadable wait_readable(fd) retry end def wait_readable(fd); wait_io(:read, fd); end def wait_io(event, fd) self[event, fd] = Fiber.current.method(:resume) Fiber.yield ensure delete(event, fd) end If there are any readable data, it reads and returns - There are cases when it's below sz Excepts when there is no readable data - IO::WaitReadable
  93. wait_io() ؝٦ٕغحؙה׃גSFTVNF׾涫ꐮ ׉׃ג'JCFSZJFME 剑穄涸חً؎ٕٝ٦فח䨱׶TFMFDUׅ׷ SFTVNFׁ׸׋הֹSFBEBCMFחז׏ג׷ def _read(fd, sz) return fd.read_nonblock(sz)

    rescue IO::WaitReadable wait_readable(fd) retry end def wait_readable(fd); wait_io(:read, fd); end def wait_io(event, fd) self[event, fd] = Fiber.current.method(:resume) Fiber.yield ensure delete(event, fd) end Register resume as callback Fiber.yield - Goes back to mainloop and selects at the end readable when resumed
  94. 㹋ꥷך،فٔ؛٦ءّٝד铡僇׃תׅ I'll show this to you on an actual application

  95. tiny_drbͷൈਮ ًحإ٦آ铣׿ד穈׫甧ג׷鿇ⴓדׅ ـٗحؙ׾孡חׇ׆⽃秪זSFBE׾⚛ץג׷ ـٗحؙׅ׷㜥さחכ农럀涸ח؝ٝذؗأ ز׾أ؎حث׃תׅ def req_drb(reader) ref = load(reader,

    false) msg = load(reader) argc = load(reader) argv = argc.times.collect {load(reader)} block = load(reader, false) [ref, msg, argv] end def load(reader, marshal=true) sz = reader.read(4) sz = sz.unpack('N')[0] data = reader.read(sz) return data unless marshal begin Marshal.load(data) rescue DRb::DRbUnknown.new($!, data) end end extracted from tiny_drb - Where it constructs with the read messages - Just lining up reads, blocks are ignored - When blocked, it context switches implicitly.
  96. צҧ͍ ❌ SFBEBCMFז׵ًحإ٦آ׾ּׅחⰋ鿇铣׭׷ ❌ XSJUBCMFז׵ًحإ٦آ׾ּׅחⰋ鿇剅ֽ׷ misinterpretation Can read all messages

    when it's readable Can write all messages when its writeable
  97. ੲΑ͘ݟͨόά TFMFDU ך֮הծًحإ٦آ׾铣׿ד穈׫甧 ג׷ꟼ侧׾؝٦ٕ 鹼䒀ծꥺ㹱ָ֮׷הةأؙأ؎حثדֹ ׆ח㔿ת׷ Bugs in the good

    Ol' days (in C) - Call functions that read & construct messages after select() -- Cannot switch tasks when there is latency and/or incident, and freezes
  98. σϞʢ͸͋Γ·ͤΜʣ 醱侧ךؙٓ؎،ٝزָ⹛ֻ圫㶨ָTBNQMFח 剅ְג֮׶תָׅ ⹛ַ׃ג׮鋅ג׻ַ׵זַ׏׋ךד (There is no) DEMO - You

    can see how multiple clients work in the sample.. -- But you couldn't really *see* it moving
  99. tiny_drb.rb irb -r drb irb -r drb tiny_drb.rb Rdv https://github.com/seki/bartender/tree/master/sample

    Skip!
  100. tiny_drbͷ͓΋͠Ζ͍ͱ͜Ζ CMPDLJTIOPOCMPDL*0 僇爙涸זأ؛آُ٦ָٓזְ ♧אך'JCFSָ醱侧ך➬穈׫TFMFDU겲ה 3EWהַהֲתֻ⹛ְגְ׷ Interesting stuff about tiny_drb -

    block-ish nonblock I/O - No written scheduler -- One Fiber is working around well with multiple nodes (select, Rdv, etc.)
  101. ଞʹࢼ࡞ͨ͠΋ͷ 'JCFS晛3JOEB5VQMF4QBDF 'JCFS晛8&#SJDL '%׾ⵃ欽׃׋5ISFBEהך䖉׍さ׻ׇ Other prototypes - Rinda::TupleSpace - Fiber

    edition - WEBrick - Fiber edition - Synchronize Threads with FD Skip!
  102. Threadͱͷ଴ͪ߹Θͤ 5ISFBEWBMVFׅ׷ה姺ת׏ג׃תֲ *0QJQFךDMPTF׾⢪׏ג穄✪׾鸐濼ׅ׷ class ThreadTask def initialize(*args, &block) left, @right

    = IO.pipe @task = Thread.new do begin block.call(*args) ensure left.close end end end def value if @right Bartender.wait_readable(@right) @right.close end @task.value ensure @right = nil end end Sync Threads with FD - Stops on Thread.value - Notifies stop with IO.pipe close Skip!
  103. Multiplexerͷ·ͱΊ ⽃秪ח♧湫简ח剅ֹ׋ְWT⚛遤Ⳣ椚 5ISFBE⤑ⵃ 'JCFSד⚛遤䏝♳־׷ךꨇ׃ְ CMPDLJTIOPOCMPDL*0׾⡲׏׋ Multiplexer Wrap-up - Simple coding

    vs parallelism - Threads are useful - Hard to implement perfect parallel with Fiber -- made block-ish non-block I/O
  104. Fiberೖ໳ͷ·ͱΊ 'JCPOBDDJ&OVNFSBUPS 3EW4J[FE2VFVF DIBOOFM .VMUJQMFYFS ꨇ׃ְ״י Fiber 101 Wrap-up -

    Difficult, ain't it?
  105. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ ࣌ؒ͋Δ͔ͳʁ ✓ ͓·͚ please wait for a moment

  106. ͓·͚ 'JCFSכꨇ׃ְהֿ׹ָ꬗涯ְ ➙㔐כֲתְֻ׏׋ֽו㣟侁׃׋鑧 Digression - Fiber is hard, and fun

    - It went well this time, but a story about it *not* working Skip?
  107. ࠓճͷྫ͸ 农鎸דֹ׷ֻ׵ְ㼭ְׁ➬穈׫׌׏׋ ➬穈׫חさ׻ׇג،فٔ؛٦ءّٝ׾剅 ֽ׷ 如ך״ֲז縐׾鼘ֽ׵׸׋ This example - Was pretty

    small, small enough to memorize it all -- Can code & adapt according to a particular application - Didn't get stuck in the following traps:
  108. ᠘ 'JCFSך⚥ד鹼ְⳢ椚׾׃גـٗحؙ ㄎ׿׌⯓ד5ISFBE⡲׏גְג'JCFS&SSPS ׌׸ַךHFNךזַדְֲֲֿֿהװ׏ג ׷ַ׮ Traps: - Get blocked inside

    Fiber by processing something slow in it - Read some place that used a thread and FiberError - Somebody might have already done this in gem somewhere
  109. ͭ·Γ ،فٔ؛٦ءّٝך⢪ֲٓ؎ـٓٔךׅ׫ ׆׫תד㹋鄲׾椚鍑׃גזְהعو׷ عو׷׏ג鋙ִגֶֽל㣐♱㣗 㹋כ'JCFSׄׯזֻג׮عو׷ The point is - that

    you need to understand everything about the library or you'll get stuck -- If you know that you'll get stuck, you're okay --- (Actually, it doesn't need to be Fiber to get stuck)
  110. ϋϚΔྫ 'JCFS晛ך5XJUUFSךة؎يٓ؎ٝך铣׫鴥׫ ⡲׹ֲ +40/QBSTFכأزٔ٦ي׾Ⰻ鿇铣׿ד 4USJOHח׃׍ׯֲ̔تً أزٔ٦يַ׵铣׬+40/QBSTFזְ Examples of getting stuck

    - Making a Fiber version of Twitter timeline reading -- JSON.parse reads and puts everything into a string →Won't work - Is there JSON.parse that reads from streams?
  111. ϋϚΓͦ͏ͳྫ 3%#.4ךسٓ؎غהַ 㔿ת׵זְך More examples of places to get stuck

    - RDBMS drivers...? -- Do they freeze?
  112. ͠͹Β͓͘଴͍ͪͩ͘͞ ✓ ͓ΘΓ please wait for a moment - Almost

    the end
  113. ·ͱΊ 5ISFBEכꨇ׃ְָ'JCFSכ׮׏הꨇ׃ְ 'JCFSך"1*הず劍ծ*0ך䘔欽⢽׾爙׃׋ 剑鵚㥨ֹז؎ر؍ؔي׾稱➜׃׋ Summary - Threads are hard, but

    Fibers are even harder - Explained Fiber API, synchronization, I/O Multiplexer application - Introduced my favorite idiom
  114. Process ↔ Thread ↔ Fiber 'JCFS 荈⹛涸חأ؎حث׃ג״ 5ISFBE ًٌٔ⥂隊׃גֻ׸זְה㔭׷׿׌ 1SPDFTT

    ----- Fiber ---- Why won't it switch automatically? --- Thread -- Can't code if it doesn't protect memory! - Process