Go で Network Programming するためのよもやま話 / Network Programming with Go

by Tomohiro Takezawa

Slide 1

Slide 1 text

Go Ͱ Network Programming ͢ΔͨΊ ͷΑ΋΍·࿩ Tomohiro Takezawa 

Slide 2

Slide 2 text

ࣗݾ঺հ • ஛ᖒ ༑ത • Github: ttakezawa • Twitter: @takezawa • גࣜձࣾKyashॴଐ • όοΫΤϯυશൠ • ಛʹ VISA ΍ QUICPay (Google Pay) ͷϓϩηγϯάγεςϜͳͲ

Slide 3

Slide 3 text

࣮຿ͱωοτϫʔΫϓϩάϥϛϯά • ࣮ࡍͷͱ͜Ζɺ࢓ࣄͰ࢖͏ػձ͋Δʁ • Kyash ͷۀ຿Ͱ͸ඞਢͳͱ͜Ζ͕͋Δ • ΫϨδοτΧʔυͷϓϩηγϯάۀ຿ • ௿ϨΠϠʔͷཧղ͕ਂ·Δͱڧ͍

Slide 4

Slide 4 text

࿩͢͜ͱͱɺ࿩͞ͳ͍͜ͱ • ࿩͢͜ͱ • ιέοτϓϩάϥϛϯάશൠ • Go ʹ͓͚Δ I/O ͷ࢓૊Έ • ࿩͞ͳ͍͜ͱ • HTTP • νϟωϧ

Slide 5

Slide 5 text

ࠓ೔ͷΞδΣϯμ • ωοτϫʔΫϓϩάϥϛϯάͷجຊతͳΠϯ λϑΣʔε • netpoll ʹΑΔ I/OଟॏԽͷ࢓૊Έ • ϒϩοΩϯά I/O ΛΩϟϯηϧ͢Δํ๏ • TLSରԠʹ͍ͭͯ

Slide 6

Slide 6 text

GoͷωοτϫʔΫϓϩάϥϛ ϯάͷجຊతͳΠϯλϑΣʔ εʹ͍ͭͯ

Slide 7

Slide 7 text

·ͣ৮Γ͔Β • جຊతͳΠϯλϑΣʔεʹ͍ͭͯ • ந৅౓ߴΊʹઆ໌

Slide 8

Slide 8 text

ίωΫγϣϯཱ֬·ͰͷϓϦϛςΟϒ $POO -JTUFO %JBM αʔόଆ جຊతͳϓϩτίϧʹ͓͍ͯڞ௨ͷྲྀΕ ίωΫγϣϯ ΫϥΠΞϯτ αʔό͸ListenͰ଴ͪड͚͠ɺΫϥΠΞϯτ͸ DialͰ઀ଓ͢Δ

Slide 9

Slide 9 text

ίωΫγϣϯཱ֬·ͰͷϓϦϛςΟϒ $POO -JTUFO %JBM αʔόଆ جຊతͳϓϩτίϧʹ͓͍ͯڞ௨ͷྲྀΕ ίωΫγϣϯ ΫϥΠΞϯτ αʔό͸ListenͰ଴ͪड͚͠ɺΫϥΠΞϯτ͸ DialͰ઀ଓ͢Δ

Slide 10

Slide 10 text

ίωΫγϣϯཱ֬·ͰͷϓϦϛςΟϒ $POO -JTUFO %JBM αʔόଆ ίωΫγϣϯ ΫϥΠΞϯτ αʔό͸ListenͰ଴ͪड͚͠ɺΫϥΠΞϯτ͸ DialͰ઀ଓ͢Δ جຊతͳϓϩτίϧʹ͓͍ͯڞ௨ͷྲྀΕ

Slide 11

Slide 11 text

ίωΫγϣϯཱ֬·ͰͷϓϦϛςΟϒ $POO -JTUFO %JBM αʔόଆ ίωΫγϣϯ ΫϥΠΞϯτ ͓ޓ͍ʹίωΫγϣϯΦϒδΣΫτͰίϛϡχ έʔγϣϯΛߦ͏ جຊతͳϓϩτίϧʹ͓͍ͯڞ௨ͷྲྀΕ

Slide 12

Slide 12 text

ίωΫγϣϯཱ֬·ͰͷϓϦϛςΟϒ $POO -JTUFO %JBM netύοέʔδʹ͸ϓϩτίϧ͝ͱʹͦΕͧΕͷ࣮૷͕͋Δ net.DialIP(…) net.DialTCP(…) net.DialUDP(…) net.DialUnix(…) net.IPConn net.TCPConn net.UDPConn net.UnixConn net.ListenIP(…) net.ListenTCP(…) net.ListenUDP(…) net.ListenUnix(…) net.ListenUnixgram(…)

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

ίωΫγϣϯཱ֬·ͰͷϓϦϛςΟϒ $POO -JTUFO %JBM netύοέʔδʹ͸ϓϩτίϧ͝ͱʹͦΕͧΕͷ࣮૷͕͋Δ net.ListenIP(…) net.ListenTCP(…) net.ListenUDP(…) net.ListenUnix(…) net.ListenUnixgram(…) net.DialIP(…) net.DialTCP(…) net.DialUDP(…) net.DialUnix(…) net.IPConn net.TCPConn net.UDPConn net.UnixConn

Slide 16

Slide 16 text

ΠϯλϑΣʔεΛ࢖͓͏ લϖʔδͰڍ͛ͨ΋ͷ͸͢΂ͯ۩ମ࣮૷ ۩ମ࣮૷Ͱ͸ͳ͘ɺΠϯλϑΣʔεΛ࢖͑͹ • ςετ͕͠΍͍͢ • ൚༻Խ͕͠΍͍͢

Slide 17

Slide 17 text

ΠϯλϑΣʔεΛ࢖͓͏ ΠϯλʔϑΣʔεͰѻ͏৔߹ʹ͸ɺେ͖͘2छྨʹ෼͔ΕΔ • ετϦʔϜܕϓϩτίϧ • TCP • ετϦʔϜܕͷUnixυϝΠϯιέοτ • σʔλάϥϜܕϓϩτίϧ • UDP • IP • σʔλάϥϜܕͷUnixυϝΠϯιέοτ

Slide 18

Slide 18 text

ΠϯλϑΣʔεΛ࢖͏ͱ $POO -JTUFO %JBM net.Listen(…) net.ListenPacket(…) net.Dial(…) net.Conn net.PacketConn • Listen • ετϦʔϜܕͷ৔߹: net.Listen() → net.Listener → net.Conn • σʔλάϥϜܕͷ৔߹: net.ListenPacket() → net.PacketConn • Dial͸͍ͣΕͷϓϩτίϧͰ΋ڞ௨ • net.Dial() → net.Conn

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

ετϦʔϜܕϓϩτίϧͷ৔߹ • αʔό/ΫϥΠΞϯτ͸ڞʹ net.Conn ΠϯλϑΣʔε ͱͯ͠ίωΫγϣϯΛѻ͏ • αʔόଆ • net.Listen() Ͱ଴ͪड͚Λ࢝Ίɺฦ͖ͬͯͨListener ΠϯλϑΣʔεͷ Accept() ͰίωΫγϣϯͷ஋Λಘ Δ • ΫϥΠΞϯτଆ  • net.Dial() Ͱ઀ଓ͢Δ

Slide 24

Slide 24 text

ετϦʔϜܕϓϩτίϧͷ৔߹ • TCPͰૹड৴Λ͢Δίʔυ͸ྫ͑͹͜͏ͳΔ αʔόଆ ΫϥΠΞϯτଆ • ετϦʔϜܕͷ৔߹ɺίωΫγϣϯཱ͕֬͢ΔͱɺαʔόଆɺΫϥΠΞϯ τଆͷ྆ํʹ net.Conn ΠϯλϑΣʔεΛຬͨ͢ΦϒδΣΫτ͕ฦͬͯ͘Δ

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

ετϦʔϜܕϓϩτίϧͷ৔߹ • TCPͰૹड৴Λ͢Δίʔυͷྫ αʔόଆ ΫϥΠΞϯτଆ • ετϦʔϜܕͷ৔߹ɺίωΫγϣϯཱ͕֬͢ΔͱɺαʔόଆɺΫϥΠΞϯ τଆͷ྆ํʹ net.Conn ΠϯλϑΣʔεΛຬͨ͢஋͕ฦͬͯ͘Δ

Slide 28

Slide 28 text

ετϦʔϜܕϓϩτίϧͷ৔߹ • αʔό/ΫϥΠΞϯτ͸ͱ΋ʹ net.Conn ΠϯλʔϑΣʔεͰίωΫ γϣϯΛѻ͏ • networkͱͯ͠ "tcp"ɺ"unix"ɺ"unixpacket" ͳͲ͕ࢦఆՄೳ OFU-JTUFO αʔόଆ  net.Listen("tcp", "127.0.0.1:8080")  net.Listen("unix", "socketﬁle") ΫϥΠΞϯτଆ  net.Dial("tcp", "127.0.0.1:8080")  net.Dial("unix", "socketﬁle") OFU-JTUFOFS "DDFQU OFU$POO OFU%JBM OFU$POO

Slide 29

Slide 29 text

σʔλάϥϜܕϓϩτίϧͷ৔߹ • αʔόଆ • net.PacketConn ΠϯλϑΣʔεͷ஋Λฦ͢ • net.PacketConn ͸௨৴૬ख͕ݶఆ͞Ε͍ͯͳ͍৔߹ʹ ࢖༻͞ΕΔίωΫγϣϯͷΠϯλϑΣʔεͰ͋Γɺ net.PacketConn Λ࢖ͬͯૹ৴͢Δͱ͖͸ɺૹ৴ઌΛࢦ ఆ͢Δඞཁ͕͋Δ • ΫϥΠΞϯτଆ • ௨৴૬ख͕ݶఆ͞ΕΔͷͰɺ net.Conn ΠϯλϑΣʔεͷ ஋͕࢖͑Δ

Slide 30

Slide 30 text

σʔλάϥϜܕϓϩτίϧͷ৔߹ • UDPͰૹड৴Λ͢Δίʔυͷྫ αʔόଆ ΫϥΠΞϯτଆ • ΫϥΠΞϯτଆ͸ɺΠϯλϑΣʔε্͸ετϦʔϜܕͱมΘΒͳ͍ • αʔόଆ͸௨৴૬ख͕ఆ·Βͳ͍ͷͰɺ net.PacketConn ΠϯλϑΣʔεͷΦϒδΣΫτͰ ௨৴͢Δ͜ͱʹͳΔ • PacketConn.ReadFrom ͕௨৴૬खͷΞυϨε΋ฦͯ͘͠ΕΔ • PacketConn.WriteTo Ͱ૬खઌͷΞυϨεΛࢦఆ͢Δඞཁ͕͋Δ

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

σʔλάϥϜܕϓϩτίϧͷ৔߹ • σʔλάϥϜܕϓϩτίϧͷ৔߹ • αʔόଆ͸ net.ListenPacket() Λ࢖ͬͯ଴ͪड͚Λ։࢝ • αʔόଆ͸ net.PacketConnɺΫϥΠΞϯτଆ͸ net.Conn ͰίωΫγϣϯΛѻ͏ • networkͱͯ͠ "udp"ɺ"unixgram"ɺ"ip" ͳͲ͕ࢦఆՄೳ OFU-JTUFO1BDLFU αʔόଆ  net.ListenPacket("udp", "127.0.0.1:8080")  net.ListenPacket("unixgram", "socketﬁle") ΫϥΠΞϯτଆ  net.Dial("tcp", "127.0.0.1:8080")  net.Dial("unix", "socketﬁle") OFU1BDLFU$POO OFU$POO OFU%JBM ΫϥΠΞϯτͱ௨৴Λ։࢝͢Δલ ʹ࡞ΒΕΔίωΫγϣϯͷΠϯλʔ ϑΣʔε  ௨৴૬ख͕1ͭʹఆ·͍ͬͯͳ͍ ͱ͖ʹ࢖ΘΕΔ ΫϥΠΞϯτ(Dial)ଆ͸ɺ௨৴૬ ख͕ఆ·͓ͬͯΓɺ net.Conn Π ϯλʔϑΣʔεʹ౷Ұ͞Ε͍ͯΔ

Slide 38

Slide 38 text

ωοτϫʔΫϓϩάϥϛϯάͷ جຊతͳΠϝʔδ͸Θ͔ͬͨ

Slide 39

Slide 39 text

Go ͷ৔߹Ͳ͏΍ͬͯ I/O ͷ ฒྻԽΛͯ͠Δʁ

Slide 40

Slide 40 text

I/O ͷॲཧΛฒྻԽ I/O ͸ඇৗʹ஗͍ • ฒྻԽ • I/O Λ଴͍ͬͯΔؒʹผͷॲཧΛߦ͍͍ͨ • I/O ͷ଴ػΛ௿ίετͰ • ΫϥΠΞϯτ1ສ୆(C10K)໰୊

Slide 41

Slide 41 text

I/O ʹؔ͢Δ༻ޠͷઆ໌ • ϒϩοΩϯά I/O • I/O ॲཧ͕׬ྃɺΤϥʔɺλΠϜΞ΢τʹͳΔ͔ɺͳͲ௨஌͕͘Δ·Ͱ ࣮ߦΛઌʹਐ·ͳ͍ॲཧ • ϊϯϒϩοΩϯά I/O ͷ৔߹͸ɺ I/O ͕ॲཧதͰ΋࣮ߦΛ͙͢ʹਐΊΔ • I/O ͷଟॏԽ • ෳ਺ͷ I/O ͷ௨஌ΛҰ౓ʹ଴ͯΔΑ͏ʹ͢Δ͜ͱ • ଟॏԽ͍ͯ͠ͳ͍৔߹ʹ͸଴ͪड͚ର৅͝ͱʹγεςϜίʔϧ͕ඞཁ ʹͳΔ • OSʹΑΓɺ selectɺpollɺepoll(Linux)ɺkqueue(BSDܥ) ͳͲͷγες ϜίʔϧͰ࣮ݱ͞ΕΔ • ΠϕϯτۦಈϞσϧͷ࣮૷ʹ͸΄΅ඞਢͳ࢓૊Έ

Slide 42

Slide 42 text

ฒྻԽΛ͢ΔͨΊͷ ϓϩάϥϛϯάϞσϧ

Slide 43

Slide 43 text

ϚϧνεϨουϞσϧ • Α͋͘ΔҰൠతͳϓϩάϥϛϯάϞσϧ ͦͷ1 • ϑϩʔۦಈϞσϧ • ฒྻʹॲཧ͍ͨ͠୯ҐͰεϨουΛ࡞ͬͯɺεϨου͝ ͱʹ I/O Λߦ͏ • ྫ͑͹ίωΫγϣϯ͝ͱʹεϨου΋͘͠͸ϓϩηε Λੜ੒͠ɺͦΕͧΕͷεϨουͰ I/O Λߦ͏ɺͱ͍͏ Α͏ͳΞϓϩʔνʹͳΔ • ฒྻ౓͕ߴ͍ͱ͖(ίωΫγϣϯ͕ଟ͍ͱ͖)ʹɺͦͷ෼ εϨουΛͨ͘͞Μ࢖͏͜ͱʹͳΔͷͰɺίετ͕େ͖ ͍

Slide 44

Slide 44 text

ΠϕϯτۦಈϞσϧ • Α͋͘ΔҰൠతͳϓϩάϥϛϯάϞσϧ ͦͷ2 • ΠϕϯτΛ଴ػ͠ɺىͬͨ͜Πϕϯτʹ͍ͭͯΠϕϯτϋϯυϥΛ࣮ߦ͠ଓ͚ Δɺͱ͍͏Ϟσϧ • Node.jsɺRubyͷEventMachineɺPythonͷTwisted ͳͲ • I/O ΛଟॏԽͯ͠γϯάϧεϨουͰͨ͘͞Μͷ I/O Λѻ͑Δ • ೉͍͠ɺͱݴΘΕ΍͍͢Ϟσϧ • ओʹγϯάϧεϨουͰಈ࡞͢ΔͷͰɺ • ͕͔͔࣌ؒΔॲཧΛ࣮ߦͯ͠͠·͏ͱଞͷॲཧ͕શ࣮͘ߦ͞Εͳ͘ͳΔ • ෳ਺ίΞͷCPUΛੜ͔ͤͳ͍ • ॲཧͷ࣮ߦ͞ΕΔॱং͕Θ͔Βͳ͍ɺͳͲ

Slide 45

Slide 45 text

GoͰͷ΍Γํ ϚϧνεϨουͱΠϕϯτۦಈͷϋΠϒϦουͷΑ͏ͳ࢓૊Έ • Go͸εϨουΑΓ΋ܰྔͳgoroutineΛɺεϨου্ʹׂΓ౰ ͯͳ͕Β࣮ߦ͢Δ • goroutine্Ͱ I/O Λ࣮ߦ͢Δͱgoroutine͸ϒϩοΫ͞ΕΔ • ϒϩοΫ͍ͨ͠਺͚ͩgoroutine͕ඞཁ • GoͷϥϯλΠϜ͸ I/O ΛଟॏԽͯ͠଴ͪड͚͢Δ • ଴ͪड͚ͷͨΊʹେྔͷεϨουΛඞཁͱ͠ͳ͍ • I/O ଴ͪͷͱ͖goroutine͸εϨουʹׂΓ౰ͯ͞Εͳ͍

Slide 46

Slide 46 text

ϢʔβͷGoίʔυࢹ఺Ͱ͍͏ͱ • GoͷϓϩάϥϛϯάΠϯλϑΣʔεͱͯ͠͸ I/O ͸ϒ ϩοΩϯά • ྫ͑͹ conn.Read(buffer) ͱ͢ΔͱΠϕϯτ͕͋Δ· Ͱɺͦͷgoroutine͸εέδϡʔϥ͔Β֎͞Εͯ଴ͪঢ় ଶ(ϒϩοΩϯά)ʹͳΔ • ͜ͷ I/O ʹ͍ͭͯ௨஌͕͘Δͱɺ conn.Read(buffer) ͕ ฦͬͯ͘Δ

Slide 47

Slide 47 text

GoͷϥϯλΠϜ͕΍ͬͯ͘Ε͍ͯΔ • Ϣʔβͷίʔυ্Ͱ͸ϒϩοΩϯάૢ࡞ʹ ͳ͍ͬͯΔ΋ͷΛɺ಺෦తʹ͸ଟॏԽͯ͠औ Γѻͬͯ͘Ε͍ͯΔ • ϒϩοΩϯά I/O Λ ଟॏԽ I/O ʹ͍ͯ͠Δ netpoll ͱ͍͏࢓૊Έ͕͋Δ

Slide 48

Slide 48 text

netpollͷ࢓૊Έʹ͍ͭͯ

Slide 49

Slide 49 text

netpollͱ͸ʁ • goroutine ͷϒϩοΩϯάI/O ΛଟॏԽͯ͠؂ࢹ͢Δ࢓૊Έ • ଟॏԽʹ͍ͭͯͷ࣮૷ͰɺҎԼͷγεςϜίʔϧΛ࢖༻͍ͯ͠Δ • epoll(Linux) • kqueue(BSDܥ) • event ports(Solaris) • IOCP(Windows)

Slide 50

Slide 50 text

netpollʹΑΔଟॏԽͷྲྀΕ ҎԼͷྲྀΕΛ Linux (epoll) ͷྫͰऔΓ্͛ͯઆ໌͢Δ 1. ιέοτΛ࡞Δͱ͖ʹ४උ͢Δ 2. I/O ॲཧ (ReadͳͲ) Λ։࢝͢Δ 3. I/O ϒϩοΩϯά͢Δ goroutine Λεέδϡʔϥ͔Β֎͢ 4. ௨஌Λड͚औͬͨΒɺgoroutineΛ࣮ߦΩϡʔʹ໭͢ 5. εέδϡʔϥ͸࣮ߦ଴ͪͷΩϡʔ͔ΒgoroutineΛऔΓग़ ͠ɺOSεϨουʹׂΓ౰࣮ͯͯߦ͢Δ

Slide 51

Slide 51 text

1. ιέοτΛ࡞Δͱ͖ͷ४උ ιέοτΛ࡞ΔλΠϛϯάͰڞ௨ͷinitؔ਺Λ࣮ߦ͍ͯ͠Δ ҎԼɺGoͷιʔείʔυ͔Βൈਮ go1.11/src/net/sock_posix.go go1.11/src/os/ﬁle_unix.go • DialɺListenɺListenPacketɺAcceptɺϑΝΠϧΛ։͘ɺͦΕͧΕͷͱ͖ʹڞ௨ͷॳظԽॲཧ͕ߦΘΕΔ • Go 1.9 ͔ΒωοτϫʔΫ I/O͚ͩͰͳ͘ϑΝΠϧ I/Oʹ͍ͭͯ΋ netpoll ͰଟॏԽ͢Δ࣮૷͕ೖͬͨ • ͱ͸͍͑ɺৗʹ͋ΒΏΔ I/O Λ netpoll ͰଟॏԽ͍ͯ͠Δͱ͍͏Θ͚Ͱ͸ͳ͘ɺos.Open() ΍ os.Pipe() ͳͲݶఆ͞ΕͨϑΝΠϧσΟεΫϦϓλʹ͍ͭͯ netpoll ͰଟॏԽ͞ΕΔ go1.11/src/net/fd_unix.go

Slide 52

Slide 52 text

1. ιέοτΛ࡞Δͱ͖ͷ४උ • ڞ௨initؔ਺ : pollDesc.init • ιέοτ΍ϑΝΠϧͷσΟεΫϦϓλॳظԽʹݺͼग़͞ΕΔɻ • epoll ͕ॳظԽ͞Ε͍ͯͳ͍৔߹ʹ͸ epoll ͷॳظԽ epoll_create Λ͢Δ • epoll ͷ ؂ࢹ଴ͪʹొ࿥ EPOLL_CTL_ADD ͢Δ go1.11/src/internal/poll/fd_poll_runtime.go • runtime_pollServerInit ʹͯ epoll_create ͕1౓͚ͩ (sync.Once Ͱอূ) ͞ΕΔ • runtime_pollOpen Ͱ؂ࢹ଴ͪʹొ࿥ (EPOLL_CTL_ADD)͢Δ

Slide 53

Slide 53 text

1. ιέοτΛ࡞Δͱ͖ͷ४උ go1.11/src/internal/poll/fd_poll_runtime.go • runtime_pollServerInit ʹͯ epoll_create ͕1౓͚ͩ (sync.Once Ͱอূ) ͞ΕΔ • runtime_pollOpen Ͱ؂ࢹ଴ͪʹొ࿥ (EPOLL_CTL_ADD)͢Δ • ڞ௨initؔ਺ : pollDesc.init • ιέοτ΍ϑΝΠϧͷσΟεΫϦϓλॳظԽʹݺͼग़͞ΕΔɻ • epoll ͕ॳظԽ͞Ε͍ͯͳ͍৔߹ʹ͸ epoll ͷॳظԽ epoll_create Λ͢Δ • epoll ͷ ؂ࢹ଴ͪʹొ࿥ EPOLL_CTL_ADD ͢Δ

Slide 54

Slide 54 text

2. I/O ॲཧΛ։࢝͢Δ I/O ॲཧ͕ϒϩοΫ͢Δ৔߹ʹɺ pollDesc.wait() Λݺͼग़͢ go1.11/src/internal/poll/fd_unix.go go1.11/src/internal/poll/fd_poll_runtime.go ReadFrom ΍ WriteTo ͳͲʹͭ ͍ͯ΋ɺଟॏԽͷඞཁ͕͋Δ৔ ߹͸ಉ༷ʹ pollDesc.wait() ͢ ΔΑ͏ʹͳ͍ͬͯΔ

Slide 55

Slide 55 text

3. I/OϒϩοΩϯά͢ΔgoroutineΛεέδϡʔϥ͔Β֎͢ go1.11/src/internal/poll/fd_poll_runtime.go runtime.gopark ʹΑΓ goroutine͕εέδϡʔϥͷ ࣮ߦΩϡʔ͔ΒऔΓআ͔ΕΔ go1.11/src/runtime/netpoll.go runtime_pollWait͔Β netpollblockΛݺͼग़͍ͯ͠Δ ϢʔβͷGoίʔυ͔ΒݟΔͱɺ I/O ॲཧͷλΠϛ ϯάͰ goroutine ͕ϒϩοΫ͞ΕΔ

Slide 56

Slide 56 text

go1.11/src/internal/poll/fd_poll_runtime.go runtime.gopark ʹΑΓ goroutine͕εέδϡʔϥͷ ࣮ߦΩϡʔ͔ΒऔΓআ͔ΕΔ go1.11/src/runtime/netpoll.go runtime_pollWait͔Β netpollblockΛݺͼग़͍ͯ͠Δ ϢʔβͷGoίʔυ͔ΒݟΔͱɺ I/O ॲཧͷλΠϛ ϯάͰ goroutine ͕ϒϩοΫ͞ΕΔ 3. I/OϒϩοΩϯά͢ΔgoroutineΛεέδϡʔϥ͔Β֎͢

Slide 57

Slide 57 text

Slide 58

Slide 58 text

4. ௨஌Λड͚औͬͨΒgoroutineΛ࣮ߦΩϡʔʹ໭͢ go1.11/src/runtime/proc.go ιέοτΛ؂ࢹͯ͠௨஌Λड͚औͬͨ৔߹ʹ͸ɺ֘౰͢ΔgoroutineΛ࣮ ߦΩϡʔʹೖΕΔ • εέδϡʔϦϯάͰ͙͢ʹ࣮ ߦͰ͖Δgoroutine͕ݟ͔ͭ Βͳ͍৔߹ʹ͸ɺnetpoll() ʹΑΓ epoll_pwait Λ࣮ߦ͠ ͯɺI/O ͷ௨஌Λ଴ͭ • netpoll() ͸௨஌͕͋ͬͨ goroutine Λࢦ͢஋gpΛฦ ͢ͷͰɺ injectglist(gp) ʹΑ Γ࣮ߦΩϡʔʹೖΕΔ ͜ͷޙɺεέδϡʔϥ͸࣮ߦ଴ͪͷΩϡʔ͔ΒgoroutineΛऔΓग़͠ɺ࠶౓OS εϨουʹׂΓ౰࣮ͯͯߦ͢Δ

Slide 59

Slide 59 text

4. ௨஌Λड͚औͬͨΒgoroutineΛ࣮ߦΩϡʔʹ໭͢ go1.11/src/runtime/proc.go • εέδϡʔϦϯάͰ͙͢ʹ࣮ ߦͰ͖Δgoroutine͕ݟ͔ͭ Βͳ͍৔߹ʹ͸ɺnetpoll() ʹΑΓ epoll_pwait Λ࣮ߦ͠ ͯɺI/O ͷ௨஌Λ଴ͭ • netpoll() ͸௨஌͕͋ͬͨ goroutine Λࢦ͢஋gpΛฦ ͢ͷͰɺ injectglist(gp) ʹΑ Γ࣮ߦΩϡʔʹೖΕΔ ιέοτΛ؂ࢹͯ͠௨஌Λड͚औͬͨ৔߹ʹ͸ɺ֘౰͢ΔgoroutineΛ࣮ ߦΩϡʔʹೖΕΔ ͜ͷޙɺεέδϡʔϥ͸࣮ߦ଴ͪͷΩϡʔ͔ΒgoroutineΛऔΓग़͠ɺ࠶౓OS εϨουʹׂΓ౰࣮ͯͯߦ͢Δ

Slide 60

Slide 60 text

4. ௨஌Λड͚औͬͨΒgoroutineΛ࣮ߦΩϡʔʹ໭͢ go1.11/src/runtime/proc.go • εέδϡʔϦϯάͰ͙͢ʹ࣮ ߦͰ͖Δgoroutine͕ݟ͔ͭ Βͳ͍৔߹ʹ͸ɺnetpoll() ʹΑΓ epoll_pwait Λ࣮ߦ͠ ͯɺI/O ͷ௨஌Λ଴ͭ • netpoll() ͸௨஌͕͋ͬͨ goroutine Λࢦ͢஋gpΛฦ ͢ͷͰɺ injectglist(gp) ʹΑ Γ࣮ߦΩϡʔʹೖΕΔ ͜ͷޙɺεέδϡʔϥ͸࣮ߦ଴ͪͷΩϡʔ͔ΒgoroutineΛऔΓग़͠ɺ࠶౓OS εϨουʹׂΓ౰࣮ͯͯߦ͢Δ ιέοτΛ؂ࢹͯ͠௨஌Λड͚औͬͨ৔߹ʹ͸ɺ֘౰͢ΔgoroutineΛ࣮ ߦΩϡʔʹೖΕΔ

Slide 61

Slide 61 text

I/O ͕ goroutine ΛϒϩοΫ ͢Δ͜ͱ͸Θ͔ͬͨɻ runtime ͕ଟॏԽ͢Δ͜ͱ΋ Θ͔ͬͨɻ

Slide 62

Slide 62 text

Ϣʔβͷ೚ҙͷλΠϛϯάͰ ϒϩοΩϯά I/O Λऴྃͤ͞ ͯɺgoroutineΛಈ͔͢ʹ ͸ʁ

Slide 63

Slide 63 text

ϒϩοΩϯά I/O Λ೚ҙͷλΠϛϯάͰऴ͍ྃͤͨ͞৔߹ ೚ҙͷλΠϛϯάͰ I/O Λதஅ͍ͤͨ͜͞ͱ͕͋Δ • ΞϓϦέʔγϣϯΛऴྃ࣌ͷॲཧΛ࡞ΓࠐΉ৔߹ • SetDeadline Ͱ͸ରԠͰ͖ͳ͍தஅॲཧΛೖΕ͍ͨ৔߹ ͳͲ • ϒϩοΩϯάૢ࡞ (Read) ͷલʹϑϥάνΣοΫ Λ௥Ճ͢Δ͚ͩͰ͸ɺϒϩοΩϯάૢ࡞͕ղআ ͞ΕΔΘ͚Ͱ͸ͳ͍ • ͜ͷΑ͏ͳ࣮૷Λͯ͠͠·͏ͱɺ goroutine ͷऴྃʹ͸࠷େͰ1෼Ҏ্଴ͬͯ͠·͏ • SetDeadline Λ୹͘͢Δͱɺ଴ͭ࣌ؒ͸୹͘ ͳΔ͕ɺγεςϜίʔϧΛݺͼग़͠·͘Δ͜ ͱʹͳΔͷͰɺίετ͕ߴ͍ ଴ͨ͞Εͯ͠·͏ྫ

Slide 64

Slide 64 text

ϒϩοΩϯά I/O Λ೚ҙͷλΠϛϯάͰऴ͍ྃͤͨ͞৔߹ ओʹ2ͭͷΞϯϒϩοΫํ๏͕͋Δ 1. context.Context ΠϯλϑΣʔεͰΩϟϯηϧ͢Δ • DialͳͲɺ৽͘͠ιέοτΛ࡞੒͢Δ৔߹ͳͲ ͸ɺCloseର৅ʹͳΔΦϒδΣΫτ͕ଘࡏ͍ͯ͠ͳ ͍ͷͰ context Λ࢖ͬͨ΍ΓํʹͳΔ 2. Read ΍ WriteΛ͍ͯ͠ΔΦϒδΣΫτΛ Close() ͢Δ

Slide 65

Slide 65 text

contextΛ࢖ͬͯΞϯϒϩοΫͤ͞Δ৔߹ context.Context ʹରԠ͍ͯ͠ΔΠϯλϑΣʔε͕͋Ε͹ɺͦΕΛ࢖͏ • Dial() Ͱ͋Ε͹ net.Dialer ͷ DialContext() ͕࢖͑Δ • Listen() ΋ net.ListenConﬁg Ͱ ΩϟϯηϧରԠ • net.ListenConﬁg ͸ Go 1.11 Ͱ௥Ճ͞Εͨ࢓૊Έ • ͱ͸͍͑ɺListen()ͰϒϩοΫ͢Δͷ͸ DNS Ͱ໊લղܾ͢Δͱ͜ΖͷΈ

Slide 66

Slide 66 text

context.Context ʹରԠ͍ͯ͠ΔΠϯλϑΣʔε͕͋Ε͹ɺͦΕΛ࢖͏ • Dial() Ͱ͋Ε͹ net.Dialer ͷ DialContext() ͕࢖͑Δ • Listen() ΋ net.ListenConﬁg Ͱ ΩϟϯηϧରԠ • net.ListenConﬁg ͸ Go 1.11 Ͱ௥Ճ͞Εͨ࢓૊Έ • ͱ͸͍͑ɺListen()ͰϒϩοΫ͢Δͷ͸ DNS Ͱ໊લղܾ͢Δͱ͜ΖͷΈ contextΛ࢖ͬͯΞϯϒϩοΫͤ͞Δ৔߹

Slide 67

Slide 67 text

Close() Λ࢖ͬͯΞϯϒϩοΫͤ͞Δ৔߹ contextରԠ͕ͳ͍৔߹ʹ͸ɺClose() ͢Δ • ྫ͑͹ net.Listener ͷ Accept() ͸ context ΛҾ਺ʹऔΒͳ͍ • Listener ʹ SetDeadline() Λ࢖͏͜ͱͰλΠϜΞ΢τͷઃఆͳΒՄೳ • Listener Λ Close() ͨ͠λΠϛϯάͰAccpet()͕ΞϯϒϩοΫ͞ΕΔ • Close() ͰΞϯϒϩοΫͨ͠ͱ͍͏͜ͱΛ఻͑ΔͨΊʹɺdoneνϟωϧͰ఻͍͑ͯΔ

Slide 68

Slide 68 text

contextରԠ͕ͳ͍৔߹ʹ͸ɺClose() ͢Δ • net.Listener ͷ Accept() ʹ͸ context ରԠ͕ͳ͍ • Listener ʹ SetDeadline() Λ࢖͏͜ͱͰλΠϜΞ΢τͷઃఆͳΒՄೳ • Listener Λ Close() ͨ͠λΠϛϯάͰAccept()͕ΞϯϒϩοΫ͞ΕΔ • Close() ͰΞϯϒϩοΫͨ͠ͱ͍͏͜ͱΛ఻͑ΔͨΊʹɺdoneνϟωϧͰ఻͍͑ͯΔ Close() Λ࢖ͬͯΞϯϒϩοΫͤ͞Δ৔߹

Slide 69

Slide 69 text

ϒϩοΩϯά I/O Λ೚ҙͷλΠϛϯάͰऴ͍ྃͤͨ͞৔߹ net.Conn ͱ os.File ͷΦϒδΣΫτʹ͍ͭͯ • net.Conn ͷ Read ΍ Write ʹ΋ context ͷ࢖͑Δ ݺͼग़͠͸ͳ͍ • Accept() ͱಉ༷ʹ net.Conn ͷ஋Λ Close() ͢Δ ͜ͱʹΑΓ೚ҙͷλΠϛϯάͰΞϯϒϩοΫ • os.File ͷ Read ΍ Write ΋ಉ༷ • os.File Λ Close() ͢Ε͹ΞϯϒϩοΫ͞ΕΔ

Slide 70

Slide 70 text

͜͜·ͰͰϒϩοΩϯά I/O ͷ࿩ʹ͍ͭͯҰ௨Γ͠·ͨ͠ɻ

Slide 71

Slide 71 text

ҙ֎ͱ؆୯ͩ͠ɺॏཁͳ TLSʹ΋ରԠͯ͠ΈΑ͏ʂ

Slide 72

Slide 72 text

TLSରԠ͸೉͘͠ͳ͍ • ॳظԽॲཧΛআ͚͹ɺετϦʔϜܕϓϩτί ϧͱಉ͡ΠϯλϑΣʔε͕࣮૷͞Ε͍ͯΔ • ࠷ॳ͔ΒΠϯλϑΣʔεΛϕʔεʹ։ൃͯ͠ ͍Δͱ؆୯ • ࣮૷ͷखؒ͸গͳ͍ͷͰɺηΩϡΞʹͨ͠ ͍ͱ͖ݕ౼͢΂͖

Slide 73

Slide 73 text

TLSରԠ: αʔόଆ • net.Listen ͷ୅ΘΓʹ tls.Listen Λ࢖͏ • αʔόଆ͸ূ໌ॻͷઃఆ͕ՃΘ Δ͘Β͍Ͱɺ͋ͱ͸ಉ͡Πϯλ ϑΣʔεͰѻ͑Δ αʔόଆͷίʔυྫ

Slide 74

Slide 74 text

• net.Listen ͷ୅ΘΓʹ tls.Listen Λ࢖͏ • αʔόଆ͸ূ໌ॻͷઃఆ͕ՃΘ Δ͘Β͍Ͱɺ͋ͱ͸ಉ͡Πϯλ ϑΣʔεͰѻ͑Δ αʔόଆͷίʔυྫ TLSରԠ: αʔόଆ

Slide 75

Slide 75 text

• net.Listen ͷ୅ΘΓʹ tls.Listen Λ࢖͏ • αʔόଆ͸ূ໌ॻͷઃఆ͕ՃΘ Δ͘Β͍Ͱɺ͋ͱ͸ಉ͡Πϯλ ϑΣʔεͰѻ͑Δ αʔόଆͷίʔυྫ TLSରԠ: αʔόଆ

Slide 76

Slide 76 text

TLSରԠ: ΫϥΠΞϯτଆ • net.Dial ͷ୅ΘΓʹ tls.Dial Λ࢖͏ • ࣗݾೝূہΛ࢖͏৔߹ʹ͸ɺূ໌ॻΛࢦ ఆ͢Δඞཁ͕͋Δ • αʔόূ໌ॻͷݕূΛεΩοϓ͢ΔΦ ϓγϣϯ΋͋Δ͕ɺӡ༻ͰࣄނΔϦε ΫΛߴΊΔͷͰɺݸਓతʹ͸։ൃ؀ڥ ͩͱͯ͠΋εΩοϓ͸Φεεϝ͠ͳ͍ ΫϥΠΞϯτଆ

Slide 77

Slide 77 text

Slide 78

Slide 78 text

Slide 79

Slide 79 text

TLSͰΑΓηΩϡΞʹ • TLSΛ࢖͏ͱͦΕ͚ͩͰ؆୯ʹ௨৴ͷ҉߸Խ͕Ͱ͖Δ͕ɺߟྀ͓͖ͯͨ͠ ͍໰୊΋͋Δ • ௨৴ઌͷαʔό͸ຊ෺ʁ ͳΓ͢·͠͞Ε͍ͯͳ͍ʁ • SymantecύʔτφʔاۀʹΑΔূ໌ॻͷޡൃߦ໰୊ͳͲ • Webϒϥ΢βͳΒ CT (Certiﬁcate Transparency) ΍ HPKP (HTTP Public Key Pinning) ͱ͍͏࢓૊Έ͕͋Δ͕ɺࣗલͷTLSΞϓϦέʔγϣ ϯͰ͸Ͳ͏͢Δʁ • ࣗ෼Ͱಉ͡Α͏ͳ࢓૊ΈΛೖΕΑ͏ʂ

Slide 80

Slide 80 text

ࣗ෼Ͱ Public Key Pinning ݕূ͚ͩ͢ΔΫϥΠΞϯτίʔυ

Slide 81

Slide 81 text

ࣗ෼Ͱ Public Key Pinning ݕূ͚ͩ͢ΔΫϥΠΞϯτίʔυ ΫϥΠΞϯτʹݕূ͍ͨ͠pOHFSQSJOUΛؚΊ͓ͯ͘

Slide 82

Slide 82 text

Public Key Pinning ݕূ͚ͩ͢ΔΫϥΠΞϯτίʔυ %JBM ͷ࣌ʹ$POpHͰݕূؔ਺Λࢦఆ͢Δ

Slide 83

Slide 83 text

Public Key Pinning ݕূ͚ͩ͢ΔΫϥΠΞϯτίʔυ w ઀ଓʹ࢖ΘΕ͍ͯΔূ໌ॻͷpOHFSQSJOUΛܭࢉ w Ұக͍ͯ͠Δ͔ݕূ

Slide 84

Slide 84 text

TLSରԠ • TLS΋ετϦʔϜܕͰ͸͋ΔͷͰɺ net.Conn ΠϯλϑΣʔεΛ࢖͑Δ • Public Key Pinning ͘Β͍͸ࣗ෼Ͱ࣮૷Ͱ͖Δ ͘Β͍ʹඪ४ϥΠϒϥϦͷॊೈੑ͸͋Δ

Slide 85

Slide 85 text

·ͱΊ

Slide 86

Slide 86 text

ΠϯλϑΣʔεΛ࢖ͬͨϓϩάϥϛϯά • GoͷιέοτϓϩάϥϛϯάͰ͸ɺந৅౓ͷ ߴ͍ΠϯλϑΣʔε͕࢖͑Δ

Slide 87

Slide 87 text

I/O ଟॏԽͷ࢓૊Έ • GoϥϯλΠϜ͸ netpoll ͱ͍͏ػೳͰ ϒ ϩοΩϯά I/O ΛଟॏԽ͍ͯ͠Δ

Slide 88

Slide 88 text

ϒϩοΩϯά I/O ͷΞϯϒϩοΫ • context.Context Λ࢖͏৔߹ • Close() Λ͢Δ৔߹

Slide 89

Slide 89 text

TLSରԠͯ͠ΈΑ͏ • ূ໌ॻΛࢦఆΛ͢Δ͙Β͍Ͱɺ͋ͱ͸ετϦʔ ϜܕͱΠϯλϑΣʔε͕ಉ͡ͳͷͰɺ͓खܰʂ • TLSΛ࢖ͬͯ΋ηΩϡϦςΟ͕֬อ͞ΕΔΘ͚ Ͱ͸ͳ͍ • Web ͡Όͳͯ͘΋ Public Key Pinning ͸؆ ୯ʹ࣮ࢪͰ͖Δ

Slide 90

Slide 90 text

Thanks!