Web サービスの信頼性を守るための取り組み / jtf-2017-site-reliability-engineering

Slide 1

Slide 1 text

Web αʔϏεͷ৴པੑΛ कΔͨΊͷऔΓ૊Έ ΫοΫύουגࣜձࣾ ΠϯϑϥετϥΫνϟʔ෦ SRE άϧʔϓ ٢઒ ཽଠ ( @rrreeeyyy ) July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 1

Slide 2

Slide 2 text

Agenda • Site Reliability Engineering (SRE) ͱ͸ • Ұൠతͳ SRE ͷ׆ಈʹ͍ͭͯ • ΫοΫύουʹ͓͚Δ SRE ͷ׆ಈʹ͍ͭͯ • JTF2017 ͷςʔϚʮIT ΤϯδχΞϦϯάͷຊ࣭ʯ • SRE ʹ͓͚ΔʮΤϯδχΞϦϯάʯͱ͸Կ͔ʁ • SRE ͷจ຺͔Β IT ΤϯδχΞϦϯάͷຊ࣭Λߟ͑Δ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 2

Slide 3

Slide 3 text

!SSSFFFZZZ !SSSFFFZZZ IUUQTSSSFFFZZZDPN :PTIJLBXB3ZPUB Me • Yoshikawa Ryota ( @rrreeeyyy [reɪ] ) • ΫοΫύουגࣜձࣾ (2017/01 ʙ) • ΠϯϑϥετϥΫνϟʔ෦ SRE άϧʔϓ • ڵຯྖҬ • ϞχλϦϯάɾ࣌ܥྻσʔλϕʔε • ෼ࢄγεςϜɾϩʔυόϥϯα • झຯ • League of Legends, ΀Α΀ΑΫϩχΫϧ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 3

Slide 4

Slide 4 text

Slide 5

Slide 5 text

July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 5

Slide 6

Slide 6 text

Site Reliability Engineering ͱ͸ • αΠτͷ৴པੑͷશͯʹ੹೚ൣғΛ࣋ͭ • ಛʹɺιϑτ΢ΣΞɾγεςϜΤϯδχΞϦϯάΛ༻͍ͯ՝୊Λղܾ͢Δ • ྫ͑͹ɺύϑΥʔϚϯε • ύϑΥʔϚϯεʹ໰୊͕͋Δίʔυ͸ SRE ࣗ਎͕ίʔυΛॻ͍ͯमਖ਼͢Δ • ྫ͑͹ɺεέʔϥϏϦςΟɾΞϕΠϥϏϦςΟ • ਓؒʹΑΔखಈͷ࡞ۀΛίʔυʹΑΓࣗಈԽ͢Δ • ଐਓੑΛഉআͯ҆͠ఆͤ͞ΔɾਓؒΛ૿΍͞ͳͯ͘΋αʔϏεΛεέʔϧͤ͞Δ • ن໛͕૿େͯ͠΋҆ఆͯ͠αʔϏεΛఏڙͰ͖Δঢ়ଶʹ͢Δ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 6

Slide 7

Slide 7 text

SRE ͷ׆ಈ SRE ͷ׆ಈ͸େ͖͘෼͚ͯ࣍ͷ 4 ͭʹ෼ྨͰ͖Δͱ͞Ε͍ͯΔ • So#ware Engineering • Systems Engineering • Toil • Overhead July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 7

Slide 8

Slide 8 text

SRE ͷ׆ಈ • So#ware Engineering • ίʔυͷ࡞੒΍मਖ਼, ͦΕʹؔ࿈͢Δઃܭ΍υΩϡϝϯςʔγϣϯͷ࡞ۀͳͲ • ࣗಈԽεΫϦϓτͷ࡞੒, πʔϧ΍ϑϨʔϜϫʔΫͷ࡞੒ ... • αʔϏε΁ͷ৴པੑΛߴΊΔػೳͷ௥Ճ ... • Systems Engineering • 1 ճͷ࡞ۀͰӬଓతͳվળΛੜΉ࣍ͷΑ͏ͳ࡞ۀ • ຊ൪γεςϜͷઃఆมߋ ... • ։ൃνʔϜͷαʔϏεΞʔΩςΫνϟઃܭ ... July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 8

Slide 9

Slide 9 text

SRE ͷ׆ಈ • Toil • αʔϏεΛՔಇͤ͞ΔͨΊ͚ͩʹ܁Γฦ͠ߦΘΕΔखಈͰͷ࡞ۀ • ௕ظతͳՁ஋Λ࣋ͨͳ͍, αʔϏεͷ੒௕ʹରͯ͠ O(n) Ͱ͋Δ ... • ͭ·Βͳ͍࢓ࣄͰ΋௕ظతͳՁ஋͕͋Ε͹ Toil Ͱ͸ͳ͍ • Overhead • αʔϏεΛՔಇͤ͞Δ͜ͱʹ௚݁͠ͳ͍؅ཧతͳ࡞ۀ • ࠾༻, ϛʔςΟϯά, ධՁ, τϨʔχϯά ... July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 9

Slide 10

Slide 10 text

ΫοΫύουʹ͓͚Δ SRE ͷ׆ಈ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 10

Slide 11

Slide 11 text

ΫοΫύουʹ͓͚Δ SRE ͷ׆ಈ • SRE ຊʹ͋ΔΑ͏ͳ׆ಈΛΫοΫύου಺Ͱ΋༷ʑߦ͍ͬͯΔ • ΫοΫύουͷ SRE ͷ׆ಈΛ͍͔ͭ͘঺հ • ৽نαʔϏεʹର͢Δ SRE ͷಈ͖ • ϞχλϦϯάͷվળ • ϩʔυόϥϯγϯάͷվળ • ৽ଔݚम • ... July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 11

Slide 12

Slide 12 text

৽نαʔϏεʹର͢Δ SRE ͷಈ͖ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 12

Slide 13

Slide 13 text

৽نαʔϏεʹର͢Δ SRE ͷಈ͖ • େ͖Ίͳػೳ͸αʔϏε୯ҐͰ෼ׂͯ͠։ൃ͢Δ • ৔߹ʹ΋ΑΔ͕ SRE ͕ 1 ਓΞαΠϯ͞ΕΔ • ౰֘ͷαʔϏεͷ৴པੑʹؔ͢Δ͜ͱ͸جຊతʹશ෦΍Δ • ઃܭͷ૬ஊɾϨϏϡʔ • ϦϦʔεલͷෛՙࢼݧɾΩϟύγςΟϓϥϯχϯά • ύϑΥʔϚϯεʹର͢ΔΞϓϩʔν • ... July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 13

Slide 14

Slide 14 text

ઃܭͷ૬ஊɾϨϏϡʔ • ։ൃͷલʹҎԼͷΑ͏ͳ͜ͱΛڞ༗ͯ͠΋Β͍ٞ࿦͢Δ • ͲͷΑ͏ͳΞϓϦέʔγϣϯΛ࡞Δ͔ • ͲͷΑ͏ͳίϯϙʔωϯτ͕ඞཁ͔ • ͲͷΑ͏ͳߏ੒ɾεέʔϧઓུʹ͢Δ͔ • Design Document Λॻ͍ͯ΋Β͍ͦΕΛϕʔεʹ࿩Λͨ͠Γ͢Δ • େ͖ΊͷػೳͳͲΛ։ൃ͢Δ࣌͸ Design Document Λॻ͘͜ͱ͕ଟ͍ (SRE ΋) July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 14

Slide 15

Slide 15 text

ෛՙࢼݧ • ͋Δఔ౓ग़དྷ͖ͯͨลΓͰෛՙࢼݧͷ࿩Λ͢Δ • Ͳͷ͘Β͍ͷϦΫΤετ͕૝ఆ͞ΕΔ͔ • ͲͷΑ͏ͳϢʔβϦΫΤετ͕૝ఆ͞ΕΔ͔ • SRE ଆͰෛՙࢼݧͷγφϦΦ΍؀ڥΛ࡞Δ • ෛՙࢼݧΛߦ͍ͳ͕Βຊ൪؀ڥͰͷΩϟύγςΟΛݟੵ΋Δ • ΞϓϦέʔγϣϯ্ͷϘτϧωοΫ͕Ͳ͜ʹ͋Δ͔ΛݟΔ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 15

Slide 16

Slide 16 text

July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 16

Slide 17

Slide 17 text

ύϑΥʔϚϯεʹର͢ΔΞϓϩʔν • ৴པੑʹؔ͢Δ໰୊͕͋Δ৔߹͸ࣗ෼ͰίʔυΛॻ͘ • ྫ͑͹ύϑΥʔϚϯε্ͷ໰୊ͳͲ • ΞϓϦέʔγϣϯͷ࢓༷Λཧղͯ͠ద੾ͳΞϓϩʔνΛऔΔ • ྫ͑͹Ͳ͏͍͏ը໘ͷͲ͏͍͏ΫΤϦͳͷ͔ • Ͳ͏͍͏৘ใ͕ཉͯ͘͠Ͳ͏͍͏੍໿͕͋Δͷ͔ • ࢖͍ͬͯΔϛυϧ΢ΣΞͷ࢓༷΋ཧղͯ͠ద੾ͳΞϓϩʔνΛऔΔ • ྫ͑͹ SQL ͷվળͳΒ MySQL ͷΠϯσοΫεͷߏ଄΍಺෦ॲཧͷ஌͕ࣝ͋Δͱྑ͍ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 17

Slide 18

Slide 18 text

July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 18

Slide 19

Slide 19 text

৽نαʔϏεʹର͢Δ SRE ͷಈ͖ • ͜ͷลΓ͸౰ͨΓલͷࣄΛී௨ʹ΍͍ͬͯΔ͚ͩ • ΋ͬͱվળग़དྷΔՕॴ͕ͨ͘͞Μ͋Δͱߟ͍͑ͯΔ • ྫ͑͹ෛՙࢼݧΛࣗಈͰߦ͏Α͏ʹ͢ΔɺͳͲ • ΞϓϦέʔγϣϯΛཧղ͠ɺඞཁͳΒ͹ίʔυΛॻ͘ • SRE ͸αΠτͷ৴པੑʹؔΘΔશͯʹ੹຿Λෛ͏ • ΞϓϦέʔγϣϯͷಛੑ΍ઃܭΛཧղ͠ϛυϧ΢ΣΞΛ૊Έ߹ΘͤΔ Systems Engineering • ࢓༷΍ϛυϧ΢ΣΞΛཧղ࣮͠ࡍʹΞϓϦέʔγϣϯͷϩδοΫΛॻ͘ So.ware Engineering July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 19

Slide 20

Slide 20 text

ϞχλϦϯάͷվળ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 20

Slide 21

Slide 21 text

ϞχλϦϯάͷվળ • ݱࡏ͸ϝτϦΫεऔಘʹ Zabbix Λ࢖༻͍ͯ͠Δ • ΑΓεέʔϧ͠ɺߴڃݴޠͰϝτϦΫε΍ΞϥʔτΛѻ͑ΔΑ͏ʹ͍ͨ͠ • SRE ຊ ͷ 10 ষʹॻ͔Ε͍ͯΔΑ͏ͳঢ়ଶʹͳΓ͍ͨ • Prometheus ͷಋೖΛঃʑʹ࣮ࢪ͍ͯ͠Δ • ϝτϦΫεͷղ૾౓΋ 15 ඵ୯ҐͳͲͰऔಘ͢ΔΑ͏ʹ͍ͯ͠Δ • ୯७ͳ HTTP ϨεϙϯεͰϝτϦΫεΛऩूग़དྷΔ • ΞϓϦέʔγϣϯࣗମͷϝτϦΫεͳͲ͕ൺֱత؆୯ʹऩूग़དྷΔ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 21

Slide 22

Slide 22 text

July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 22

Slide 23

Slide 23 text

July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 23

Slide 24

Slide 24 text

July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 24

Slide 25

Slide 25 text

ϞχλϦϯάͷվળ • Ͳ͏͍ͬͨϛυϧ΢ΣΞΛબఆ͢Δ͔ • ϞχλϦϯάͰ͋Ε͹࣌ܥྻσʔλϕʔεͳͲʹؔ͢Δਂ͍ཧղΛ͔ͯ͠ΒબͿ • Ͳ͏͍ͬͨಛੑ͕͋Δ͔ʁࣗ෼ͨͪͷن໛ʹ߹͍ͬͯΔ͔ʁεέʔϧ͢Δ͔ʁ • ଍Γͳ͍ίϯϙʔωϯτ͸ͳ͍͔ʁແ͍ͳΒࣗ࡞Ͱ͖ͦ͏ͳن໛͔ʁ • Prometheus ͳΒྫ͑͹ Long-Term Storage पΓͷίϯϙʔωϯτͳͲ • ͔ͬ͠Γͱͨ͠ Systems Engineering ͷ஌ݟΛར༻ͯ͠બఆ͢Δ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 25

Slide 26

Slide 26 text

July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 26

Slide 27

Slide 27 text

ϩʔυόϥϯγϯάͷվળ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 27

Slide 28

Slide 28 text

ϩʔυόϥϯγϯάͷվળ • ैདྷ͸ΞϓϦέʔγϣϯαʔόͷϩʔυόϥϯγϯάʹ haproxy Λར༻͍ͯͨ͠ • Weighted Round-robin (wrr) Λར༻ͯ͠όϥϯγϯάΛߦ͍ͬͯͨ • ෛՙͷ௿͍ΤϯυϙΠϯτͱߴ͍ΤϯυϙΠϯτ͕͋Δ৔߹ෛՙ͕͹Βͭ͘ • wrr ͩͱಛఆͷαʔόʹෛՙͷߴ͍ΤϯυϙΠϯτ͕ूத͢ΔՄೳੑ͕͋Δ • leastconn Λ࢖͑͹ྑͦ͞͏͕ͩ HTTP keepalive ͍ͯ͠Δ৔߹ʹ໰୊͕͋Δ • ίωΫγϣϯ͕࢒ΔͷͰ࣮ࡍʹॲཧ͍ͯ͠ΔϦΫΤετ਺ͱണ཭͢Δɹ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 28

Slide 29

Slide 29 text

ryotarai/simproxy • h#ps:/ /github.com/ryotarai/simproxy • Go Ͱॻ͔ΕͨγϯϓϧͳϦόʔεϓϩΩγ • ϔϧενΣοΫͳͲͷجຊతͳػೳ͸͋Δ • Balancing Method ʹ leastreq Λ࣮૷͍ͯ͠Δ • ࢦඪͱͯ͠୯७ͳίωΫγϣϯ਺Ͱ͸ͳ࣍͘ͷΑ͏ͳ஋Λར༻͢Δ • όοΫΤϯυ͕ड͚औ͕ͬͨԠ౴͕ฦ͖͍ͬͯͯͳ͍ϦΫΤετͷ਺ • ͜ΕʹΑΓ HTTP keepalive ͕͋ͬͯ΋ leastconn ͷΑ͏ͳόϥϯγϯά͕Մೳ • ϩʔυόϥϯγϯάͷΞϧΰϦζϜ౳ʹؔ͢Δ Systems Engineering ͱ࣮૷͢Δ So.ware Engineering July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 29

Slide 30

Slide 30 text

before simproxy July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 30

Slide 31

Slide 31 text

a"er simproxy July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 31

Slide 32

Slide 32 text

৽ଔݚम • ৽ଔʹରͯ͠ΠϯϑϥͷݚमΛ 3 ೔ؒߦ͍ͬͯΔ • MySQL ͷΠϯσοΫεͷߏ଄Λཧղ͠ΫΤϦνϡʔχϯάΛߦ͏ ... • AWS ͷ ALB ΍ RDS ͳͲΛ࢖͍ࣗ෼ͷΞϓϦέʔγϣϯΛεέʔϧͤ͞Δ ... • ࣮ࡍͷΞϓϦέʔγϣϯΛॻ͘ϝϯόʔ͕Πϯϑϥͷཧղ͕͋Δํ͕ྑ͍ • શͯͷΞϓϦέʔγϣϯίʔυΛ SRE ͕νΣοΫ͍ͯ͠Δͱεέʔϧ͠ͳ͍ • ෼ྨ্͸ Overhead ͔΋͠Εͳ͍͕αʔϏεΛεέʔϧͤ͞ΔͨΊʹ͸ඞཁ • ࣮ࡍʹ͸ Systems Engineering ͷਂ͍஌ݟ͕ແ͍ͱڭ͑Δ͜ͱ͕ग़དྷͳ͍ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 32

Slide 33

Slide 33 text

July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 33

Slide 34

Slide 34 text

July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 34

Slide 35

Slide 35 text

ࠓճͷ JTF ͷςʔϚ ʮITΤϯδχΞϦϯάͷຊ࣭ʯ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 35

Slide 36

Slide 36 text

ࠓճͷ JTF ͷςʔϚ ʮITΤϯδχΞϦϯάͷຊ࣭ʯ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 36

Slide 37

Slide 37 text

SRE ʹ͓͚Δ "ΤϯδχΞϦϯά" • The work of reducing toil and scaling up services is the "Engineering" in Site Reliability Engineering. • Engineering work is novel and intrinsically requires human judgment. It produces a permanent improvement in your service, and is guided by a strategy. It is frequently creaBve and innovaBve, taking a design-driven approach to solving a problem—the more generalized, the beFer. — Site Reliability Engineering, Chapter 5 - EliminaBng Toil July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 37

Slide 38

Slide 38 text

SRE ʹ͓͚Δ "ΤϯδχΞϦϯά" • Toil ΛݮΒ͠αʔϏεΛεέʔϧͤ͞Δ͜ͱ͕ͦ͜ SRE ͷ "ΤϯδχΞϦϯά" • ΤϯδχΞϦϯάͷ࡞ۀ͸... • ৽͍͜͠ͱΛ͢Δ΋ͷͰ͋Δ • ຊ࣭తʹਓؒͷ൑அΛඞཁͱ͢Δ • αʔϏεʹ߃ٱతͳվળΛ༩͑Δ • ઓུʹΑͬͯಋ͔ΕΔ • ΫϦΤΠςΟϒ͔ͭΠϊϕʔςΟϒͰ͋Δ • ໰୊Λղܾ͢ΔͨΊʹઃܭओಋͷΞϓϩʔνΛऔΔ • ൚༻ੑ͕͋Δ΄Ͳ༏Ε͍ͯΔ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 38

Slide 39

Slide 39 text

ࣗ෼͕ߟ͑Δ IT ΤϯδχΞϦϯάͷຊ࣭ • IT ΤϯδχΞϦϯά͸޻ֶͷதͰ΋ಛʹʮ৘ใʯΛऔΓѻ͏ • ৘ใͱ͸(weblioࣙॻΑΓ): • ࣄ෺ɾग़དྷࣄͳͲͷ಺༰ɾ༷ࢠɻ·ͨɼͦͷ஌Βͤɻ • ػցܥ΍ੜମܥʹ༩͑ΒΕΔࢦྩ΍৴߸ɻ • ͋Δಛఆͷ໨తʹ͍ͭͯɼద੾ͳ൑அΛԼͨ͠Γɼߦಈͷҙ ࢥܾఆΛ͢ΔͨΊʹ໾ཱͭࢿྉ΍஌ࣝɻ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 39

Slide 40

Slide 40 text

ࣗ෼͕ߟ͑Δ IT ΤϯδχΞϦϯάͷຊ࣭ • Web αʔϏεͳͲͷ৘ใ఻ୡͷखஈΛ࢖ͬͯϢʔβʹՁ஋Λఏڙ͢Δ • ྫ͑͹ɺશੈքʹ͓͚ΔϨγϐͷڞ༗ͳͲ • ػցʹ༷ʑͳࢦྩ΍৴߸Λ༩͑ɺਓؒͷ࡞ۀΛޮ཰Խͤ͞Δ • ྫ͑͹ɺϓϩάϥϛϯάʹΑΔࣗಈԽ΍༏ΕͨઃܭʹΑΔޮ཰Խ • → ৘ใ఻ୡ΍ਓؒͷࢥߟ΍൑அΛػցʹߦΘͤΔ͜ͱʹΑͬͯՁ஋Λग़͢ • SRE ͱ͍͏จԽ͸ैདྷΑΓࣗ෼ͷߟ͑Δ͜ͷຊ࣭ʹ͍ۙͱࢥ͍ͬͯΔ • ߴ౓ͳࣗಈԽɺੜମͷΑ͏ʹࣗಈͰಈ࡞͢ΔγεςϜ͸ࣗ෼ͷڵຯྖҬ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 40

Slide 41

Slide 41 text

·ͱΊ • Ұൠతͳ SRE ͷ׆ಈͱΫοΫύουʹ͓͚Δ SRE ͷऔΓ૊ΈΛ঺հ • Ұൠతͳ SRE ͷ׆ಈ͸ So'ware, Systems Engineering Λओ࣠ʹ Toil, Overhead ͕͋Δ • SRE ຊʹ΋͋ΔΑ͏ͳΤοηϯεΛऔΓೖΕͭͭࣗ෼ͨͪͷ໨తɾن໛ʹ͋ͬͨΤϯ δχΞϦϯάΛΫοΫύουͰ͸΍͍ͬͯΔ • SRE ʹ͓͚ΔΤϯδχΞϦϯάͱɺࣗ෼͕ߟ͑Δ IT ΤϯδχΞϦϯάͷຊ࣭ʹ͍ͭͯ • Toil ΛݮΒ͠αʔϏεΛεέʔϧͤ͞Δ͜ͱ • ৘ใ఻ୡ΍ਓؒͷࢥߟ΍൑அΛػցʹޮ཰తʹߦΘͤΔ͜ͱͰՁ஋Λग़͢͜ͱ July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 41

Slide 42

Slide 42 text

PR July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 42

Slide 43

Slide 43 text

We are hiring! • ΫοΫύουͰ͸ຊ೔঺հͨ͠Α͏ͳऔΓ૊ΈΛߦ͍ͬͯ·͢ • ·ͩ·ͩղܾ͍ͨ͠໰୊ɾऔΓ૊Έ͍ͨ՝୊͕ͨ͘͞Μ͋Δ • ָ͠Έͳ͕ΒҰॹʹ΍ΕΔϝϯόʔΛืू͍ͯ͠·͢ → h$ps:/ /cookpad.jobs July Tech Festa 2017 (2017/08/27) | Yoshikawa Ryota ( @rrreeeyyy ) 43