SRE NEXT 2023 https://sre-next.dev/2023/schedule/#jp011
։ൃऀͱͱʹ࡞ΔSite Reliability EngineeringTakeshi Kondo / @chaspy2023/09/29SRE NEXT 2023
View Slide
Who am Ichaspy chaspy_Senior Engineering Manager (10/1~)StudySapuri K12at Recruit Co., Ltd.SRE Team ͷϚωʔδϟͱͯ͠ࠓ͕࠷ऴͰ͢👋Takeshi Kondohttps://chaspy.me
ʰελσΟαϓϦʱখதߴ(*)ͷ։ൃΛ͍ͯ͠·͢ *খֶɾதֶɾߴߍɾେֶडݧߨ࠲ͷུͰ͢
SRE NEXT օۈͰ͢ʂ͍ͭ͋Γ͕ͱ͏͍͟͝·͢ʂ
ࠓ͢͜ͱSRE Λ։ൃऀͱ”ͱʹ”࣮ݱ͢ΔͨΊͷࣄྫ
Key MessagesSREϓϥΫςΟεͱ৫จԽ
·ͱΊ• લఏ: SREing ͷ࣮ݱ = ։ൃऀ͕৴པੑΛίϯτϩʔϧͰ͖͍ͯΔ͜ͱ• SREs ։ൃऀΛ Enabling / Platform ྆໘Ͱαϙʔτ͢Δ• ͦͷͨΊʹ SREs ։ൃऀͷཁٻΛਖ਼͘͠ཧղ͢Δඞཁ͕͋Δ• ࣄྫհ: SRE ͕։ൃऀͷཁٻΛਖ਼͘͠ཧղ͢ΔͨΊͷΞϓϩʔν• 1. ϑΟʔυόοΫΛಘΔ• 2. ίϥϘϨʔγϣϯ͢Δ• 3. ࣮ࡍʹମݧ͢Δ• ͜ΕΒΛԼࢧ͑͢Δ৫จԽ• ৼΓฦΓ / ϑΟʔυόοΫ / ઓ
Agenda• ͜Ε·Ͱͷ͋Β͢͡• SRE νʔϜ͕ࢦ͍ͯͨ͜͠ͱ• ࣄྫհ• ߟ• ·ͱΊ
SRE NEXT 2020/2022 2020 / SLI/SLO ͱ͍͏ݴ༿͕ͳ͍ঢ়ଶ͔Βٕज़ɾจԽ྆໘Ͱܒͨ͠ࣄྫ2022 / Error Budget ӡ༻ͷͨΊʹͦͷͨΊͷ༧ࢉͱܭըΛߦ͏ٕज़ઓུ͕ඞཁͰ͋Δͱ͍͏ߟ
SRE NEXT 2022 ͷ·ͱΊεϥΠυΑΓ
SRE Team at StudySapuri K12• 7໊Ͱߏ• ର৫ʰελσΟαϓϦʱখதߴͱւ֎ Quipper• αϙʔτ͢Δ Developer TPM, Native Devs ؚΉͱ100໊Ҏ্• ৴པੑͷͨΊʹ։ൃऀΛ Enabling ͢Δۀͱɺ։ൃੜ࢈ੑͷͨΊͷ Platform Λ࡞Δۀ྆ํΛߦ͏• Team Topology ༻ޠͰ͢
ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱSRE ~ @chaspy ೖࣾޙ• 2018: @chaspy ೖࣾ• 2019: Application Platform Λ Kubernetes Ҡ• 2020: Microservices Readiness ͷඋ• αʔϏεΦʔφʔγοϓͷࡦఆ• Design Doc / Production Readiness Checklist• Self-services Infrastructure (terraform monorepo)• SLI/SLO• 2021: SLI/SLO ӡ༻Λ։ൃνʔϜʹશҠৡPlatform Team ͱͯ͠ Platform Λ࡞͍ͬͯΔEnabling Team ͱͯ͠։ൃ৫ʹ SLI/SLOͳͲͷΧϧνϟʔৢ
ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱSRE ~ 2021• COVID-19 ྲྀߦɺΞΫηε૿େ• Platform ͷਐԽ• Terraform monorepo• Loadtest Platform• GitHub Actions ʹΑΔ monorepo CI • ৫ͷมԽ• ٕज़ઓུάϧʔϓൃ• ࣄۀҠʹΑΓϦΫϧʔτస੶ɺQuipper ຊࢧళਗ਼ࢉ• chaspy EM ༻
ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱSRE ~ 2022• ৽͍͠ SRE ࣮ફͷܗ• Partially Embedded SRE• Partially Enabling SRE in development team
ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱSRE ~ 2023• ʮ։ൃऀ͔ΒϑΟʔυόοΫΛಘΔʯ͜ͱʹϑΥʔΧε• ͜ͷޙ͠·͢ʂ
SRE νʔϜ͕ࢦ͍ͯͨ͜͠ͱ• ՝: SRE νʔϜ͕Δ͜ͱ SRE νʔϜ͕ܾΊ͍ͯͨ• దͳ՝ઃఆ͕Ͱ͖͍ͯͳ͍Մೳੑ͕͋ͬͨ• ࠶ݱੑʹ͚͍ܽͯͨ / ଐਓੑ͕͋ͬͨ• ։ൃऀ͔ΒϑΟʔυόοΫΛಘΔ͜ͱʹྗ• ෳͷΞϓϩʔνΛࢼͨ͠• ׆ಈΛਐΊΔ͏ͪʹɺ3ͭͷύλʔϯ͕͋Δ͜ͱʹؾ͍ͮͨ
ࣄྫհ• 1. ϑΟʔυόοΫΛಘΔ• 2. ίϥϘϨʔγϣϯ͢Δ• 3. ࣮ࡍʹମݧ͢Δ
1. ϑΟʔυόοΫΛಘΔ• SRE αʔϕΠͷ࣮ࢪ• ʮ݄ץSREʯ- Update ͷڞ༗• ظؒΛܾΊͯνʔϜʹೖΔ (Embedding)
SRE αʔϕΠͷ࣮ࢪ• Google form Ͱ࣮ࢪ / ʹ1 / ղ60%ఔ• ࣭༰• νʔϜ͝ͱͷࣗݾ݁ / πʔϧ SaaS ͷशख़• Կ͕Ͱ͖͍ͯΕαʔϏεΛࣗͨͪͰӡ༻Ͱ͖͍ͯΔɺͱݴ͑·͔͢ʁ• ΑΓϓϩμΫτΛྑ͍ͯͨ͘͘͠ΊʹɺԿ͕ϘτϧωοΫͰ͔͢ʁ• ͠ɺ࠷ۙ໘ͨ͠SREʹؔ࿈ͦ͠͏ͳ՝/͕͋Εڭ͍͑ͯͩ͘͞• ීஈͷ։ൃͷதͰ࠷ετϨεΛײ͍ͯ͡ΔఔɾखॱԿͰ͔͢ʁ• ։ൃͷఔͷதͰɺ͜Ε͕͘ͳͬͨΒخ͍͠ͱ͍͏෦͋Γ·͔͢ʁ• ࣗͨͪͷαʔϏεʹ͓͍ͯɺෛ࠴ͩͱײ͍ͯ͡Δ / ফ͠ڈΓ͍ͨཁૉԿ͔͋Γ·͔͢?• ݱࡏͷ(SRE͕։ൃ/ӡ༻͍ͯ͠Δ)ϓϥοτϑΥʔϜͰ͔Βͳ͍͜ͱ͕͋ͬͨΒɺԿΛݟ·͔͢?• SREͷରԠऔΓΈʹ͍ͭͯͷཁ/ίϝϯτ͕͋Ε͓ئ͍͠·͢• SREͷ͍߹ΘͤରԠʹ͍ͭͯཁ/ίϝϯτ͕͋Ε͓ئ͍͠·͢• SREνʔϜ͔Βґཔ͕͋Δͱ͖ɺͲͷΑ͏ͳܗͰ࿈བྷ/ґཔ͞ΕΔͱྑ͍Ͱ͔͢?
SRE αʔϕΠͷ࣮ࢪ• Google form Ͱ࣮ࢪ / ʹ1• ࣭༰• νʔϜ͝ͱͷࣗݾ݁ / πʔϧ SaaS ͷशख़• Կ͕Ͱ͖͍ͯΕαʔϏεΛࣗͨͪͰӡ༻Ͱ͖͍ͯΔɺͱݴ͑·͔͢ʁ• ΑΓϓϩμΫτΛྑ͍ͯͨ͘͘͠ΊʹɺԿ͕ϘτϧωοΫͰ͔͢ʁ• ͠ɺ࠷ۙ໘ͨ͠SREʹؔ࿈ͦ͠͏ͳ՝/͕͋Εڭ͍͑ͯͩ͘͞• ීஈͷ։ൃͷதͰ࠷ετϨεΛײ͍ͯ͡ΔఔɾखॱԿͰ͔͢ʁ• ։ൃͷఔͷதͰɺ͜Ε͕͘ͳͬͨΒخ͍͠ͱ͍͏෦͋Γ·͔͢ʁ• ࣗͨͪͷαʔϏεʹ͓͍ͯɺෛ࠴ͩͱײ͍ͯ͡Δ / ফ͠ڈΓ͍ͨཁૉԿ͔͋Γ·͔͢?• ݱࡏͷ(SRE͕։ൃ/ӡ༻͍ͯ͠Δ)ϓϥοτϑΥʔϜͰ͔Βͳ͍͜ͱ͕͋ͬͨΒɺԿΛݟ·͔͢?• SREͷରԠऔΓΈʹ͍ͭͯͷཁ/ίϝϯτ͕͋Ε͓ئ͍͠·͢• SREͷ͍߹ΘͤରԠʹ͍ͭͯཁ/ίϝϯτ͕͋Ε͓ئ͍͠·͢• SREνʔϜ͔Βґཔ͕͋Δͱ͖ɺͲͷΑ͏ͳܗͰ࿈བྷ/ґཔ͞ΕΔͱྑ͍Ͱ͔͢?शख़ΛνʔϜผʹͰूܭ։ൃੜ࢈ੑΛ્͢ΔཁҼΛώΞϦϯάSRE ͷࠒͷৼΔ͍ʹ͍ͭͯϑΟʔυόοΫΛಘΔ
νʔϜ͝ͱͷࣗݾ݁ / πʔϧ SaaS ͷशख़ओ؍ɾఆੑධՁͯ͠Β͏
νʔϜ͝ͱͷࣗݾ݁ / πʔϧ SaaS ͷशख़eNPS ͷΑ͏ʹมԽΛݟΔͨΊʹ͏͜ͱΛఆ
νʔϜ͝ͱͷࣗݾ݁ / πʔϧ SaaS ͷशख़0 - શ͘Θ͔Βͳ͍1 - ͍ͬͯΔ2- ͳΜͱ͔͏͜ͱ͕Ͱ͖Δ3- ͏͜ͱ͕Ͱ͖Δ4- มߋ͢Δ͜ͱ͕Ͱ͖Δ5- શʹཧղ͍ͯ͠Δ
ΑΓϓϩμΫτΛྑ͍ͯͨ͘͘͠ΊʹɺԿ͕ϘτϧωοΫͰ͔͢ʁϦιʔεআʹ՝͋Γͦ͏
ΑΓϓϩμΫτΛྑ͍ͯͨ͘͘͠ΊʹɺԿ͕ϘτϧωοΫͰ͔͢ʁCI ͷ࣌ؒɺ҆ఆੑϩʔϧόοΫͷ࣌ؒʹ՝͕͋Γͦ͏
SREͷରԠऔΓΈʹ͍ͭͯͷཁ/ίϝϯτ͕͋Ε͓ئ͍͠·͢ຬߴͦ͏😌
SREͷରԠऔΓΈʹ͍ͭͯͷཁ/ίϝϯτ͕͋Ε͓ئ͍͠·͓͢ޓ͍ͷظʹ՝͕͋Γͦ͏
αʔϕΠ݁Ռ͔ΒɺΞΫγϣϯ࣮ࢪ• ճʹ͚ͯ SRE Team ͰαʔϕΠ݁ՌΛோΊͯٞͨ͠• ·ͣൃࢄతʹײΛهࡌ• ՝ʹͳΓͦ͏ͳཁૉΛநग़͠ vote• ࣮ࢪͨ͠ΞΫγϣϯ• Terraform CI/CD վળ• SRE ͔Βݟͨ։ൃऀͷظΛ໌ࣔ
SRE ͔Βݟͨ։ൃऀͷظΛ໌ࣔ - ࣗݾ݁νʔϜʹ͍ͭͯલఏͱͯ͠ɺSRE ͕ࢦࣗ͢ݾ݁ͳ৫ʹ͍ͭͯઆ໌
SRE ͔Βݟͨ։ൃऀͷظΛ໌ࣔ - ࣗݾ݁νʔϜͷԸܙͱ
SRE ͔Βݟͨ։ൃऀͷظΛ໌ࣔରʹରͯ͠ઢΛҾ͘ͷͰͳ͘ɺߟ͑ํΛఏࣔ
݄ץSRE• ຖिՐ༵”random-tech-talk”ͱ͍͏ࣗ༝ʹࡶʹٕज़తͳΛ͢Δ͕͋ΓɺͳΜͱͳ͘։ൃऀ͕ू·͍ͬͯΔ• ݄ʹҰ SRE ͔Β Update Λൃ৴͠ɺϑΟʔυόοΫΛΒ͍ͬͯΔ
݄ץSRE
݄ץSRE - Envoy ഇࢭʹΑΔ Metrics มߋͷΞφϯε
݄ץSRE - AWS Profile ߋ৽ͷґཔ
ظؒΛܾΊͯνʔϜʹೖΔ(Embedding)Πϯϑϥʹ͍ͭͯνʔϜͷ capability͕ෆ͍ͯ͠ΔͨΊࢧԉͷґཔʢ͜͏͍͏ίϛϡχέʔγϣϯΛͯ͘͠Εͨ͜ͱࣗମ͕خ͍͠ʣ͍ۙϛογϣϯΛ͍࣋ͬͯͯɺಘҙͳϝϯόʔ͕͍ͨͷͰ͓ئ͍ͨ͠
ظؒΛܾΊͯνʔϜʹೖΔ(Embedding)• 2िؒݶఆͰ։ൃνʔϜʹೖΓࠐΜͰɺΠϯϓοτΛߦͬͨ• Kubernetes AWS ʹؔ͢Δجຊࣄ߲ͷઆ໌• ϞϒϓϩͰ k8s manifest / terraform ͷมߋ࡞ۀ࣮ࢪ• ݁ՌνʔϜͰղܾͰ͖Δ͜ͱ͕૿͑ͨ🎉
2. ίϥϘϨʔγϣϯ͢Δ• 1. ৽ػೳϦϦʔεલͷσϓϩΠύΠϓϥΠϯͷߏங• 2. ٕज़ઓུάϧʔϓͰͷٞ• 3. ڞ௨ͷ՝ΛϖΞϓϩɾϞϒϓϩͰਐΊΔ
৽ػೳϦϦʔεલͷσϓϩΠύΠϓϥΠϯͷߏங• 2022ʰελσΟαϓϦதֶߨ࠲ʱϑϧϦχϡʔΞϧ• ֶशମݧΛϑϧεΫϥονͰ࡞Γͨ͠• Repository ৽نͰ࡞ΓɺCI/CD 0 ͔Β࡞ͬͨ• SRE / ։ൃऀͰཁ݅ఆٛͱ࣮Λڞʹߦͬͨ• Branch strategy• Fast-forward merge Λ͍ͨ͠• ࣌ϝΠϯͷϓϩμΫτ CD ͕ GitOps(ArgoCD)͓ͯ͠ΒͣɺSRE ͱͯ͠ྑ͍࣮ݧతϓϩμΫτͱͳͬͨ
2. ίϥϘϨʔγϣϯ͢Δ• 1. ৽ػೳϦϦʔεલͷσϓϩΠύΠϓϥΠϯͷߏங• 2. ٕज़ઓུάϧʔϓͰͷٞ• 3. ڞ௨ͷ՝ΛϖΞϓϩɾϞϒϓϩͰਐΊΔ• CNDF2023 ࢿྉΛࢀর 👉https://speakerdeck.com/chaspy/toward-the-acquisition-of-self-diagnostic-skills
3. ࣮ࡍʹମݧ͢Δ• 1. ։ൃνʔϜ͔Β SRE νʔϜͷظཹֶ• 2. SRE νʔϜ͔Β։ൃνʔϜͷظཹֶ• 3. SRE Ϛωʔδϟ͕։ൃνʔϜͷϚωδϝϯτ݉43&։ൃνʔϜ"։ൃνʔϜ#։ൃνʔϜ$
1. ։ൃνʔϜ͔Β SRE νʔϜͷظཹֶ• ΞϥʔτϋϯυϦϯάͳͲ SRE ͷීஈͷۀ࣮ࢪ• Kubernetes Cluster ͷ Upgrade Λ(DeveloperͰ্࢙ॳ)• ཹֶऴྃޙڞ௨ͷϓϩδΣΫτΛϞϒϓϩͰ࣮ࢪ͢ΔͳͲؔܧଓ͍ͯ͠Δ• e.g. ઌ΄Ͳհͨ͠ Envoy ഇࢭͷ݅͜ͷϝϯόʔͱ SRE Ͱߦͬͨ
2. SRE νʔϜ͔Β։ൃνʔϜͷظཹֶ• εΫϥϜ։ൃΛܦݧ• Release Λ࣮ࡍʹମݧ͢Δ͜ͱͰϓϩηεͷෛΛମײ• ։ൃνʔϜ͕Πϯϑϥ/ϓϥοτϑΥʔϜͰͲ͜ʹ٧·Δͷ͔Λʹ؍ଌ
3. SRE Ϛωʔδϟ͕։ൃνʔϜͷϚωδϝϯτ݉• ࣗͷͰ͢• Ref: 2022-10-02 Web Application ։ൃͷ EM Λ݉͢Δ͜ͱʹͳͬͨ• ͕͍͍͔݉Ͳ͏͔ผͷ…(͓͢͢Ί͠ͳ͍)• ։ൃνʔϜͱ SRE νʔϜ྆ํݟΔ͜ͱͰɺ։ൃνʔϜ͕Ͳ͜ʹࠔΔͷ͔͕Θ͔ΔΑ͏ʹͳͬͨ• ࣮ࡍʹ͕ࣗσϓϩΠ͢Δ͜ͱͰ՝Λײͨ͡• ։ൃऀ͕ීஈͲ͏ͬͯΤϥʔΛ͍ͯ͠Δͷ͔Λͬͨ
ߟ• Ϛωδϝϯτͷ• จԽతͳഎܠ
Ϛωδϝϯτͷ• ૂ͍ͬͯͨ͜ͱɾҙ͍ࣝͯͨ͜͠ͱ• ઓུతʹ։ൃऀ͔ΒϑΟʔυόοΫΛಘΔ͜ͱʹྗͨ͜͠ͱ• ։ൃऀͱ SRE ͷίϥϘϨʔγϣϯΛҙਤతʹΜͩ͜ͱ• νʔϜؒͷظཹֶΛߦ͑ΔΑ͏ʹௐͨ͜͠ͱ• νʔϜΛ͑ͯ՝ΛͤΔΛσβΠϯͨ͜͠ͱ(ٕज़ઓུ)• ͕ɺ͍ͨͨ͜͠ͱͯ͠ͳ͍ͳ…ͱ͍͏࣮ײ• Ϛωδϝϯτͱͦ͏͍͏ΜͰ͋Δ͔͠Εͳ͍
จԽతͳഎܠ• ৼΓฦΔจԽ• ϑΟʔυόοΫ͢ΔจԽ• ઓ͢Δɾड͚ೖΕΔจԽ
ৼΓฦΔจԽ• ৼΓฦΓͷจԽ• Ref: SRE νʔϜΛࢧ͑Δ;Γ͔͑ΓͷจԽ• ΄΅શͯͷνʔϜͰ2िʹ1ৼΓฦΓ͕ߦΘΕ͍ͯΔ• ϓϩδΣΫτ͕ऴΘͬͨΓɺোൃੜޙඞͣৼΓฦΓ͕ߦΘΕΔ• ৼΓฦΔ͔Βͦ͜ɺ՝Λݟ͚ͭΒΕΔ• ݟ͚ͭͨ՝ͷΞΫγϣϯͱ͍ͯ͠ΖΜͳͷ࣋ͪࠐΈʹܨ͕Δ• Πϯϑϥαϙʔτͷґཔ͓ͦΒͦ͘ͷΑ͏ͳձ͕͋ͬͨͷͰͳ͍͔
ϑΟʔυόοΫ͢ΔจԽ - 360 degree Feedback• 360 degree Feedback ͱ͍͏ӦΈΛ3Ҏ্ܧଓ͍ͯ͠Δ• ධՁͱؔͳ͘ɺಉ྅ʹྑ͍ͱͬͱྑ͘ͳΔΛ͑Δ
ϑΟʔυόοΫ͢ΔจԽ - 360 degree Feedback
ϑΟʔυόοΫ͢ΔจԽ - ਖ਼͍͠૬खʹɺਖ਼͘͠ϑΟʔυόοΫ͢Δ• ࠒ͔ΒνʔϜɾάϧʔϓɾ෦Λ͑ͯϑΟʔυόοΫΛ͢ΔจԽ͕͋Δ• શ෦͕ղܾͰ͖ΔΘ͚Ͱͳ͍͕ɺগͳ͘ͱ͖߹ΘΕΔ• ҆ശͷΑ͏ͳΈΛ࣋ͭάϧʔϓ͕͋Εɺslack ελϯϓͰ՝Λ͛ࠐΊΔΈ(reacjiΛΩϟονͯ͠ϋϯυϦϯά)͕͋ͬͨΓ͢Δ• Blameless ʹϑΟʔυόοΫ͍ͯ͠Δ• ϦΫϧʔτશମͷԣஅ৫ʹରͯ͠ɺվળཁͳͲΑ͘ૹ͍ͬͯΔʢਓࣄɺ࿑ɺICTɺηΩϡϦςΟਪਐ etcʣ
ઓΛड͚ೖΕΔจԽ• ͋ΒΏΔઓ͕ड͚ೖΕΒΕ͍ͯΔ• ϚωδϝϯτͦΕΛαϙʔτ͢ΔɺഎதΛԡ͢• ઓ͢ΔਓΛνʔϜड͚ೖΕɺαϙʔτ͢Δ• ࣗͨ͘͞ΜઓΛͤͯ͞Β͍·ͨ͠• SRE NEXT 2020 ͷ SLI/SLO ͷಋೖେ͖ͳઓͷ1ͭ
·ͱΊ• ࠓͨ͜͠ͱ• SRE ͱ։ൃऀͰ”ͱʹ” SREing Λ࣮ݱ͢ΔΞϓϩʔνͷ۩ମྫΛհ͠·ͨ͠• ͦͷഎܠͱͳΔ৫จԽʹ͍ͭͯߟ͠·ͨ͠• ߟ• SREing Λ࣮ݱ͢Δʹɺಛఆͷख๏ΛΔ͚ͩͰͳ͘ɺԼࢧ͑͢ΔจԽৢηοτͰߟ͑Δඞཁ͕͋Γ·͢• ʰελσΟαϓϦʱ։ൃ৫ʹݩʑ͋ͬͨจԽతૉཆʹॿ͚ΒΕͨେ͖͍ͱࢥ͍·͢• Έͳ͞Μͷ৫ʹͲΜͳจԽ͕͋Δ͔ɺͦΕ͕ SREing ࣮ݱʹͲ͏ؔ͢Δ͔Λߟ͑ͯΈ͍͔͕ͯͰ͠ΐ͏͔
ϓϥΫςΟε͕ઌ͔ɺ৫จԽ͕ઌ͔🐓🥚• ϓϥΫςΟεͱ৫จԽηοτ• ͲͪΒ͔͚ͩͰ͏·͍͔͘ͳ͍͔͠Ε·ͤΜ• ͔ͩΒͦ͜ਖ਼ղ͕ͳ͍͠ɺ͔ͩΒͦ͜͜ͷΧϯϑΝϨϯε͕͋Δ👍• օ͞Μͷ৫จԽʹ͍ͭͯڭ͍͑ͯͩ͘͞• ࠓͷൃදΛͭ·Έʹ࠙ձͰͤΔͱخ͍͠Ͱ͢
࠷ޙʹ• SRE ͱͯ͠աͨ͝͠5ؒɺͷΩϟϦΞ SRE Lounge /SRE NEXT ͱͱʹ͋Γ·ͨ͠• ࠷ޙʹൃදͰ͖ͯخ͍͠Ͱ͢• ӡӦͷΈͳ͞Μ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ• ίϛϡχςΟʹԸฦ͠Ͱ͖͍ͯͨΒ͍Ͱ͢• ࠙ձͰͨ͘͞Μ͠·͠ΐ͏ʂ
Thank you!chaspy chaspy_Takeshi Kondohttps://chaspy.meSenior Engineering Manager (10/1~)StudySapuri K12at Recruit Co., Ltd.͜Εʹͯ SRE ҾୀͰ͢ʂ࠙ձͰձ͍·͠ΐ͏🍻
chaspy
csswizardry
denniskardys
jeffersonlam
lynnandtonic
zakiyama
philhawksworth
nonsquared
tammyeverts
jennybc
marktimemedia
kastner
chriscoyier