Link
Embed
Share
Beginning
This slide
Copy link URL
Copy link URL
Copy iframe embed code
Copy iframe embed code
Copy javascript embed code
Copy javascript embed code
Share
Tweet
Share
Tweet
Slide 1
Slide 1 text
গ৫Ͱ։ൃ͢ΔେنαʔϏε גࣜձࣾ TVer / Ճզ وࢤ (SRE) גࣜձࣾ TVer / ւ ܙհ (Backend) 2022/07/21 Developers Summit 2022 Summer
Slide 2
Slide 2 text
͜ͷηογϣϯʹ͍ͭͯ ຊ͓͢Δ͜ͱ ● TVerϦχϡʔΞϧʹ͓͚ΔϞχλϦϯάվળ ● গਓͰ։ൃͱվળΛߦ͏ͨΊͷ New Relic׆༻ํ๏
Slide 3
Slide 3 text
ࣗݾհ ɾՃզ وࢤ (@TAKA_0411) ɾגࣜձࣾTVer ɹαʔϏεࣄۀຊ෦ SRE ɾझຯαφͱήʔϜͱVTuber ɾϗʔϜαφߔޱԹઘ تָཬ
Slide 4
Slide 4 text
ΞδΣϯμ ● TVerͷհ ● αʔϏεϦχϡʔΞϧʹ͍ͭͯ ● ϦϦʔε४උฤ ● ϦϦʔεޙͷվળ ● ·ͱΊ
Slide 5
Slide 5 text
ΞδΣϯμ ● TVerͷհ ● αʔϏεϦχϡʔΞϧʹ͍ͭͯ ● ϦϦʔε४උฤ ● ϦϦʔεޙͷվળ ● ·ͱΊ
Slide 6
Slide 6 text
TVerͷհ
Slide 7
Slide 7 text
TVerͷհ [TVer] 2022 3 ݄ͷಈը࠶ੜ͕ 2 ԯ 5 ઍສճ/݄Λಥഁ! https://tver.co.jp/news/20220426-1.html
Slide 8
Slide 8 text
TVerͷհ TVerͷ ٕज़ελοΫ
Slide 9
Slide 9 text
TVerͷٕज़ελοΫ ※ϑϩϯτΤϯυɺTVΞϓϦɺϞόΠϧΞϓϦέʔγϣϯ ֎෦ͷڠྗձࣾͷ͓ੈʹͳ͓ͬͯΓ·͢
Slide 10
Slide 10 text
TVerͷհ TVerͷ ΠϯϑϥετϥΫνϟ
Slide 11
Slide 11 text
TVerͷΠϯϑϥετϥΫνϟ WebαʔϏεͱͯ͠ͷTVer ൪࿈ಈγεςϜͰ Քಇ͍ͯ͠ΔϚΠΫϩαʔϏε
Slide 12
Slide 12 text
TVer (WebαʔϏε) ͷΠϯϑϥετϥΫνϟ
Slide 13
Slide 13 text
ΞδΣϯμ ● TVerͷհ ● αʔϏεϦχϡʔΞϧʹ͍ͭͯ ● ϦϦʔε४උฤ ● ϦϦʔεޙͷվળ ● ·ͱΊ
Slide 14
Slide 14 text
αʔϏεϦχϡʔΞϧʹ͍ͭͯ
Slide 15
Slide 15 text
αʔϏεϦχϡʔΞϧʹ͍ͭͯ [TVer] ςϨϏίϯςϯπͷࢹௌମݧࠂମݧͷ্Λࢦ͢ ʮTVer IDʯͷఏڙΛ։࢝ https://tver.co.jp/news/20220404-1.html
Slide 16
Slide 16 text
αʔϏεϦχϡʔΞϧʹ͍ͭͯ γεςϜͷมߋ ● ΠϯϑϥͷԽ ● όοΫΤϯυͷ৽ ● ϞχλϦϯάͷ৽
Slide 17
Slide 17 text
αʔϏεϦχϡʔΞϧʹ͍ͭͯ γεςϜͷมߋ ● ΠϯϑϥͷԽ ● όοΫΤϯυͷ৽ ● ϞχλϦϯάͷ৽ɹ←ࠓճ͜͜ͷ ϦχϡʔΞϧΛػʹ New Relic OneΛϑϧ׆༻
Slide 18
Slide 18 text
ϞχλϦϯάͷ৽ ϞχλϦϯάͷ৽ ● ͷཧ ● ಋೖ͢Δػೳͷऔࣺબ ● μογϡϘʔυͱσʔλͷ׆༻
Slide 19
Slide 19 text
ϞχλϦϯάͷ৽ ϞχλϦϯάͷ৽ ● ͷཧ ● ಋೖ͢Δػೳͷऔࣺબ ● μογϡϘʔυͱσʔλͷ׆༻
Slide 20
Slide 20 text
ϞχλϦϯάͷ৽ Before (ʙ202112݄) ʮNew RelicΛ ยखؒ ͰݟΔͷ͕ͭΒ͍ʂʯ Backend γε Πϯϑϥ New Relic पΓͷରԠ New Relic पΓͷରԠ New Relic पΓͷରԠ APIͷ։ൃ Fargateݕূ ෛՙςετ ΞΧϯτ ઃܭͱඋ ίετࢼࢉ ͓Αͼܖ ֤෦ॺ͔Β ͷ͍߹Θ ͤରԠ νϡʔχϯά CI/CDඋ ཁ݅ཧ ΞϓϦϑϩ ϯτΤϯυͱ ͷௐ Πϯϑϥߏங ͱCDKपΓͷ උ
Slide 21
Slide 21 text
After (20221݄ʙ) ϞχλϦϯάͷ৽ SRE (ࢲ) New RelicपΓͷλεΫ ·Δ͝ͱҾ͖ड͚·͢Ͷ ΈΜͳ ͡Ό͋ϦʔυΑΖ͘͠ʂ ଞͷλεΫਐΊΔͶ
Slide 22
Slide 22 text
࠷ۙ ϞχλϦϯάͷ৽ Backend SRE μογϡϘʔυͰ ͜ΜͳϝτϦΫε͕ݟ ͍ͨΜ͚ͩͲʙ ͜ͷ߹ʹΞϥʔτ ඈͯ͠ཉ͍͠ΑͶ Γ͍ͨࣄΛԕྀͤͣʹ૬ஊɾ࣮ݱͰ͖Δମ੍ ○○ͱ͍͏ΛΩʔʹ͠ ͯΤϥʔΛௐ͍ࠪͨ͠ ͱࢥͬͯΔΘ͚Α Γ·͢
Slide 23
Slide 23 text
ϞχλϦϯάͷ৽ ϞχλϦϯάͷ৽ ● ͷཧ ● ಋೖ͢Δػೳͷऔࣺબ ● μογϡϘʔυͱσʔλͷ׆༻
Slide 24
Slide 24 text
ϞχλϦϯάͷ৽ NRU101 New Relic One ೖ https://newrelic.com/jp/resources/presentations/nru101_20211027
Slide 25
Slide 25 text
ϞχλϦϯάͷ৽ ࣗͷཧ ● MELTΛશͯຬ͍ͨͨ͠ ○ Metrics, Events, Logs, Traces ● ΤϥʔௐࠪͷͨΊʹ Traces(APM)Λಋೖ͍ͨ͠
Slide 26
Slide 26 text
ϞχλϦϯάͷ৽ ݱ࣮ ● New RelicͷGo Agent(APM)ͷ ಋೖ͕एׯ໘ ● APMͷΦʔόʔϔου͕ෆ҆ ● pprofͰྑ͘ͳ͍ʁ ● ͍ͬͯ͏͔ࠓඞཁͳΜ͚ͩͬʁ ϦϦʔε·Ͱ͋ͱ2ϲ݄ɾɾɾ
Slide 27
Slide 27 text
ϞχλϦϯάͷ৽ ͱΓ͋͑ͣ New Relic Go Agent Λ ඞཁ࠷খݶͷՕॴʹ͚ͩಋೖͨ͠
Slide 28
Slide 28 text
ϞχλϦϯάͷ৽
Slide 29
Slide 29 text
ϞχλϦϯάͷ৽ Dev (Backend)ɹɹOps (SRE) ಋೖ݁Ռ ● EchoͷIntegrationΛઃఆ͢Δͩ ͚Ͱେ෯ʹՄࢹԽͰ͖ͨ ● Τϥʔൃੜ࣌ͷτϦΞʔδ͕ Մೳʹͳͬͨ ● ࠷খݶͷ࿑ྗͰ࠷େͷޮՌΛಘ Δ͜ͱ͕Ͱ͖ͨ
Slide 30
Slide 30 text
ϞχλϦϯάͷ৽ গ৫ × New Relic One ● ػೳ͕๛ͰΦʔϧΠϯϫϯ ● ಋೖͷϋʔυϧ͕͍ ● υΩϡϝϯτɾαϙʔτɾίϛϡχςΟ͕ॆ࣮ ● ͱΓ͋͑ͣಋೖ͢Δ͚ͩͰେ෯ʹՄࢹԽͰ͖Δ ಋೖɾӡ༻ͷίετ͕࠷খݶͰࡁΜͩ
Slide 31
Slide 31 text
ϞχλϦϯάͷ৽ ϞχλϦϯάͷ৽ ● ͷׂ ● ಋೖ͢Δػೳͷऔࣺબ ● μογϡϘʔυͱσʔλͷ׆༻
Slide 32
Slide 32 text
ϞχλϦϯάͷ৽ ΫϥΠΞϯτͷಉ࣌ଓΫϥογϡͷϝτϦΫε
Slide 33
Slide 33 text
ϞχλϦϯάͷ৽ ि1ͰϞόΠϧΞϓϦͷΫϥογϡωοτϫʔΫΤϥʔΛௐࠪ͠ɺࣾ͘ʹڞ༗
Slide 34
Slide 34 text
ϞχλϦϯάͷ৽ μογϡϘʔυͱσʔλͷ׆༻ ● ඞཁͳϝτϦΫεͰμογϡϘʔυΛඋ ● ϑϩϯτΤϯυ͔ΒόοΫΤϯυ·ͰΧόʔ ● ΤϥʔΛௐࠪͯࣾ͠ʹڞ༗ ● APIͷվળʹAPMΛ׆༻
Slide 35
Slide 35 text
ࣗݾհ ɾւ ܙհ (@voidofglans) ɾגࣜձࣾTVer αʔϏεࣄۀຊ෦ ɹϦʔυ ΤϯδχΞ ɾ͖ͳͷ:ՐͰম͍ͨϗϧϞϯ ɾಸྑࡏॅͷԕִۈத ɾNFCຒΊࠐΜͩΓ͢Δͷ͕͖
Slide 36
Slide 36 text
ΞδΣϯμ ● TVerͷհ ● αʔϏεϦχϡʔΞϧʹ͍ͭͯ ● ϦϦʔε४උฤ ● ϦϦʔεޙͷվળ ● ·ͱΊ
Slide 37
Slide 37 text
ϦϦʔε४උฤ ● ϑΣʔζʹΑͬͯNew Relicͷ͍͚ ● গਓͰεϐʔυײΛམͱ͞ͳ͍ͨΊʹ ● ύϑΥʔϚϯεɾνϡʔχϯά/ෛՙରࡦ
Slide 38
Slide 38 text
ϦϦʔε४උฤ ● ϑΣʔζʹΑͬͯNew Relicͷ͍͚ ● গਓͰεϐʔυײΛམͱ͞ͳ͍ͨΊʹ ● ύϑΥʔϚϯεɾνϡʔχϯά/ෛՙରࡦ
Slide 39
Slide 39 text
ϑΣʔζʹΑͬͯNew Relicͷ͍͚ APMಋೖޙͷϦϦʔε·ͰͷϑΣʔζ ● 2ϲ݄લ ● 1ϲ݄લ ● 2िؒલ
Slide 40
Slide 40 text
ϑΣʔζʹΑͬͯNew Relicͷ͍͚(2ϲ݄લ ● Δ͖͜ͱόά௵͠ ● ओʹར༻ͨ͠ػೳ ○ Errors Inbox ○ Errors ○ NRQL ● 4xx/5xxܥΤϥʔͷ༰Λੳ ○ 4xx: ΫϥΠΞϯτ͕ఆ͍ͯ͠ͳ͍ύϥϝʔλ͕ͳ͍͔ ○ 5xx: ఆ͍ͯ͠ͳ͍ॲཧλΠϜΞτ͕ͳ͍͔
Slide 41
Slide 41 text
ϑΣʔζʹΑͬͯNew Relicͷ͍͚(1ϲ݄લ ● Δ͖͜ͱෛՙࢼݧʹΑΔϘτϧωοΫௐࠪ ● ओʹར༻ͨ͠ػೳ ○ Transactions ○ Go Runtime ○ CloudWatch Metrics ● ߴෛՙ࣌ͷΞϓϦέʔγϣϯશମͷڍಈΛੳ ○ TransactionsͰͷSlowest95percentileͳͲΛࢦඪʹ ○ ϝϞϦϦʔΫɺ์ஔ͞Εͨgoroutineͷଘࡏ ○ GC AllocationʹΑΔஅଓతͳੑೳྼԽ
Slide 42
Slide 42 text
ϑΣʔζʹΑͬͯNew Relicͷ͍͚(2िؒલ ● Δ͖͜ͱཁٻੑೳͷୡ ● ओʹར༻ͨ͠ػೳ ○ Transactions ○ CloudWatch Metrics ● શମΛ௨ͨ͠ෛՙࢼݧͰͷ҆ఆύϑΥʔϚϯε ○ 10msҎ্͔͔ΔAPI ○ ֤ϛυϧΣΞͷΫΤϦଓͷ ○ ॻ͖ࠐΈܥAPIͰͷLock,Waitͷ
Slide 43
Slide 43 text
ϦϦʔε४උฤ ● ϑΣʔζʹΑͬͯNew Relicͷ͍͚ ● গਓͰεϐʔυײΛམͱ͞ͳ͍ͨΊʹ ● ύϑΥʔϚϯεɾνϡʔχϯά/ෛՙରࡦ
Slide 44
Slide 44 text
গਓͰεϐʔυײΛམͱ͞ͳ͍ͨΊʹ ૣ͘ߦ͖͚ͨΕɺͻͱΓͰߦ͚ɻ ԕ͘·Ͱߦ͖͚ͨΕɺΈΜͳͰߦ͚ɻ If you want to go fast, go alone. If you want to go far, go together.
Slide 45
Slide 45 text
গਓͰεϐʔυײΛམͱ͞ͳ͍ͨΊʹ ͓͚ͬʂ ͡Ό͋ͪΐͬͱ1ਓͰߦͬͯ͘Δʂ
Slide 46
Slide 46 text
গਓͰεϐʔυײΛམͱ͞ͳ͍ͨΊʹ ϦϦʔε·Ͱʹ ΈΜͳݱू߹Ͱʂ
Slide 47
Slide 47 text
● ͦΕͧΕ͕ͦΕͧΕͷ࠷Ͱతʹ ● ͷ໌֬ͳղ ● ໌֬ͳతͷڞ༗ গਓͰεϐʔυײΛམͱ͞ͳ͍ͨΊʹ
Slide 48
Slide 48 text
● ίέͳ͍͜ͱ͕ॏཁ ● Backend/Infra/SREͦΕͧΕ͕ઐੑΛ࣋ͬͯಠཱ͠ ͨϝϯόʔ ● ίϛϡχέʔγϣϯίετ͕࠷ݶʹ͑ΒΕͨ গਓͰεϐʔυײΛམͱ͞ͳ͍ͨΊʹ
Slide 49
Slide 49 text
গਓͰεϐʔυײΛམͱ͞ͳ͍ͨΊʹ ● ൫ͬͨ ● ࣍ΈΜͳͰԕ͘ߦ͜͏
Slide 50
Slide 50 text
ϦϦʔε४උฤ ● ϑΣʔζʹΑͬͯNew Relicͷ͍͚ ● গਓͰεϐʔυײΛམͱ͞ͳ͍ͨΊʹ ● ύϑΥʔϚϯεɾνϡʔχϯά/ෛՙରࡦ
Slide 51
Slide 51 text
● Α͋͘ΔಥൃతͳεύΠΫ ○ ্࿈ಈ์ૹΕʹΑΔྲྀೖ ■ 1minޙ͖ͬ͞·Ͱͷ10ഒҎ্ͷϦΫΤετ ■ ૿͑Δಉ࣌ଓ ○ ΞΠυϧࣄॴλϨϯτ༷ग़ԋ൪ ○ ϦΞϧλΠϜ৴൪ऴྃ࣌ ■ Ϣʔβʔ͕Ұ੪ʹβοϐϯά ■ ಉظͷͱΕͨສʙेສͷϢʔβʔΞΫγϣϯ ύϑΥʔϚϯεɾνϡʔχϯά/ෛՙରࡦ
Slide 52
Slide 52 text
ύϑΥʔϚϯεɾνϡʔχϯά/ෛՙରࡦ
Slide 53
Slide 53 text
● ϦϦʔεલʹSRE͕͔ͬ͠Γઐͯ͘͠Εͨ ○ New RelicΛݟ͓͚ͯҰൠతͳࢦඪશ෦ݟΕΔ ○ ఆ͞ΕΔΞΫηεύλʔϯʹ͓͚ΔMetricsͷมԽΛࣄ લʹೝࣝ͢Δ ○ ෛՙࢼݧ/ରࡦʹूத͢Δ͜ͱ͕Ͱ͖ͨ ύϑΥʔϚϯεɾνϡʔχϯά/ෛՙରࡦ
Slide 54
Slide 54 text
ΞδΣϯμ ● TVerͷհ ● αʔϏεϦχϡʔΞϧʹ͍ͭͯ ● ϦϦʔε४උฤ ● ϦϦʔεޙͷվળ ● ·ͱΊ
Slide 55
Slide 55 text
ϦϦʔεޙͷվળ ● ԣஅతʹՄࢹԽ͞ΕͨMetricsͷԸܙ ● ఆظతʹ࣮ࢪ͢ΔμογϡϘʔυோΊΔձ ● ͷʹΑΔରԠͷ୲อ
Slide 56
Slide 56 text
ԣஅతʹՄࢹԽ͞ΕͨMetricsͷԸܙ ● ҆ఆՔಇ͍ͯ͠Δ͜ͱ͕ԕͰΘ͔Δ҆৺ײ
Slide 57
Slide 57 text
ԣஅతʹՄࢹԽ͞ΕͨMetricsͷԸܙ ● σόΠε͝ͱͷ࣮ʹΑΔࠩҟͷݕ ○ σόΠεछผΛCustomAttributeʹ֨ೲ͢Δ ○ ϦΫΤετΤϥʔͷѲ ■ iOSΞϓϦ͚ͩಛఆͷϖʔδͰͷAPIϦΫΤετ͕ҟৗʹଟ͍ ⿌ ϦετදࣔͰཁૉ͝ͱʹAPIΛݺͼग़͍ͯͨ͠ ⿌ 30minΒͣͰݪҼಛఆˠ։ൃϕϯμʔͷڞ༗ ⿌ 3ޙͷϦϦʔεʹแؚ͠ɺແࣄղফ🎉
Slide 58
Slide 58 text
ԣஅతʹՄࢹԽ͞ΕͨMetricsͷԸܙ ● ։ൃॳఆ͞Ε͍ͯͳ͍ϢʔβʔγφϦΦ ○ ࢥͬͨͷͷ3ഒ͘Β͍ݕࡧػೳ͕ΘΕ͍ͯΔʂʂ ○ ࢥͬͨΑΓxxͷػೳ͕ΘΕͯͳ͍ʂʂ ఆͱӡ༻ဃ͢Δͷ
Slide 59
Slide 59 text
● աͳϦιʔε͕͙͢Θ͔Δ ○ ϦϦʔεॳظ҆৺ΛۚͰങ͍ͬͯͨͷͰ५ͳϦιʔε Λ༻ҙ͍ͯͨ͠ ○ ίετΠϯύΫτΛ࣠ʹॱ࣍Ϧιʔεͷ࠷దԽΛߦ͑ͨ ■ ͪΖΜ͜ΕՄࢹԽ͞Ε͍ͯΔͷͰμϝͳ߹ͷݕਝ ԣஅతʹՄࢹԽ͞ΕͨMetricsͷԸܙ
Slide 60
Slide 60 text
● ࡢࠓؾʹͳͬͨࢦඪΛμογϡϘʔυʹՃ ● Metricsͷҧײʹؾͮ͘ᄿ֮ͷ܇࿅ ● োޙτϥϒϧൃੜ࣌ͷݟͷूੵ ఆظతʹ࣮ࢪ͢ΔμογϡϘʔυோΊΔձ
Slide 61
Slide 61 text
ͷʹΑΔରԠͷ୲อ ● ਖ਼͘͠ઐʹͤΔ ○ த్ʹ͍݉ͯ͠ΔͱɺԱଌओ؍͕ೖͬͯ͠·͍͕ ͪ ○ ໌֬ͳͷ ■ SREࣄ࣮ͱϩάΛ͢ͱ͜Ζʹઐ೦
Slide 62
Slide 62 text
ΞδΣϯμ ● TVerͷհ ● αʔϏεϦχϡʔΞϧʹ͍ͭͯ ● ϦϦʔε४උฤ ● ϦϦʔεޙͷվળ ● ·ͱΊ
Slide 63
Slide 63 text
·ͱΊ ● 1ਓͰ๊͑ࠐΉͷΑ͘ͳ͍ ● ૣ͘ΔͨΊʹΛ໌֬ʹղ͢Δඞཁ͕͋Δ ● ͰͬͺΓΈΜͳͰૣ͘ԕ͘ߦ͖͍ͨʂʂ
Slide 64
Slide 64 text
·ͱΊ ● ແࣄϦϦʔεˍ҆ఆӡ༻͕࢝·ͬͨ🎉 ● NewRelicΛ։ൃڥͰ׆༻ ○ ։ൃ࣌ͷόάτϥοΩϯά ○ ՝ཧͱͷ࿈ܞ ○ QAνʔϜʹΑΔNewRelicͰͷࣄ࣮ڞ༗ ● ։ൃ৫ͷϦϑΝΫλϦϯάΛ࢝Ί͍͖ͯ·͢ ○ ࠓ·Ͱͷ୲ΛਓνʔϜͰ୲ ○ ઐνʔϜʹΑΔεϧʔϓοτΛ୲อͨ͠։ൃ৫
Slide 65
Slide 65 text
No content
Slide 66
Slide 66 text
No content