TVerのサービスリニューアルにおける 信頼性向上への取り組み

2022/05/14
SRE NEXT 2022 (Day1)

イベントURL :
https://sre-next.dev/2022/

Transcript

1. [εϙϯαʔηογϣϯ]

   TVerͷαʔϏεϦχϡʔΞϧʹ͓͚Δ
   ৴པੑ޲্΁ͷऔΓ૊Έ
   גࣜձࣾ TVer / Ճզ وࢤ
   2022/05/14
   SRE NEXT 2022 ONLINE
   

   View Slide

2. ΞδΣϯμ
   ● TVerͷ঺հ
   ● αʔϏεϦχϡʔΞϧʹ͍ͭͯ
   ● ࠓޙͷ࿩
   

   View Slide

3. ΞδΣϯμ
   ● TVerͷ঺հ
   ● αʔϏεϦχϡʔΞϧʹ͍ͭͯ
   ● ࠓޙͷ࿩
   

   View Slide

4. TVerͷ঺հ
   TVerͷ
   αʔϏε
   

   View Slide

5. TVerͷ঺հ
   

   View Slide

6. TVerͷ঺հ
   

   View Slide

7. TVerͷ঺հ
   TVerͷ
   ٕज़ελοΫ
   

   View Slide

8. TVerͷٕज़ελοΫ
   ※ϑϩϯτΤϯυͱϞόΠϧΞϓϦέʔγϣϯ͸

   ֎෦ͷڠྗձࣾͷ͓ੈ࿩ʹͳ͓ͬͯΓ·͢
   

   View Slide

9. TVerͷΠϯϑϥετϥΫνϟ
   TVerͷ
   ΠϯϑϥετϥΫνϟ
   

   View Slide

10. TVerͷΠϯϑϥετϥΫνϟ
    ɾ଱ো֐ੑʹ༏Εͨ

    ɹϚΠΫϩαʔϏε
    ɾϘτϧωοΫ͕ൃੜ

    ɹ͠ʹ͍͘ػೳઃܭ
    ɾϋΠύϑΥʔϚϯε

    ɹͳϛυϧ΢ΣΞ(ޙड़)
    TVer΍൪૊࿈ಈγεςϜͱͯ͠
    

    Քಇ͍ͯ͠Δ΋ͷͨͪ
    

    View Slide

11. TVer (WebαʔϏε) ͷΠϯϑϥετϥΫνϟ
    

    View Slide

12. ΞδΣϯμ
    ● TVerͷ঺հ
    ● αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    ● ࠓޙͷ࿩
    

    View Slide

13. αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    

    View Slide

14. αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    αʔϏεͷมߋ఺
    ● TVer IDʹΑΔϩάΠϯػೳͷఏڙ
    ○ σόΠεؒͰͷࢹௌঢ়گͷಉظ
    ○ σόΠεؒͰͷϦετͷಉظ
    ○ ௥͔͚ͬ࠶ੜػೳ΁ͷରԠ
    

    View Slide

15. αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    αʔϏεͷมߋ఺
    ● TVer IDʹΑΔϩάΠϯػೳͷఏڙ
    ○ σόΠεؒͰͷࢹௌঢ়گͷಉظ
    ○ σόΠεؒͰͷϦετͷಉظ
    ○ ௥͔͚ͬ࠶ੜػೳ΁ͷରԠ
    

    View Slide

16. αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    αʔϏεͷมߋ఺
    ● TVer IDʹΑΔϩάΠϯػೳͷఏڙ
    ○ σόΠεؒͰͷࢹௌঢ়گͷಉظ
    ○ σόΠεؒͰͷϦετͷಉظ
    ○ ௥͔͚ͬ࠶ੜػೳ΁ͷରԠ
    

    View Slide

17. αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    αʔϏεͷมߋ఺
    ● TVer IDʹΑΔϩάΠϯػೳͷఏڙ
    ○ σόΠεؒͰͷࢹௌঢ়گͷಉظ
    ○ σόΠεؒͰͷϦετͷಉظ
    ○ ௥͔͚ͬ࠶ੜػೳ΁ͷରԠ
    

    View Slide

18. αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    [TVer] ςϨϏίϯςϯπͷࢹௌମݧ΍޿ࠂମݧͷ޲্Λ໨ࢦ͢ ʮTVer IDʯͷఏڙΛ։࢝

    https://tver.co.jp/news/20220404-1.html
    

    View Slide

19. αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    γεςϜͷมߋ఺
    ● Πϯϑϥͷ಺੡Խ
    ● όοΫΤϯυͷ࡮৽
    ● ϞχλϦϯάͷ࡮৽
    ● ҆શͳϚΠάϨʔγϣϯ
    

    View Slide

20. αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    γεςϜͷมߋ఺
    ● Πϯϑϥͷ಺੡Խ
    ● όοΫΤϯυͷ࡮৽
    ● ϞχλϦϯάͷ࡮৽
    ● ҆શͳϚΠάϨʔγϣϯ
    

    View Slide

21. Πϯϑϥͷ಺੡Խ
    ಺੡Խͷഎܠ
    ● ౰ॳ͸֎෦ͷڠྗձ͕ࣾߏஙɾӡ༻
    ● TVerͷαʔϏε͕ӈݞ্͕ΓͰ੒௕
    ● ࣗ෼ͨͪͰγεςϜΛίϯτϩʔϧ͍ͨ͠
    ● ࡢ೥4݄ʹTVer Technologiesϝϯόʔ͕߹ྲྀ
    ຊؾͰ಺੡Խʹ଩Λ੾Δ؀ڥ͕੔ͬͨ
    

    View Slide

22. Πϯϑϥͷ಺੡Խ
    ɹ“޷ҙ౓޲্ʹ༗རͳ؀ڥԼɺTVer͸σʔλʹجͮ͘

    ɹ޿ࠂ഑৴΋ڧԽͭͭ͋͠Δɻࢹௌཤྺͷ؅ཧͳͲɺ

    ɹIDج൫ɾσʔλࣄۀΛࢧ͍͑ͯͨɺTVer TechnologiesΛ

    ɹຊମʹٵऩ߹ซͨ͠ͷ͕ࡢ೥4݄ͷ͜ͱɻҰମԽ͢Δ͜ͱ

    ɹͰσʔλϚʔέςΟϯάΛՃ଎͍ͤͯ͞Δɻ”
    TVerɺ4500ສDL௒͑ɹσʔλʹΑΔϒϥϯυࢧԉ൫ੴʹ

    https://www.advertimes.com/20220328/article379481/
    

    View Slide

23. Πϯϑϥͷ಺੡Խ
    before
    ● ֎෦ͷڠྗձࣾʹߏஙɾӡ༻Λґཔ
    ● มߋ࡞ۀ͸ґཔϕʔεͰ΍ΓऔΓ
    ○ LBͷஆػਃ੥
    ○ Πϯελϯε୆਺ͷ૿ڧ
    ○ αʔόʔͷઃఆมߋ
    

    View Slide

24. Πϯϑϥͷ಺੡Խ (before)
    ஆػਃ੥
    ● ֎෦ͷڠྗձࣾ΁
    ஆػਃ੥Λґཔ
    ● ۓٸ࣌ʹਃ੥Λ

    ग़͍ͨ࣌͠͸

    ʲۓٸʳͰґཔ🙇
    

    View Slide

25. Πϯϑϥͷ಺੡Խ (before)
    Πϯελϯε୆਺૿ڧ
    ● ֎෦ͷڠྗձࣾ΁

    Πϯελϯε૿ڧ

    Λґཔ
    ● େ͖ΊͳΠϕϯτͰ
    ͸ۓٸରԠΛආ͚Δ
    ͨΊɺ૝ఆҎ্ͷ୆
    ਺Ͱґཔ
    

    View Slide

26. Πϯϑϥͷ಺੡Խ (after)
    after
    ● ࣗ෼ͨͪͷAWSΞΧ΢ϯτͰશͯΛ؅ཧ
    ● CDKʹΑΔ҆৺ɾ҆શͳΠϯϑϥ؅ཧ
    ● ૝ఆ֎͕ى͖ͯ΋ॊೈʹରԠͰ͖Δମ੍
    ● ద੾ͳΩϟύγςΟʹଈ࣌มߋ
    

    View Slide

27. Πϯϑϥͷ಺੡Խ (after)
    

    View Slide

28. αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    γεςϜͷมߋ఺
    ● Πϯϑϥͷ಺੡Խ
    ● όοΫΤϯυͷ࡮৽
    ● ϞχλϦϯάͷ࡮৽
    ● ҆શͳϚΠάϨʔγϣϯ
    

    View Slide

29. όοΫΤϯυͷ࡮৽
    before
    ● ಈ࡞؀ڥ͸EC2
    ● Node.jsΛ࠾༻ͨ͠

    ΞϓϦέʔγϣϯαʔόʔ
    after
    ● ಈ࡞؀ڥ͸Fargate
    ● GoΛ࠾༻ͨ͠

    ΞϓϦέʔγϣϯαʔόʔ
    

    View Slide

30. αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    γεςϜͷมߋ఺
    ● Πϯϑϥͷ಺੡Խ
    ● όοΫΤϯυͷ࡮৽
    ● ϞχλϦϯάͷ࡮৽
    ● ҆શͳϚΠάϨʔγϣϯ
    

    View Slide

31. ϞχλϦϯάͷ࡮৽ (before)
    before
    ● CloudWatchϕʔεͷϞχλϦϯά
    ● Ξϥʔτ͸֎෦ͷڠྗձ͔ࣾΒͷ࿈བྷϕʔε
    ● ϦΞΫςΟϒͳϞχλϦϯά
    

    View Slide

32. ϞχλϦϯάͷ࡮৽ (before)
    Ξϥʔτͷ࿈བྷ
    ● ֎෦ͷڠྗձࣾ

    ܦ༝ͰΞϥʔτΛ
    ड͚Δ
    ● ࿈བྷ͕͔͋ͬͯΒ
    ௐࠪ։࢝
    

    View Slide

33. ϞχλϦϯάͷ࡮৽ (after)
    after
    ● New Relic OneΛಋೖ
    ● ϝτϦΫεΛऩूɾ෼ੳ͢Δ͜ͱͰ

    σʔλυϦϒϯͳ։ൃ΍ӡ༻͕Մೳʹ
    ● ϓϩΞΫςΟϒͳϞχλϦϯά
    ● ΦϒβʔόϏϦςΟ
    

    View Slide

34. ϞχλϦϯάͷ࡮৽ (after)
    

    View Slide

35. ϞχλϦϯάͷ࡮৽ (after)
    ݄ؒಈը࠶ੜ਺2ԯճΛࢧ͑Δ TVerͷΦϒβʔόϏϦςΟ

    https://speakerdeck.com/techtver/20220311-observability-conference-2022-how-tver-achieve-observability
    

    View Slide

36. αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    γεςϜͷมߋ఺
    ● Πϯϑϥͷ಺੡Խ
    ● όοΫΤϯυͷ࡮৽
    ● ϞχλϦϯάͷ࡮৽
    ● ҆શͳϚΠάϨʔγϣϯ
    

    View Slide

37. ҆શͳϚΠάϨʔγϣϯ
    3ߦͰ·ͱΊ
    ● طʹಈ͍͍ͯΔڊେαʔϏεΛ
    ● طଘͷϚΠΫϩαʔϏεͷதʹ௥Ճͯ͠
    ● ϊʔτϥϒϧͰҠߦ੒ޭʂ
    

    View Slide

38. ҆શͳϚΠάϨʔγϣϯ
    ● ϚΠΫϩαʔϏεͷ
    தʹ௒ڊେαʔϏε
    Λ௥Ճ
    ● ଞͷϚΠΫϩαʔϏ
    εʹτϥϑΟοΫ͕
    ೾ٴ

    (ձһ৘ใDBͳͲ)
    ● ແࣄ৐Γ੾ͬͨ💪
    

    View Slide

39. ΞδΣϯμ
    ● TVerͷ঺հ
    ● αʔϏεϦχϡʔΞϧʹ͍ͭͯ
    ● ࠓޙͷ࿩
    

    View Slide

40. ࠓޙͷ࿩
    ࠓޙ΍Γ͍ͨ͜ͱ / վળ͍͖͍ͯͨ͜͠ͱ
    ● CI / CDपΓͷվળ
    ● CDKʹΑΔܧଓతͳΠϯϑϥͷίʔυԽ
    ● ίϯςφԽͷਪਐ
    ● શγεςϜͰͷΦϒβʔόϏϦςΟ࣮ݱ
    ● SLI / SLOʹΑΔ৴པੑͷՄࢹԽ
    

    View Slide

41. ࠓޙͷ࿩
    ࠓޙ΍Γ͍ͨ͜ͱ / վળ͍͖͍ͯͨ͜͠ͱ
    ● CI / CDपΓͷվળ
    ● CDKʹΑΔܧଓతͳΠϯϑϥͷίʔυԽ
    ● ίϯςφԽͷਪਐ
    ● શγεςϜͰͷΦϒβʔόϏϦςΟ࣮ݱ
    ● SLI / SLOʹΑΔ৴པੑͷՄࢹԽ
    

    View Slide

42. ڵຯ

    ͋Γ·͢ΑͶʁ
    

    View Slide

43. View Slide

44. ࣗݾ঺հ
    ɾՃզ وࢤ (@TAKA_0411)
    ɾגࣜձࣾTVer

    ɹࣄۀຊ෦ ٕज़։ൃ෦ (2022/1ʙ)
    ɾझຯ͸α΢φͱΧϝϥͱήʔϜ
    ɾ޷͖ͳSREจԽ͸ϙετϞʔςϜ

    ɹ(ඇ೉ͷͳ͍ৼΓฦΓ)
    

    View Slide

45. View Slide