$30 off During Our Annual Pro Sale. View Details »

Amazon OpenSearch Serviceで 構築するTVerの次世代検索基盤

Amazon OpenSearch Serviceで 構築するTVerの次世代検索基盤

2022/10/08
JAWS DAYS 2022 - Satellites

イベントURL :
https://jawsdays2022.jaws-ug.jp/

TVer Inc.
PRO

October 08, 2022
Tweet

More Decks by TVer Inc.

Other Decks in Technology

Transcript

  1. Amazon OpenSearch ServiceͰ
    ߏங͢ΔTVerͷ࣍ੈ୅ݕࡧج൫
    2022.10.8 JAWS DAYS 2022
    גࣜձࣾ TVer ࠇ੉

    View Slide

  2. ࣗݾ঺հ
    ● ໊લɿࠇ੉ ༞ฏ
    ● ॴଐɿגࣜձࣾ TVer
    ● ୲౰ɿ
    ○ σʔλγεςϜ։ൃʢETLʣ
    ○ όοΫΤϯυ։ൃʢݕࡧج൫ʣ

    View Slide

  3. ΞδΣϯμ
    ● TVerʹ͍ͭͯ
    ● TVerϦχϡʔΞϧͱ࣍ੈ୅ݕࡧج൫
    ○ ϦχϡʔΞϧޙͷݕࡧػೳ
    ○ ։ൃମ੍
    ○ Amazon OpenSearch Service
    ● ࣮ݱʹ޲͚ͯͷऔΓ૊Έ
    ○ σʔλ؅ཧ
    ○ ετϨʔδ؅ཧ
    ○ αʔϏεͷӡ༻ɾ։ൃ
    ● ·ͱΊ

    View Slide

  4. ΞδΣϯμ
    ● TVerʹ͍ͭͯ
    ● TVerϦχϡʔΞϧͱ࣍ੈ୅ݕࡧج൫
    ○ ϦχϡʔΞϧޙͷݕࡧػೳ
    ○ ։ൃମ੍
    ○ Amazon OpenSearch Service
    ● ࣮ݱʹ޲͚ͯͷऔΓ૊Έ
    ○ σʔλ؅ཧ
    ○ ετϨʔδ؅ཧ
    ○ αʔϏεͷӡ༻ɾ։ൃ
    ● ·ͱΊ

    View Slide

  5. TVerʹ͍ͭͯ
    TVer͸ຽ์ςϨϏہͷίϯςϯπΛ഑৴͢Δຽ์ެࣜςϨϏ഑৴αʔϏε


    ຖि໿600൪૊ΛແྉͰύιίϯɾεϚʔτϑΥϯɾλϒϨοτɾςϨϏʹ഑৴

    View Slide

  6. TVerʹ͍ͭͯ

    View Slide

  7. ΞδΣϯμ
    ● TVerʹ͍ͭͯ
    ● TVerϦχϡʔΞϧͱ࣍ੈ୅ݕࡧج൫
    ○ ϦχϡʔΞϧޙͷݕࡧػೳ
    ○ ։ൃମ੍
    ○ Amazon OpenSearch Service
    ● ࣮ݱʹ޲͚ͯͷऔΓ૊Έ
    ○ σʔλ؅ཧ
    ○ ετϨʔδ؅ཧ
    ○ αʔϏεͷӡ༻ɾ։ൃ
    ● ·ͱΊ

    View Slide

  8. TVerϦχϡʔΞϧ
    [TVer] ςϨϏίϯςϯπͷࢹௌମݧ΍޿ࠂମݧͷ޲্Λ໨ࢦ͢ ʮTVer IDʯͷఏڙΛ։࢝

    https://tver.co.jp/news/20220404-1.html

    View Slide

  9. ݕࡧػೳ
    ● ݕࡧΩʔϫʔυΛೖྗ͢Δͱɺίϯςϯπ৘ใ͕Ϛονͨ͠ެ։த
    ͷίϯςϯπΛҰཡͰฦ͢ػೳ

    View Slide

  10. ݕࡧػೳ
    ● ैདྷ͸ίϯςϯπ৘ใͷDBΛ௚઀ݕࡧ
    ● ػೳ֦ுͷ๦͛ʹͳ͍ͬͯͨ
    ➢ αʔϏεͷ੒௕ʹ͋Θͤͨػೳ֦ுʹ՝୊
    ݕࡧAPI
    Amazon
    Aurora


    (MySQL)
    ݕࡧ݁Ռ
    CMS
    ίϯςϯπ


    ৘ใ

    View Slide

  11. ϦχϡʔΞϧͰͷ։ൃମ੍ͷڧԽ
    ● αʔϏε੒௕ʹ͋ΘͤͯΤϯδχΞʹΑΔ಺੡ԽΛਪਐ
    ● ΤϯδχΞ͕ઐ໳֎ͷ։ൃʹ΋௅ઓͰ͖ΔΑ͏ʹ
    ○ σʔλγεςϜ୲౰͕खΛڍ͛ͯόοΫΤϯυ΋։ൃ
    ➢ΤϯδχΞ͕։ൃͷମ੍΍αΠΫϧΛίϯτϩʔϧ
    ϦχϡʔΞϧ

    View Slide

  12. ݕࡧج൫
    ํ਑
    ● ࣗࣾΤϯδχΞओಋͰγεςϜߏ੒Λ࡮৽
    ○ ݕࡧػೳΛΠϯϑϥ͝ͱಠཱͤ͞Δ
    ○ શจݕࡧʹಛԽͨ͠ϛυϧ΢ΣΞΛಋೖ
    ➢ Amazon OpenSearch ServiceΛ࠾༻
    ݕࡧAPI
    ݕࡧ݁Ռ
    CMS
    ίϯςϯπ


    ৘ใ
    શจݕࡧ


    ϛυϧ΢ΣΞ

    View Slide

  13. Amazon OpenSearch Service
    ● OpenSearchͷϚωʔδυαʔϏε
    ● AWSͷػೳɾαʔϏεͱγʔϜϨεʹ࿈
    ܞՄೳ
    ○ VPC
    ○ RI
    ○ etc.
    ● ϩά΍ϝτϦΫε΋ఏڙ
    Amazon
    OpenSearch Service

    View Slide

  14. ΞδΣϯμ
    ● TVerʹ͍ͭͯ
    ● TVerϦχϡʔΞϧͱ࣍ੈ୅ݕࡧج൫
    ○ ϦχϡʔΞϧޙͷݕࡧػೳ
    ○ ։ൃମ੍
    ○ Amazon OpenSearch Service
    ● ࣮ݱʹ޲͚ͯͷऔΓ૊Έ
    ○ σʔλ؅ཧ
    ○ ετϨʔδ؅ཧ
    ○ αʔϏεͷӡ༻ɾ։ൃ
    ● ·ͱΊ

    View Slide

  15. ໨ࢦ͢΋ͷ
    ● σʔλͷ؅ཧ؍఺
    ○ MySQLͷσʔλมߋʹ΋ॊೈʹରԠͰ͖Δ
    ● ετϨʔδ؍఺
    ○ ӡ༻தͷετϨʔδ࢖༻ྔ૿ՃΛ཈ࢭ͢Δ
    ● αʔϏεӡ༻؍఺
    ○ ݕࡧAPIͷϝϯςφϯείετΛ࠷খԽ͢Δ

    View Slide

  16. ैདྷͷγεςϜߏ੒
    ݕࡧAPI
    CMS
    ݕࡧ݁Ռ
    ίϯςϯπ
    ࢀর
    σʔλͷྲྀΕ

    View Slide

  17. ݱࡏͷγεςϜߏ੒
    ݕࡧAPI
    औΓࠐΈ
    όον
    CMS
    ݕࡧ݁Ռ
    ެ։։࢝


    ίϯςϯπ
    ެ։։࢝


    ίϯςϯπ
    ެ։த


    ίϯςϯπ
    new
    old
    Alias
    ެ։த


    ίϯςϯπ
    ࢀর
    ࢀর
    ࢀর
    σʔλͷྲྀΕ

    View Slide

  18. σʔλͷ؅ཧ
    ● CMSͷσʔλ͸ܗࣜมߋ͞ΕΔ͜ͱ͕͋Δ
    ެ։தίϯςϯπΛ


    ಉظ
    όοΫΞοϓͳͲઐ
    ༻ͷσʔλͷ؅ཧ͕
    ඞཁ
    ➢ େݩͷίϯςϯπ৘ใͷ؅ཧ͸MySQLʹ೚ͤɺίϐʔΛOpenSearch
    ʹఆظతʹ౤ೖ͢Δ
    ݕࡧAPI
    ݕࡧ݁Ռ

    View Slide

  19. σʔλͷ؅ཧ
    ࢀর
    σʔλͷྲྀΕ
    ݕࡧAPI
    औΓࠐΈ
    όον
    CMS
    ެ։։࢝


    ίϯςϯπ
    ެ։த


    ίϯςϯπ
    Index
    ެ։த


    ίϯςϯπ
    ࢀর
    ͦͷ೔ެ։தͷίϯ
    ςϯπ৘ใΛఆظత
    ʹMySQL͔Βऔಘ͠
    ͯ౤ೖ͢Δ
    ݕࡧ݁Ռ

    View Slide

  20. σʔλͷ؅ཧ
    ݕࡧAPI
    औΓࠐΈ
    όον
    CMS
    ݕࡧ݁Ռ
    ެ։։࢝


    ίϯςϯπ
    ެ։։࢝


    ίϯςϯπ
    ެ։த


    ίϯςϯπ
    ެ։த


    ίϯςϯπ
    ࢀর
    ࢀর
    σʔλͷྲྀΕ
    Index
    ࢀর
    ͦͷ೔ެ։ɾมߋ͞Εͨ
    ίϯςϯπΛMySQLͱ
    OpenSearchʹ౤ೖ͢Δ

    View Slide

  21. ετϨʔδ؍఺
    ● ݟಀ͠഑৴ϝΠϯͷͨΊɺଟ͘ͷίϯςϯπ͕1िؒ΄ͲͰެ։ظ
    ͕ؒऴྃ͠ɺݕࡧର৅֎ʹͳΔ
    ● ݕࡧର৅֎͕஝ੵ͞ΕɺແବͳσΟεΫ࢖༻ྔ͕೔ʑ૿Ճ
    ➢ ެ։ظ͕ؒऴྃͨ͠ίϯςϯπ͸Index͔Β࡟আ
    Index
    2022/09/01 ެ։
    2022/09/08 ެ։
    2022/09/15 ެ։
    Index
    2022/09/01 ެ։
    2022/09/08 ެ։
    2022/09/15 ެ։
    1िؒޙ

    View Slide

  22. ● औΓࠐΈόον͸ຖճ৽͍͠IndexΛ࡞੒͠ɺͦͪΒʹίϯςϯ
    π৘ใΛ౤ೖ͢Δ
    ● ݹ͍Index͸࡟আ͢Δ
    ετϨʔδ؍఺
    ݕࡧAPI
    औΓࠐΈ
    όον
    ݕࡧ݁Ռ
    ެ։த


    ίϯςϯπ
    new
    old
    ࢀর

    View Slide

  23. αʔϏεӡ༻؍఺
    ● IndexΛ࡞Γସ͑Δ͝ͱʹࢀরΛߋ৽͢Δඞཁ͕͋Δ
    ݕࡧAPI
    چIndex
    ৽Index
    ࢀরͷ੾Γସ͕͑


    ඞཁ
    ݹ͍


    ίϯςϯπ
    ৽͍͠


    ίϯςϯπ
    ➢ ݕࡧAPIʹ͸AliasΛݟ͓͖ͤͯɺཪͰIndexΛߋ৽͢Δ

    View Slide

  24. ● ݕࡧ༻AliasΛݕࡧAPIʹݟ͓ͤͯ͘
    Index੾Γସ͑
    ݕࡧAPI
    औΓࠐΈ
    όον
    ݕࡧ݁Ռ
    old
    Alias
    ࢀর
    ࢀর
    ࢀর
    σʔλͷྲྀΕ

    View Slide

  25. ● AliasͷཪͰ࠷৽ͷIndexʹ੾Γସ͑
    Index੾Γସ͑
    ݕࡧAPI
    औΓࠐΈ
    όον
    ݕࡧ݁Ռ
    ެ։த


    ίϯςϯπ
    new
    old
    Alias
    ࢀর
    ࢀর
    ࢀর
    σʔλͷྲྀΕ

    View Slide

  26. ࣮ࡍͷӡ༻
    ݕࡧAPI
    औΓࠐΈ
    όον
    CMS
    ݕࡧ݁Ռ
    ެ։։࢝


    ίϯςϯπ
    ެ։։࢝


    ίϯςϯπ
    ެ։த


    ίϯςϯπ
    new
    old
    Alias
    ެ։த


    ίϯςϯπ
    ࢀর
    ࢀর
    ࢀর
    σʔλͷྲྀΕ

    View Slide

  27. ࣮ࡍͷӡ༻
    ݕࡧAPI
    औΓࠐΈ
    όον
    CMS
    ݕࡧ݁Ռ
    ެ։։࢝


    ίϯςϯπ
    ެ։։࢝


    ίϯςϯπ
    ެ։த


    ίϯςϯπ
    new
    old
    Alias
    ެ։த


    ίϯςϯπ
    ࢀর
    ࢀর
    ࢀর
    σʔλͷྲྀΕ
    ͦͷ೔ެ։தͷίϯςϯ
    π৘ใΛຖே౤ೖ͠ɺ
    IndexΛ࡞Γସ͑

    View Slide

  28. ࣮ࡍͷӡ༻
    ݕࡧAPI
    औΓࠐΈ
    όον
    CMS
    ݕࡧ݁Ռ
    ެ։։࢝


    ίϯςϯπ
    ެ։։࢝


    ίϯςϯπ
    ެ։த


    ίϯςϯπ
    new
    old
    Alias
    ެ։த


    ίϯςϯπ
    ࢀর
    ࢀর
    ࢀর
    σʔλͷྲྀΕ
    ίϯςϯπऔΓࠐΈ࣌ʹ
    ࢀর͢ΔIndexΛมߋ

    View Slide

  29. ࣮ࡍͷӡ༻
    ެ։ɾมߋ͞Εͨί
    ϯςϯπΛ౎౓౤ೖ
    ݕࡧAPI
    औΓࠐΈ
    όον
    CMS
    ݕࡧ݁Ռ
    ެ։։࢝


    ίϯςϯπ
    ެ։։࢝


    ίϯςϯπ
    ެ։த


    ίϯςϯπ
    new
    old
    Alias
    ެ։த


    ίϯςϯπ
    ࢀর
    ࢀর
    ࢀর
    σʔλͷྲྀΕ

    View Slide

  30. ΞδΣϯμ
    ● TVerʹ͍ͭͯ
    ● TVerϦχϡʔΞϧͱ࣍ੈ୅ݕࡧج൫
    ○ ϦχϡʔΞϧޙͷݕࡧػೳ
    ○ ։ൃମ੍
    ○ Amazon OpenSearch Service
    ● ࣮ݱʹ޲͚ͯͷऔΓ૊Έ
    ○ σʔλ؅ཧ
    ○ ετϨʔδ؅ཧ
    ○ αʔϏεͷӡ༻ɾ։ൃ
    ● ·ͱΊ

    View Slide

  31. ·ͱΊ
    ● TVerͷϦχϡʔΞϧʹ൐͍ɺݕࡧػೳͷΠϯϑϥͱͯ͠
    Amazon OpenSearch ServiceΛಋೖͨ͠
    ● TVerʹదͨ͠ར༻ํ๏Λ࠾༻ͨ͠
    ○ ఆظతʹMySQL͔Βίϯςϯπ৘ใΛ౤ೖ͢Δ
    ○ Index͝ͱ࡞Γସ͑Δ
    ○ ݕࡧAPIʹ͸Alias͚ͩΛݟͤΔ
    ●͜ΕΒͷࢪࡦʹΑΓɺ҆৺ͯ͠ӡ༻Ͱ͖Δݕࡧج൫Λ࣮ݱ

    View Slide

  32. ࠓޙͷ՝୊
    ● ͞ΒͳΔӡ༻ϊ΢ϋ΢ͷ஝ੵ
    ● Մ༻ੑͷ޲্
    ● ϨίϝϯυͳͲଞͷࢪࡦͱͷ࿈ಈ

    View Slide

  33. View Slide