Upgrade to Pro — share decks privately, control downloads, hide ads and more …

日経テレコンを支えるElasticsearch

 日経テレコンを支えるElasticsearch

第38回Elasticsearch勉強会 2020.10.29 #elasticsearchjp
https://www.meetup.com/ja-JP/Tokyo-Elastic-Fantastics/events/273562532/

Taisuke Hinata

October 29, 2020
Tweet

Other Decks in Technology

Transcript

  1. ೔ܦςϨίϯΛࢧ͑Δ
    &MBTUJDTFBSDI
    ೔ຊܦࡁ৽ฉࣾɹ೔ᙛହี
    ୈ38ճ Elasticsearchษڧձ #elasticsearchjp
    2020/10/29

    View Slide


  2. w ໊લ
    w ೔ᙛହี 5BJTVLF)JOBUB

    w !IJOBUBEFT
    w ॴଐ
    w ೔ຊܦࡁ৽ฉࣾΤϯδχΞ৽ଔ೥໨
    w ࢓ࣄ
    w αʔόαΠυɾΠϯϑϥத৺
    w 8FC"1*ݕࡧ
    w (PMBOH1ZUIPO&MBTUJDTFBSDI
    ࣗݾ঺հ

    View Slide

  3. ೔ܦͷσʔλϕʔεࣄۀ
    w ࣾҎ্ͷϝσΟΞاۀ༷ͱίϯςϯπ࿈ܞ
    w ৽ฉ શࠃࢴ஍ํࢴۀքࢴεϙʔπࢴ

    w اۀ৘ใ
    w ۀքϨϙʔτ
    w ਓ෺ɾਓࣄ৘ใ
    w 104 ߪങ
    σʔλ

    View Slide

  4. ࠓ೔࿩͢͜ͱ
    w ೔ܦςϨίϯͱ͸
    w &MBTUJDTFBSDIΛಋೖͨ͠എܠ
    w Ͳ͏࡞ͬͨͷ͔
    w ΞʔΩςΫνϟ
    w Πϯϑϥઃܭ
    w ؂ࢹ
    w Ͳ͏ͳͬͨͷ͔

    View Slide

  5. ೔ܦςϨίϯ
    w ೥ ʂ
    αʔϏε։࢝
    w ࠃ಺࠷େڃͷϏδωεσʔλϕʔεαʔϏε
    w ౦ূҰ෦্৔اۀͷ໿ׂ͕ಋೖ
    w Ҏ্ͷهࣄഔମ
    w ສࣾҎ্ͷࠃ಺֎اۀ৘ใ
    w ສਓҎ্ͷΩʔύʔιϯ৘ใ
    w ͦͷଞଟ਺ ϦϯΫࢀর

    w ίϯςϯπΛશจݕࡧͰϫϯετοϓऩू
    IUUQTUFMFDPNOJLLFJDPKQHVJEF

    View Slide

  6. ࠓ೥݄͔Β&MBTUJDTFBSDI΁
    w ೔ܦςϨίϯݕࡧΤϯδϯͷྺ࢙

    هࣄ
    1999 2000 2001 2002 2003 2004 2005 2006 2007 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022
    2008
    Pana Search
    Pana Search
    اۀ
    ਓࣄ
    +
    ೥શจݕࡧαʔϏε։࢝

    View Slide

  7. ೥݄શίϯςϯπΛ&MBTUJDTFBSDIʹҠߦ
    ࠓ೥݄͔Β&MBTUJDTFBSDI΁
    w ೔ܦςϨίϯݕࡧΤϯδϯͷྺ࢙

    هࣄ
    1999 2000 2001 2002 2003 2004 2005 2006 2007 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022
    2008
    Pana Search
    Pana Search
    اۀ
    ਓࣄ
    +

    View Slide

  8. &MBTUJDTFBSDIΛಋೖͨ͠എܠ

    View Slide

  9. ᶃ୤ɾΦϯϓϨ

    w ݕࡧΤϯδϯΛ୆Ҏ্ͷ෺ཧαʔόͰӡ༻
    w ػثߋ৽ͷͨͼʹ೥Ҏ্͔͔Γίετେ
    w Ϋϥ΢υ্Ͱӡ༻ՄೳͳݕࡧΤϯδϯ͕ඞཁʹ

    View Slide

  10. ᶄίϯςϯπϓϥοτϑΥʔϜߏ૝

    w Ϋϥ΢υҠߦ͠ɺ͞ΒʹͦΕΛશαʔϏεڞ௨ͷ౷߹ݕࡧج൫ʹ
    w ΑΓεέʔϥϏϦςΟͷ͋ΔݕࡧΤϯδϯ͕ඞཁʹ
    2016೥ 2020೥

    View Slide

  11. ͳͥ&MBTUJDTFBSDIͳͷ͔
    w ᶃΫϥ΢υԽ
    w %PDLFS*NBHFͷ഑෍
    w Ϋϥ΢υ1MVHJO͕๛෋ ˞ޙड़

    w ᶄϓϥοτϑΥʔϜԽ
    w ෼ࢄܕεέʔϥϏϦςΟ
    w ϊʔυ௥Ճ͕؆୯
    w σʔλ෼ࢄͱϨϓϦΧࣗಈ഑ஔ

    View Slide

  12. Ͳ͏࡞ͬͨͷ͔

    View Slide

  13. ΞʔΩςΫνϟ֎؍

    ɾ
    ɾ
    ɾ
    ݕࡧ༻υϝΠϯ
    "1*(BUFXBZ
    API
    $%/

    View Slide

  14. ΞʔΩςΫνϟ֎؍

    ɾ
    ɾ
    ɾ
    "1*(BUFXBZ
    API
    ೝূ
    ྲྀྔ੍ݶ

    View Slide

  15. ΞʔΩςΫνϟ֎؍

    ɾ
    ɾ
    ɾ
    "1*(BUFXBZ
    API
    ΫΤϦόϦσʔγϣϯ
    ΫΤϦͷࣙॻల։
    &4ΫΤϦ΁ͷॻ͖׵͑

    View Slide

  16. ΞʔΩςΫνϟ֎؍

    ɾ
    ɾ
    ɾ
    "1*(BUFXBZ
    API
    &MBTUJDTFBSDI༻υϝΠϯ
    ϦΫΤετઌͷΫϥελΛεϫοϓ

    View Slide

  17. &4ΫϥελͷΠϯϑϥઃܭ֓ཁ
    w &MBTUJDTFBSDIPO%PDLFS
    w "NB[PO&$4PO&$
    w ϩʔϦϯάΞοϓσʔτ
    w ΦʔτεέʔϦϯά
    w 4IBSE"MMPDBUJPO"XBSFOFTT

    D D D
    D
    D
    D
    AZ-a AZ-c AZ-d
    .BTUFS %BUB $PPSEJOBUJOH
    Indexing
    Search
    Kibana
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ

    View Slide

  18. &MBTUJDTFBSDIPO%PDLFS

    D D D
    D
    D
    D
    AZ-a AZ-c AZ-d
    .BTUFS %BUB $PPSEJOBUJOH
    Indexing
    Search
    Kibana
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    w Մൖੑɾ؀ڥ࠶ݱੑͷ޲্
    w *NBHF͸%PDLFSϨδετϦͰอଘ
    w όʔδϣϯ؅ཧ
    Dockerfile CI
    Amazon ECR

    View Slide

  19. "NB[PO&$4PO&$

    w "NB[PO&$4
    w ίϯςφΦʔέετϨʔγϣϯ
    w "NB[PO&$
    w αʔόʔ
    w "NB[PO&$"VUP4DBMJOH
    w ϧʔϧϕʔεͰεέʔϦϯά
    D D D
    D
    D
    D
    AZ-a AZ-c AZ-d
    .BTUFS %BUB $PPSEJOBUJOH
    Indexing
    Search
    Kibana
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ

    View Slide

  20. "NB[PO&$4PO&$

    w "NB[PO&$4
    w ίϯςφΦʔέετϨʔγϣϯ
    w "NB[PO&$
    w αʔόʔ
    w "NB[PO&$"VUP4DBMJOH
    w ϧʔϧϕʔεͰεέʔϦϯά
    D D D
    D
    D
    D
    AZ-a AZ-c AZ-d
    .BTUFS %BUB $PPSEJOBUJOH
    Indexing
    Search
    Kibana
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ

    View Slide

  21. "NB[PO&$4PO&$

    w "NB[PO&$4
    w ίϯςφΦʔέετϨʔγϣϯ
    w "NB[PO&$
    w αʔόʔ
    w "NB[PO&$"VUP4DBMJOH
    w ϧʔϧϕʔεͰεέʔϦϯά
    D D D
    D
    D
    D
    AZ-a AZ-c AZ-d
    .BTUFS %BUB $PPSEJOBUJOH
    Indexing
    Search
    Kibana
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ

    View Slide

  22. ᶃϩʔϦϯάΞοϓσʔτ

    w &$4͕ίϯςφ܊ΛΑΖ͘͠؅ཧ
    w ίϯςφͷҰׅߋ৽
    w ίϯςφͷࣗಈ෮چ
    w ϚΠφʔόʔδϣϯΞοϓ ୆
    ͕਺෼Ͱ
    w ˞ॠஅ͕ൃੜ͢ΔͨΊຊ൪؀ڥͰ͸
    D D D
    D
    D
    D
    AZ-a AZ-c AZ-d
    .BTUFS %BUB $PPSEJOBUJOH
    Indexing
    Search
    Kibana
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ

    View Slide

  23. ᶄΦʔτεέʔϦϯά
    w ";ো֐౳Ͱγϟοτμ΢ϯ࣌ʹࣗಈ෮چ
    w ෮چޙɺ&4Ϋϥελ΁ͷొ࿥͕ඞཁ
    w &$%JTDPWFSZ1MVHJO
    w 5BHʹ&4Ϋϥελ໊ͷొ࿥

    D D D
    D
    D
    D
    AZ-a AZ-c AZ-d
    .BTUFS %BUB $PPSEJOBUJOH
    Indexing
    Search
    Kibana
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ

    View Slide

  24. εςʔτϑϧΞϓϦ͸
    σʔλӬଓԽͷ؍఺͔Β
    ͪ͜ΒΛબͿ͸ͣ
    ετϨʔδ͸ԿΛ࢖͏͔

    w &MBTUJDTFBSDI͸εςʔτϑϧͰ͋ΓετϨʔδ*0͕ॏཁ
    w "NB[PO&$
    w
    w σʔλӬଓԽՄೳ
    w ΠϯελϯεετΞ
    w شൃੑ͕ͩߴੑೳ
    ετϨʔδI/O
    Մ༻ੑ
    ΠϯελϯεετΞ

    View Slide

  25. ετϨʔδ͸ԿΛ࢖͏͔

    w &MBTUJDTFBSDI͸εςʔτϑϧͰ͋ΓετϨʔδ*0͕ॏཁ
    w "NB[PO&$
    w
    w σʔλӬଓԽՄೳ
    w ΠϯελϯεετΞ
    w شൃੑ͕ͩߴੑೳ
    ετϨʔδI/O
    Մ༻ੑ
    ΠϯελϯεετΞ

    &4͸γϟʔυ͔Βσʔλ͕
    ࣗಈ෮ݩɻࣗಈ෮چΛલఏ
    ͱ͢Ε͹બ୒Մೳʹ

    View Slide

  26. 4IBSE"MMPDBUJPO"XBSFOFTT

    AZ-a AZ-c AZ-d
    ϓϥΠϚϦγϟʔυ
    ϨϓϦΧγϟʔυ
    ϓϥΠϚϦʹର͢Δ
    ϨϓϦΧ͕ผͷ";ʹ
    ෼ࢄ͞ΕߴՄ༻
    w ෺ཧαʔόͷ;POFΛߟྀͨ͠ϓϥΠϚϦɾϨϓϦΧγϟʔυͷ෼ࢄ഑ஔ
    w ྫ"; ϊʔυ ϓϥΠϚϦγϟʔυ ϨϓϦΧ

    View Slide

  27. 4IBSE"MMPDBUJPO"XBSFOFTT
    w ෺ཧαʔόͷ;POFΛߟྀͨ͠ϓϥΠϚϦɾϨϓϦΧγϟʔυͷ෼ࢄ഑ஔ
    w ྫ"; ϊʔυ ϓϥΠϚϦγϟʔυ ϨϓϦΧ

    AZ-a AZ-c AZ-d
    ϓϥΠϚϦγϟʔυ
    ϨϓϦΧγϟʔυ
    ಉ";಺ʹ*OEFY͕׬੒͠ɺ
    ݕࡧ࣌ʹ";ؒ௨৴͕ෆཁʹ

    View Slide

  28. ؂ࢹʹ͍ͭͯ

    View Slide

  29. ઐ༻ϞχλϦϯάΫϥελ
    w αʔϏεΫϥελͷෛՙ௿ݮ
    w (PMEϥΠηϯεҎ্Ͱར༻Մೳ

    AZ-a
    4JOHMF
    D D D
    D
    D
    D
    AZ-a AZ-c AZ-d
    .BTUFS %BUB $PPSEJOBUJOH
    Indexing
    Search
    Kibana
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    ɾ
    xpack.monitoring.exporters.my_remote.type: http
    xpack.monitoring.exporters.my_remote.host: https://XXX:9200

    View Slide

  30. ,JCBOB%BTICPBSEͷ׆༻
    w &YQPSUͰ͖ΔͷͰҰ౓࡞ͬͯ͠·͑͹࠶ར༻Մೳ

    View Slide

  31. ,JCBOB%BTICPBSEͷ׆༻
    w ,JCBOBϞχλϦϯάʹࡌͬͯͳ͍ϝτϦΫεΛ࡞੒ˍ؂ࢹ

    *OEFYผͷ214؂ࢹ

    View Slide

  32. ӡ༻਍அαʔϏε ΞΫϩΫΤετςΫϊϩδʔࣾ

    w िճͷΫϥελΠϯσοΫε਍அ
    w ݄ճͷϨϙʔτใࠂ
    w ਍அʹඞཁͳ,JCBOB%BTICPBSEͷ࡞੒
    [email protected]@EJBHOPTJT

    View Slide

  33. Ͳ͏ͳͬͨͷ͔

    View Slide

  34. ᶃόʔδϣϯΞοϓ͕Մೳʹ
    w ΦϯϓϨ࣌୅͸όʔδϣϯΞοϓͷϋʔυϧ͕ߴ͘ɺಉόʔδϣϯΛػثߋ৽·Ͱอक
    w όʔδϣϯΞοϓ͕਺෼Ͱ࣮ߦՄೳʹ

    ߋ৽ස౓͕ߴ͍͕
    ͔ͬ͠Γ௥ਵͯ͠
    ͍͖͍ͨ
    IUUQTXXXFMBTUJDDPKQTVQQPSUFPM

    View Slide

  35. ᶄ෮چ࣌ؒͷ୹ॖ
    w ΦϯϓϨ࣌୅͸ো֐ϊʔυͷ੾Γ཭͠ɺ෮ؼ·Ͱ͕͢΂ͯख࡞ۀ
    w &MBTUJDTFBSDIͷϨϓϦέʔγϣϯ΍1MVHJOʹΑΓશࣗಈ෮چՄೳʹ
    w ઌि
    ͷ"84ো֐࣌΋ແࣄࣗಈ෮چ☺

    αʔόγϟʔυ࠶࡞੒Ͱݩͷঢ়ଶʹ

    View Slide

  36. ᶅϥϯχϯάίετ࡟ݮ
    w ෺ཧαʔό୆͔ΒΫϥ΢υαʔό୆ʹ

    ݕࡧΤϯδϯ
    ೥ؒϥϯχϯάίετ
    ໿22.5%ݮ
    ݕࡧΤϯδϯͷ
    ੑೳΞοϓͷ
    د༩͕େ͖͍ͱߟ࡯

    View Slide

  37. ·ͱΊ
    w ೔ܦςϨίϯ͸&MBTUJDTFBSDIʹେ͍ʹࢧ͑ΒΕ͍ͯΔ
    w ͋Γ͕ͱ͏ɺ&MBTUJDTFBSDI

    View Slide

  38. ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ

    View Slide