日経テレコンを支えるElasticsearch

第38回Elasticsearch勉強会 2020.10.29 #elasticsearchjp
https://www.meetup.com/ja-JP/Tokyo-Elastic-Fantastics/events/273562532/

Transcript

1. ೔ܦςϨίϯΛࢧ͑Δ
   &MBTUJDTFBSDI ೔ຊܦࡁ৽ฉࣾɹ೔ᙛହี ୈ38ճ Elasticsearchษڧձ #elasticsearchjp 2020/10/29
2.  w ໊લ
   w ೔ᙛହี
   5BJTVLF
   )JOBUB
   w !IJOBUBEFT
   w ॴଐ
   w ೔ຊܦࡁ৽ฉࣾ
   ΤϯδχΞ
   ৽ଔ೥໨
   w ࢓ࣄ
   w αʔόαΠυɾΠϯϑϥத৺
   w 8FC
   "1*ݕࡧ
   
   w (PMBOH1ZUIPO&MBTUJDTFBSDI ࣗݾ঺հ
3. ೔ܦͷσʔλϕʔεࣄۀ w ࣾҎ্ͷϝσΟΞاۀ༷ͱίϯςϯπ࿈ܞ
   w ৽ฉ
   શࠃࢴ஍ํࢴۀքࢴεϙʔπࢴ
   w اۀ৘ใ
   w ۀքϨϙʔτ
   w ਓ෺ɾਓࣄ৘ใ
   w 104 ߪങ σʔλ 
4. ࠓ೔࿩͢͜ͱ w ೔ܦςϨίϯͱ͸
   w &MBTUJDTFBSDIΛಋೖͨ͠എܠ
   w Ͳ͏࡞ͬͨͷ͔
   w ΞʔΩςΫνϟ
   w Πϯϑϥઃܭ
   w ؂ࢹ
   w Ͳ͏ͳͬͨͷ͔ 
5. ೔ܦςϨίϯ w ೥ ʂ αʔϏε։࢝
   w ࠃ಺࠷େڃͷϏδωεσʔλϕʔεαʔϏε
   w ౦ূҰ෦্৔اۀͷ໿ׂ͕ಋೖ
   w Ҏ্ͷهࣄഔମ
   w  ສࣾҎ্ͷࠃ಺֎اۀ৘ใ
   w ສਓҎ্ͷΩʔύʔιϯ৘ใ
   w ͦͷଞଟ਺ ϦϯΫࢀর
   w ίϯςϯπΛશจݕࡧͰϫϯετοϓऩू  IUUQTUFMFDPNOJLLFJDPKQHVJEF
6. ࠓ೥݄͔Β&MBTUJDTFBSDI΁ w ೔ܦςϨίϯ
   ݕࡧΤϯδϯͷྺ࢙  هࣄ 1999 2000 2001 2002 2003 2004 2005 2006 2007 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2008 Pana Search Pana Search اۀ ਓࣄ + ೥
   શจݕࡧαʔϏε
   ։࢝
7. ೥݄
   શίϯςϯπΛ&MBTUJDTFBSDIʹҠߦ ࠓ೥݄͔Β&MBTUJDTFBSDI΁ w ೔ܦςϨίϯ
   ݕࡧΤϯδϯͷྺ࢙  هࣄ 1999 2000 2001 2002 2003 2004 2005 2006 2007 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2008 Pana Search Pana Search اۀ ਓࣄ +
8. &MBTUJDTFBSDIΛಋೖͨ͠എܠ 
9. ᶃ୤ɾΦϯϓϨ  w ݕࡧΤϯδϯΛ୆Ҏ্ͷ෺ཧαʔόͰӡ༻
   w ػثߋ৽ͷͨͼʹ೥Ҏ্͔͔Γίετେ
   w Ϋϥ΢υ্Ͱӡ༻ՄೳͳݕࡧΤϯδϯ͕ඞཁʹ
10. ᶄίϯςϯπϓϥοτϑΥʔϜߏ૝  w Ϋϥ΢υҠߦ͠ɺ͞ΒʹͦΕΛશαʔϏεڞ௨ͷ౷߹ݕࡧج൫ʹ
    w ΑΓεέʔϥϏϦςΟͷ͋ΔݕࡧΤϯδϯ͕ඞཁʹ 2016೥ 2020೥
11. ͳͥ&MBTUJDTFBSDIͳͷ͔ w ᶃΫϥ΢υԽ
    w %PDLFS
    *NBHFͷ഑෍
    w Ϋϥ΢υ1MVHJO͕๛෋
    ˞ޙड़
    w ᶄϓϥοτϑΥʔϜԽ
    w ෼ࢄܕεέʔϥϏϦςΟ
    w ϊʔυ௥Ճ͕؆୯
    w σʔλ෼ࢄͱϨϓϦΧࣗಈ഑ஔ 
12. Ͳ͏࡞ͬͨͷ͔ 
13. ΞʔΩςΫνϟ֎؍  ɾ ɾ ɾ ݕࡧ༻υϝΠϯ "1*
    (BUFXBZ API $%/
14. ΞʔΩςΫνϟ֎؍  ɾ ɾ ɾ "1*
    (BUFXBZ API ೝূ
    ྲྀྔ੍ݶ
15. ΞʔΩςΫνϟ֎؍  ɾ ɾ ɾ "1*
    (BUFXBZ API ΫΤϦόϦσʔγϣϯ
    ΫΤϦͷࣙॻల։
    &4ΫΤϦ΁ͷॻ͖׵͑
16. ΞʔΩςΫνϟ֎؍  ɾ ɾ ɾ "1*
    (BUFXBZ API &MBTUJDTFBSDI༻υϝΠϯ
    ϦΫΤετઌͷΫϥελΛεϫοϓ
17. &4ΫϥελͷΠϯϑϥઃܭ
    ֓ཁ w &MBTUJDTFBSDI
    PO
    %PDLFS
    w "NB[PO
    &$4
    PO
    &$
    w ϩʔϦϯάΞοϓσʔτ
    w ΦʔτεέʔϦϯά
    w 4IBSE
    "MMPDBUJPO
    "XBSFOFTT  D D D D D D AZ-a AZ-c AZ-d .BTUFS %BUB $PPSEJOBUJOH Indexing Search Kibana ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ
18. &MBTUJDTFBSDI
    PO
    %PDLFS  D D D D D D AZ-a AZ-c AZ-d .BTUFS %BUB $PPSEJOBUJOH Indexing Search Kibana ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ w Մൖੑɾ؀ڥ࠶ݱੑͷ޲্
    w *NBHF͸%PDLFSϨδετϦͰอଘ
    w όʔδϣϯ؅ཧ Dockerfile CI Amazon ECR
19. "NB[PO
    &$4
    PO
    &$  w "NB[PO
    &$4
    w ίϯςφΦʔέετϨʔγϣϯ
    w "NB[PO
    &$
    w αʔόʔ
    w "NB[PO
    &$
    "VUP
    4DBMJOH
    w ϧʔϧϕʔεͰεέʔϦϯά D D D D D D AZ-a AZ-c AZ-d .BTUFS %BUB $PPSEJOBUJOH Indexing Search Kibana ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ
20. "NB[PO
    &$4
    PO
    &$  w "NB[PO
    &$4
    w ίϯςφΦʔέετϨʔγϣϯ
    w "NB[PO
    &$
    w αʔόʔ
    w "NB[PO
    &$
    "VUP
    4DBMJOH
    w ϧʔϧϕʔεͰεέʔϦϯά D D D D D D AZ-a AZ-c AZ-d .BTUFS %BUB $PPSEJOBUJOH Indexing Search Kibana ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ
21. "NB[PO
    &$4
    PO
    &$  w "NB[PO
    &$4
    w ίϯςφΦʔέετϨʔγϣϯ
    w "NB[PO
    &$
    w αʔόʔ
    w "NB[PO
    &$
    "VUP
    4DBMJOH
    w ϧʔϧϕʔεͰεέʔϦϯά D D D D D D AZ-a AZ-c AZ-d .BTUFS %BUB $PPSEJOBUJOH Indexing Search Kibana ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ
22. ᶃϩʔϦϯάΞοϓσʔτ  w &$4͕ίϯςφ܊ΛΑΖ͘͠؅ཧ
    w ίϯςφͷҰׅߋ৽
    w ίϯςφͷࣗಈ෮چ
    w ϚΠφʔόʔδϣϯΞοϓ ୆ ͕਺෼Ͱ
    w ˞ॠஅ͕ൃੜ͢ΔͨΊຊ൪؀ڥͰ͸ D D D D D D AZ-a AZ-c AZ-d .BTUFS %BUB $PPSEJOBUJOH Indexing Search Kibana ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ
23. ᶄΦʔτεέʔϦϯά w ";ো֐౳Ͱγϟοτμ΢ϯ࣌ʹࣗಈ෮چ
    w ෮چޙɺ&4Ϋϥελ΁ͷొ࿥͕ඞཁ
    w &$
    %JTDPWFSZ
    1MVHJO
    w 5BHʹ&4Ϋϥελ໊ͷొ࿥  D D D D D D AZ-a AZ-c AZ-d .BTUFS %BUB $PPSEJOBUJOH Indexing Search Kibana ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ
24. εςʔτϑϧΞϓϦ͸
    σʔλӬଓԽͷ؍఺͔Β ͪ͜ΒΛબͿ͸ͣ ετϨʔδ͸ԿΛ࢖͏͔  w &MBTUJDTFBSDI͸εςʔτϑϧͰ͋ΓετϨʔδ*0͕ॏཁ
    w "NB[PO
    &$
    w &#4
    w σʔλӬଓԽՄೳ
    w ΠϯελϯεετΞ
    w شൃੑ͕ͩߴੑೳ ετϨʔδI/O Մ༻ੑ ΠϯελϯεετΞ &#4
25. ετϨʔδ͸ԿΛ࢖͏͔  w &MBTUJDTFBSDI͸εςʔτϑϧͰ͋ΓετϨʔδ*0͕ॏཁ
    w "NB[PO
    &$
    w &#4
    w σʔλӬଓԽՄೳ
    w ΠϯελϯεετΞ
    w شൃੑ͕ͩߴੑೳ ετϨʔδI/O Մ༻ੑ ΠϯελϯεετΞ &#4 &4͸γϟʔυ͔Βσʔλ͕ ࣗಈ෮ݩɻࣗಈ෮چΛલఏ ͱ͢Ε͹બ୒Մೳʹ
26. 4IBSE
    "MMPDBUJPO
    "XBSFOFTT  AZ-a AZ-c AZ-d ϓϥΠϚϦγϟʔυ ϨϓϦΧγϟʔυ ϓϥΠϚϦʹର͢Δ
    ϨϓϦΧ͕ผͷ";ʹ
    ෼ࢄ͞ΕߴՄ༻ w ෺ཧαʔόͷ;POFΛߟྀͨ͠ϓϥΠϚϦɾϨϓϦΧγϟʔυͷ෼ࢄ഑ஔ
    w ྫ
    ";
    ϊʔυ
    ϓϥΠϚϦγϟʔυ
    ϨϓϦΧ
27. 4IBSE
    "MMPDBUJPO
    "XBSFOFTT w ෺ཧαʔόͷ;POFΛߟྀͨ͠ϓϥΠϚϦɾϨϓϦΧγϟʔυͷ෼ࢄ഑ஔ
    w ྫ
    ";
    ϊʔυ
    ϓϥΠϚϦγϟʔυ
    ϨϓϦΧ  AZ-a AZ-c AZ-d ϓϥΠϚϦγϟʔυ ϨϓϦΧγϟʔυ ಉ";಺ʹ*OEFY͕׬੒͠ɺ
    ݕࡧ࣌ʹ";ؒ௨৴͕ෆཁʹ
28. ؂ࢹʹ͍ͭͯ 
29. ઐ༻ϞχλϦϯάΫϥελ w αʔϏεΫϥελͷෛՙ௿ݮ
    w (PMEϥΠηϯεҎ্Ͱར༻Մೳ  AZ-a 4JOHMF D D D D D D AZ-a AZ-c AZ-d .BTUFS %BUB $PPSEJOBUJOH Indexing Search Kibana ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ xpack.monitoring.exporters.my_remote.type: http xpack.monitoring.exporters.my_remote.host: https://XXX:9200
30. ,JCBOB
    %BTICPBSEͷ׆༻ w &YQPSUͰ͖ΔͷͰҰ౓࡞ͬͯ͠·͑͹࠶ར༻Մೳ 
31. ,JCBOB
    %BTICPBSEͷ׆༻ w ,JCBOBϞχλϦϯάʹࡌͬͯͳ͍ϝτϦΫεΛ࡞੒ˍ؂ࢹ  *OEFYผͷ214؂ࢹ
32. ӡ༻਍அαʔϏε ΞΫϩΫΤετςΫϊϩδʔࣾ w िճͷΫϥελΠϯσοΫε਍அ
    w ݄ճͷϨϙʔτใࠂ
    w ਍அʹඞཁͳ,JCBOB
    %BTICPBSEͷ࡞੒  IUUQTXXXFOEPTOJQFDPNFMBTUJD@DMVTUFS@EJBHOPTJT
33. Ͳ͏ͳͬͨͷ͔ 
34. ᶃόʔδϣϯΞοϓ͕Մೳʹ w ΦϯϓϨ࣌୅͸όʔδϣϯΞοϓͷϋʔυϧ͕ߴ͘ɺಉόʔδϣϯΛػثߋ৽·Ͱอक
    w όʔδϣϯΞοϓ͕਺෼Ͱ࣮ߦՄೳʹ  ߋ৽ස౓͕ߴ͍͕
    ͔ͬ͠Γ௥ਵͯ͠
    ͍͖͍ͨ IUUQTXXXFMBTUJDDPKQTVQQPSUFPM
35. ᶄ෮چ࣌ؒͷ୹ॖ w ΦϯϓϨ࣌୅͸ো֐ϊʔυͷ੾Γ཭͠ɺ෮ؼ·Ͱ͕͢΂ͯख࡞ۀ
    w &MBTUJDTFBSDIͷϨϓϦέʔγϣϯ΍1MVHJOʹΑΓશࣗಈ෮چՄೳʹ
    w ઌि  ͷ"84ো֐࣌΋ແࣄࣗಈ෮چ☺  αʔόγϟʔυ࠶࡞੒Ͱݩͷঢ়ଶʹ
36. ᶅϥϯχϯάίετ࡟ݮ w ෺ཧαʔό ୆͔ΒΫϥ΢υαʔό୆ʹ  ݕࡧΤϯδϯ ೥ؒϥϯχϯάίετ ໿22.5%ݮ ݕࡧΤϯδϯͷ
    ੑೳΞοϓͷ
    د༩͕େ͖͍ͱߟ࡯
37. ·ͱΊ w ೔ܦςϨίϯ͸&MBTUJDTFBSDIʹେ͍ʹࢧ͑ΒΕ͍ͯΔ
    w ͋Γ͕ͱ͏ɺ&MBTUJDTFBSDI 
38. ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ