Upgrade to Pro — share decks privately, control downloads, hide ads and more …

まさかEC2にインストールなんかしていないよね?~Amazon ESのメリット/デメリット~

まさかEC2にインストールなんかしていないよね?~Amazon ESのメリット/デメリット~

2020/2/7(金)に開催された「Amazon Elasticsearch Service & Open Distro for Elasticsearch Meetup」にて発表した資料です。

NAVITIME JAPAN

February 07, 2020
Tweet

More Decks by NAVITIME JAPAN

Other Decks in Technology

Transcript

  1. Our Company • גࣜձࣾ φϏλΠϜδϟύϯ • ઃཱ 2000 ೥ 3

    ݄ 1 ೔ • ۀ຿಺༰ • φϏήʔγϣϯαΠτɾΞϓϦͷӡӦ։ൃ • ௨৴ΧʔφϏήʔγϣϯࣄۀ • ܦ࿏୳ࡧΤϯδϯͷϥΠηϯεࣄۀ ͳͲ • ैۀһ਺ ໿ 540 ໊ (ΤϯδχΞ 80% Ҏ্ʣ
  2. Our Products NAVITIME ৐׵NAVITIME όεNAVITIME ͜ΈΕΆ ެڞަ௨ υϥΠϒ ΧʔφϏλΠϜ υϥΠϒαϙʔλʔ

    τϥοΫΧʔφϏ πʔϦϯά πʔϦϯάαϙʔλʔ ࣗసंNAVITIME Travel/Inbound JapanTravel NAVITIME Transit NAVITIME Travel Plat ϔϧεέΞ ALKOO PC/SP Web
  3. Our Products Φ΢ϯυϝσΟΞ ޿ࠂ / ΞϥΠΞϯε ϏδωεφϏλΠϜ ಈଶ؅ཧιϦϡʔγϣϯ ϏδωεφϏλΠϜ ަ௨අਫ਼ࢉύοέʔδ

    ςϨϚςΟΫεαʔϏε API / SDK ిమɾόεࣄۀऀ޲͚ ιϦϡʔγϣϯ ަ௨ίϯαϧςΟϯά ϏδωεφϏλΠϜ ަ௨අਫ਼ࢉύοέʔδ ϏδωεφϏλΠϜ ಈଶ؅ཧιϦϡʔγϣϯ ޿ࠂ/ΞϥΠΞϯε Φ΢ϯυϝσΟΞ ަ௨ίϯαϧςΟϯά ిమ/όεࣄۀऀ޲͚ ιϦϡʔγϣϯ API / SDK ςϨϚςΟΫεαʔϏε
  4. ʲ࿩͢͜ͱʳ • Amazon Elasticsearch Serviceͷ؂ࢹ/ӡ༻ํ๏ͷ঺հ • EC2্ͰElasticsearchΛߏஙͨ͠ͱ͖ͷ೉఺ • UltraWarm for

    Amazon Elasticsearch Serviceʹ͍ͭͯຊ౰ʹগ͠ ʲ࿩͞ͳ͍͜ͱʳ • Elasticsearchࣗମͷઆ໌ • Elasticsearchࣗମͷվળํ๏
  5. 2013೥ Ŗ 2015೥ Ŗ 2016೥ Ŗ σʔλ෼ੳج൫ͷҰ෦ΛAWS؀ڥͰߏங։࢝ ަ௨ίϯαϧςΟϯάࣄۀɺNAVITIME TransitͰAWSར༻։࢝ φϏλΠϜαʔϏεͷAWSҠߦϓϩδΣΫτελʔτ

    ৐׵NAVITIME͕AWSར༻։࢝ σʔλ෼ੳج൫ΛAthena+S3ʹҠߦ ॱ࣍ɺφϏλΠϜαʔϏεΛAWS؀ڥ΁Ҡߦ ܦ࿏୳ࡧΤϯδϯʹAWSͷGPUΛ࠾༻ ΄΅͢΂ͯͷαʔϏε͕AWS্ͰՔಇ 2017೥ Ŗ 2019೥ Ŗ φϏλΠϜδϟύϯͷAWSͷ׆༻
  6. v3. ݱࡏͷߏ੒ • Amazon ES΁Ҡߦ • ΞΫηε͸VPCΤϯυϙΠϯτܦ༝ υϥΠϒαϙʔλʔ ΞϓϦέʔγϣϯαʔό APIαʔό

    ֤छσʔλ(DB) ஍ਤ഑৴αʔό ܦ࿏୳ࡧαʔό ֤छσʔλ(ES) ֤छσʔλ(Solr)
  7. • Πϯελϯεͷ؅ཧΛAWS͕΍ͬͯ͘ΕΔ • ઐ༻ϚελʔϊʔυΛઃఆ͢Δ͜ͱͰෛՙ͕ܰݮͰ͖Δ • ݁ՌɺΠϯελϯε୆਺͕࡟ݮͰ͖ͨ • EC2࣌ΑΓ΋ίετ͕໿3ׂݮ Pros Cons

    • Amazon ES࢖ͬͯ΋Φʔτεέʔϧ͸Ͱ͖ͳ͍ • ElasticsearchόʔδϣϯΞοϓ࣌ʹ੾Γ໭͕ͮ͠͠Β͍ • ઃఆมߋ࣌ʹBlue/GreenσϓϩΠͷΑ͏ͳڍಈʹͳΔ(͕͔͔࣌ؒΔ)
  8. • CloudWatchμογϡϘʔυ • ElasticsearchRequestsɿϦΫΤετ਺(Sum) • ClusterStatusɹɹɹɹ ɿΫϥελঢ়ଶ(Avg) • Nodesɹɹɹɹɹɹɹɹɿϊʔυ਺(Avg/Min) •

    4xx/5xxErrorɹɹɹɹɹɿΤϥʔ਺(Sum) • Read/Search LatencyɹɿϨΠςϯγ(Avg/p99) • CPU UtilizationɹɹɹɹɿCPU࢖༻཰(Avg) • JVMMemoryPressureɹɿJavaώʔϓ%(Max) Amazon ESͷ؂ࢹ
  9. • Amazon ESͱ͍͏ΑΓ΋Elasticsearchͱͯ͠Φʔτεέʔϧ ʹ͸ରԠ͍ͯ͠ͳ͍† • Amazon ESͰ͸σʔλϊʔυͷ਺Λ૿ݮͤ͞Δ͜ͱ͸Մೳ • σʔλϊʔυΛม͑ΔͱϨϓϦΧ਺Λมߋ͢Δඞཁ͕͋Δ Amazon

    ESͷӡ༻ ʙAutoScaleʙ † https://www.elastic.co/guide/en/cloud/current/ec-faq.html#faq-autoscale LambdaΛ༻͍ͯΦʔτεέʔϧΛྗٕͰ࣮૷
  10. Amazon ESͷӡ༻ ʙAutoScaleʙ • SearchLatencyΛ؂ࢹ • SearchLatency͕ѱԽͨ͠ͷΛݕ஌ͨ͠Β LambdaΛൃՐ • Lambda͔ΒAmazon

    ESͷσʔλϊʔυΛ૿ Ճͤ͞Δ • ૿Ճ͕׬ྃޙɺϨϓϦΧ਺Λมߋ͢ΔϦΫΤ ετΛAmazon ESͷΤϯυϙΠϯτʹରͯ͠ ౤͛Δ
  11. • μ΢ϯάϨʔυ • Elasticsearch6ܥˠ7ܥ΁ͷΞοϓάϨʔυ࣌ʹϨεϙϯε ͷ಺༰͕มΘΔมߋ͕ೖͬͨͷʹؾ͔ͮͣΞοϓάϨʔυ
 -> hits.total is now an

    object in the search response† • Amazon ES΁ͷϦΫΤετͰΤϥʔ͕ଟൃ… • μ΢ϯάϨʔυํ๏Λ໛ࡧ… Amazon ESͷӡ༻ ʙUpgrade&Downgradeʙ † https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#hits-total-now-object-search-response
  12. ࠓޙͷ՝୊ • Φʔτεέʔϧ৚݅ͷݟ௚͠ • όʔδϣϯΞοϓ࣌ʹSearchLatency͕ѱԽ͢Δ͜ͱ͕͋ ΔͷͰɺSearchLatencyҎ֎ͷ৚݅ΛࢦఆͰ͖ͳ͍͔ݕ౼ • Elasticsearch on EC2ͷ๾໓

    • ·ͩҰ෦࢒͍ͬͯΔ؀ڥ͕͋ΔͷͰݱࡏҠߦத • 2݄தʹαʔϏεར༻தͷ΋ͷ͸׬શҠߦͰ͖ΔݟࠐΈ • OpenDistroࢼͯ͠ΈΔ