Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AWS上に構築する メンテ容易なElasticsearch System / Maintainable Elasticsearch system on AWS

moznion
August 24, 2019

AWS上に構築する メンテ容易なElasticsearch System / Maintainable Elasticsearch system on AWS

Kyoto.なんか #5 (https://kyoto-nanka.connpass.com/event/141982/) の資料です

moznion

August 24, 2019
Tweet

More Decks by moznion

Other Decks in Technology

Transcript

  1. Amazon ES service - ߏங؆୯ - εέʔϧΞοϓɾεέʔϧΞ΢τ͠์୊ - ఆظతͳindexόοΫΞοϓ΋औͬͯ͘ΕΔ -

    όʔδϣϯΞοϓ͕νϣʔ؆୯ʂʂ - ESࣗମͷϝτϦΫεʹԠͯ͡AlartΛ্͛Δͱ͔΋Ͱ͖Δ - Error Log΍Slow Query Log·Ͱ͋Δ (͜Ε͸CloudWatch Logsʹग़Δ)
  2. Kinesis Firehoseͱͷ૊Έ߹Θͤ - Kinesis FirehoseΛσϑΥϧτͷঢ়ଶͰ࢖͏ͱྲྀྔ੍ݶ͕͋Δ - > For EU (Paris),

    Asia Pacific (Mumbai), US East (Ohio), EU (Frankfurt), South America (São Paulo), Asia Pacific (Seoul), EU (London), Asia Pacific (Tokyo), US West (N. California), Asia Pacific (Singapore), Asia Pacific (Sydney), and Canada (Central): 1,000 records/second, 1,000 transactions/second, and 1 MiB/second. - Ҿ͔͔ͬͬͨ৔߹͸Kinesis StreamΛผ్༻ҙ͠·͠ΐ͏
  3. DynamoDB Streamsͱͷ૊Έ߹Θͤ - ϓϥΠϚϦετϨʔδ (DynamoDB) ͱElasticsearchͷ
 ಉظΛͱΔͷ͕νϣʔָ - DynamoDBͷόοΫΞοϓ͔ΒͷESͷ෮ݩ΋νϣʔָ -

    ͳΜΒ͔೚ҙͷσʔλݕࡧΛఏڙ͢Δ༻్޲͖ - ͜͜Ͱ࿩ͨ͠
 https://www.slideshare.net/SORACOM/soracom-ug-tokyo-9-2018iot-lt- 87355107
  4. Index template updater - Χλ͘ESΛ࢖͓͏ͱ͢Δͱindex template͕΄΅ඞਢ - αʔϏε͝ͱͷindex templateͷద༻Λ͍͍ͪͪ୭͔ʹ
 ґཔ͍ͯͯ͠͸໘౗

    - ʮ͓ئ͍͠·͢೴ͷڪා™ʯ - S3ʹtemplate jsonஔ͚͹ࣗಈͰద༻͞ΕΔͱศར͡ΌΜʁ - ࣮ࡍศར
  5. ࠔΓ͝ͱ - ࣗಈͰϦγϟʔσΟϯάͯ͘͠Εͳ͍ - 1 nodeͰ΋Disk FullʹͳΔͱॻ͖ࠐΈ͕ϒϩοΫ͢Δ - ʮଞͷnode͸disk͕५୔ʹ͋Δͷʹ……ʯ -

    ϦγϟʔσΟϯά͸ҎԼʹΑͬͯىͤ͜Δ - εέʔϧΞοϓɾεέʔϧΞ΢τ - ESͷόʔδϣϯΞοϓ - Diskෆ଍ݕग़࣍ୈࣗಈͰεέʔϧΞ΢τͯ͠εέʔϧΠϯ͢Δ
 Lambdaͳ͍͠͸Step FunctionsͰղܾ (๫ྗ)
  6. Q?