Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AWSサービスで実現するバッチ実行環境のコンテナ/サーバレス化/ Container service of batch execution environment realized by AWS service

AWSサービスで実現するバッチ実行環境のコンテナ/サーバレス化/ Container service of batch execution environment realized by AWS service

AWS DevDay Tokyo 2019での発表資料です

Shoichiro Nagai(shnagai)

October 04, 2019
Tweet

More Decks by Shoichiro Nagai(shnagai)

Other Decks in Technology

Transcript

  1. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. T O K Y O 2 0 1 9 . 1 0 . 0 3 - 0 4 AWSαʔϏεͰ࣮ݱ͢Δ όον࣮ߦ؀ڥͷίϯςφ/αʔόϨεԽ ӬҪ উҰ࿠ ίωώτגࣜձࣾ © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. T O K Y O 2 0 1 9 . 1 0 . 0 3 - 0 4 AWSαʔϏεͰ࣮ݱ͢Δ όον࣮ߦ؀ڥͷίϯςφ/αʔόϨεԽ ӬҪ উҰ࿠ ίωώτגࣜձࣾ G - 5
  2. ࣗݾ঺հ ӬҪউҰ࿠ ओͳ׆ಈ
 +BQBO$POUBJOFS%BZTW "844VNNJU
 ͸͡Ίͯͷ1)1ϓϩϑΣογϣφϧ։ൃ ೔ܦ4:45&.4 ϚϚϦ͕࣮ફ͢Δɹίϯ ςφҠߦͷਖ਼߈๏ 


    ίωώτΤϯδχΞϒϩάIUUQTUFDIDPOOFIJUPDPNBSDIJWFBVUIPSOBHBJT ίωώτגࣜձࣾɹΤϯδχΞ
 Πϯϑϥ΢ΣϒΦϖϨʔγϣϯػցֶश ࠷ۙ  !TIOBHBJ
  3.   ˞ʮӾཡ਺ʯʮར༻ऀ਺ʯ͸ϝσΟΞͱΞϓϦͷ߹ܭ஋ʢ೥݄݄ͷฏۉ஋ʣ ˞ʮϚϚ޲͚/PΞϓϦʯ͸೥݄Πϯςʔδௐ΂ɹௐࠪର৅ɿ೛৷தʙ̎ࡀ̌ϲ݄ͷࢠڙΛ࣋ͭঁੑ O  Λநग़ ˞*OTUBHSBNͷϑΥϩϫʔ਺ɺ'BDFCPPLͷ͍͍Ͷ਺ɺ-*/&ͷͱ΋ͩͪ਺ͷ߹ܭ஋ ೥݄࣌఺ 

    ϚϚϦ ΞϓϦɾ8FC 4/4 *OTUBHSBNɾ-*/&ɾ'BDFCPPL هࣄ ϚϚಉ࢜Ͱ೰ΈΛ૬ஊ͠߹͏2"ίϛϡχςΟΛத৺ʹ ϢʔβʔΛ֦େ͍ͯ͠·͢ ʮϚϚϦʯͰϢʔβʔಉ͕࢜ ͲΜͲΜܨ͕͍ͬͯ·͢ ϚϚͷੜ׆ʹ໾ཱͭهࣄΛ ෯޿͍δϟϯϧͰ഑৴͍ͯ͠·͢ ϚϚ޲͚/P̍ΞϓϦʹબग़  ਓͷϚϚ͕બͿʮݱࡏ࢖͍ͬͯΔΞϓϦʯʹͯɺ ߲໨ ଞͷϚϚʹΦεεϝ͍ͨ͠ɺೝ஌౓ɺ
 ར༻཰ɺརศੑɺ޷ײ౓ Ͱ̍ҐΛ֫ಘ͠·ͨ͠ هࣄ਺ 6,000 هࣄҎ্ ྦྷܭϑΝϯ਺ ໿ 91 ສਓ ˞ ݄ؒӾཡ਺ ໿ 1.5ԯճ ˞ ݄ؒར༻ऀ਺ ໿ 650ສਓ ˞ ˞ l೰ΈzͱzڞײzΛ࣠ʹϚϚʹدΓఴ͍ ΞϓϦɾ8FCɾ4/4ͱଟ֯తʹαʔϏεΛల։͍ͯ͠·͢ "CPVUϚϚϦ
  4. 0 450,000 900,000 1,350,000 1,800,000 2014/4 2014/5 2014/6 2014/7 2014/8

    2014/9 2014/10 2014/11 2014/12 2015/1 2015/2 2015/3 2015/4 2015/5 2015/6 2015/7 2015/8 2015/9 2015/10 2015/11 2015/12 2016/1 2016/2 2016/3 2016/4 2016/5 2016/6 2016/7 2016/8 2016/9 2016/10 2016/11 2016/12 2017/1 2017/2 2017/3 2017/4 2017/5 2017/6 2017/7 2017/8 2017/9 2017/10 2017/11 2017/12 2018/1 2018/2 2018/3 2018/4 2018/5 2018/6 2018/7 2018/8   ݄ؒ౤ߘ਺ ໿ 150ສ݅ िʹ೔Ҏ্ىಈ͢Δ ΞΫςΟϒϢʔβʔ ໿ 50 ਓʹਓ 57$. ์ө ΞϓϦ૯%-਺ສ ਓʹਓ ਓʹਓ ਓʹਓ ਓʹਓ ˞ ˞ʮϚϚϦʯ಺ͷग़࢈༧ఆ೔Λઃఆͨ͠Ϣʔβʔ਺ͱɺްੜ࿑ಇলൃදʮਓޱಈଶ౷ܭʯͷग़ੜ਺͔Βࢉग़
 ˞िʹճҎ্ىಈ͢ΔϢʔβʔ ˞ ೥ʹग़࢈ͨ͠ϚϚͷʮਓʹਓʯ͕ϚϚϦΛར༻த ೔ຊ࠷େڃن໛ΛތΔϒϥϯυ΁ͱ੒௕͍ͯ͠·͢ ˞ ೔ຊ࠷େڃن໛ΛތΔϒϥϯυ΁
  5. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. ϗετܕόον࣮ߦ؀ڥͷ՝୊ΛৼΓฦΔ
  6. ୆ͷόοναʔό αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 1୆ͷEC2Ͱӡ༻ cronͰεέδϡʔϧ؅ཧ ༷ʑͳόονॲཧΛू໿ େ͖ͳίϯϐϡʔςΟϯάϦιʔε
  7. ୆ߏ੒ʹͯ͠৑௕Խ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2
  8. όονຖʹ"DUJWF4UBOECZʹ͢Δ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 # αʔϏεAόον: cron(10 0 * * *) # αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό
  9. ୆ࢮΜͩΒखಈͰϑΣΠϧΦʔόʔ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό ☓ ίϝϯτΞ΢τ ֎ͯ͠Activeʹ
  10. ෳ਺ͷαʔόʹશ͘ಉ͡มߋΛߦ͏ඞཁ͕͋Δ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 # αʔϏεAόον: cron(10 0 * * *) # αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό ΞϓϦέʔγϣϯσϓϩΠ ϓϥοτϑΥʔϜͷมߋ
  11. ෳ਺ͷαʔόʹશ͘ಉ͡มߋΛߦ͏ඞཁ͕͋Δ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 # αʔϏεAόον: cron(10 0 * * *) # αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό ΞϓϦέʔγϣϯσϓϩΠ ϓϥοτϑΥʔϜͷมߋ όοναʔόʹ͔͔Δӡ༻ෛՙ͸େ͖͍
  12. w ϦιʔεΛڞ༗͢Δ؍఺͔Β៛ີͳεέδϡʔϧ؅ཧ͕ඞཁ
 ݶΒΕͨϚγϯϦιʔεΛڞ༗͢ΔͷͰॏ͍ॲཧ͕ॏͳΒͳ͍Α͏ߟྀ
 ៛ີͳεέδϡʔϧ؅ཧ όον؅ཧ୆ா 
 δϣϒεέδϡʔϥͷಋೖˠδϣϒεέδϡʔϥࣗମͷӡ༻͕ൃੜ
 εέδϡʔϦϯάͷܦҢ౳͸Ұ෦ʹଐਓԽ͢Δ͜ͱ΋͓͖͕ͪ w අ༻͕ߴ͘ͳΓ͕ͪ


    Ұ൪ϚγϯϦιʔεΛ࢖͏ॲཧΛϕʔεʹΠϯελϯεΛܾఆ͠ৗ࣌ىಈ
 ৑௕Խ΍໾ׂຖόοναʔόΛ࡞Δͱߋʹඅ༻͕ߴ͘ͳΔ
 Ϋϥ΢υͷ৔߹͸ɺඞཁͳ࣌ʹ࢖ͬͨ෼͚ͩͷඅ༻Λ෷͏ܗʹ͍ͨ͠ ϗετܕόονॲཧͷ՝୊
  13. w ϦιʔεΛڞ༗͢Δ؍఺͔Β៛ີͳεέδϡʔϧ؅ཧ͕ඞཁ
 ݶΒΕͨϚγϯϦιʔεΛڞ༗͢ΔͷͰॏ͍ॲཧ͕ॏͳΒͳ͍Α͏ߟྀ
 ៛ີͳεέδϡʔϧ؅ཧ όον؅ཧ୆ா 
 δϣϒεέδϡʔϥͷಋೖˠδϣϒεέδϡʔϥࣗମͷӡ༻͕ൃੜ
 εέδϡʔϦϯάͷܦҢ౳͸Ұ෦ʹଐਓԽ͢Δ͜ͱ΋͓͖͕ͪ w අ༻͕ߴ͘ͳΓ͕ͪ


    Ұ൪ϚγϯϦιʔεΛ࢖͏ॲཧΛϕʔεʹΠϯελϯεΛܾఆ͠ৗ࣌ىಈ
 ৑௕Խ΍໾ׂຖόοναʔόΛ࡞Δͱߋʹඅ༻͕ߴ͘ͳΔ
 Ϋϥ΢υͷ৔߹͸ɺඞཁͳ࣌ʹ࢖ͬͨ෼͚ͩͷඅ༻Λ෷͏ܗʹ͍ͨ͠ ϗετܕόονॲཧͷ՝୊ όονॲཧΛαʔόϨεԽ͢Δ͜ͱͰղܾ
  14. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. ʲࣄྫ1ʳ εέδϡʔϧܥόονॲཧΛίϯςφԽͯ͠ ECS ScheduleTaskʹҠߦ
  15. ᶃՄൖੑͷߴ͞Λ׆͔͢͜ͱͰϓϥοτϑΥʔϜมߋͷෑډ͕Լ͕Δ εςʔδϯά؀ڥ ϓϩμΫγϣϯ؀ڥ ։ൃ؀ڥ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ όοναʔό ࣮ߦ؀ڥ

    όοναʔό ࣮ߦ؀ڥ ΞϓϦέʔγϣϯ ࣮ߦ؀ڥ ΞϓϦέʔγϣϯ ࣮ߦ؀ڥ ઃఆมߋ ઃఆมߋ σϓϩΠ σϓϩΠ σϓϩΠ σϓϩΠ ઃఆมߋ͸ίϯςφ಺Ͱ׬݁
  16. ᶃՄൖੑͷߴ͞Λ׆͔͢͜ͱͰϓϥοτϑΥʔϜมߋͷෑډ͕Լ͕Δ εςʔδϯά؀ڥ ϓϩμΫγϣϯ؀ڥ ։ൃ؀ڥ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ όοναʔό ࣮ߦ؀ڥ

    όοναʔό ࣮ߦ؀ڥ ΞϓϦέʔγϣϯ ࣮ߦ؀ڥ ΞϓϦέʔγϣϯ ࣮ߦ؀ڥ ઃఆมߋ ઃఆมߋ σϓϩΠ σϓϩΠ σϓϩΠ σϓϩΠ ઃఆมߋ͸ίϯςφ಺Ͱ׬݁ σϓϩΠཻ౓͕ΞϓϦέʔγϣϯίʔυ͔Β ΞϓϦέʔγϣϯ࣮ߦج൫ͦͷ΋ͷʹมΘΔ
  17. w ίϯςφPODSPO -BNCEB "84#BUDI౳Λൺֱݕ౼
 ౰࣌ͷΞʔΩςΫνϟબఆ࣌ͷQSPTDPOT͸ͪ͜Βʹ
 ʲ"NB[PO&$44DIFEVMF5BTLͰ࣮ݱ͢ΔεϚʔτͳ%PDLFSϕʔεͷόον࣮ߦ؀ڥʳ 
 IUUQTUFDIDPOOFIJUPDPNFOUSZ  w

    શͯͷཁ݅Λຬͨͨ͠ͷͰ&$44DIFEVMF5BTLͷ࠾༻Λܾఆ
 ͜Ε͔Β΍ΔͳΒཁ݅ʹΑͬͯ͸"84#BUDIͰ΋͍͍ͷ͔΋ େن໛σʔλ౳ 
 ౰࣌͸εέδϡʔϧىಈʹରԠ͍ͯ͠ͳ͔ͬͨ w ಋೖ౰ॳ͸ଞࣾࣄྫ͕͋·Γͳ͍ͱ͍͏ෆ҆͸͋ͬͨ
 4"ʹ૬ஊͯ͠ΞʔΩςΫνϟͷϨϏϡʔΛड͚Δ
 &$4ͷӡ༻࣮੷͸͋Γ஌ݟ͕͋ͬͨͷͰಋೖΛܾఆ ٕज़બఆ
  18. ΞʔΩςΫνϟ ECR ECS ScheduleTask Task CloudWatch Event 
 (time-based) Aόον:

    cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Task Task Aόον Bόον Cόον pull pull pull EC2 EC2Λ৑௕ߏ੒ʹ͠SPOFճආ ECSλεΫ഑ஔઓུʹଇΓɺόϥ ϯεΑ͘λεΫ͸ىಈ͞ΕΔ
  19. όον࣮ߦͷྲྀΕ +45 ECR ECS ScheduleTask Task CloudWatch Event 
 (time-based)

    Aόον: cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Aόον pull EC2
  20. όον࣮ߦͷྲྀΕ +45 ECR ECS ScheduleTask Task CloudWatch Event 
 (time-based)

    Aόον: cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Task Aόον Bόον pull EC2 Bόονىಈ࣌͸ɺ Aόονىಈ࣌ʹग़དྷͨEC2্ͷ DockerϨΠϠΩϟογϡ͕ޮ͘ͷ Ͱߴ଎ىಈ
  21. Cόον͸ɺσϓϩΠ͞Εͨ৽͠ ͍DockerΠϝʔδͰόονΛىಈ σϓϩΠ࣌ʹىಈ͍ͯͨ͠Bόο ν͸ɺӨڹͳ͘ॲཧΛଓ͚Δ όον࣮ߦͷྲྀΕ +45 ECR ECS CloudWatch Event

    
 (time-based) Aόον: cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Task Task Bόον Cόον pull (new image) EC2
  22. w Ϋϥελ
 λεΫΛಈ͔͢ϗετ &$ ͷू߹ମ
 'BSHBUFͷ࣌͸ɺͨͩͷശ w %PDLFSΠϝʔδ &$3 

    w λεΫఆٛ
 %PDLFSίϯςφͷىಈύϥϝʔλͷఆٛ w Ϋϥελ಺ͷ4DIFEVMF5BTL
 $MPVE8BUDI&WFOUTͱཪͰ࿈ܞ͍ͯ͠Δ &$44DIFEVMF5BTLͰόονΛ૊ΉͨΊͷཁૉ
  23. w Ϋϥελ
 λεΫΛಈ͔͢ϗετ &$ ͷू߹ମ
 'BSHBUFͷ࣌͸ɺͨͩͷശ w %PDLFSΠϝʔδ &$3 

    w λεΫఆٛ
 %PDLFSίϯςφͷىಈύϥϝʔλͷఆٛ w Ϋϥελ಺ͷ4DIFEVMF5BTL
 $MPVE8BUDI&WFOUTͱཪͰ࿈ܞ͍ͯ͠Δ λεΫఆٛΛ5FSSBGPSNͰ؅ཧ λεΫఆٛΛTerraformͰ؅ཧ
  24. όονͷ։ൃϑϩʔ ίϯςφϕʔεͰ όονॲཧΛϩʔΧϧ։ൃ APͷϦϙδτϦͰ όονॲཧͷPR terraformͷϦϙδτϦͰ όονॲཧఆٛͷPR terraformܦ༝Ͱ ຊ൪ʹλεΫఆٛొ࿥ εςʔδϯάͷECSͰ

    Ұ౓खಈ࣮ߦ͠ಈ࡞֬ೝ terraformܦ༝Ͱ εςʔδϯάʹλεΫఆٛొ࿥ ϓϥοτϑΥʔϜมߋ͸ίϯςφͰ׬݁ DockerͷՄൖੑͷߴ͞Ͱͦͷ··ผ؀ڥ΁
  25. όονͷ։ൃϑϩʔ ίϯςφϕʔεͰ όονॲཧΛϩʔΧϧ։ൃ APͷϦϙδτϦͰ όονॲཧͷPR terraformͷϦϙδτϦͰ όονॲཧఆٛͷPR terraformܦ༝Ͱ ຊ൪ʹλεΫఆٛొ࿥ εςʔδϯάͷECSͰ

    Ұ౓खಈ࣮ߦ͠ಈ࡞֬ೝ terraformܦ༝Ͱ εςʔδϯάʹλεΫఆٛొ࿥ ϓϥοτϑΥʔϜมߋ͸ίϯςφͰ׬݁ DockerͷՄൖੑͷߴ͞Ͱͦͷ··ผ؀ڥ΁ ։ൃσϓϩΠ֬ೝͱ͍͏։ൃϑϩʔΛ ͢΂ͯ։ൃऀͷ੹຿Ͱߦ͏͜ͱ͕Ͱ͖Δ
  26. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. ʲࣄྫ2ʳ ػցֶशͷֶशσʔλલॲཧϑϩʔ(Glue + Fargate) ΛStep FunctionsͰඪ४Խ
  27. w ॲཧຖʹεέδϡʔϧ࣮ߦ
 (MVFεέδϡʔϧ࣮ߦ ". 
 'BSHBUFεέδϡʔϧ࣮ߦ ". 
 ࠷ॳͷॲཧ͕͚ͯ͜ޙଓόον͕ಈ͍ͯ΋͍͍৔߹ͷΈʹ࢖͑Δ w

    ॲཧຖʹΤϥʔ௨஌΍ϦτϥΠ੍ޚΛೖΕΔ
 (MVFͰͷϦτϥΠ੍ޚͱΤϥʔ௨஌
 όονॲཧͷίʔυͰϦτϥΠ੍ޚͱΤϥʔ௨஌ wॲཧεςοϓ͕૿͑Δ΄Ͳશମ૾Λ೺Ѳ͢Δͷ͕ࠔ೉ʹͳΔ εέδϡʔϧ࣮ߦύλʔϯ
  28. w ॲཧຖʹεέδϡʔϧ࣮ߦ
 (MVFεέδϡʔϧ࣮ߦ ". 
 'BSHBUFεέδϡʔϧ࣮ߦ ". 
 ࠷ॳͷॲཧ͕͚ͯ͜ޙଓόον͕ಈ͍ͯ΋͍͍৔߹ͷΈʹ࢖͑Δ w

    ॲཧຖʹΤϥʔ௨஌΍ϦτϥΠ੍ޚΛೖΕΔ
 (MVFͰͷϦτϥΠ੍ޚͱΤϥʔ௨஌
 όονॲཧͷίʔυͰϦτϥΠ੍ޚͱΤϥʔ௨஌ wॲཧεςοϓ͕૿͑Δ΄Ͳશମ૾Λ೺Ѳ͢Δͷ͕ࠔ೉ʹͳΔ εέδϡʔϧ࣮ߦύλʔϯ ΋ͬͱศརʹग़དྷΔํ๏͸ͳ͍ͷ͔
  29. w ॲཧຖʹεέδϡʔϧ࣮ߦ
 (MVFεέδϡʔϧ࣮ߦ ". 
 'BSHBUFεέδϡʔϧ࣮ߦ ". 
 ࠷ॳͷॲཧ͕͚ͯ͜ޙଓόον͕ಈ͍ͯ΋͍͍৔߹ͷΈʹ࢖͑Δ w

    ॲཧຖʹΤϥʔ௨஌΍ϦτϥΠ੍ޚΛೖΕΔ
 (MVFͰͷϦτϥΠ੍ޚͱΤϥʔ௨஌
 όονॲཧͷίʔυͰϦτϥΠ੍ޚͱΤϥʔ௨஌ wॲཧεςοϓ͕૿͑Δ΄Ͳશମ૾Λ೺Ѳ͢Δͷ͕ࠔ೉ʹͳΔ εέδϡʔϧ࣮ߦύλʔϯ 4UFQ'VODUJPOTΛ࢖͏͜ͱͰ࣮ݱͰ͖ͨ
  30. w TUBUF
 ࣮ߦ͢Δॲཧ୯Ґ-BNCEB &$4λεΫ (MVFFUD
 5ZQF۟Ͱॲཧ಺༰Λࢦఆ
 5BTL ͳΜΒ͔ͷॲཧ࣮ߦ 
 $IPJDF

    ೖྗʹج͖ͮॲཧ෼ذ 
 8BJU Ұఆ࣌ؒεϦʔϓ  w TUBUFNBDIJOF
 TUFUFͷू߹ମ ϫʔΫϑϩʔຊମ 
 +40/ϕʔεͷ"NB[POεςʔτϝϯτݴޠͰఆٛ
 ͜ͷ୯ҐͰϏδϡΞϥΠζ͞ΕΔ 4UFQ'VODUJPOTͷߏ੒ཁૉ
  31. "-#ϩάΛ,JCBOBͰݟΔ࢓૊ΈΛαʔόϨεʹ Amazon Elasticsearch Service ALB S3 Bucket Lambda Event 


    (time-based) 5෼ຖʹ όέοτʹϩάΛग़ྗ ObjectCreatedτϦΨ ͰLambda͕ىಈ ElasticserachʹBulkLoad GrokͰύʔε͠ɺΠϯ σοΫεొ࿥ ΤϯδχΞ͸ɺ Kibanaܦ༝ͰϩάΛݟΔ աڈ෼ͷΠϯσοΫεΛ ఆظతʹ࡟আ
  32. w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ
 ˠ΋͔ͨ͠͠ΒόονॲཧͰͳͯ͘΋Α͍Մೳੑ΋ඞવతʹαʔόϨεߏ੒ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ
 ˠϥϯλΠϜͷ؅ཧ͕ෆཁͳͷͰ΍Γ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ
 ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w

    ಠࣗͷϥϯλΠϜΛ࢖ͬͯόονॲཧ͕͍ͨ͠
 ˠΞϓϦέʔγϣϯͷίϯςφԽ
 ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ࢖͏ LTͱ͍͏બ୒ࢶ΋ग़ͯ͘Δ  w "84্ͷॲཧͰϫʔΫϑϩʔΛ૊Έ͍ͨ
 ˠ4UFQ'VODUJPOTΛ࢖͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ਑
  33. w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ
 ˠ΋͔ͨ͠͠ΒόονॲཧͰͳͯ͘΋Α͍Մೳੑ΋ඞવతʹαʔόϨεߏ੒ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ
 ˠϥϯλΠϜͷ؅ཧ͕ෆཁͳͷͰ΍Γ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ
 ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w

    ಠࣗͷϥϯλΠϜΛ࢖ͬͯόονॲཧ͕͍ͨ͠
 ˠΞϓϦέʔγϣϯͷίϯςφԽ
 ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ࢖͏ LTͱ͍͏બ୒ࢶ΋ग़ͯ͘Δ  w "84্ͷॲཧͰϫʔΫϑϩʔΛ૊Έ͍ͨ
 ˠ4UFQ'VODUJPOTΛ࢖͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ਑ ӡ༻όον࣮ߦͷ੹຿Λ Ͱ͖Δ͚ͩ"84ʹد͍ͤͯ͘͜ͱͰ ຊདྷՁ஋ͷ͋Δ։ൃʹ஫ྗ͢Δͷ͕େࣄ
  34. w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ
 ˠ΋͔ͨ͠͠ΒόονॲཧͰͳͯ͘΋Α͍Մೳੑ΋ඞવతʹαʔόϨεߏ੒ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ
 ˠϥϯλΠϜͷ؅ཧ͕ෆཁͳͷͰ΍Γ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ
 ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w

    ಠࣗͷϥϯλΠϜΛ࢖ͬͯόονॲཧ͕͍ͨ͠
 ˠΞϓϦέʔγϣϯͷίϯςφԽ
 ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ࢖͏ LTͱ͍͏બ୒ࢶ΋ग़ͯ͘Δ  w "84্ͷॲཧͰϫʔΫϑϩʔΛ૊Έ͍ͨ
 ˠ4UFQ'VODUJPOTΛ࢖͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ਑ )PXͱͯ͠ίϯςφ΍αʔόϨεͱ͍ͬͨ ΞʔΩςΫνϟΛཧղ͠ ͦΕΒΛ࣮αʔϏεʹམͱ͠ࠐΉ࣌ʹ
  35. w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ
 ˠ΋͔ͨ͠͠ΒόονॲཧͰͳͯ͘΋Α͍Մೳੑ΋ඞવతʹαʔόϨεߏ੒ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ
 ˠϥϯλΠϜͷ؅ཧ͕ෆཁͳͷͰ΍Γ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ
 ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w

    ಠࣗͷϥϯλΠϜΛ࢖ͬͯόονॲཧ͕͍ͨ͠
 ˠΞϓϦέʔγϣϯͷίϯςφԽ
 ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ࢖͏ LTͱ͍͏બ୒ࢶ΋ग़ͯ͘Δ  w "84্ͷॲཧͰϫʔΫϑϩʔΛ૊Έ͍ͨ
 ˠ4UFQ'VODUJPOTΛ࢖͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ਑ "84ͩͬͨΓଞͷΫϥ΢υαʔϏεΛ ͍͔ʹ͏·͘࢖͍͚ͬͯΔ͔͕ ͜Ε͔ΒߋʹٻΊΒΕͯ͘ΔεΩϧͩͱࢥ͏
  36. w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ
 ˠ΋͔ͨ͠͠ΒόονॲཧͰͳͯ͘΋Α͍Մೳੑ΋ඞવతʹαʔόϨεߏ੒ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ
 ˠϥϯλΠϜͷ؅ཧ͕ෆཁͳͷͰ΍Γ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ
 ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w

    ಠࣗͷϥϯλΠϜΛ࢖ͬͯόονॲཧ͕͍ͨ͠
 ˠΞϓϦέʔγϣϯͷίϯςφԽ
 ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ࢖͏ LTͱ͍͏બ୒ࢶ΋ग़ͯ͘Δ  w "84্ͷॲཧͰϫʔΫϑϩʔΛ૊Έ͍ͨ
 ˠ4UFQ'VODUJPOTΛ࢖͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ਑ ਐԽͷૣ͍֤αʔϏεΛΩϟονΞοϓ͠ ࣮ࡍʹखΛಈ͔ͯ͠෢ثΛ૿΍͢͜ͱ͕େࣄ Ϋϥ΢υωΠςΟϒͳࢥ૝΁
  37. Thank you! © 2019, Amazon Web Services, Inc. or its

    affiliates. All rights reserved. ӬҪ উҰ࿠ ίωώτגࣜձࣾ