ECS/Fargateの活用事例 (Ruby on Rails編)

【アソビュー×ランサーズ】AWSでのオーケストレーションツールの活用事例(EKS vs ECS)

https://lancersrecruit.connpass.com/event/187989/

Transcript

1. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
   ECS/Fargateͷ׆༻ࣄྫ
   (Ruby on Railsฤ)
   ϥϯαʔζגࣜձࣾ
   SRE/҆ୡ ྋ
   

   View Slide

2. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
   ΞδΣϯμ
   ɾࣗݾ঺հ
   ɾݱࡏऔΓ૊ΜͰ͍Δ͜ͱ
   ɾECS/FargateͷϝϦοτσϝϦοτ
   ɾ։ൃ؀ڥΛ࡞ΔϙΠϯτ
   ɾStg/ຊ൪؀ڥͷߏ੒ʹ͍ͭͯ
   ɾSSH/࡞ۀ༻ίϯςφʮdevopsʯʹ͍ͭͯ
   ɾCircleCIʹΑΔίϯςφσϓϩΠ
   ɾECS Scheduled TasksʹΑΔόον࣮૷ʹ͍ͭͯ
   ɾϩά/ίϯςφ؂ࢹʹ͍ͭͯ
   ɾ·ͱΊ
   

   View Slide

3. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
   ࣗݾ঺հ
   

   View Slide

4. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
   ࣗݾ঺հ
   ɾ҆ୡ ྋ(adachin)
   ɾLancers SRE 2018~
   ɾ@adachin0817
   ɾRVIRUS0817
   ɾblog.adachin.me
   ɾwiki.adachin.me
   ɾOSS :
   ɾCakePHPͰݸਓαʔϏε։ൃத
   

   View Slide

5. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
   ݱࡏऔΓ૊ΜͰ͍Δ͜ͱ
   

   View Slide

6. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
   ݱࡏऔΓ૊ΜͰ͍Δ͜ͱ
   ɾڈ೥ळ
   ɾάϧʔϓձࣾ(γΫϩϚʔέςΟϯά)
   ɾ͘͞ΒͷΫϥ΢υͰӡ༻
   ɾॳΊͯECS/FargateʹҠߦνϟϨϯδ
   ɾࠓ೥4݄ʹҠߦ׬ྃ
   ɹɾࠓճ͸ͪ͜Βͷ࿩Λ͠·͢!!
   ɾݱࡏ(5݄͔Β)
   ɾάϧʔϓձࣾ(Lancers Agency)ɹ
   ɾECS/FargateͰҠߦத!!
   

   View Slide

7. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
   EC2ͱൺֱͨ͠ECS/Fargate
   ͷϝϦοτσϝϦοτ
   

   View Slide

8. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
   EC2ͱൺֱͨ͠ECS/FargateͷϝϦοτσϝϦοτ
   ϝϦοτ
   ɾଐਓԽ͕ൃੜ͠ͳ͍
   ɾImmutableͳαʔόʔͷӡ༻
   ɹɾ҆ఆӡ༻ɺӡ༻ޮ཰ͷΞοϓ
   ɾϗετΠϯελϯεͷ؅ཧ͕ল͚Δ
   ɾλεΫ୯ҐͰࣗ༝ʹϦιʔεมߋՄೳ
   ɾεέʔϧΞ΢τͷ͠΍͢͞
   σϝϦοτ
   ɾσόοά͕ͮ͠Β͍
   ɾdocker exec΍SSH͕Ͱ͖ͳ͍
   ɾλεΫͷىಈ଎౓͕஗Ί
   ɹɾόον࣮ߦ΋਺෼஗͘ͳΔ
   ɾྉ͕͓ۚߴΊ
   ɹɾEC2ΑΓׂߴʢ1ׂʙ2ׂʣ
   

   View Slide

9. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
   ։ൃ؀ڥΛ࡞Δ
   ϙΠϯτ
   

   View Slide

10. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    ։ൃ؀ڥΛ࡞ΔϙΠϯτ
    ɾຊ൪؀ڥͱಉ౳ͷߏ੒Λ࠶ݱ
    ɾELB(H2O)ίϯςφ(ϦόʔεϓϩΩγ)
    ɾApp(ίϯςφ)
    ɹɾruby:2.5.7
    ɹɾRails4.2.11
    ɹɾNginx
    ɾMySQLίϯςφ
    ɹɾ5.7
    ɾ֤ΞϓϦέʔγϣϯͷىಈ͸SupervisorͰ؅ཧ
    ɾbundle install͸AppίϯςφͰϩάΠϯ࣌ʹ࣮ߦ
    ɹɾsupervisorctlͰΞϓϦΛrestart͢ΔΑ͏ʹ
    

    View Slide

11. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    Stg/ຊ൪؀ڥͷ
    ߏ੒ʹ͍ͭͯ
    

    View Slide

12. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    Stg/ຊ൪؀ڥͷߏ੒ʹ͍ͭͯ
    ɾTerraformͰશϦιʔείʔυԽ
    ɾStg؀ڥ͸λεΫ਺͸1ͭ
    ɾdevopsίϯςφͷར༻
    ɾDB͸RDS AuroraʹҠߦ
    ɾը૾͸S3ʹอଘ
    ɾϩά͸CloudWatch Logs
    ɾόον͸ECS Scheduled Tasks
    ɾ಺෦؂ࠪ(ϩάͷू໿)
    ɹɾAmazon Kinesis
    ɾσϓϩΠ͸CircleCIͷΈͰ࣮૷
    ɾ؂ࢹ͸Datadog
    

    View Slide

13. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    SSH/࡞ۀ༻ίϯςφ
    ʮdevopsʯʹ͍ͭͯ
    

    View Slide

14. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    SSH/࡞ۀ༻ίϯςφʮdevopsʯʹ͍ͭͯ
    ɾAppʹ͸SSH͠ͳ͍
    ɾSSH/࡞ۀ༻ίϯςφ(Appͱಉ౳)
    ɹɾdevopsͱ໋໊
    ɾίϚϯυͰRDS΁઀ଓ
    ɹɾdump΍restore
    ɾaws cliͷར༻
    ɾrails consoleͰͷσόοά
    ɾSSMͷSession Manager ΋͋Δ͕
    ɹɾखܰʹ౿Έ୆͔ΒSSHΛ͍ͨͨ͠Ί
    ɾVPC PeeringͰϥϯαʔζͷωοτϫʔΫ͔ΒSSHϩάΠϯ
    

    View Slide

15. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    CircleCIʹΑΔ
    ίϯςφσϓϩΠʹ͍ͭͯ
    

    View Slide

16. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    CircleCIʹΑΔStg/ຊ൪σϓϩΠʹ͍ͭͯ(ॳظ)
    ɾCircleCIͷOrbs
    ɹɾDockerfileͷbuild
    ɹɾECR΁push
    ɹɾECSϦϏδϣϯͷߋ৽
    ɹɾαʔϏελεΫఆٛͷߋ৽
    ɹɾDBϚΠάϨʔγϣϯ
    ɹɾrspec
    ɾgit flow
    ɾdevelopϒϥϯν
    ɹɾϚʔδ͢ΔͱStgʹϦϦʔε
    ɾmasterϒϥϯν
    ɹɾϚʔδ͢Δͱຊ൪ʹϦϦʔε
    

    View Slide

17. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    APIܦ༝ͰͷStg؀ڥͷσϓϩΠ(ݱࡏ)
    ɾStg؀ڥͷσϓϩΠ
    ɹɾgithub flow
    ɹɾγΣϧεΫϦϓτ
    ɹɹɾ೚ҙͷϒϥϯνΛࢦఆ
    ɾૉૣ͘σϓϩΠ͕Մೳ
    ɹɾ։ൃޮ཰͕Ξοϓ
    ɾTerraform CI؀ڥΛ࣮૷
    $ sh deploy.sh
    ԼهͷΑ͏ʹϒϥϯνΛࢦఆ࣮ͯ͠ߦ͍ͯͩ͘͠͞ɻ
    (ྫ) sh deploy.sh ϒϥϯν໊
    $ sh deploy.sh fix-nginx
    {
    "number" : 74,
    "state" : "pending",
    "id" : "xxxxxx-xxxxx-xxxxx-xxx-xxxxxxxx",
    "created_at" : "2020-08-27T06:14:43.973Z"
    }
    ϒϥϯν໊/ fix-nginx ͷStgσϓϩΠΛ։࢝͠·ͨ͠!
    

    View Slide

18. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    όονͷ࣮૷ʹ͍ͭͯ
    

    View Slide

19. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    ECS Scheduled TasksʹΑΔόον࣮૷ʹ͍ͭͯ
    ɾࠓ·Ͱ
    ɹɾgem whenever
    ɹɾschedule.rb
    ɾݱࡏ
    ɹɾECS Scheduled Tasks
    ɹɹɾCloudWatchͷΠϕϯτϧʔϧ
    ɹɹɾTerraformԽ
    {
    "containerOverrides": [
    {
    "name": "HogeWorker",
    "command": ["bundle","exec","rails","runner","HogeWorker.new.execute"]
    }
    ]
    }
    

    View Slide

20. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    ϩάʗίϯςφ؂ࢹ
    ʹ͍ͭͯ
    

    View Slide

21. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    ϩά/ίϯςφ؂ࢹʹ͍ͭͯ
    ɾΞΫηε/ΞϓϦϩάɺRDS؂ࠪ/Τϥʔϩά
    ɹɾCloudWatch Logsʹ
    ɹɾࠓ·Ͱ
    ɹɹɾLambdaͰS3ʹγϯΫɹɹ
    ɹɹɾPythonͰ࣮૷͍ͯͨͨ͠Ίӡ༻ίετΞοϓ
    ɹɾࠓճ
    ɹɹɾAmazon Kinesis Data FirehoseͰS3ʹू໿
    ɹɹɾTerraformԽ
    ɾ؂ࢹ
    ɹɾMackerel→DatadogʹҠߦ
    ɹɾλεΫఆٛʹdatadog-agentΛಈ࡞ͤ͞ΔΑ͏ʹ
    ɹɾcpu:10ɺmemory:256 TerraformԽ
    

    View Slide

22. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    ·ͱΊ
    

    View Slide

23. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    ·ͱΊ
    ɾγϯϓϧͳߏ੒Ͱ࣮ݱͰ͖ͨ
    ɾখ͍͞αʔϏε͔ΒECS/FargateԽத
    ɾDockerΠϝʔδͱλεΫఆٛͰӡ༻Մೳ
    ɾॳΊͯίϯςφຊ൪ӡ༻͢Δʹ͸͔ͳΓϋϚΔ
    ɹɾϔϧενΣοΫ͕௨Βͳ͍ͱίϯςφ͕མͪͨΓ
    ɹɾσϓϩΠͷ࣮૷͕େม
    ɹɾLinuxྗ͕ࢼ͞ΕΔ
    ɾࠓޙ
    ɹɾRuby2.7.1/Rails6΁ͷόʔδϣϯΞοϓ
    ɹɾίϯςφͷ੬ऑੑݕ஌ (Trivy)
    ɹɾσϓϩΠ͕10෼͔͔ΔͷͰվળ
    ɹɾChatOpsͰͷϦϦʔε
    ɹɾDatadogͰϩά࿈ܞ
    ɹɾϥϯαʔζຊՈ΋ίϯςφҠߦʹνϟϨϯδ
    

    View Slide

24. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    SREٻΉʂʂ
    ECS/Fargate/PHPͰͷ؀ڥΛ͓࿩͠·͢ʂ
    

    View Slide

25. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)
    

    View Slide