Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Amazon ECS デプロイツール ecspresso 開発5年の歩み

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →

Amazon ECS デプロイツール ecspresso 開発5年の歩み

AWS Dev Day Tokyo 2023

Avatar for FUJIWARA Shunichiro

FUJIWARA Shunichiro

June 22, 2023

More Decks by FUJIWARA Shunichiro

Other Decks in Technology

Transcript

  1. @fujiwara SREνʔϜ ISUCON 1,2,5,11 ! ༏উ4ճ ISUCON 3,8,12,13 ӡӦ(ग़୊)4ճ github.com/kayac/ecspresso

    Amazon ECS σϓϩΠπʔϧ github.com/fujiwara/lambroll AWS Lambda σϓϩΠπʔϧ
  2. ecspresso ͱ͸ github.com/kayac/ecspresso ൃԻ͸ʮΤεϓϨοιʯ ☕ Amazon Elastic Container Service (Amazon

    ECS) ͷͨΊͷσϓϩΠπʔϧ ʮλεΫఆٛʯͱʮαʔϏεʯΛϑΝΠϧͰ؅ཧ͢Δ Go࣮૷ γϯάϧόΠφϦͰಈ࡞͢ΔCLI λεΫఆٛͱαʔϏεҎ֎͸(جຊతʹ)؅ཧ͠ͳ͍
  3. λεΫఆٛɺαʔϏεͷઃఆ͸AWS Command Line Interface (AWS CLI)ޓ׵ͷJSONϑΝΠϧ ϑΝΠϧ͸ςϯϓϨʔτͱͯ͠ѻΘΕΔ ྫ: σϓϩΠ͝ͱʹՄมͷΠϝʔδλάΛ؀ڥม਺Ͱࢦఆ //

    ඞਢͷ؀ڥม਺ "image": "debian:{{ must_env `IMAGE_TAG` }}" // ઃఆ͞Εͯͳ͍৔߹ͷσϑΥϧτ஋Λࢦఆ "image": "nginx:{{ env `IMAGE_TAG` `latest` }}"
  4. AWS CloudFormation, AWS Cloud Development Kit (CDK), Terraformͱ࿈ܞՄೳ ґଘ͢ΔϦιʔεͷIDΛϋʔυίʔυͤͣʹղܾ #

    ઃఆϑΝΠϧ ecspresso.yml plugins: - name: tfstate config: url: s3://my-bucket/terraform.tfstate // αʔϏεఆٛϑΝΠϧ "securityGroups": [ "{{ tfstate `aws_security_group.default.id` }}" ],
  5. ӡ༻ʹศརͳαϒίϚϯυ͕ͨ͘͞Μ run: ୯ମλεΫΛ࣮ߦ (RunTask) status: αʔϏεͷঢ়ଶΛදࣔ scale: αʔϏεͷλεΫ਺Λมߋ refresh: αʔϏεͷλεΫΛશͯೖΕ׵͑

    diff: σϓϩΠલʹมߋࠩ෼Λදࣔ verify: σϓϩΠલʹ࢖༻͢Δ֎෦ϦιʔεΛ֬ೝ exec: Amazon ECS ExecͰλεΫʹ઀ଓ (·ͩ·ͩ͋Γ·͢)
  6. ecspressoΛ͝ར༻ͷօ༷ 2 ϒοΫ΢ΥʔΧʔ, ϕΨίʔϙϨʔγϣϯ, LayerX ϑΝϯϑΝʔϨ, ίωώτ, εϚʔτϥ΢ϯυ, ͸ͯͳ CARTA

    HOLDINGS, ΧϯϜ, Classi, ϑΥʔΠοτ PR TIMES, ArkEdge Space, τϥετόϯΫ, λΠϛʔ Retty, MICIN, ηʔϑΟʔ, AmebaϚϯΨ, BASE ϚωʔϑΥϫʔυ, STORES, Kakedas, mediba Nature, Connected Base, (ΧϠοΫ) 2 ॱෆಉɺܝࡌڐՄΛಘͨ΋ͷͷΈ
  7. AWS Summit Tokyo 2023 ʮχϯςϯυʔΞΧ΢ϯτ ϦϊϕʔγϣϯϓϩδΣΫτʯͷൃද3 www.itmedia.co.jp/news/articles/ 2305/11/news048.html ΑΓҾ༻ ҰํͰFargateͷઃఆ͸Amazon

    CDKΛ࢖ͬͯ ͍ͳ͍ͱ͍͏ɻίʔυͷઃஔΛࣗಈԽ͢Δπʔϧ ͱͷ૬ੑͷ໰୊͕ྑ͘ͳ͍͜ͱ͕ཧ༝ͱ͍͍ɺ ୅ΘΓʹΧϠοΫͷΦʔϓϯιʔειϑτ΢ΣΞ ʮecspressoʯʢΤεϓϨοιʣΛ׆༻ͨ͠ ͱ͍ͯ͠Δɻ 3 https://pages.awscloud.com/rs/112-TZM-766/images/CUS-20AWS- Summit-2023Nintendo-Systems_FINAL.pdf
  8. ଞͷπʔϧ΋ௐ΂ͯΈΔ • ecs-deploy: λεΫఆٛࣗମͷίʔυ؅ཧ͸Ͱ͖ͳ͍ github.com/silinternational/ecs-deploy • ecs-cli: ϩʔϧόοΫ͕Ͱ͖ͳ͍ github.com/aws/amazon-ecs-cli ϩʔϧόοΫ

    = 1ͭલͷλεΫఆٛΛαʔϏεʹઃఆ͢Δ ! ʮAPI/CLIΛ਺ճݺͿ͚ͩͳΒࣗ࡞Ͱ͖ΔͷͰ͸…?ʯ ʮҰݟ؆୯ͦ͏ͳ΋ͷͰ΋ɺࣗલ࣮૷͸প΁ͷୈҰาʯ
  9. ઃܭࢥ૝(1) αʔϏεɺλεΫఆٛͷ؅ཧʹಛԽ ʲཧ༝ʳଞͷϦιʔεͱϥΠϑαΠΫϧ ͕ҧ͏͜ͱ͕ଟ͍ͨΊ • Amazon VPC, ALB, Amazon RDS...

    ௕ظؒ࢖͏ɺมߋ͸සൟͰ͸ͳ͍ • λεΫఆٛɺαʔϏε σϓϩΠ͢Δͨͼʹߋ৽ ؅ཧ͢Δओମ͕ҧ͏͜ͱ΋ଟ͍ ୯ҰͷπʔϧͰશͯΛ؅ཧ͢Δͱɺҙਤ ͠ͳ͍มߋ΍ࣄނ͕ى͜ΔՄೳੑ͕ߴ͍
  10. v0.1.0 ecspresso --help usage: ecspresso --config=CONFIG [<flags>] <command> [<args> ...]

    Flags: --help Show context-sensitive help (also try --help-long and --help-man). --config=CONFIG config file Commands: help [<command>...] Show help. deploy [<flags>] deploy service status [<flags>] show status of service rollback [<flags>] rollback service
  11. v0.1.0 $ find . -name '*.go' | xargs wc -l

    32 ./service.go 35 ./config.go 56 ./cmd/ecspresso/main.go 288 ./ecspresso.go 411 ߹ܭ
  12. 2017.11.29 AWS Fargate GA6 2017.12.07 ecspresso v0.2.2 AWS FargateରԠ σϓϩΠπʔϧΛࣗ࡞͢Δ

    → Amazon ECSͷ৽ػೳʹରԠ͢ΔͨΊʹ͸ ࣗ෼Ͱ࣮૷͢Δඞཁ͕͋Δ 6 https://aws.amazon.com/jp/blogs/aws/aws-fargate/
  13. 2018೥ 2018.03.30 v0.4.0 deleteίϚϯυ࣮૷ 2018.04.05 v0.5.0 runίϚϯυ࣮૷ @ijin͞Μ͔Βrun taskΛ࣮૷ͨ͠ Pull

    Request 2018.11.15 Secrets Support for Amazon ECS7 2018.11.20 v0.6.5 SecretsରԠ 7 https://aws.amazon.com/jp/about-aws/whats-new/2018/11/aws-launches- secrets-support-for-amazon-elastic-container-servic/
  14. 2019೥ 2019.04.10 CircleCIͱecspressoʹΑΔ ECS΁ͷσϓϩΠϝϯτύΠϓϥΠϯ8 ॳΊͯͷଞࣾࣄྫ 2019.09.11 --no-wait Pull Request @Songmu9

    2019.09.12 v0.9.0 waitίϚϯυ࣮૷ CI/CDͰͷར༻͕૿͖͑ͯͨ 9 https://github.com/kayac/ecspresso/pull/38 8 https://devblog.thebase.in/entry/2019/04/10/110000
  15. 2019.10.08 AWS Dev Day Tokyo 2019 ୈ1ճ AWS Fargate ͔ΜͨΜσϓϩΠબखݖ10

    @torics ͞Μʹ঺հ͞ΕΔ 10 https://speakerdeck.com/toricls/the-very-first-aws-fargate-easy-deployment- tooling-championship
  16. ecspresso init ͷൃ໌ ecspresso͕ಈ࡞͢ΔͨΊʹ͸࠷௿3ͭͷϑΝΠϧ͕ඞཁ͕ͩ… • ઃఆϑΝΠϧ: READMEΛݟͯงғؾͰ • λεΫఆٛϑΝΠϧ: aws

    ecs describe-task-definition • αʔϏεఆٛϑΝΠϧ: ࣗ෼ͰͳΜͱ͔ ͜Ε͕໘౗ͩͬͨ ! طଘͷαʔϏεͷ৘ใΛݩʹ࡞ΕΔͷͰ͸?
  17. 2019.10.12 ecspresso v0.11.0 initίϚϯυ࣮૷ طଘͷΫϥελ໊ɺαʔϏε໊Λࢦఆͯ͠ઃఆϑΝΠϧΛੜ੒ $ ecspresso init --cluster default

    --service myapp 2023/05/31 14:46:00 myapp/default save service definition to ecs-service-def.json 2023/05/31 14:46:00 myapp/default save task definition to ecs-task-def.json 2023/05/31 14:46:00 myapp/default save config to ecspresso.yml $ tree . ├── ecs-service-def.json ├── ecs-task-def.json └── ecspresso.yml
  18. init ҎલʗҎޙ Ҏલ: ઃఆϑΝΠϧΛखॻ͖͢Δͱ࢖͑Δ → ڧ͍ڵຯ͕͋Δਓ͔͠࢖͑ͳ͍ Ҏޙ: ECSͰಈ͘αʔϏεΛ͍࣋ͬͯΔਓ͸ͩΕͰ΋ ίϚϯυҰൃͰ࢖͑Δ ެࣜπʔϧ΍ஶ໊OSS:

    ֶश͢Δಈػ͕ڧ͍(ඞཁੑ͕͋Δ) ऑখOSS: ϋʔυϧ͕ߴ͍ͱ୯ʹ࢖ΘΕͳ͍(ಈػ͕ऑ͍) ʲڭ܇ʳઃఆϑΝΠϧΛ޷ΜͰॻౕ͖͍ͨ͸͍ͳ͍
  19. Dev Day 2019Ҏ߱ GitHub ΋։ൃ΋Ճ଎ 2019.10.16 v0.12.0 CodeDeployʹΑΔBlue/GreenσϓϩΠ 2020.03.31 v0.14.0

    tfstateࢀরػೳ௥Ճ 2020೥2݄͔ΒίϩφՒ ༵ۚ໷ʹҿΈʹߦ͔ͳ͘ͳΔ → ՈͰOSS։ൃΛ͢ΔΑ͏ʹ
  20. Amazon ECSͷػೳ௥Ճʹߴ଎ʹ௥ै͍ͯ͘͠ ͜ͷ͜Ζɺ৽ػೳ͕ଟ͔ͬͨ Fargate Spot, Amazon Elastic File System (EFS)

    mount, Fargate Platoform 1.4.0, S3 envfile... ৽ػೳϦϦʔεޙɺ͙͢ʹαϙʔτ͠ଓ͚͍ͯ͘ 2020.01.17 Amazon ECS͔ΒAmazon EFSϚ΢ϯτՄೳʹ13 2020.01.20 ecspresso v0.13.5 EFSϚ΢ϯταϙʔτ ͳͥ͜Ε͕Մೳͳͷ͔ 13 https://aws.amazon.com/jp/about-aws/whats-new/2020/01/amazon-ecs-preview-support-for-efs-file-systems-now-available/
  21. ઃܭࢥ૝ ࠶ܝ (1) Amazon ECSαʔϏεͱλεΫఆٛͷ؅ཧʹಛԽ → มߋ͢ΔՕॴ͕࠷খݶʹͳΔ (2) Amazon ECSઐ༻ͷπʔϧͱͯ͠࡞Δ

    → Լखʹந৅Խ͍ͯ͠ͳ͍ͨΊɺSDK͕ఏڙ͢ΔΞοϓσʔτ Λͦͷ··ड͚ೖΕΒΕΔ มߋʹߴ଎ʹ௥ैͰ͖Δઃܭͩͬͨ(݁Ռ࿦)
  22. v1.0 ! 2020.09.05 v0.99.0 (v1ͷͨΊͷpre-release൛) 2020.10.20 AWS Dev Day Tokyo

    ୈ2ճ AWS Fargate ͔ΜͨΜσϓϩΠબखݖ15 2020.10.20 ecspresso v1.0.0 ͍ͭ·Ͱ΋v0.xͩͬͨͷͰɺ֮ޛΛܾΊͯv1ϦϦʔε େ͖ͳඇޓ׵͸ͳ͠ 15 https://speakerdeck.com/toricls/the-easiest-deployment-championship-2020-find-your-winner-for-aws-fargate
  23. όʔδϣχϯάํ਑ جຊతʹ͸Semantic Versioning v{major}.{minor}.{patch} • ϝδϟʔόʔδϣϯ͕ಉҰͳΒඇޓ׵͸ೖΕͳ͍ (v1, v2...) • όάमਖ਼ʹΑΔڍಈมߋ͸΍ΉΛಘͳ͍৔߹͕͋Δ

    • ػೳ͕૿͑ͨΒϚΠφʔόʔδϣϯΛ্͛Δ (v1.1, v1.2...) • όάमਖ਼͸ύονόʔδϣϯͰ (v1.1.1, v1.1.2...) • খ͍͞ػೳ௥Ճ͸ύονόʔδϣϯͰೖΔ͜ͱ͕͋Δ
  24. 2021 ػೳ௥Ճ͕ଓ͘ظ 2021.01.12 v1.3.0 CloudFormation plugin 2021.03.15 Amazon ECS Exec

    GA17 2021.04.10 v1.5.0 tasks, exec ίϚϯυ௥Ճ 2021.10.29 v1.7.0 ఆٛϑΝΠϧྨʹJsonnetαϙʔτ 17 https://aws.amazon.com/jp/about-aws/whats-new/2021/03/amazon-ecs-now-allows-you-to-run-commands-in-a-container- running-on-amazon-ec2-or-aws-fargate/
  25. ecspresso exec Amazon ECS Exec (ಈ࡞தͷλεΫʹ઀ଓͯ͠ίϚϯυΛ࣮ߦͰ͖Δ) ʹରԠ͢Δ ecspresso exec Λ࣮૷

    (ޙ͔Βࢥ͑͹)͜Ε͸ͪΐͬͱҳ୤ؾຯ ઃܭࢥ૝(1) αʔϏεͱλεΫఆٛͷ؅ཧʹಛԽ ❌ exec͸ಈ࡞தͷλεΫͦͷ΋ͷΛ૬खʹ͢Δػೳ σϓϩΠΛ؅ཧ͢Δ = ฏ࣌ ಈ࡞தͷλεΫʹԿ͔͢Δ = ҟৗ࣌
  26. 2022 ͦͯ͠v2΁ 2021-01-19 aws-sdk-go-v2 GA18 ecspresso͸aws-sdk-goʹґଘ͍ͯ͠Δ SDK͕ߋ৽͞Εͳ͍ = ৽ػೳʹ௥ैͰ͖ͳ͍ =

    ࠔΔ(ࣗ෼͕) aws-sdk-go (v1)͸Ͳ͏ͳΔ?? 18 https://aws.amazon.com/jp/about-aws/whats-new/2021/01/aws-sdk-for-go-version-2-now-generally-available/
  27. AWS SDKϝϯςφϯεϙϦγʔ20 1. General Availability (GA) GAʹͳͬͨSDK͸࠷௿24ϲ݄ؒ͸αϙʔτ 2. Maintenance Announcement

    SDK͕ϝϯςφϯεϞʔυʹͳΔ6ϲ݄લʹΞφ΢ϯε 3. Maintenance ϝϯςφϯεϞʔυʹೖΔͱॏཁͳόάमਖ਼ͷΈ ৽ػೳͷ௥Ճ͸ͳ͠ → ৽ػೳͷߋ৽(อূ)͸ऴྃΞφ΢ϯε͔Β൒೥ؒ 20 https://docs.aws.amazon.com/sdkref/latest/guide/maint-policy.html
  28. github.com/fujiwara/ecsta ecspresso͔ΒλεΫૢ࡞ػೳΛ੾Γग़ͨ͠πʔϧ(ϥΠϒϥϦ) ೚ҙͷλεΫʹexecͳͲͷૢ࡞͕Ͱ͖ͯศར describe: λεΫͷ৘ใΛJSONͰදࣔ exec: ECS ExecͰλεΫʹ઀ଓ list: Ϋϥελ಺ͷλεΫҰཡΛදࣔ

    logs: λεΫ͕CloudWatch Logsʹग़ྗͨ͠ϩάΛදࣔ portforward: ECS ExecΛ࢖ͬͨϙʔτϑΥϫʔυ stop: λεΫΛఀࢭ trace: λεΫ͕ىಈ→ఀࢭ·ͰͷΠϕϯτɺϩάΛ࣌ܥྻදࣔ