Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Amazon ECS デプロイツール ecspresso 開発5年の歩み

Amazon ECS デプロイツール ecspresso 開発5年の歩み

AWS Dev Day Tokyo 2023

FUJIWARA Shunichiro

June 22, 2023
Tweet

More Decks by FUJIWARA Shunichiro

Other Decks in Technology

Transcript

  1. @fujiwara SREνʔϜ ISUCON 1,2,5,11 ! ༏উ4ճ ISUCON 3,8,12,13 ӡӦ(ग़୊)4ճ github.com/kayac/ecspresso

    Amazon ECS σϓϩΠπʔϧ github.com/fujiwara/lambroll AWS Lambda σϓϩΠπʔϧ
  2. ecspresso ͱ͸ github.com/kayac/ecspresso ൃԻ͸ʮΤεϓϨοιʯ ☕ Amazon Elastic Container Service (Amazon

    ECS) ͷͨΊͷσϓϩΠπʔϧ ʮλεΫఆٛʯͱʮαʔϏεʯΛϑΝΠϧͰ؅ཧ͢Δ Go࣮૷ γϯάϧόΠφϦͰಈ࡞͢ΔCLI λεΫఆٛͱαʔϏεҎ֎͸(جຊతʹ)؅ཧ͠ͳ͍
  3. λεΫఆٛɺαʔϏεͷઃఆ͸AWS Command Line Interface (AWS CLI)ޓ׵ͷJSONϑΝΠϧ ϑΝΠϧ͸ςϯϓϨʔτͱͯ͠ѻΘΕΔ ྫ: σϓϩΠ͝ͱʹՄมͷΠϝʔδλάΛ؀ڥม਺Ͱࢦఆ //

    ඞਢͷ؀ڥม਺ "image": "debian:{{ must_env `IMAGE_TAG` }}" // ઃఆ͞Εͯͳ͍৔߹ͷσϑΥϧτ஋Λࢦఆ "image": "nginx:{{ env `IMAGE_TAG` `latest` }}"
  4. AWS CloudFormation, AWS Cloud Development Kit (CDK), Terraformͱ࿈ܞՄೳ ґଘ͢ΔϦιʔεͷIDΛϋʔυίʔυͤͣʹղܾ #

    ઃఆϑΝΠϧ ecspresso.yml plugins: - name: tfstate config: url: s3://my-bucket/terraform.tfstate // αʔϏεఆٛϑΝΠϧ "securityGroups": [ "{{ tfstate `aws_security_group.default.id` }}" ],
  5. ӡ༻ʹศརͳαϒίϚϯυ͕ͨ͘͞Μ run: ୯ମλεΫΛ࣮ߦ (RunTask) status: αʔϏεͷঢ়ଶΛදࣔ scale: αʔϏεͷλεΫ਺Λมߋ refresh: αʔϏεͷλεΫΛશͯೖΕ׵͑

    diff: σϓϩΠલʹมߋࠩ෼Λදࣔ verify: σϓϩΠલʹ࢖༻͢Δ֎෦ϦιʔεΛ֬ೝ exec: Amazon ECS ExecͰλεΫʹ઀ଓ (·ͩ·ͩ͋Γ·͢)
  6. ecspressoΛ͝ར༻ͷօ༷ 2 ϒοΫ΢ΥʔΧʔ, ϕΨίʔϙϨʔγϣϯ, LayerX ϑΝϯϑΝʔϨ, ίωώτ, εϚʔτϥ΢ϯυ, ͸ͯͳ CARTA

    HOLDINGS, ΧϯϜ, Classi, ϑΥʔΠοτ PR TIMES, ArkEdge Space, τϥετόϯΫ, λΠϛʔ Retty, MICIN, ηʔϑΟʔ, AmebaϚϯΨ, BASE ϚωʔϑΥϫʔυ, STORES, Kakedas, mediba Nature, Connected Base, (ΧϠοΫ) 2 ॱෆಉɺܝࡌڐՄΛಘͨ΋ͷͷΈ
  7. AWS Summit Tokyo 2023 ʮχϯςϯυʔΞΧ΢ϯτ ϦϊϕʔγϣϯϓϩδΣΫτʯͷൃද3 www.itmedia.co.jp/news/articles/ 2305/11/news048.html ΑΓҾ༻ ҰํͰFargateͷઃఆ͸Amazon

    CDKΛ࢖ͬͯ ͍ͳ͍ͱ͍͏ɻίʔυͷઃஔΛࣗಈԽ͢Δπʔϧ ͱͷ૬ੑͷ໰୊͕ྑ͘ͳ͍͜ͱ͕ཧ༝ͱ͍͍ɺ ୅ΘΓʹΧϠοΫͷΦʔϓϯιʔειϑτ΢ΣΞ ʮecspressoʯʢΤεϓϨοιʣΛ׆༻ͨ͠ ͱ͍ͯ͠Δɻ 3 https://pages.awscloud.com/rs/112-TZM-766/images/CUS-20AWS- Summit-2023Nintendo-Systems_FINAL.pdf
  8. ଞͷπʔϧ΋ௐ΂ͯΈΔ • ecs-deploy: λεΫఆٛࣗମͷίʔυ؅ཧ͸Ͱ͖ͳ͍ github.com/silinternational/ecs-deploy • ecs-cli: ϩʔϧόοΫ͕Ͱ͖ͳ͍ github.com/aws/amazon-ecs-cli ϩʔϧόοΫ

    = 1ͭલͷλεΫఆٛΛαʔϏεʹઃఆ͢Δ ! ʮAPI/CLIΛ਺ճݺͿ͚ͩͳΒࣗ࡞Ͱ͖ΔͷͰ͸…?ʯ ʮҰݟ؆୯ͦ͏ͳ΋ͷͰ΋ɺࣗલ࣮૷͸প΁ͷୈҰาʯ
  9. ઃܭࢥ૝(1) αʔϏεɺλεΫఆٛͷ؅ཧʹಛԽ ʲཧ༝ʳଞͷϦιʔεͱϥΠϑαΠΫϧ ͕ҧ͏͜ͱ͕ଟ͍ͨΊ • Amazon VPC, ALB, Amazon RDS...

    ௕ظؒ࢖͏ɺมߋ͸සൟͰ͸ͳ͍ • λεΫఆٛɺαʔϏε σϓϩΠ͢Δͨͼʹߋ৽ ؅ཧ͢Δओମ͕ҧ͏͜ͱ΋ଟ͍ ୯ҰͷπʔϧͰશͯΛ؅ཧ͢Δͱɺҙਤ ͠ͳ͍มߋ΍ࣄނ͕ى͜ΔՄೳੑ͕ߴ͍
  10. v0.1.0 ecspresso --help usage: ecspresso --config=CONFIG [<flags>] <command> [<args> ...]

    Flags: --help Show context-sensitive help (also try --help-long and --help-man). --config=CONFIG config file Commands: help [<command>...] Show help. deploy [<flags>] deploy service status [<flags>] show status of service rollback [<flags>] rollback service
  11. v0.1.0 $ find . -name '*.go' | xargs wc -l

    32 ./service.go 35 ./config.go 56 ./cmd/ecspresso/main.go 288 ./ecspresso.go 411 ߹ܭ
  12. 2017.11.29 AWS Fargate GA6 2017.12.07 ecspresso v0.2.2 AWS FargateରԠ σϓϩΠπʔϧΛࣗ࡞͢Δ

    → Amazon ECSͷ৽ػೳʹରԠ͢ΔͨΊʹ͸ ࣗ෼Ͱ࣮૷͢Δඞཁ͕͋Δ 6 https://aws.amazon.com/jp/blogs/aws/aws-fargate/
  13. 2018೥ 2018.03.30 v0.4.0 deleteίϚϯυ࣮૷ 2018.04.05 v0.5.0 runίϚϯυ࣮૷ @ijin͞Μ͔Βrun taskΛ࣮૷ͨ͠ Pull

    Request 2018.11.15 Secrets Support for Amazon ECS7 2018.11.20 v0.6.5 SecretsରԠ 7 https://aws.amazon.com/jp/about-aws/whats-new/2018/11/aws-launches- secrets-support-for-amazon-elastic-container-servic/
  14. 2019೥ 2019.04.10 CircleCIͱecspressoʹΑΔ ECS΁ͷσϓϩΠϝϯτύΠϓϥΠϯ8 ॳΊͯͷଞࣾࣄྫ 2019.09.11 --no-wait Pull Request @Songmu9

    2019.09.12 v0.9.0 waitίϚϯυ࣮૷ CI/CDͰͷར༻͕૿͖͑ͯͨ 9 https://github.com/kayac/ecspresso/pull/38 8 https://devblog.thebase.in/entry/2019/04/10/110000
  15. 2019.10.08 AWS Dev Day Tokyo 2019 ୈ1ճ AWS Fargate ͔ΜͨΜσϓϩΠબखݖ10

    @torics ͞Μʹ঺հ͞ΕΔ 10 https://speakerdeck.com/toricls/the-very-first-aws-fargate-easy-deployment- tooling-championship
  16. ecspresso init ͷൃ໌ ecspresso͕ಈ࡞͢ΔͨΊʹ͸࠷௿3ͭͷϑΝΠϧ͕ඞཁ͕ͩ… • ઃఆϑΝΠϧ: READMEΛݟͯงғؾͰ • λεΫఆٛϑΝΠϧ: aws

    ecs describe-task-definition • αʔϏεఆٛϑΝΠϧ: ࣗ෼ͰͳΜͱ͔ ͜Ε͕໘౗ͩͬͨ ! طଘͷαʔϏεͷ৘ใΛݩʹ࡞ΕΔͷͰ͸?
  17. 2019.10.12 ecspresso v0.11.0 initίϚϯυ࣮૷ طଘͷΫϥελ໊ɺαʔϏε໊Λࢦఆͯ͠ઃఆϑΝΠϧΛੜ੒ $ ecspresso init --cluster default

    --service myapp 2023/05/31 14:46:00 myapp/default save service definition to ecs-service-def.json 2023/05/31 14:46:00 myapp/default save task definition to ecs-task-def.json 2023/05/31 14:46:00 myapp/default save config to ecspresso.yml $ tree . ├── ecs-service-def.json ├── ecs-task-def.json └── ecspresso.yml
  18. init ҎલʗҎޙ Ҏલ: ઃఆϑΝΠϧΛखॻ͖͢Δͱ࢖͑Δ → ڧ͍ڵຯ͕͋Δਓ͔͠࢖͑ͳ͍ Ҏޙ: ECSͰಈ͘αʔϏεΛ͍࣋ͬͯΔਓ͸ͩΕͰ΋ ίϚϯυҰൃͰ࢖͑Δ ެࣜπʔϧ΍ஶ໊OSS:

    ֶश͢Δಈػ͕ڧ͍(ඞཁੑ͕͋Δ) ऑখOSS: ϋʔυϧ͕ߴ͍ͱ୯ʹ࢖ΘΕͳ͍(ಈػ͕ऑ͍) ʲڭ܇ʳઃఆϑΝΠϧΛ޷ΜͰॻౕ͖͍ͨ͸͍ͳ͍
  19. Dev Day 2019Ҏ߱ GitHub ΋։ൃ΋Ճ଎ 2019.10.16 v0.12.0 CodeDeployʹΑΔBlue/GreenσϓϩΠ 2020.03.31 v0.14.0

    tfstateࢀরػೳ௥Ճ 2020೥2݄͔ΒίϩφՒ ༵ۚ໷ʹҿΈʹߦ͔ͳ͘ͳΔ → ՈͰOSS։ൃΛ͢ΔΑ͏ʹ
  20. Amazon ECSͷػೳ௥Ճʹߴ଎ʹ௥ै͍ͯ͘͠ ͜ͷ͜Ζɺ৽ػೳ͕ଟ͔ͬͨ Fargate Spot, Amazon Elastic File System (EFS)

    mount, Fargate Platoform 1.4.0, S3 envfile... ৽ػೳϦϦʔεޙɺ͙͢ʹαϙʔτ͠ଓ͚͍ͯ͘ 2020.01.17 Amazon ECS͔ΒAmazon EFSϚ΢ϯτՄೳʹ13 2020.01.20 ecspresso v0.13.5 EFSϚ΢ϯταϙʔτ ͳͥ͜Ε͕Մೳͳͷ͔ 13 https://aws.amazon.com/jp/about-aws/whats-new/2020/01/amazon-ecs-preview-support-for-efs-file-systems-now-available/
  21. ઃܭࢥ૝ ࠶ܝ (1) Amazon ECSαʔϏεͱλεΫఆٛͷ؅ཧʹಛԽ → มߋ͢ΔՕॴ͕࠷খݶʹͳΔ (2) Amazon ECSઐ༻ͷπʔϧͱͯ͠࡞Δ

    → Լखʹந৅Խ͍ͯ͠ͳ͍ͨΊɺSDK͕ఏڙ͢ΔΞοϓσʔτ Λͦͷ··ड͚ೖΕΒΕΔ มߋʹߴ଎ʹ௥ैͰ͖Δઃܭͩͬͨ(݁Ռ࿦)
  22. v1.0 ! 2020.09.05 v0.99.0 (v1ͷͨΊͷpre-release൛) 2020.10.20 AWS Dev Day Tokyo

    ୈ2ճ AWS Fargate ͔ΜͨΜσϓϩΠબखݖ15 2020.10.20 ecspresso v1.0.0 ͍ͭ·Ͱ΋v0.xͩͬͨͷͰɺ֮ޛΛܾΊͯv1ϦϦʔε େ͖ͳඇޓ׵͸ͳ͠ 15 https://speakerdeck.com/toricls/the-easiest-deployment-championship-2020-find-your-winner-for-aws-fargate
  23. όʔδϣχϯάํ਑ جຊతʹ͸Semantic Versioning v{major}.{minor}.{patch} • ϝδϟʔόʔδϣϯ͕ಉҰͳΒඇޓ׵͸ೖΕͳ͍ (v1, v2...) • όάमਖ਼ʹΑΔڍಈมߋ͸΍ΉΛಘͳ͍৔߹͕͋Δ

    • ػೳ͕૿͑ͨΒϚΠφʔόʔδϣϯΛ্͛Δ (v1.1, v1.2...) • όάमਖ਼͸ύονόʔδϣϯͰ (v1.1.1, v1.1.2...) • খ͍͞ػೳ௥Ճ͸ύονόʔδϣϯͰೖΔ͜ͱ͕͋Δ
  24. 2021 ػೳ௥Ճ͕ଓ͘ظ 2021.01.12 v1.3.0 CloudFormation plugin 2021.03.15 Amazon ECS Exec

    GA17 2021.04.10 v1.5.0 tasks, exec ίϚϯυ௥Ճ 2021.10.29 v1.7.0 ఆٛϑΝΠϧྨʹJsonnetαϙʔτ 17 https://aws.amazon.com/jp/about-aws/whats-new/2021/03/amazon-ecs-now-allows-you-to-run-commands-in-a-container- running-on-amazon-ec2-or-aws-fargate/
  25. ecspresso exec Amazon ECS Exec (ಈ࡞தͷλεΫʹ઀ଓͯ͠ίϚϯυΛ࣮ߦͰ͖Δ) ʹରԠ͢Δ ecspresso exec Λ࣮૷

    (ޙ͔Βࢥ͑͹)͜Ε͸ͪΐͬͱҳ୤ؾຯ ઃܭࢥ૝(1) αʔϏεͱλεΫఆٛͷ؅ཧʹಛԽ ❌ exec͸ಈ࡞தͷλεΫͦͷ΋ͷΛ૬खʹ͢Δػೳ σϓϩΠΛ؅ཧ͢Δ = ฏ࣌ ಈ࡞தͷλεΫʹԿ͔͢Δ = ҟৗ࣌
  26. 2022 ͦͯ͠v2΁ 2021-01-19 aws-sdk-go-v2 GA18 ecspresso͸aws-sdk-goʹґଘ͍ͯ͠Δ SDK͕ߋ৽͞Εͳ͍ = ৽ػೳʹ௥ैͰ͖ͳ͍ =

    ࠔΔ(ࣗ෼͕) aws-sdk-go (v1)͸Ͳ͏ͳΔ?? 18 https://aws.amazon.com/jp/about-aws/whats-new/2021/01/aws-sdk-for-go-version-2-now-generally-available/
  27. AWS SDKϝϯςφϯεϙϦγʔ20 1. General Availability (GA) GAʹͳͬͨSDK͸࠷௿24ϲ݄ؒ͸αϙʔτ 2. Maintenance Announcement

    SDK͕ϝϯςφϯεϞʔυʹͳΔ6ϲ݄લʹΞφ΢ϯε 3. Maintenance ϝϯςφϯεϞʔυʹೖΔͱॏཁͳόάमਖ਼ͷΈ ৽ػೳͷ௥Ճ͸ͳ͠ → ৽ػೳͷߋ৽(อূ)͸ऴྃΞφ΢ϯε͔Β൒೥ؒ 20 https://docs.aws.amazon.com/sdkref/latest/guide/maint-policy.html
  28. github.com/fujiwara/ecsta ecspresso͔ΒλεΫૢ࡞ػೳΛ੾Γग़ͨ͠πʔϧ(ϥΠϒϥϦ) ೚ҙͷλεΫʹexecͳͲͷૢ࡞͕Ͱ͖ͯศར describe: λεΫͷ৘ใΛJSONͰදࣔ exec: ECS ExecͰλεΫʹ઀ଓ list: Ϋϥελ಺ͷλεΫҰཡΛදࣔ

    logs: λεΫ͕CloudWatch Logsʹग़ྗͨ͠ϩάΛදࣔ portforward: ECS ExecΛ࢖ͬͨϙʔτϑΥϫʔυ stop: λεΫΛఀࢭ trace: λεΫ͕ىಈ→ఀࢭ·ͰͷΠϕϯτɺϩάΛ࣌ܥྻදࣔ