Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ECSからEKSへの移行への移行事例の紹介

Masato Oshima
November 20, 2020

 ECSからEKSへの移行への移行事例の紹介

発表資料の中に出てくるURLです。

* [Amazon EKSでのArgoCDを使ったGitOps CD](https://tech.recruit-mp.co.jp/infrastructure/gitops-cd-by-using-argo-cd-at-eks/)
* [EnvoyとAmazon ECS Service Discoveryを利用したgRPCの負荷分散](https://tech.recruit-mp.co.jp/infrastructure/post-17098/)
* [Amazon EKSでAWS App Meshを利用してgRPCサーバーを運用する](https://tech.recruit-mp.co.jp/infrastructure/post-20765/)
* [https://tech.recruit-mp.co.jp/infrastructure/post-19364/](https://tech.recruit-mp.co.jp/infrastructure/post-19364/)
* [JenkinsのJobを CronJobに移した話](https://speakerdeck.com/yutachaos/the-story-of-moving-jenkins-job-to-cronjob)
* [EKS クラスタの CronJob を Fargate 上で実行する方法](https://tech.recruit-mp.co.jp/infrastructure/post-20631/)

Masato Oshima

November 20, 2020
Tweet

More Decks by Masato Oshima

Other Decks in Technology

Transcript

  1. &$4͔Β&,4΁
    ͷҠߦࣄྫͷ঺հ
    !@NQPO
    +"846(ίϯςφࢧ෦

    View full-size slide

  2. Masato Oshima
    github.com/mpon
    @_mpon
    Software Engineer

    View full-size slide

  3. ίϯςφࢧ෦ͷൃද͸3ճ໨Ͱ͢
    2017/12 2018/12

    View full-size slide

  4. ίϝϯτ΍࣭໰ੋඇ͓ئ͍͠·͢
    ➤ ਖ਼௚ɺ΋ͬͱ͍͍ํ๏͋Δ͔΋ͱ͍͏ෆ҆ɾɾ
    ➤ ͳͷͰποίϛ΋Β͑Δͱخ͍͠Ͱ͢ʂ
    ➤ ͦΜͳ͜ͱ͠ͳͯ͘΋ࠓͳΒ͜ΕͰͰ͖Δͷʹɻͱ͔
    ➤ ͬͪ͜ͷπʔϧ࢖͑͹Α͔ͬͨͷʹɻͱ͔
    ➤ ͳΜͰ͜͏͍͏ߏ੒ʹͨ͠ͷʁͱ͔
    ➤ ͜͏͍͏έʔεͬͯରԠͰ͖ͯ·͔͢ʁͱ͔
    ➤ ͕͜͜Α͘෼͔Βͳ͔ͬͨɻͱ͔

    View full-size slide

  5. AGENDA
    ➤ ελσΟαϓϦENGLISHʹ͍ͭͯ
    ➤ ୈҰ෦: ECS͔ΒEKS΁Ҡߦͷഎܠ
    ➤ ՝୊1: ؀ڥ૿΍͢ͷ͕ਏ͔ͬͨ
    ➤ ՝୊2: Ͳ͜ʹԿ͕σϓϩΠ͞ΕͯΔ͔೺Ѳ͠ʹ͔ͬͨ͘
    ➤ ՝୊3: gRPCͷෛՙ෼ࢄ
    ➤ ୈೋ෦: EKSҠߦޙͷߏ੒ʹ͍ͭͯ
    ➤ Spot OceanΛ࢖ͬͨΫϥελʔ؅ཧ
    ➤ JenkinsͷJobΛCronJobʹҠߦʢFargateͷ࿩΋͋ΔΑʣ
    ➤ ·ͱΊɿίϯςφ͸͍͍ͧ

    View full-size slide

  6. ελσΟαϓϦENGLISH
    ➤ ΦϯϥΠϯӳޠֶशαʔϏε
    ➤ ӳձ࿩Ի੠΍ը૾ɾಈըͳͲΛ഑৴
    ➤ ΫΠζܗࣜͰֶशσʔλɺཤྺΛอଘ
    ➤ iOS, Android, WebͰར༻Մೳ

    View full-size slide

  7. αʔϏεߏ੒ͷ֓ཁ

    View full-size slide

  8. αʔϏεߏ੒ͷ֓ཁ
    ໿50αʔϏε
    ɾɾɾ
    ɾ
    ɾ
    ɾ
    dev1, dev2...
    ໿20؀ڥ

    View full-size slide

  9. ୈҰ෦
    ECS͔ΒEKS΁Ҡߦ͠
    ͨഎܠʹ͍ͭͯ

    View full-size slide

  10. ՝୊1
    ؀ڥ૿΍͢ͷ͕ਏ
    ͔ͬͨ

    View full-size slide

  11. ECSͷͱ͖ͷߏ੒؅ཧํ๏
    ෼ྨ ίϯϙʔωϯτ ઃఆํ๏
    Ծ૝Ϋϥελʔ ECS Cluster Terraform
    Πϯελϯε Autoscaling Group Terraform
    αʔϏεεέδϡʔϥ ECS Service Terraform
    αʔϏεσΟεΧόϦ Cloud Map Terraform
    ίϯςφఆٛ ECS Task Definition Terraform + ಠࣗπʔϧ
    ϩʔυόϥϯαʔ ALB Terraform
    αʔϏεͻ΋͚ͮ ALB Target Group Terraform
    ϧʔςΟϯά ALB Listenr Rule Terraform

    View full-size slide

  12. ECSͰ؀ڥΛͨ͘͞Μ૿΍͢ʹ͸ʁ
    ➤ TerraformͰECS ClusterɺECS ServiceɺALBͳͲ৭ʑ࡞Δ
    ➤ Task Definitionʢಠࣗπʔϧͷ࢓༷ͷYAMLʣΛେྔʹίϐʔ
    ➤ ςϯϓϨʔτͷػೳ͕ͳ͘؀ڥ͝ͱʹ؀ڥݻ༗ͷ஋ʹॻ͖׵͑Δ
    ➤ ؀ڥ૿΍͢ͷʹTerraformͰΠϯελϯε΍ALBΛ૿΍͞ͳ͍ͱ͍͚ͳ͍
    ➤ Terraform͸ModuleԽ͸͚ͨ͠Ͳɺ͋ͱ͔ΒมߋೖΕΔͱޓ׵ੑอͭͷ
    ʹେมͳ͜ͱ͕͋ͬͯؾܰʹmoduleΛมߋͰ͖ͳ͍
    ➤ σϓϩΠͰมߋ͕ى͖Δ΋ͷΛTerraformͰ؅ཧͯ͠͠·͍ͬͯͨͷͰ
    ignore_changesͰແཧ΍Γແࢹ
    ➤ ϥΠϑαΠΫϧͷҧ͍ΛݟۃΊΒΕ͍ͯͳ͔ͬͨɾɾ

    View full-size slide

  13. EKSͷߏ੒؅ཧํ๏
    ෼ྨ ίϯϙʔωϯτ ઃఆํ๏
    Ծ૝Ϋϥελʔ Namespace YAML(k8s)
    Πϯελϯε EC2Πϯελϯε Terraform + Spot Ocean
    αʔϏεεέδϡʔϥ Deployment YAML(k8s)
    αʔϏεσΟεΧόϦ Service YAML(k8s)
    ίϯςφఆٛ Pod YAML(k8s)
    ϩʔυόϥϯαʔ Ingress YAML(k8s)
    αʔϏεͻ΋͚ͮ Ingress YAML(k8s)
    ϧʔςΟϯά Ingress YAML(k8s)

    View full-size slide

  14. EKSͰ؀ڥΛͨ͘͞Μ૿΍͢ʹ͸ʁ
    ➤ Terraformͷग़൪͸EKS ClusterΛ࠷ॳʹ࡞Δ͚ͩͰ؀ڥ૿΍͢ͱ
    ͖ʹ͸͍Βͳ͍
    ➤ ؀ڥ૿΍͢৔߹͸NamespaceΛ૿΍͢
    ➤ KustomizeΛ࢖ͬͯ؀ڥ͝ͱʹมΘΔ෦෼͚ͩΛ௥Ճ
    ➤ ֎෦͔ΒͷϦΫΤετͷϚοϐϯά͸Ingress
    ➤ ಺෦௨৴͸ServiceΦϒδΣΫτͰ؆୯αʔϏεσΟεΧόϦɺ
    CoreDNS͕͋ΔͷͰRoute53ͳ͠Ͱ΋಺෦ͰDNS͕༻ҙ
    ➤ ΄ͱΜͲKubernetesͷΦϒδΣΫτͰ׬݁͢ΔͷͰએݴతʹ؅ཧ
    ͠΍͍͢

    View full-size slide

  15. ΫϥελʔΛ࿦ཧతʹ෼ׂͰ͖Δ
    Namespace
    dev1
    EC2 EC2
    EC2
    Cluster
    EC2 EC2 EC2
    Namespace
    dev2
    Namespace
    dev3
    ➤ Kubernetesʹ͸Namespace
    ͱ͍͏ΦϒδΣΫτ͕͋Δ
    ➤ Cluster͑͞࡞ͬͯ͠·͑͹
    ͋ͱ͸NamespaceΛ૿΍͢
    ͚ͩͰ؀ڥ͕૿΍ͤΔ
    ➤ Namespace͝ͱͷDNSΤϯ
    τϦ΋࡞੒͞ΕΔ
    ➤ ͜Μͳ୯७ͳYAMLͰOK

    View full-size slide

  16. Kustomize, HelmͳͲσϑΝΫτͳߏ੒؅ཧ͕͋Δ

    View full-size slide

  17. Kustomizeͷoverlays

    View full-size slide

  18. ՝୊2
    Ͳ͜ʹԿ͕σϓϩΠ͞Ε
    ͯΔ͔೺Ѳ͠ʹ͔ͬͨ͘

    View full-size slide

  19. ECSͷͱ͖ʹײ͍ͯͨ͡՝୊
    ➤ hoge؀ڥͷfugaαʔϏεͬͯࠓσϓϩΠ͞ΕͯΔͷ͸Ͳͷϒϥϯνʁ
    ➤ ίϯςφͷ؀ڥม਺͸Կ͕ઃఆ͞ΕͯΔʁ
    ➤ aws ecs list-servicesͱ͔ͰҰൃͰͲͷtag͕σϓϩΠ͞ΕͯΔ͔೺ѲͰ͖
    ͳ͍ͷͰࣗ࡞πʔϧ࡞͙ͬͯ྇
    ➤ σϓϩΠͷઃఆʢJenkinsfileʣʹઃఆ͕ͲΜͲΜ૿͍͑ͯͬͯΧΦεʹ
    ͳ͖ͬͯͨ
    ➤ Α͘σϓϩΠ͢ΔAPIͳͲͷαʔόʔΞϓϦέʔγϣϯͱ୯ͳΔnginxͱ
    ͔ͷσϓϩΠ͸σϓϩΠํ๏͕ผʑʹ༻ҙͯ͠ϝϯς͍ͯͨ͠
    ➤ gitϦϙδτϦΛݟΕ͹Կ͕σϓϩΠ͞ΕͯΔ͔෼͔ΔΑ͏ʹ͍ͨ͠
    ➤ gitops͍ͨ͠

    View full-size slide

  20. GITOPS͍ͨ͠

    View full-size slide

  21. gitopsπʔϧͱͯ͠Argo CDͷಋೖ
    https://tech.recruit-mp.co.jp/infrastructure/gitops-cd-by-using-argo-cd-at-eks/

    View full-size slide

  22. Argo CDͷArchitecture - k8sͷReconciliation Loop
    gitϦϙδτϦΛ
    ݟΕ͹Կ͕σϓϩΠ͞Ε
    ͍ͯΔ͔೺ѲͰ͖Δ
    gitϦϙδτϦͱ
    ࠩ෼Λఆظతʹ
    νΣοΫͯ͠
    ࠩ෼͕͋Ε͹
    σϓϩΠ

    View full-size slide

  23. Dashboard͕͋ΔͷͰDeveloper Friendly
    GUI͕͋Δͱ։ൃऀʹ΋
    ֓೦Λཧղͯ͠΋Β͍΍͍͢

    View full-size slide

  24. Sync Phase and WavesͰσϓϩΠͷґଘؔ܎ΛදݱͰ͖Δ

    View full-size slide

  25. Clusterʹඞཁͳresource΋gitopsͰ
    ➤ Ingress Controller΍
    Datadog AgentͳͲ΋؅ཧ
    Ͱ͖Δ
    ➤ Helm Chart Repositoriesͷ
    ػೳ͕͋ΔͷͰArgo CDͰ
    Helm ChartΛιʔεʹ؅ཧ
    Մೳ
    ➤ ArgoCD ApplicationΛ࡞੒
    ͢Ε͹ߏங׬ྃ

    View full-size slide

  26. ެࣜπʔϧͷkubectlͰ͙͢ʹ೺ѲͰ͖Δ
    ➤ ྫ͑͹ͲͷImage͕σϓϩΠ͞ΕͯΔ͔֬ೝ͢Δͷʹ͜Ε͚ͩͰOK
    ➤ `kubectl get deploy -o wide`

    View full-size slide

  27. ՝୊3
    GRPCͷෛՙ෼ࢄ

    View full-size slide

  28. ౰࣌ͷgRPCͷෛՙ෼ࢄ
    https://tech.recruit-mp.co.jp/infrastructure/post-17098/

    View full-size slide

  29. ECSͷͱ͖ͷߏ੒

    View full-size slide

  30. σϓϩΠ࣌ʹϦΫΤετ͕མͪΔ໰୊
    Blue
    gRPC
    Green
    gRPC
    Blue
    gRPC
    Green
    gRPC
    blue.grpc.internal green.grpc.internal
    Cloud Map
    lb.grpc.internal
    ➤ grpcαʔόʔΛRolling Update͢Δ
    ͱSTOPࡁΈͷίϯςφͷIP͕ฦͬ
    ͖ͯͯϦΫΤετ͕མͪͯ͠·͏
    ➤ JenkinsͷpipelineͱshΛ૊Έ߹Θ
    ͤͯBlue/GreenσϓϩΠ
    ➤ ECS Task Definitionͷ؀ڥม਺Λ
    sedͰBlue->Greenʹॻ͖׵͑ͯ
    envoyΛRolloing Update
    ➤ Sleep 30͔ͯ͠ΒBlueΛ࡟আ
    ➤ ϫʔΫΞϥ΢ϯυײຬࡌͳͷͰͳ
    Μͱ͔͔ͨͬͨ͠

    View full-size slide

  31. App Meshͷ࠾༻
    https://tech.recruit-mp.co.jp/infrastructure/post-20765/

    View full-size slide

  32. App Meshͷ࠾༻
    ➤ gRPC routingΛαϙʔτ͍ͯͨ͠
    ➤ কདྷతʹαʔϏεϝογϡͱͯ͠׆༻Ͱ͖Δ
    ➤ ͲͪΒʹ͠Ζenvoyͷ؅ཧ͸୭͔͕΍Δඞཁ͸͋Δ
    ➤ envoyͷretryPolicyɺkubernetesͷPod LifecycleΛ׆༻ͯ͠ϦΫ
    ΤετΛམͱͣ͞ʹσϓϩΠͰ͖Δ

    View full-size slide

  33. App Meshͷߏ੒؅ཧ
    ➤ AWS App Mesh Controller
    for k8s͕༻ҙ͞Ε͍ͯΔ
    ➤ APIͷίϯςφͷYAMLΛ͍
    ͡Δ͜ͱͳ͘envoyΛInject
    ͯ͘͠ΕΔ
    ➤ k8sͷCRDͱͯ͠؅ཧͰ͖
    ΔͷͰgitopsͰ͖Δ
    ➤ ։ൃ؀ڥΛ૿΍͢ͷ΋؆୯

    View full-size slide

  34. σϓϩΠ࣌ʹϦΫΤετ͕མͪΔ໰୊
    ➤ App Mesh best practicesͱ
    ͍͏υΩϡϝϯτ͕༻ҙ͞
    Ε͍ͯͯͦͷ௨Γʹ
    RetryPolicyΛઃఆ

    View full-size slide

  35. σϓϩΠ࣌ʹϦΫΤετ͕མͪΔ໰୊
    ➤ PodͷpreStopͰsleepΛೖΕΔ͜ͱͰ
    Service͔Β੾Γ཭͞ΕΔͷΛ଴ͭ
    ➤ AWSαϙʔτͷํʹΞυόΠεΛ΋
    Β͍ղܾ͠·ͨ͠ɻେײँ
    ➤ AWSαʔϏεͷ͜ͱ͔͠ฉ͍ͪΌ͍
    ͚ͳ͍ͱࢥ͚ͬͯͨͲͦΜͳ͜ͱ͸ͳ
    ͔ͬͨ
    ➤ Kubernetes΍Envoyͷ͜ͱ΋αϙʔτ
    ͯ͘͠Εͨ
    ➤ Argo CDʹΑΔSync(Deploymentͷ
    Rolling Update)Ͱγϯϓϧʹσϓϩ
    ΠͰ͖ΔΑ͏ʹͳͬͨ

    View full-size slide

  36. ༨ஊ: ਓྨ͕଴๬͍ͯͨ͠ALBͷgRPCαϙʔτʂʂ
    ΋ͬͱૣ͘ཉ͔ͬͨ͠

    View full-size slide

  37. ୈೋ෦
    EKSҠߦޙͷ
    ߏ੒ʹ͍ͭͯ

    View full-size slide

  38. EKSҠߦޙͷߏ੒
    ECS EKS

    View full-size slide

  39. EKSҠߦޙͷߏ੒
    ALB Ingress
    Controllerͱ
    Nginx Ingrss
    Controllerͷ
    ૊Έ߹Θͤ
    App MeshʹΑΔ
    EnvoyͷInject
    Cloud MapͰ
    Service
    Discovery

    View full-size slide

  40. SPOT OCEANΛ࢖ͬͨ
    Ϋϥελʔ؅ཧ

    View full-size slide

  41. ৄࡉ͸ͪ͜Βͷϒϩά
    https://tech.recruit-mp.co.jp/infrastructure/post-19364/

    View full-size slide

  42. Spot Ocean
    ੲ͸Spotinstͱ͍͏αʔϏεͩͬͨ
    ࠓ͸NetAppʹങऩ͞ΕͯSpotͱ͍͏໊લʹ

    View full-size slide

  43. Pod-Driven Scaling
    ➤ Pod͕UnscheduleʹͳΒͳ͍Α͏ʹ
    NodeΛΦʔτεέʔϦϯάͯ͘͠
    ΕΔ
    ➤ PodʹׂΓ౰ͯΔrequests͚ͩΛߟ
    ͑Ε͹OK
    ➤ Node͸ҙࣝ͠ͳͯ͘Α͍ͷͰ
    FargateͷΑ͏ͳ࢖͍উख
    ➤ Spot Instance͔Β࠷దͳΠϯελϯ
    εΛબΜͰ͘ΕΔ
    ➤ ΦϯσϚϯυͰಈ͍ͯཉ͍͠podʹ
    ͸annotation෇༩͢Δ͚ͩͰOK

    View full-size slide

  44. Terraform΍Custom ControllerͰߏ੒؅ཧͰ͖Δ
    ➤ Terraformͷprovider͕༻ҙ
    ͞Ε͍ͯΔ
    ➤ Spot OceanͷCustom
    Controller͕Helm ChartͰ
    ༻ҙ͞Ε͍ͯΔͷͰ؆୯Π
    ϯετʔϧ

    View full-size slide

  45. Headroom
    ➤ Headroom͸༨৒ͷϊʔυ
    Λ֬อ͓͍ͯͯ͘͠ΕΔػ

    ➤ ϦΫΤετͷεύΠΫ౳Ͱ
    PodͷεέʔϧΛૉૣ͘͢Δ

    View full-size slide

  46. Cluster Roll
    ➤ Cluster Roll͸Nodeͷ҆શ
    ͳRolling UpdateΛͯ͘͠Ε
    Δ
    ➤ NodeͷೖΕସ͍͑ͨ͠ࣄ৘
    ͕ൃੜͨ͠৔߹ͳͲʹศར

    View full-size slide

  47. JENKINSͷJOBΛ
    CRONJOBʹҠߦ

    View full-size slide

  48. ৄࡉ͸ͪ͜Β
    https://tech.recruit-mp.co.jp/infrastructure/post-20631/
    https://speakerdeck.com/yutachaos/the-story-of-moving-jenkins-job-to-cronjob

    View full-size slide

  49. όον͸JenkinsͰ΍͍ͬͯͨ
    ➤ αʔόʔΞϓϦέʔγϣϯ
    ͱಉ͡ίϯςφImageΛ
    ࢖ͬͯλεΫΛىಈ
    ➤ JenkinsͷఆظτϦΨʔͰ࣮
    ߦ
    ➤ ௨஌΋ϦτϥΠ΋ศརɺͳ
    ΜͰ΋Ͱ͖Δɻ͔͠͠ɾɾ

    View full-size slide

  50. Jenkinsศར͗͢໰୊
    ➤ ศརա͗ͯWebը໘্Ͱ
    ͡ΌΜ͡ΌΜδϣϒ͕࡞Β
    Ε͍ͯ͘
    ➤ ҰମԿ͕͍ͭىಈͯ͠Δ͔
    Α͘෼͔Βͳ͍
    ➤ ාͯ͘updateͰ͖ͣʹԘ௮
    ͚ʹɾɾ

    View full-size slide

  51. k8sͷCronJobʹ͢Δ͜ͱͰYAML؅ཧ
    ➤ CronJob͸ఆظతʹJobΛ࣮ߦ
    ͢Δk8sͷϦιʔε
    ➤ EKSҠߦͷӡ༻͸·ͣδϣϒҠ
    ߦ͔Βߦ͕ͬͨ͏·͍ͬͨ͘
    ➤ YAMLͰ؅ཧͰ͖ΔͷͰgitops
    Ͱ͖Δ
    ➤ kubectlͰݟͯ΋͍͍͠ɺgitϨ
    ϙδτϦΛgrep͢Δ͚ͩͰԿ͕
    ͍ͭͲ͜ͰJob͕ಈ͍ͯΔ͔͢
    ͙ʹ෼͔Δ

    View full-size slide

  52. όον࣮ߦͷͨͼʹNodeͷ૿ݮͯ͠͠·͏
    ➤ ఆظతʹόον࣮ߦͷͨͼ
    ʹPodͷRequestsʹԠͯ͡
    Node͕૿ݮ
    ➤ όον࣮ߦதʹNode͕མͪ
    ΔͱࠔΔ
    ➤ όονͷ࣮ߦʹΑͬͯαʔ
    ϏεʹӨڹ͸༩͑ͨ͘ͳ͍

    View full-size slide

  53. EKS FargateͰCronJobΛ࣮ߦ͢Δ
    ➤ όον͸ίϯςφ͕ىಈ͠
    ͯऴྃ͢Ε͹Α͍ͷͰ
    FargateʹͽͬͨΓ
    ➤ kubernetes͔Β͸Fargate͸
    1ͭͷnodeͷΑ͏ʹݟ͑Δ
    ➤ FargateͰىಈͨ͠όον͕
    طଘͷΫϥελʔͷϦιʔ
    εΛ࢖͏͜ͱ͸ͳ͍

    View full-size slide

  54. Sidecarίϯςφͷఀࢭ͕೉͍͠
    ➤ Datadog Agent΍FluentdͳͲΛ
    Sidecarʹ͓͔͘͠ͳ͍
    ➤ JobͷϝΠϯͷίϯςφ͕ఀࢭͯ͠΋
    sidecarίϯςφ͸ఀࢭ͠ͳ͍ͷͰJob
    ͕ऴྃ͠ͳ͍ʂʂ
    ➤ ऴྃ࣌ʹsidecarίϯςφΛఀࢭͤ͞
    ΔͳͲͷ޻෉͕ඞཁ
    ➤ ECSͰ͍͏essential:trueͷػೳ͕ͳ͍
    ➤ Sidecar Containers͕Kubernetes
    v1.19ʹೖΔ༧ఆ͕ͩͬͨະఆʹͳͬ
    ͯ͠·ͬͨʁ

    View full-size slide

  55. ·ͱΊͱࡶஊ
    ➤ ECS͔ΒEKSϔҠߦͨ͠࿩Λ͠·ͨ͠
    ➤ ͋͘·Ͱ΋զʑͷ৔߹ͷ࿩
    ➤ gitops΍KubernetesͷΤίγεςϜͰಠࣗ࢓༷Λͳͯ࣋͘͠ଓੑͷ͋ΔγεςϜ΁
    ➤ App Mesh͸ຊ൪ར༻΋໰୊ͳ͠
    ➤ ECS͸ࠓͳΒFargate Spot΋͋Δ͠ALB͕gRPCαϙʔτͨ͠͠࠶ߟͯ͠Έ͍ͨ
    ➤ ecspressoΛ࢖͑͹ϥΠϑαΠΫϧͱςϯϓϨʔτͱ͔΋ͬͱ͏·͘Ͱ͖͔ͨ΋
    ➤ ECSͰɺALBͱͷͻ΋͚ͮɺECS ServiceɺTask Definitionͱ͔Λ౷Ұతʹѻ͑Δπʔϧ
    ͕͋Ε͹͍͍ͷ͔΋
    ➤ copilot͕ͦ͏ͳͷ͔ʁ
    ➤ ίϯςφ͸͍͍ͧ
    ➤ ͨͩ͠ɺRDSɺClodwatch LogsɺS3ͳͲϚωʔδυαʔϏε͕͋ͬͯͦ͜

    View full-size slide