違いから見る Kubernetes #k8sjp / See Kubernetes through an Amazon ECS Lens

84907687e50c8ac2a09b02e0d1b36ab1?s=47 Tori
May 27, 2020

違いから見る Kubernetes #k8sjp / See Kubernetes through an Amazon ECS Lens

Presented at Kubernetes Meetup Tokyo #31, (当日の録画?もいつかどこかに上がるのではないかと思います)

ECS Agent にアイコンが欲しいですイシューはこちら 👉 https://github.com/aws/containers-roadmap/issues/914

84907687e50c8ac2a09b02e0d1b36ab1?s=128

Tori

May 27, 2020
Tweet

Transcript

  1. twitter.com/toricls 違いから⾒る Kubernetes Tori May. 27, 2020 - See Kubernetes

    through an Amazon ECS Lens - #k8sjp
  2. twitter.com/toricls ポジティブな Tori / Sr. Product Developer Advocate Containers Product,

    Amazon Web Services ❤ AWS Fargate & AWS Lambda toricls
  3. twitter.com/toricls Kubernetes and Amazon ECS

  4. twitter.com/toricls Kubernetes and Amazon ECS Initial Announcement 2014.6.7 2014.11.13 GA

    2015.7.21 (version 1.0) 2015.4.9 Developers(s) CNCF
 (originally developed by Google) AWS Amazon EKS
 (2017.11.29 Ann. / 2018.6.5 GA)
  5. twitter.com/toricls From 1,000 ft view • Amazon EKS • kubeadm

    • Kind • … } “create” Control-plane (master) Data-plane (workers) Agent Container
 Runtime (*) *) Amazon ECS Agent にはアイコンがないので… Containers Node Node
  6. twitter.com/toricls From 1,000 ft view • Amazon EKS • kubeadm

    • Kind • … } “create” Control-plane (master) Data-plane (workers) Agent Container
 Runtime Containers (*) *) Amazon ECS Agent にはアイコンがないので… Node Node “docker run” “docker run”
  7. twitter.com/toricls The Constructs

  8. twitter.com/toricls Menu Of the Day LB App ~ Keep app

    running with load balancer ~
  9. twitter.com/toricls in Kubernetes apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress

    spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App
  10. twitter.com/toricls in Amazon ECS Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition:

    cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 Service Task Definition LB App
  11. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
  12. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 Pod spec
  13. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
  14. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 ReplicaSet spec
  15. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
  16. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
  17. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
  18. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 これは?
  19. twitter.com/toricls in Amazon ECS LB App How to route `paths`?

    Application Load Balancer (L7 LB) Listener(s) Listener Rule(s) Target Group Target Group Target Group path: /yay Default route path: /cooool ECS Service (containers) \EC2 !!/ ルーティングの世界 コンテナの世界 仮想マシンの世界 関数の世界 \Lambda !!/
  20. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
  21. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 これは?
  22. twitter.com/toricls Where is “Cluster”? • Amazon EKS • kubeadm •

    Kind • … } “create” Control-plane (master) Data-plane (workers) (*) *) Amazon ECS Agent にはアイコンがないので… Node x N Node x N
  23. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
  24. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress

    metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 ?? ]
  25. twitter.com/toricls in Amazon ECS LB App Task Definition is immutable

    and it has “Revisions”
  26. twitter.com/toricls Example GitOps Flow in Kubernetes and Amazon ECS App

    Repo
 (w/ Dockerfile) Code changes git push CI process docker push Image Registry Manifests Repo
 (w/ manifests) update manifests, create pull request CD process kubectl apply App Repo
 (w/ Dockerfile, w/ Task-def.) Code changes git push CI process docker push Image Registry Manifests Repo
 (w/ Service-def.) update definition files, create pull request CD process aws ecs update-service (タスク定義は CI 処理理で ECS 側に登録済みなので CD 処理理には不不要) ECS API register task-def
  27. twitter.com/toricls App Logging in Kubernetes and Amazon ECS ▶ ログの収集はノードレベル・クラスタレベルの意思決定/オペレーション

    ▶ DaemonSet によってログ収集エージェントをノード群に展開 ▶ Docker ログドライバー(*) をサポートしないという⽅向性 ▶ ログの収集をアプリケーションレベルで意思決定/オペレーションすることがあり得る ▶ Docker ログドライバー機能をタスク定義から設定する ▶ アプリケーションごとにログの送付先が異なるケースが想定されている ▶ 2018.06 に追加されたデーモンスケジューラー機能を使ってログ収集エージェントを配布する⽅式も選択可能ではある *) https://docs.docker.com/config/containers/logging/configure/
  28. twitter.com/toricls Lots of Fun Diffs in Kubernetes and Amazon ECS

    ▶ Kubernetes Jobs ▶ ECS では Step Functions という別サービスと ECS タスクを組み合わせて実現する ▶ Kubernetes CronJobs ▶ ECS では EventBridge + Step Functions + ECS タスクの組み合わせ ▶ ECS タスク - Container Dependency ▶ Kubernetes では initContainers を利⽤したり ▶ コンテナライフサイクルフック(preStop とか)を組み合わせ ▶ Kubernetes 1.19(?) の containers.lifecycle.type: SideCar に期待… ▶ Kubernetes Admission Controllers ▶ ECS でこれにピッタリと該当する機能/やり⽅は存在しない気がする
  29. twitter.com/toricls AWS Container Services - Public Roadmap https://github.com/aws/containers-roadmap/projects/1

  30. twitter.com/toricls AWS Container Services - Public Roadmap https://github.com/aws/containers-roadmap/projects/1

  31. twitter.com/toricls Thank you :)