違いから見る Kubernetes #k8sjp / See Kubernetes through an Amazon ECS Lens

Talked at Kubernetes Meetup Tokyo #31

ECS Agent にアイコンが欲しいですイシューはこちら 👉 https://github.com/aws/containers-roadmap/issues/914

Transcript

1. twitter.com/toricls 違いから⾒る Kubernetes Tori May. 27, 2020 - See Kubernetes through an Amazon ECS Lens - #k8sjp
2. twitter.com/toricls ポジティブな Tori / Sr. Product Developer Advocate Containers Product, Amazon Web Services ❤ AWS Fargate & AWS Lambda toricls
3. twitter.com/toricls Kubernetes and Amazon ECS
4. twitter.com/toricls Kubernetes and Amazon ECS Initial Announcement 2014.6.7 2014.11.13 GA 2015.7.21 (version 1.0) 2015.4.9 Developers(s) CNCF
 (originally developed by Google) AWS Amazon EKS
 (2017.11.29 Ann. / 2018.6.5 GA)
5. twitter.com/toricls From 1,000 ft view • Amazon EKS • kubeadm • Kind • … } “create” Control-plane (master) Data-plane (workers) Agent Container
 Runtime (*) *) Amazon ECS Agent にはアイコンがないので… Containers Node Node
6. twitter.com/toricls From 1,000 ft view • Amazon EKS • kubeadm • Kind • … } “create” Control-plane (master) Data-plane (workers) Agent Container
 Runtime Containers (*) *) Amazon ECS Agent にはアイコンがないので… Node Node “docker run” “docker run”
7. twitter.com/toricls The Constructs
8. twitter.com/toricls Menu Of the Day LB App ~ Keep app running with load balancer ~
9. twitter.com/toricls in Kubernetes apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App
10. twitter.com/toricls in Amazon ECS Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 Service Task Definition LB App
11. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
12. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 Pod spec
13. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
14. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 ReplicaSet spec
15. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
16. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
17. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
18. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 これは？
19. twitter.com/toricls in Amazon ECS LB App How to route `paths`? Application Load Balancer (L7 LB) Listener(s) Listener Rule(s) Target Group Target Group Target Group path: /yay Default route path: /cooool ECS Service (containers) ＼EC2 !!／ ルーティングの世界 コンテナの世界 仮想マシンの世界 関数の世界 ＼Lambda !!／
20. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
21. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 これは？
22. twitter.com/toricls Where is “Cluster”? • Amazon EKS • kubeadm • Kind • … } “create” Control-plane (master) Data-plane (workers) (*) *) Amazon ECS Agent にはアイコンがないので… Node x N Node x N
23. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080
24. twitter.com/toricls in Kubernetes and Amazon ECS apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: hey-yo—ingress spec: rules: - http: paths: - path: /cooool backend: serviceName: cool-service servicePort: 80 apiVersion: v1 kind: Service metadata: name: cool-service labels: app: demo spec: ports: - targetPort: 8080 port: 80 protocol: TCP selector: app: i-am-cool apiVersion: apps/v1 kind: Deployment metadata: name: cool-deployment labels: app: i-am-cool spec: replicas: 3 template: metadata: labels: app: i-am-cool spec: containers: - name: super-duper-fantastic image: toricls/my-super-fantastic-app:v1 Ingress Service Deployment LB App Type: AWS::ECS::TaskDefinition Properties: Family: cool-task ContainerDefinitions: - Name: super-duper-fantastic Image: toricls/my-super-fantastic-app:v1 ECS Service Task Definition Type: AWS::ECS::Service Properties: Cluster: super-cluster TaskDefinition: cool-task:3 DesiredCount: 3 LoadBalancers: - TargetGroupArn: my-alb-arn ContainerPort: 8080 ?? ]
25. twitter.com/toricls in Amazon ECS LB App Task Definition is immutable and it has “Revisions”
26. twitter.com/toricls Example GitOps Flow in Kubernetes and Amazon ECS App Repo
 (w/ Dockerfile) Code changes git push CI process docker push Image Registry Manifests Repo
 (w/ manifests) update manifests, create pull request CD process kubectl apply App Repo
 (w/ Dockerfile, w/ Task-def.) Code changes git push CI process docker push Image Registry Manifests Repo
 (w/ Service-def.) update definition files, create pull request CD process aws ecs update-service (タスク定義は CI 処理理で ECS 側に登録済みなので CD 処理理には不不要) ECS API register task-def
27. twitter.com/toricls App Logging in Kubernetes and Amazon ECS ▶ ログの収集はノードレベル・クラスタレベルの意思決定/オペレーション ▶ DaemonSet によってログ収集エージェントをノード群に展開 ▶ Docker ログドライバー(*) をサポートしないという⽅向性 ▶ ログの収集をアプリケーションレベルで意思決定/オペレーションすることがあり得る ▶ Docker ログドライバー機能をタスク定義から設定する ▶ アプリケーションごとにログの送付先が異なるケースが想定されている ▶ 2018.06 に追加されたデーモンスケジューラー機能を使ってログ収集エージェントを配布する⽅式も選択可能ではある *) https://docs.docker.com/config/containers/logging/configure/
28. twitter.com/toricls Lots of Fun Diffs in Kubernetes and Amazon ECS ▶ Kubernetes Jobs ▶ ECS では Step Functions という別サービスと ECS タスクを組み合わせて実現する ▶ Kubernetes CronJobs ▶ ECS では EventBridge + Step Functions + ECS タスクの組み合わせ ▶ ECS タスク - Container Dependency ▶ Kubernetes では initContainers を利⽤したり ▶ コンテナライフサイクルフック(preStop とか)を組み合わせ ▶ Kubernetes 1.19(?) の containers.lifecycle.type: SideCar に期待… ▶ Kubernetes Admission Controllers ▶ ECS でこれにピッタリと該当する機能/やり⽅は存在しない気がする
29. twitter.com/toricls AWS Container Services - Public Roadmap https://github.com/aws/containers-roadmap/projects/1
30. twitter.com/toricls AWS Container Services - Public Roadmap https://github.com/aws/containers-roadmap/projects/1
31. twitter.com/toricls Thank you :)