Share
Embed
Deep Dive on Containers
Learn how to deploy and manage containers at scale in the cloud.
This session covers everything from Linux container internals to Amazon Elastic Container Services with and without Fargate,
Amazon Elastic Kubernetes Service and AWS App Mesh.
Frank Munz
October 02, 2019
Transcript
60 slides
-
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Dr Frank Munz Senior Technical Evangelist Amazon Web Services Deep Dive on Containers Brian Bordini Cloud Architect Richemont -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T About me • Software Architect / DevOps Engineer • Technical Evangelist @ AWS • Published an AWS book • Containers, serverless and a sprinkle of ML & big / fast data @frankmunz -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. Level – 300 “Sessions dive deeper into the selected topic. Presenters assume that the audience has some familiarity with the topic, but may or may not have direct experience implementing a similar solution” -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Container Amazon Elastic Container Service Amazon Elastic Kubernetes Service AWS App Mesh Brian Bordini, Richemont Agenda -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Linux Kernel Container runtime Container 1 Container 2 Container 3 Container 4 Container 5 Container 6 Control Groups Namespaces Union filesystem -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T What can you use cgroups for? • Linux kernel feature that defines resource limits, priotization, control and accounting for processes • Implemented by subsystems • Typically mounted at /sys/fs/cgroup or /cgroup Examples of subsystems: • Memory • CPU time • Block I/O • Number of discrete processes (pids) • CPU & memory pinning • Freezer (used by docker pause) • Devices • Network priority -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T And namespaces? • Partition kernel resources like hostname, proc IDs etc. • Namespaces are visible in /proc organised by PID • Files are symbolic links to the namespace $ readlink /proc/$$/ns/* cgroup:[4026531835] ipc:[4026531839] mnt:[4026531840] net:[4026531993] pid:[4026531836] user:[4026531837] uts:[4026531838] -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Network namespace • Docker uses a separate network namespace per container • Multiple containers can share a network namespace, used with K8s pods and Amazon ECS task • Improve isolation by creating dedicated network interfaces: ECS awsvpc networking, EKS amazon-vpc-cni-k8s plugin -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Union Filesystems • Used by Docker to implement layers • Efficient use of storage -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Container Base Images -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Go • Modern language • Docker is implemented in Go • Native concurrency • Low startup time • Small, static binaries • Easy cross compilation -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Java – Open Source Cloud Native Stacks Polyglot, declarative, reactive, microservices frameworks like Micronaut or Quarkus based on Graal VM. • Dead code eliminiation • Aggressive Advance of Time Compliation (AOT) • Native image generation • Docker and Kubernetes YAML file generation • Startup times in milliseconds https://quarkus.io/vision/container-first -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T “Just launch 10 copies of my container distributed across three availability zones and connect them to this load balancer” X 10 -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. Amazon ECS Easiest way to deploy and manage containers at scale Integration with entire AWS platform ALB, Auto Scaling, Batch, Elastic Beanstalk, CloudFormation, CloudTrail, CloudWatch Events, CloudWatch Logs, CloudWatch Metrics, ECR, EC2 Spot, IAM, NLB, Parameter Store, and VPC Scales to support clusters of any size Service integrations (like ALB and NLB) are at container level 1 2 3 -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. ECS EC2 vs. ECS Fargate EC2 Fargate Managed by Customer AWS Storage Ephemeral or Persistent Only Ephemeral Sidecar pattern Yes Yes Network Mode Bridge or VPC Mode VPC Mode Daemons Yes No SSH into host Yes No Privileged Containers Yes No -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Task Definition: CPU & MEMORY SPECIFICATION { "family": "scorekeep", "cpu": "1 vCpu", "memory": "2 gb", "containerDefinitions": [ { "name":“scorekeep-frontend", "image":"xxx.dkr.ecr.us-east-1.amazonaws.com/fe“, "cpu": 256, "memoryReservation": 512 }, { "name":“scorekeep-api", "image":"xxx.dkr.ecr.us-east-1.amazonaws.com/api", "cpu": 768, "memoryReservation": 512 } ] } Units • CPU : cpu-units. 1 vCPU = 1024 cpu-units • Memory : MB Task Level Resources: • Total Cpu/Memory across all containers • Required fields • Billing axis Container Level Resources: • Defines sharing of task resources among containers • Optional fields Task Level Resources Container Level Resources Task Definition Snippet -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T CONFIGURABLE NETWORKING 172.31.0.0/16 Subnet 172.31.1.0/24 Internet Other Entities in VPC EC2 LB DB etc. Private IP 172.31.1.164 Explicit control for your containers networking: • Subnet placement, specific IP address ranges • Private IP address (optional public IP address) • Security group inbound access only from specific sources on specific ports Under the hood : • We create an Elastic Network Interface (ENI) • The ENI is allocated a private IP from your subnet • The ENI is attached to your task • Your task now has a private IP from your subnet! • Optionally you can also give it a public IP address if its in a public subnet with internet access ENI Fargate Task Public / 208.57.73.13 / -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. AWS CDK Contribute Code AWS Cloud Development Kit Define cloud infrastructure and reusable components in “real code“ and provision through AWS CloudFormation Stack(s) CDK Application Construct Construct CloudFormation template Resources -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. ECS Pattern: Load Balanced Fargate Service Basic constructs: Cluster, Task Definition, Task, Service, etc. Common architecture patterns: load balanced service …this TypeScript generates 568 CloudFormation LOC -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. CDK Ressource: https://garbe.io/ Philipp Garbe AWS Container Hero -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. “Run Kubernetes for me.” -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. Availability Zone 1 Availability Zone 2 Availability Zone 3 Kubectl Amazon EKS Architecture CloudWatch Container Insights for Amazon EKS and ECS -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. Amazon EKS Features Bring your own instances / types Packer configuration for building a custom EKS AMI Market place instances with Tensorflow and NVIDIA packages CloudWatch Container Insights for EKS and ECS -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. https://eksctl.io/ Tip: Easy way to create an EKS cluster: $ eksctl create cluster --name meshtest --appmesh-access -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. OSSC libraries: code changes required, language specific Service Mesh: decentral, language agnostic, polyglot, light-weight https://www.infoq.com/articles/microservices-post-kubernetes Need for a Service Mesh -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. … EKS is open source, upstream Kubernetes and supports Istio / Envoy -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. But you asked us for more… -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. A mesh for all compute services -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. App Mesh works across compute services Amazon ECS AWS Fargate Amazon EKS Amazon EC2 Kubernetes on EC2 Based on Envoy proxy It‘s free (you only pay for resources used) -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. Demo: Install AppMesh # AppMesh Installation with Grafana/Prometheus # and Envoy, AWS X-Ray daemon sidecar # and StatsD Prometheus exporter $ helm install -n aws-appmesh \ --namespace appmesh-system \ https://github.com/PaulMaddox/ \ aws-appmesh-helm/releases/ \ latest/download/aws-appmesh.tgz https://github.com/PaulMaddox/aws-appmesh-helm -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. Install Demo App # create namespace & enable auto-injection $ kubectl create ns appmesh-demo $ kubectl label namespace appmesh-demo appmesh.k8s.aws/sidecarInjectorWebhook=enabled # deploy the demo $ helm install -n aws-appmesh-demo \ --namespace appmesh-demo \ https://github.com/.../aws-appmesh-demo.tgz -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. Observability: AWS X-Ray Service Map -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. AWS X-Ray: Traces -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. Open-Source Grafana: AppMesh Overview -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. Public Container Road Map https://github.com/aws/containers-roadmap -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. -
Who Am I ? • Developer converted to Ops and
Architecture • Working since one year with Kubernetes • "In the cloud" since two years • Worked in banking and insurance industries • Now in luxury industry (Geneva and Shanghai) Twitter: @brnmori LinkedIn: Brian Bordini -
Challenges of this journey A web product with a global
user base • Customer satisfaction • Time to market • Unpredictable load • Security and compliance • Cost efficiency Chéserex, Switzerland -
Availability Zone Amazon Simple Queue Service Amazon Elastic Container Service
for Kubernetes Amazon Aurora Amazon ElastiCache CDN Load Balancer Customers Availability Zone Amazon Aurora Amazon ElastiCache Auto Scaling Groups Instances Spot instances Spot instances Instances Amazon Simple Storage Service (S3) DB group AWS Direct Connect Architecture -
23 Pizzas Team 8 Developers 1 Quality Assurance Engineer 1
Lead Architect 1 Solution Architect / Automation Engineer 1 Product Owner 1 Business Analyst 1 Agile Coach -
Customer satisfaction, Time to Market Fast provisioning and low operational
overhead with managed services Repeatability and consistency with Infrastructure as Code Seamless experience from dev machine to production with Docker -
Unpredictable Load First worldwide launch, design for elasticity and simplicity
Elastic horizontal workloads with Amazon EKS Workers AutoScaling Groups with stateless Kubernetes Pods CDN and Amazon ElastiCache Redis for caching -
Security Encryption at rest and in-transit with one click by
template Automated certificate rotation for AWS services Accounts centrally managed in IAM -
Cost Optimization Spot Instances save up to 70-80% compared to
On-Demand Shutdown non-production services in non-business hours Purchase Reserved Instances -
Time Line January 2018 • project inception March 2018 •
start with AWS Container Service July 2018 • installation of self managed Kubernetes January 2019 • switch to AWS EKS May 2019 • go-live -
Outcome Customers satisfied Scalable infrastructure with fewer cost than on-premises
Team ownership from inception to production Faster time to market with better quality than traditional deployment Reusable infrastructure modules and CI/CD pipelines for future projects -
Q & A Any feedback is welcome J You can
reach me on: • LinkedIn: Brian Bordini • Twitter: @brnmori -
© 2019, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. Summary • Architect wisely (Serverless vs ECS vs EKS vs EC2) • Running K8s is hard, use a managed K8s service • AWS App Mesh is free to use and works across compute services -
Thank you! S U M M I T © 2019,
Amazon Web Services, Inc. or its affiliates. All rights reserved. frankmunz @frankmunz https://medium.com/@frank.munz (Blog) https://speakerdeck.com/fmunz (Slides) !