Tweet
Tweet

Slide 1

Slide 1 text

Container Solutions on AWS Mustafa Akın Site Reliability Engineer (SRE) @opsgenie @mustafaakin https://www.linkedin.com/in/mustafaakin/ #OgInsights

Slide 2

Slide 2 text

Outline - Virtualization & Container Primitives - Containers, Microservices and Use Cases - Amazon Container Services: - Amazon Elastic Container Registry (ECR) - Amazon Elastic Container Service (ECS) and Fargate - Amazon Elastic Container Service for Kubernetes #OgInsights @mustafaakin

Slide 3

Slide 3 text

Virtualization “In computing, virtualization refers to the act of creating a virtual (rather than actual) version of something, including virtual computer hardware platforms, storage devices, and computer network resources.” -- Wikipedia #OgInsights @mustafaakin

Slide 4

Slide 4 text

Virtualization Timeline: Timesharing 1961: Time slicing/sharing mainframe by IBM 7091 #OgInsights @mustafaakin

Slide 5

Slide 5 text

Time-sharing #OgInsights @mustafaakin

Slide 6

Slide 6 text

Time Slice Process #1 Process #2 Process #3 #OgInsights @mustafaakin

Slide 7

Slide 7 text

Time Slice: Scheduling Problem P1 P2 P3 P2 P1 P2 P2 P2 P1 P3 Effectiveness Fairness Correctness #OgInsights @mustafaakin

Slide 8

Slide 8 text

Virtualization Timeline: Timesharing 1971: Unix V1 by Bell Labs Ken Thompson & Dennis Ritchie with a PDP-11 #OgInsights @mustafaakin

Slide 9

Slide 9 text

Virtualization Timeline: Timesharing 1979: Unix V7 with chroot #OgInsights @mustafaakin

Slide 10

Slide 10 text

Virtualization Timeline: Timesharing 1984: HP 9000 Workstation and Server, running HP-UX, which is based on UNIX System V. #OgInsights @mustafaakin

Slide 11

Slide 11 text

Ability to Run Many Applications #OgInsights @mustafaakin

Slide 12

Slide 12 text

Virtualization Timeline: Hypervisors 1998: First company to successfully virtualize x86 Architecture #OgInsights @mustafaakin

Slide 13

Slide 13 text

Virtualization Timeline: Hypervisors 2003: First open source x86 Hypervisor, Xen #OgInsights @mustafaakin

Slide 14

Slide 14 text

AWS Virtual Machines: EC2 #OgInsights @mustafaakin

Slide 15

Slide 15 text

Digital Ocean Virtual Machines: Droplets #OgInsights @mustafaakin

Slide 16

Slide 16 text

VM for your PC: VirtualBox #OgInsights @mustafaakin

Slide 17

Slide 17 text

Virtualization Timeline: Isolation 2000: BSD Jails 2001: Linux-VServer 2004: Solaris Zones 2005: OpenVZ #OgInsights @mustafaakin

Slide 18

Slide 18 text

Virtualization Timeline: Isolation V2 2002: Linux Namespaces 2006: cgroups for Linux 2008: Linux Containers #OgInsights @mustafaakin

Slide 19

Slide 19 text

Virtualization Timeline: Isolation V3 2013: Docker 2014: Kubernetes by Google #OgInsights @mustafaakin

Slide 20

Slide 20 text

Kubernetes Dashboard #OgInsights @mustafaakin

Slide 21

Slide 21 text

Why Virtualization is required? Physical Server 48 CPU x 192 GB Memory VM #1 4 CPU - 8 GB VM #2 12 CPU 16 GB VM #3 8 CPU 2 GB VM #4 16 CPU 48 GB VM #5 0.5 CPU - 512 MB #OgInsights @mustafaakin

Slide 22

Slide 22 text

Why Virtualization is required? Physical Server 48 CPU x 192 GB Memory Container #1 4 CPU - 8 GB Container #2 12 CPU 16 GB Container #3 8 CPU 2 GB Container #4 16 CPU 48 GB Container #5 0.5 CPU - 512 MB #OgInsights @mustafaakin

Slide 23

Slide 23 text

Why Virtualization is required? #OgInsights @mustafaakin

Slide 24

Slide 24 text

What is a container? - A lightweight virtualization system - Makes use of operating system level isolation - Hardware is not virtualized #OgInsights @mustafaakin

Slide 25

Slide 25 text

Virtual Machine - Boots in 15-50 seconds in EC2 - Emulation of CPU, RAM, Disks, Network - Complete isolation - Complete root access Docker - Starts in sub-seconds if image is present - Isolation of existing devices - Shared kernel - Defaults to limited privileged root user #OgInsights @mustafaakin

Slide 26

Slide 26 text

Virtual Machine vs Container #OgInsights @mustafaakin

Slide 27

Slide 27 text

Cattle vs Pets - cow-432-21072018.mydomain.com - Not important individually - Old and sick ones are shot/killed - lassie.mydomain.com - Personal relationship, love them - You cry when they die #OgInsights @mustafaakin

Slide 28

Slide 28 text

Container Internals ● Namespaces ○ Isolate processes in the Linux kernel ● Cgroups ○ Account and Limit Resource / Devices usage ■ CPU, Memory, Disk, Network #OgInsights @mustafaakin

Slide 29

Slide 29 text

Regular Linux Process Tree #OgInsights @mustafaakin

Slide 30

Slide 30 text

Process Tree inside a Docker container #OgInsights @mustafaakin

Slide 31

Slide 31 text

Why would you use Docker? - Scale up & down fast - Single process = Single responsibility - Image Version = Update & Rollback - Clean environments - Utilization - Easy Packaging #OgInsights @mustafaakin

Slide 32

Slide 32 text

Actual Container

Slide 33

Slide 33 text

Container Operator

Slide 34

Slide 34 text

Container Ship

Slide 35

Slide 35 text

Ship Fleet

Slide 36

Slide 36 text

Microservices and Container Mega Application 150+ Services Deployed as a whole Also fails as a whole How to monitor a single part? Simple update causes complete refresh Com piles at 15 M inutes Scale up requires another copy of all Requires too many CPU & RAM Boots in 5 minutes #OgInsights @mustafaakin

Slide 37

Slide 37 text

Microservices and Container #OgInsights @mustafaakin

Slide 38

Slide 38 text

Why not VM per microservice then? - Slow boot time - Performance overhead - No isolation - Monitoring is harder - Utilization #OgInsights @mustafaakin

Slide 39

Slide 39 text

Containers should not be treated like VMs - No SSH - One top level process - No snapshot & restore of containers - Root accounts should not be used - No Ansible/Puppet/Chef for configuring container, use Dockerfile - Containers should not embed credentials, should get them externally #OgInsights @mustafaakin

Slide 40

Slide 40 text

Dockerfile #OgInsights @mustafaakin

Slide 41

Slide 41 text

Docker image building #OgInsights @mustafaakin

Slide 42

Slide 42 text

Docker images are Layered java:8 maven my-app-v1 mybuilder java:9 uservice ubuntu imagemagick image-resizer micro-service install-go my-app-v2 new-container new-container new-container new-container #OgInsights @mustafaakin

Slide 43

Slide 43 text

Amazon ECR: Elastic Container Registry - The Docker image repository backed by S3 - Can delete old images by lifecycle policies - Push/Pull Protected by IAM - Integration with ECS $ IMAGE_NAME=137175318439.dkr.ecr.eu-west-1.amazonaws.com/mywebapp:1.0.1 $ docker build . -t ${IMAGE_NAME} $ docker push ${IMAGE_NAME} #OgInsights @mustafaakin

Slide 44

Slide 44 text

Docker Image Dev-Stage-Prod Parity - Prepare the image locally - Can run locally - Better visibility for developers #OgInsights @mustafaakin

Slide 45

Slide 45 text

#OgInsights @mustafaakin

Slide 46

Slide 46 text

#OgInsights @mustafaakin

Slide 47

Slide 47 text

#OgInsights @mustafaakin

Slide 48

Slide 48 text

#OgInsights @mustafaakin

Slide 49

Slide 49 text

#OgInsights @mustafaakin

Slide 50

Slide 50 text

#OgInsights @mustafaakin

Slide 51

Slide 51 text

#OgInsights @mustafaakin

Slide 52

Slide 52 text

#OgInsights @mustafaakin

Slide 53

Slide 53 text

Container Use Cases Fast & Clean Environments java:8 my-app-v1 new-container new-container new-container #OgInsights @mustafaakin

Slide 54

Slide 54 text

Container Use Cases Continuous Integration #OgInsights @mustafaakin

Slide 55

Slide 55 text

Container Use Cases Continuous Delivery #OgInsights @mustafaakin

Slide 56

Slide 56 text

Container Use Cases Local Development & Testing #OgInsights @mustafaakin

Slide 57

Slide 57 text

Container Use Cases Microservices app:v1 app:v1 app:v2 image-resize-service app:v2 image-resize-service file-server app:v1 app:v1 app:v1 app:v1 image-resize-service #OgInsights @mustafaakin

Slide 58

Slide 58 text

Container Use at OpsGenie - Continuous Integration Tests - Packaging and Compiling - Staging environment - Branch deploy for developers - Deployment of some server software - Elasticsearch, Redis, Local DynamoDB, Jaeger #OgInsights @mustafaakin

Slide 59

Slide 59 text

Kubernetes@OpsGenie #OgInsights @mustafaakin

Slide 60

Slide 60 text

Kubernetes@OpsGenie #OgInsights @mustafaakin

Slide 61

Slide 61 text

Kubernetes@OpsGenie #OgInsights @mustafaakin

Slide 62

Slide 62 text

Kubernetes@OpsGenie #OgInsights @mustafaakin

Slide 63

Slide 63 text

Kubernetes@OpsGenie #OgInsights @mustafaakin

Slide 64

Slide 64 text

Kubernetes@OpsGenie Personal OpsGenie Instances with 20+ Services - app.mustafa.opgenie-test-domain.com - alert-service.mustafa.opgenie-test-domain.com Production, coming soon :) #OgInsights @mustafaakin

Slide 65

Slide 65 text

Docker is not the ultimate solution. The concept to embrace is containerization. Docker is just a tool. #OgInsights @mustafaakin

Slide 66

Slide 66 text

Multi Node Orchestration Historical problem of Docker and Everyone

Slide 67

Slide 67 text

Multi Node Orchestration History #OgInsights @mustafaakin

Slide 68

Slide 68 text

Why Virtualization is required? #OgInsights @mustafaakin

Slide 69

Slide 69 text

Resource Management is hard DEAD ? #OgInsights @mustafaakin

Slide 70

Slide 70 text

Resource Management Problems at Scale - State of cluster - Number of tasks - Efficient usage of remaining resources - Recovering from errors - Over-fitting & under-utilization - Multi-tenancy #OgInsights @mustafaakin

Slide 71

Slide 71 text

Multi Node Orchestration of Docker - Since Docker arise, people tried to create orchestration tools - Docker tried Swarm → not a success - Swarm Mode → better, but still has shortcomings - Google already had Omega, Borg and converted them to Kubernetes which gained much traction - Amazon created ECS, Fargate, EKS - Third party solutions: Rancher, Nomad #OgInsights @mustafaakin

Slide 72

Slide 72 text

Multi Node Orchestration of Docker Server #1 #OgInsights @mustafaakin

Slide 73

Slide 73 text

Multi Node Orchestration of Docker Server #1 Master #OgInsights @mustafaakin

Slide 74

Slide 74 text

Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 #OgInsights @mustafaakin

Slide 75

Slide 75 text

Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 #OgInsights @mustafaakin

Slide 76

Slide 76 text

Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master #OgInsights @mustafaakin

Slide 77

Slide 77 text

Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 #OgInsights @mustafaakin

Slide 78

Slide 78 text

Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 LOAD BALANCER #OgInsights @mustafaakin

Slide 79

Slide 79 text

Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 LOAD BALANCER SERVICE DISCOVERY #OgInsights @mustafaakin

Slide 80

Slide 80 text

Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 LOAD BALANCER SERVICE DISCOVERY STATEFUL SERVICES #OgInsights @mustafaakin

Slide 81

Slide 81 text

Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 LOAD BALANCER SERVICE DISCOVERY CONFIGURATION AND SECRETS STATEFUL SERVICES #OgInsights @mustafaakin

Slide 82

Slide 82 text

#OgInsights @mustafaakin

Slide 83

Slide 83 text

#OgInsights @mustafaakin

Slide 84

Slide 84 text

#OgInsights @mustafaakin

Slide 85

Slide 85 text

Additional AWS Services - Lambda - CodeBuild - XRay - Load Balancers (ALB, ELB, NLB) - Secret Manager - Parameter Store - SQS, SNS, MQ #OgInsights @mustafaakin

Slide 86

Slide 86 text

- Wraps Docker API - ECS agent in EC2 - Virtual clusters managed by AWS - Task definitions that defines 1 to many containers - ECS cluster run-task command - VPC Task networking awsvpc - Tasks can have IAM roles independent of the host - Cloudwatch Logs integration - Placement: - Availability zone, instance types, bin-packing, spreading #OgInsights @mustafaakin

Slide 87

Slide 87 text

Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 EC2 + Docker EC2 + Docker EC2 + Docker EC2 + Docker Managed by AWS Managed by You PLEASE RUN MY CONTAINER #OgInsights @mustafaakin

Slide 88

Slide 88 text

ECS Task definition #OgInsights @mustafaakin

Slide 89

Slide 89 text

ECS Task definition, Docker Equivalent #OgInsights @mustafaakin

Slide 90

Slide 90 text

Scheduling in ECS - Task: - Run and die - Batch processing - Service: - Ensures a number of containers are run - Can work with Elastic Load Balancer - Daemon: - Runs in every worker node - Logging, monitoring, backups #OgInsights @mustafaakin

Slide 91

Slide 91 text

Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 EC2 + Docker EC2 + Docker EC2 + Docker EC2 + Docker Managed by AWS PLEASE RUN MY CONTAINER #OgInsights @mustafaakin

Slide 92

Slide 92 text

- Extension for ECS - Worker nodes are managed by AWS - Auto-scale services, set desired task count #OgInsights @mustafaakin

Slide 93

Slide 93 text

AWS Lambda - Function as a service - Runs in security tightened Linux containers - Wrappers for Java, Python, Go, Nodejs PLEASE RUN MY FUNCTION WHEN SOME EVENT HAPPENS #OgInsights @mustafaakin

Slide 94

Slide 94 text

Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 EC2 + Kubelet EC2 + Kubelet EC2 + Kubelet EC2 + Kubelet Managed by AWS PLEASE MAKE MY CONFIGURATION HAPPEN #OgInsights @mustafaakin

Slide 95

Slide 95 text

- Based on Kubernetes - Master nodes are managed by AWS for a fee - Worker nodes are added by user - CNCF Kubernetes Conformant #OgInsights @mustafaakin

Slide 96

Slide 96 text

kops - Official Kubernetes product - Custom clusters on AWS - Any network driver - Custom etcd and master configuration - Alpha/Beta APIs can be enabled #OgInsights @mustafaakin

Slide 97

Slide 97 text

kops #OgInsights @mustafaakin

Slide 98

Slide 98 text

Pod #OgInsights @mustafaakin

Slide 99

Slide 99 text

Deployment #OgInsights @mustafaakin

Slide 100

Slide 100 text

Apply #OgInsights @mustafaakin

Slide 101

Slide 101 text

- Declarative API - Simple Objects - Combine objects to make more complex objects - Pod → Replica Set → Deployment - Service Discovery - Service, Ingress - Data - Volume, Secrets - Role Based Access - User, Role, ServiceAccount #OgInsights @mustafaakin

Slide 102

Slide 102 text

Thanks Mustafa Akın @mustafaakin join Follow ENGINEERING BLOG #OgInsights @mustafaakin