Container Solutions on AWS

Containers improved the deployments of both simple and complex applications by providing repeatable and isolated environments. In this presentation, we will talk about what containers really are, and the underlying technology that differentiates them from Virtual Machines and explain why they are faster and perfect for agile development and deployments. Afterwards, we will focus on container solutions of ECS (Elastic Container Service) which includes a variety of tools that suits different levels of use cases. We will talk about how to orchestrate your services on AWS ECS, manage images, lifecycle of containers, service discovery and monitoring. We will also talk about newly introduced container services of ECS, Fargate and EKS (Elastic Kubernetes Service)

Transcript

1. Container Solutions on AWS Mustafa Akın Site Reliability Engineer (SRE) @opsgenie @mustafaakin https://www.linkedin.com/in/mustafaakin/ #OgInsights
2. Outline - Virtualization & Container Primitives - Containers, Microservices and Use Cases - Amazon Container Services: - Amazon Elastic Container Registry (ECR) - Amazon Elastic Container Service (ECS) and Fargate - Amazon Elastic Container Service for Kubernetes #OgInsights @mustafaakin
3. Virtualization “In computing, virtualization refers to the act of creating a virtual (rather than actual) version of something, including virtual computer hardware platforms, storage devices, and computer network resources.” -- Wikipedia #OgInsights @mustafaakin
4. Virtualization Timeline: Timesharing 1961: Time slicing/sharing mainframe by IBM 7091 #OgInsights @mustafaakin
5. Time-sharing #OgInsights @mustafaakin
6. Time Slice Process #1 Process #2 Process #3 #OgInsights @mustafaakin
7. Time Slice: Scheduling Problem P1 P2 P3 P2 P1 P2 P2 P2 P1 P3 Effectiveness Fairness Correctness #OgInsights @mustafaakin
8. Virtualization Timeline: Timesharing 1971: Unix V1 by Bell Labs Ken Thompson & Dennis Ritchie with a PDP-11 #OgInsights @mustafaakin
9. Virtualization Timeline: Timesharing 1979: Unix V7 with chroot #OgInsights @mustafaakin
10. Virtualization Timeline: Timesharing 1984: HP 9000 Workstation and Server, running HP-UX, which is based on UNIX System V. #OgInsights @mustafaakin
11. Ability to Run Many Applications #OgInsights @mustafaakin
12. Virtualization Timeline: Hypervisors 1998: First company to successfully virtualize x86 Architecture #OgInsights @mustafaakin
13. Virtualization Timeline: Hypervisors 2003: First open source x86 Hypervisor, Xen #OgInsights @mustafaakin
14. AWS Virtual Machines: EC2 #OgInsights @mustafaakin
15. Digital Ocean Virtual Machines: Droplets #OgInsights @mustafaakin
16. VM for your PC: VirtualBox #OgInsights @mustafaakin
17. Virtualization Timeline: Isolation 2000: BSD Jails 2001: Linux-VServer 2004: Solaris Zones 2005: OpenVZ #OgInsights @mustafaakin
18. Virtualization Timeline: Isolation V2 2002: Linux Namespaces 2006: cgroups for Linux 2008: Linux Containers #OgInsights @mustafaakin
19. Virtualization Timeline: Isolation V3 2013: Docker 2014: Kubernetes by Google #OgInsights @mustafaakin
20. Kubernetes Dashboard #OgInsights @mustafaakin
21. Why Virtualization is required? Physical Server 48 CPU x 192 GB Memory VM #1 4 CPU - 8 GB VM #2 12 CPU 16 GB VM #3 8 CPU 2 GB VM #4 16 CPU 48 GB VM #5 0.5 CPU - 512 MB #OgInsights @mustafaakin
22. Why Virtualization is required? Physical Server 48 CPU x 192 GB Memory Container #1 4 CPU - 8 GB Container #2 12 CPU 16 GB Container #3 8 CPU 2 GB Container #4 16 CPU 48 GB Container #5 0.5 CPU - 512 MB #OgInsights @mustafaakin
23. Why Virtualization is required? #OgInsights @mustafaakin
24. What is a container? - A lightweight virtualization system - Makes use of operating system level isolation - Hardware is not virtualized #OgInsights @mustafaakin
25. Virtual Machine - Boots in 15-50 seconds in EC2 - Emulation of CPU, RAM, Disks, Network - Complete isolation - Complete root access Docker - Starts in sub-seconds if image is present - Isolation of existing devices - Shared kernel - Defaults to limited privileged root user #OgInsights @mustafaakin
26. Virtual Machine vs Container #OgInsights @mustafaakin
27. Cattle vs Pets - cow-432-21072018.mydomain.com - Not important individually - Old and sick ones are shot/killed - lassie.mydomain.com - Personal relationship, love them - You cry when they die #OgInsights @mustafaakin
28. Container Internals • Namespaces ◦ Isolate processes in the Linux kernel • Cgroups ◦ Account and Limit Resource / Devices usage ▪ CPU, Memory, Disk, Network #OgInsights @mustafaakin
29. Regular Linux Process Tree #OgInsights @mustafaakin
30. Process Tree inside a Docker container #OgInsights @mustafaakin
31. Why would you use Docker? - Scale up & down fast - Single process = Single responsibility - Image Version = Update & Rollback - Clean environments - Utilization - Easy Packaging #OgInsights @mustafaakin
32. Actual Container
33. Container Operator
34. Container Ship
35. Ship Fleet
36. Microservices and Container Mega Application 150+ Services Deployed as a whole Also fails as a whole How to monitor a single part? Simple update causes complete refresh Com piles at 15 M inutes Scale up requires another copy of all Requires too many CPU & RAM Boots in 5 minutes #OgInsights @mustafaakin
37. Microservices and Container #OgInsights @mustafaakin
38. Why not VM per microservice then? - Slow boot time - Performance overhead - No isolation - Monitoring is harder - Utilization #OgInsights @mustafaakin
39. Containers should not be treated like VMs - No SSH - One top level process - No snapshot & restore of containers - Root accounts should not be used - No Ansible/Puppet/Chef for configuring container, use Dockerfile - Containers should not embed credentials, should get them externally #OgInsights @mustafaakin
40. Dockerfile #OgInsights @mustafaakin
41. Docker image building #OgInsights @mustafaakin
42. Docker images are Layered java:8 maven my-app-v1 mybuilder java:9 uservice ubuntu imagemagick image-resizer micro-service install-go my-app-v2 new-container new-container new-container new-container #OgInsights @mustafaakin
43. Amazon ECR: Elastic Container Registry - The Docker image repository backed by S3 - Can delete old images by lifecycle policies - Push/Pull Protected by IAM - Integration with ECS $ IMAGE_NAME=137175318439.dkr.ecr.eu-west-1.amazonaws.com/mywebapp:1.0.1 $ docker build . -t ${IMAGE_NAME} $ docker push ${IMAGE_NAME} #OgInsights @mustafaakin
44. Docker Image Dev-Stage-Prod Parity - Prepare the image locally - Can run locally - Better visibility for developers #OgInsights @mustafaakin
45. #OgInsights @mustafaakin
46. #OgInsights @mustafaakin
47. #OgInsights @mustafaakin
48. #OgInsights @mustafaakin
49. #OgInsights @mustafaakin
50. #OgInsights @mustafaakin
51. #OgInsights @mustafaakin
52. #OgInsights @mustafaakin
53. Container Use Cases Fast & Clean Environments java:8 my-app-v1 new-container new-container new-container #OgInsights @mustafaakin
54. Container Use Cases Continuous Integration #OgInsights @mustafaakin
55. Container Use Cases Continuous Delivery #OgInsights @mustafaakin
56. Container Use Cases Local Development & Testing #OgInsights @mustafaakin
57. Container Use Cases Microservices app:v1 app:v1 app:v2 image-resize-service app:v2 image-resize-service file-server app:v1 app:v1 app:v1 app:v1 image-resize-service #OgInsights @mustafaakin
58. Container Use at OpsGenie - Continuous Integration Tests - Packaging and Compiling - Staging environment - Branch deploy for developers - Deployment of some server software - Elasticsearch, Redis, Local DynamoDB, Jaeger #OgInsights @mustafaakin
59. Kubernetes@OpsGenie #OgInsights @mustafaakin
60. Kubernetes@OpsGenie #OgInsights @mustafaakin
61. Kubernetes@OpsGenie #OgInsights @mustafaakin
62. Kubernetes@OpsGenie #OgInsights @mustafaakin
63. Kubernetes@OpsGenie #OgInsights @mustafaakin
64. Kubernetes@OpsGenie Personal OpsGenie Instances with 20+ Services - app.mustafa.opgenie-test-domain.com - alert-service.mustafa.opgenie-test-domain.com Production, coming soon :) #OgInsights @mustafaakin
65. Docker is not the ultimate solution. The concept to embrace is containerization. Docker is just a tool. #OgInsights @mustafaakin
66. Multi Node <?> Orchestration Historical problem of Docker and Everyone
67. Multi Node Orchestration History #OgInsights @mustafaakin
68. Why Virtualization is required? #OgInsights @mustafaakin
69. Resource Management is hard DEAD ? #OgInsights @mustafaakin
70. Resource Management Problems at Scale - State of cluster - Number of tasks - Efficient usage of remaining resources - Recovering from errors - Over-fitting & under-utilization - Multi-tenancy #OgInsights @mustafaakin
71. Multi Node Orchestration of Docker - Since Docker arise, people tried to create orchestration tools - Docker tried Swarm → not a success - Swarm Mode → better, but still has shortcomings - Google already had Omega, Borg and converted them to Kubernetes which gained much traction - Amazon created ECS, Fargate, EKS - Third party solutions: Rancher, Nomad #OgInsights @mustafaakin
72. Multi Node Orchestration of Docker Server #1 #OgInsights @mustafaakin
73. Multi Node Orchestration of Docker Server #1 Master #OgInsights @mustafaakin
74. Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 #OgInsights @mustafaakin
75. Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 #OgInsights @mustafaakin
76. Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master #OgInsights @mustafaakin
77. Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 #OgInsights @mustafaakin
78. Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 LOAD BALANCER #OgInsights @mustafaakin
79. Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 LOAD BALANCER SERVICE DISCOVERY #OgInsights @mustafaakin
80. Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 LOAD BALANCER SERVICE DISCOVERY STATEFUL SERVICES #OgInsights @mustafaakin
81. Multi Node Orchestration of Docker Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 LOAD BALANCER SERVICE DISCOVERY CONFIGURATION AND SECRETS STATEFUL SERVICES #OgInsights @mustafaakin
82. #OgInsights @mustafaakin
83. #OgInsights @mustafaakin
84. #OgInsights @mustafaakin
85. Additional AWS Services - Lambda - CodeBuild - XRay - Load Balancers (ALB, ELB, NLB) - Secret Manager - Parameter Store - SQS, SNS, MQ #OgInsights @mustafaakin
86. - Wraps Docker API - ECS agent in EC2 - Virtual clusters managed by AWS - Task definitions that defines 1 to many containers - ECS cluster run-task command - VPC Task networking awsvpc - Tasks can have IAM roles independent of the host - Cloudwatch Logs integration - Placement: - Availability zone, instance types, bin-packing, spreading #OgInsights @mustafaakin
87. Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 EC2 + Docker EC2 + Docker EC2 + Docker EC2 + Docker Managed by AWS Managed by You PLEASE RUN MY CONTAINER #OgInsights @mustafaakin
88. ECS Task definition #OgInsights @mustafaakin
89. ECS Task definition, Docker Equivalent #OgInsights @mustafaakin
90. Scheduling in ECS - Task: - Run and die - Batch processing - Service: - Ensures a number of containers are run - Can work with Elastic Load Balancer - Daemon: - Runs in every worker node - Logging, monitoring, backups #OgInsights @mustafaakin
91. Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 EC2 + Docker EC2 + Docker EC2 + Docker EC2 + Docker Managed by AWS PLEASE RUN MY CONTAINER #OgInsights @mustafaakin
92. - Extension for ECS - Worker nodes are managed by AWS - Auto-scale services, set desired task count #OgInsights @mustafaakin
93. AWS Lambda - Function as a service - Runs in security tightened Linux containers - Wrappers for Java, Python, Go, Nodejs PLEASE RUN MY FUNCTION WHEN SOME EVENT HAPPENS #OgInsights @mustafaakin
94. Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Master Master Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 Server #1 Server #2 Server #3 Server #4 EC2 + Kubelet EC2 + Kubelet EC2 + Kubelet EC2 + Kubelet Managed by AWS PLEASE MAKE MY CONFIGURATION HAPPEN #OgInsights @mustafaakin
95. - Based on Kubernetes - Master nodes are managed by AWS for a fee - Worker nodes are added by user - CNCF Kubernetes Conformant #OgInsights @mustafaakin
96. kops - Official Kubernetes product - Custom clusters on AWS - Any network driver - Custom etcd and master configuration - Alpha/Beta APIs can be enabled #OgInsights @mustafaakin
97. kops #OgInsights @mustafaakin
98. Pod #OgInsights @mustafaakin
99. Deployment #OgInsights @mustafaakin
100. Apply #OgInsights @mustafaakin
101. - Declarative API - Simple Objects - Combine objects to make more complex objects - Pod → Replica Set → Deployment - Service Discovery - Service, Ingress - Data - Volume, Secrets - Role Based Access - User, Role, ServiceAccount #OgInsights @mustafaakin
102. Thanks Mustafa Akın @mustafaakin join Follow ENGINEERING BLOG #OgInsights @mustafaakin