Microservices on AWS ECS

Slide 1

Slide 1 text

SAY MICROSERVICES ONE MORE GODDAMN TIME

Slide 2

Slide 2 text

ON M****SERVICES AWS ECS

Slide 3

Slide 3 text

Maciej Winnicki mthenw

Slide 4

Slide 4 text

What is AWS EC2 Container Service? How does it work? Service discovery Configuration Deployment Scaling What is missing?

Slide 5

Slide 5 text

What is AWS EC2 Container Service?

Slide 6

Slide 6 text

… is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html “

Slide 7

Slide 7 text

Amazon Docker As A Service https://www.expeditedssl.com/aws-in-plain-english

Slide 8

Slide 8 text

How does it work?

Slide 9

Slide 9 text

AZ AZ EC2 EC2 EC2 EC2

Slide 10

Slide 10 text

Cluster EC2 EC2 EC2 EC2

Slide 11

Slide 11 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent

Slide 12

Slide 12 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler

Slide 13

Slide 13 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task

Slide 14

Slide 14 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Task

Slide 15

Slide 15 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Task Task

Slide 16

Slide 16 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Task Task

Slide 17

Slide 17 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Task Task Docker container Docker container

Slide 18

Slide 18 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Task Task Docker container Docker container Task Definition {“containerDefinitions": [ { "name": "web", "image": "nginx", "cpu": 99, "memory": 100, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] } ]}

Slide 19

Slide 19 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task

Slide 20

Slide 20 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Service 1 Task Service 1 Task Service 1

Slide 21

Slide 21 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Service 1 Task Service 1 Task Service 1 ELB

Slide 22

Slide 22 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Service 1 Task Service 1 Task Service 1 ELB :80 :8001 :8001 :8001

Slide 23

Slide 23 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent AWS ECS Task Service 1 Task Service 1 Task Service 1 ELB Key/Value Store Cluster Management Engine Agent Communication Service API http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html

Slide 24

Slide 24 text

Microservices

Slide 25

Slide 25 text

ECS Cluster AZ EC2 ECS Agent EC2 ECS Agent AZ EC2 ECS Agent EC2 ECS Agent

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

ECS Cluster (c4.xlarge) AZ EC2 ECS Agent EC2 ECS Agent AZ EC2 ECS Agent EC2 ECS Agent Stateful cluster (non-ECS, m4.xlarge) EC2 w/ EBS EC2 w/ EBS EC2 w/ EBS MongoDB MongoDB ElasticSearch ElasticSearch

Slide 28

Slide 28 text

Service discovery Configuration https://www.consul.io

Slide 29

Slide 29 text

Service discovery via DNS or HTTP Key/Value Storage Failure Detection https://www.consul.io

Slide 30

Slide 30 text

Service discovery Configuration https://www.consul.io

Slide 31

Slide 31 text

ECS Cluster AZ EC2 ECS Agent Consul EC2 ECS Agent Consul AZ EC2 ECS Agent Consul EC2 ECS Agent Consul Stateful cluster EC2 w/ EBS EC2 w/ EBS EC2 w/ EBS MongoDB MongoDB ElasticSearch ElasticSearch Consul Consul Consul

Slide 32

Slide 32 text

ECS Cluster AZ EC2 ECS Agent Consul (Server) EC2 ECS Agent Consul (Server) AZ EC2 ECS Agent Consul (Server) EC2 ECS Agent Consul Stateful cluster EC2 w/ EBS EC2 w/ EBS EC2 w/ EBS MongoDB MongoDB ElasticSearch ElasticSearch Consul Consul Consul Docker container Docker container Consul DNS

Slide 33

Slide 33 text

Service discovery via DNS Consul as a default DNS for every container

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Registrator + Consul = 20MB RAM

Slide 36

Slide 36 text

Service discovery Configuration https://www.consul.io

Slide 37

Slide 37 text

JSON file for local development Consul Key/Value for any other environment Dynamic feature flags stored in Consul K/V

Slide 38

Slide 38 text

Deployment

Slide 39

Slide 39 text

build image commit click! run ecs deploy deploy ECS

Slide 40

Slide 40 text

Small docker images! (e.g. mhart/alpine-node: 35MB) ECS service update strategy (add before remove)

Slide 41

Slide 41 text

Scaling

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Task (Service 1) Task (Service 1) Task (Service 1) ELB HAProxy