Microservices on AWS ECS

AWS User Group Poland
Video (in PL): https://www.youtube.com/watch?v=OBjeHrebaWY

Transcript

1. SAY MICROSERVICES ONE MORE GODDAMN TIME
2. ON M****SERVICES AWS ECS
3. Maciej Winnicki mthenw
4. What is AWS EC2 Container Service? How does it work? Service discovery Configuration Deployment Scaling What is missing?
5. What is AWS EC2 Container Service?
6. … is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html “
7. Amazon Docker As A Service https://www.expeditedssl.com/aws-in-plain-english
8. How does it work?
9. AZ AZ EC2 EC2 EC2 EC2
10. Cluster EC2 EC2 EC2 EC2
11. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent
12. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler
13. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task
14. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Task
15. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Task Task
16. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Task Task
17. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Task Task Docker container Docker container
18. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Task Task Docker container Docker container Task Definition {“containerDefinitions": [ { "name": "web", "image": "nginx", "cpu": 99, "memory": 100, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] } ]}
19. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task
20. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Service 1 Task Service 1 Task Service 1
21. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Service 1 Task Service 1 Task Service 1 ELB
22. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Scheduler Task Service 1 Task Service 1 Task Service 1 ELB :80 :8001 :8001 :8001
23. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent AWS ECS Task Service 1 Task Service 1 Task Service 1 ELB Key/Value Store Cluster Management Engine Agent Communication Service API http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html
24. Microservices
25. ECS Cluster AZ EC2 ECS Agent EC2 ECS Agent AZ EC2 ECS Agent EC2 ECS Agent
27. ECS Cluster (c4.xlarge) AZ EC2 ECS Agent EC2 ECS Agent AZ EC2 ECS Agent EC2 ECS Agent Stateful cluster (non-ECS, m4.xlarge) EC2 w/ EBS EC2 w/ EBS EC2 w/ EBS MongoDB MongoDB ElasticSearch ElasticSearch
28. Service discovery Configuration https://www.consul.io
29. Service discovery via DNS or HTTP Key/Value Storage Failure Detection https://www.consul.io
30. Service discovery Configuration https://www.consul.io
31. ECS Cluster AZ EC2 ECS Agent Consul EC2 ECS Agent Consul AZ EC2 ECS Agent Consul EC2 ECS Agent Consul Stateful cluster EC2 w/ EBS EC2 w/ EBS EC2 w/ EBS MongoDB MongoDB ElasticSearch ElasticSearch Consul Consul Consul
32. ECS Cluster AZ EC2 ECS Agent Consul (Server) EC2 ECS Agent Consul (Server) AZ EC2 ECS Agent Consul (Server) EC2 ECS Agent Consul Stateful cluster EC2 w/ EBS EC2 w/ EBS EC2 w/ EBS MongoDB MongoDB ElasticSearch ElasticSearch Consul Consul Consul Docker container Docker container Consul DNS
33. Service discovery via DNS Consul as a default DNS for every container
34. ECS Cluster AZ EC2 ECS Agent Consul (Server) EC2 ECS Agent Consul (Server) AZ EC2 ECS Agent Consul (Server) EC2 ECS Agent Consul Stateful cluster EC2 w/ EBS EC2 w/ EBS EC2 w/ EBS MongoDB MongoDB ElasticSearch ElasticSearch Consul Consul Consul Registrator New task Consul Docker (socket) Register new task/service
35. Registrator + Consul = 20MB RAM
36. Service discovery Configuration https://www.consul.io
37. JSON file for local development Consul Key/Value for any other environment Dynamic feature flags stored in Consul K/V
38. Deployment
39. build image commit click! run ecs deploy deploy ECS
40. Small docker images! (e.g. mhart/alpine-node: 35MB) ECS service update strategy (add before remove)
41. Scaling
43. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS Agent ECS Agent Task (Service 1) Task (Service 1) Task (Service 1) ELB HAProxy
45. What is still missing?
46. Exactly one task per instance Dynamic port allocation on ELB Stateful services