Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Microservices on AWS ECS

5ff4f1dbfd3888fba963ec7cc629e6f3?s=47 mthenw
September 24, 2015

Microservices on AWS ECS

AWS User Group Poland
Video (in PL): https://www.youtube.com/watch?v=OBjeHrebaWY

5ff4f1dbfd3888fba963ec7cc629e6f3?s=128

mthenw

September 24, 2015
Tweet

More Decks by mthenw

Other Decks in Technology

Transcript

  1. SAY MICROSERVICES ONE MORE GODDAMN TIME

  2. ON M****SERVICES AWS ECS

  3. Maciej Winnicki mthenw

  4. What is AWS EC2 Container Service? How does it work?

    Service discovery Configuration Deployment Scaling What is missing?
  5. What is AWS EC2 Container Service?

  6. … is a highly scalable, fast, container management service that

    makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html “
  7. Amazon Docker As A Service https://www.expeditedssl.com/aws-in-plain-english

  8. How does it work?

  9. AZ AZ EC2 EC2 EC2 EC2

  10. Cluster EC2 EC2 EC2 EC2

  11. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent
  12. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Scheduler
  13. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Scheduler Task
  14. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Scheduler Task Task
  15. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Scheduler Task Task Task
  16. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Scheduler Task Task Task
  17. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Scheduler Task Task Task Docker container Docker container
  18. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Scheduler Task Task Task Docker container Docker container Task Definition {“containerDefinitions": [ { "name": "web", "image": "nginx", "cpu": 99, "memory": 100, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] } ]}
  19. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Scheduler Task
  20. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Scheduler Task Service 1 Task Service 1 Task Service 1
  21. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Scheduler Task Service 1 Task Service 1 Task Service 1 ELB
  22. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Scheduler Task Service 1 Task Service 1 Task Service 1 ELB :80 :8001 :8001 :8001
  23. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent AWS ECS Task Service 1 Task Service 1 Task Service 1 ELB Key/Value Store Cluster Management Engine Agent Communication Service API http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html
  24. Microservices

  25. ECS Cluster AZ EC2 ECS Agent EC2 ECS Agent AZ

    EC2 ECS Agent EC2 ECS Agent
  26. None
  27. ECS Cluster (c4.xlarge) AZ EC2 ECS Agent EC2 ECS Agent

    AZ EC2 ECS Agent EC2 ECS Agent Stateful cluster (non-ECS, m4.xlarge) EC2 w/ EBS EC2 w/ EBS EC2 w/ EBS MongoDB MongoDB ElasticSearch ElasticSearch
  28. Service discovery Configuration https://www.consul.io

  29. Service discovery via DNS or HTTP Key/Value Storage Failure Detection

    https://www.consul.io
  30. Service discovery Configuration https://www.consul.io

  31. ECS Cluster AZ EC2 ECS Agent Consul EC2 ECS Agent

    Consul AZ EC2 ECS Agent Consul EC2 ECS Agent Consul Stateful cluster EC2 w/ EBS EC2 w/ EBS EC2 w/ EBS MongoDB MongoDB ElasticSearch ElasticSearch Consul Consul Consul
  32. ECS Cluster AZ EC2 ECS Agent Consul (Server) EC2 ECS

    Agent Consul (Server) AZ EC2 ECS Agent Consul (Server) EC2 ECS Agent Consul Stateful cluster EC2 w/ EBS EC2 w/ EBS EC2 w/ EBS MongoDB MongoDB ElasticSearch ElasticSearch Consul Consul Consul Docker container Docker container Consul DNS
  33. Service discovery via DNS Consul as a default DNS for

    every container
  34. ECS Cluster AZ EC2 ECS Agent Consul (Server) EC2 ECS

    Agent Consul (Server) AZ EC2 ECS Agent Consul (Server) EC2 ECS Agent Consul Stateful cluster EC2 w/ EBS EC2 w/ EBS EC2 w/ EBS MongoDB MongoDB ElasticSearch ElasticSearch Consul Consul Consul Registrator New task Consul Docker (socket) Register new task/service
  35. Registrator + Consul = 20MB RAM

  36. Service discovery Configuration https://www.consul.io

  37. JSON file for local development Consul Key/Value for any other

    environment Dynamic feature flags stored in Consul K/V
  38. Deployment

  39. build image commit click! run ecs deploy deploy ECS

  40. Small docker images! (e.g. mhart/alpine-node: 35MB) ECS service update strategy

    (add before remove)
  41. Scaling

  42. None
  43. Cluster EC2 EC2 EC2 EC2 ECS Agent ECS Agent ECS

    Agent ECS Agent Task (Service 1) Task (Service 1) Task (Service 1) ELB HAProxy
  44. None
  45. What is still missing?

  46. Exactly one task per instance Dynamic port allocation on ELB

    Stateful services
  47. None