Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Operating ECS in production
Search
Michael Wittig
April 12, 2018
Technology
0
61
Operating ECS in production
https://github.com/widdix/aws-cf-templates
Michael Wittig
April 12, 2018
Tweet
Share
More Decks by Michael Wittig
See All by Michael Wittig
AWS Cost Optimization 101
michaelwittig
0
61
DVC02 - Cutting-edge architectures based on AWS AppSync, Lambda, and Fargate
michaelwittig
0
83
Review: AWS Global Accelerator - AWS Meetup Stuttgart 2019
michaelwittig
0
330
Rapid Docker on AWS
michaelwittig
0
73
Serverless Workflows with AWS Step Functions
michaelwittig
0
41
Your Lambda function might execute twice. Be prepared! (ServerlessDays Zurich)
michaelwittig
0
52
Cutting-Edge Architectures Based on AppSync, Lambda, and Fargate
michaelwittig
1
490
Your Lambda function might execute twice. Be prepared!
michaelwittig
0
670
Network Security on AWS
michaelwittig
0
49
Other Decks in Technology
See All in Technology
モノレポ開発のエラー、誰が見る?Datadog で実現する適切なトリアージとエスカレーション
biwashi
6
770
エンジニアのためのドキュメント力基礎講座〜構造化思考から始めよう〜(2025/02/15jbug広島#15発表資料)
yasuoyasuo
15
5.5k
依存関係があるコンポーネントは Barrel ファイルでまとめよう
azukiazusa1
3
530
CZII - CryoET Object Identification 参加振り返り・解法共有
tattaka
0
240
Ask! NIKKEIの運用基盤と改善に向けた取り組み / NIKKEI TECH TALK #30
kaitomajima
1
450
第13回 Data-Centric AI勉強会, 画像認識におけるData-centric AI
ksaito_osx
0
360
開発スピードは上がっている…品質はどうする? スピードと品質を両立させるためのプロダクト開発の進め方とは #DevSumi #DevSumiB / Agile And Quality
nihonbuson
1
1.3k
開発者が自律的に AWS Security Hub findings に 対応する仕組みと AWS re:Invent 2024 登壇体験談 / Developers autonomously report AWS Security Hub findings Corresponding mechanism and AWS re:Invent 2024 presentation experience
kaminashi
0
190
自動テストの世界に、この5年間で起きたこと
autifyhq
10
7.1k
技術負債の「予兆検知」と「状況異変」のススメ / Technology Dept
i35_267
1
1k
飲食店予約台帳を支えるインタラクティブ UI 設計と実装
siropaca
6
1.4k
リーダブルテストコード 〜メンテナンスしやすい テストコードを作成する方法を考える〜 #DevSumi #DevSumiB / Readable test code
nihonbuson
11
5.8k
Featured
See All Featured
Typedesign – Prime Four
hannesfritz
40
2.5k
KATA
mclloyd
29
14k
Navigating Team Friction
lara
183
15k
Embracing the Ebb and Flow
colly
84
4.6k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
232
17k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
20
2.4k
Being A Developer After 40
akosma
89
590k
Reflections from 52 weeks, 52 projects
jeffersonlam
348
20k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
44
9.4k
Practical Orchestrator
shlominoach
186
10k
Fireside Chat
paigeccino
34
3.2k
Transcript
https://github.com/widdix/aws-cf-templates Operating ECS in production
Hello! I am Michael Wittig AWS in Action (2nd ed)
cloudonaut.io AWS Community Hero Independent AWS Consultant Twitter @hellomichibye 2
ECS Orchestrates Docker containers for you Manages Network and per
Task Security 3
ECS Cluster 4 ECS Cluster ECS Instance = EC2 Instance
running ecs-agent ECS optimized AMI ECS Instance ECS Instance
Task Definition 5 ECS Cluster aws ecs run-task \ --count
2 Task Definition Image ... ECS Task 1..N Container ECS Task 1..N Container
ECS Scheduling 6 ECS Cluster ECS Instance ECS Instance ECS
Instance ECS Task 1..N Container ECS Task 1..N Container Placement constraints and strategies
ECS Networking ◎ Public/Private Load Balancer ◎ Elastic Network Interface
(ENI) per task ◦ Public IP ◦ Private IP ◦ Per Task Security Group 7
ECS Service 8 ECS Cluster Task Definition ECS Task ECS
Task ECS Service ◎ Observer ◎ ENI ◎ Load Balancer ◎ Deployment
Operating ECS Challenges 9
1. Spinning up a cluster 10 Demo
Fault Tolerant Auto Scaling Group, Availability Zones 11 Demo
2. Updating a cluster New ECS optimized AMIs are released
frequently! 12
Rolling Update CloudFormation replaces EC2 instances in Auto Scaling Groups
in small batches. 13 Demo
But what about inflight requests? 14
Instance Draining Move all tasks from ECS instance before Instance
is terminated. 15 Demo
Implementing Instance Draining ◎ Auto Scaling Lifecycle hook ◦ Drain
Instance ◦ Wait until drained ◦ Complete Lifecycle hook 16
Tasks are not rescheduled once places! Your last batch of
ECS instances will end up with 0 tasks! 17
3. Scaling a cluster Or adding/removing EC2 instances. 18 Demo
We don’t know how many tasks we can schedule! 19
20 ECS Cluster Available: CPU 100 Memory 200 Available: CPU
100 Memory 200 Available: CPU 100 Memory 200 Available: CPU 300 Memory 600 Task CPU 200 Memory 200
Schedulable Containers 1. Define largest possible task (CPU/memory) 2. For
each instance: a. Calculate how many largest possible tasks would fit b. Report to CloudWatch 3. Scale based on the sum of this metric 21 Credits http://garbe.io/blog/2017/04/12/a-better-solution-to-ecs-autoscaling/
22 ECS Cluster Available: CPU 100 Memory 200 Schedulable 0
Available: CPU 100 Memory 200 Schedulable 0 Available: CPU 100 Memory 200 Schedulable 0 Available: CPU 300 Memory 600 Schedulable 0 Largest Task CPU 200 Memory 200
No CloudWatch Events emitted when task launch failed due to
capacity shortage. 23
4. Public load balancing 24
Public load balancing 25 ECS Cluster ECS Instance ECS Instance
ECS Task 1..N Container ECS Task 1..N Container ECS Task 1..N Container Load Balancer (ALB) ◎ Path based ◎ Host based DNS
5. Internal service discovery / load balancing 26
Internal load balancing 27 ECS Task Frontend LB (internet-facing) ECS
Task ECS Task Catalog (internal) ECS Task ECS Task Shopping card (internal) ECS Task ECS Task DNS DNS DNS
Catalog Catalog Internal Route 53 (with per task ENI) 28
ECS Task Frontend LB (internet-facing) ECS Task ECS Task ECS Task ECS Task ECS Task ECS Task DNS DNS DNS
6. Logging & Monitoring CloudWatch. 29
Credits Special thanks to all the people who made and
released these awesome resources for free: ◎ Presentation template by SlidesCarnival ◎ Photographs by Pexels 30
Thanks! http://bit.ly/amazon-web-services-in-action-2nd-edition https://github.com/widdix/aws-cf-templates https://cloudonaut.io Twitter @hellomichibye Mail
[email protected]
31