Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling infrastructure beyond containers
Search
Mateusz Gajewski
September 24, 2015
Technology
1
160
Scaling infrastructure beyond containers
This is the talk about using Apache Mesos to effectively manage resources in modern infrastructure.
Mateusz Gajewski
September 24, 2015
Tweet
Share
More Decks by Mateusz Gajewski
See All by Mateusz Gajewski
Modern java for decade-old problems
wendigo
0
150
3 kroki do tyłu, 2 lata w przód
wendigo
2
170
JDD - Dług techniczny
wendigo
2
1.2k
WJUG 2017: 4 lata rewolucji mikrousługowej w Allegro
wendigo
0
340
4 lata rewolucji mikrousługowej w Allegro
wendigo
2
800
Project Rubicon
wendigo
0
370
RxJava - Reactive extensions for the JVM
wendigo
1
310
Building distributed systems with OSS
wendigo
1
170
Microservices architecture pitfalls
wendigo
1
210
Other Decks in Technology
See All in Technology
自社製CMSからmicroCMSへのリプレースがプロダクトグロースを加速させた話
nextbeatdev
0
140
実践アプリケーション設計 ②トランザクションスクリプトへの対応
recruitengineers
PRO
3
170
コスト削減の基本の「キ」~ コスト消費3大リソースへの対策 ~
smt7174
2
130
kintone開発チームの紹介
cybozuinsideout
PRO
0
73k
アジャイルテストで高品質のスプリントレビューを
takesection
0
120
EKS Pod Identity における推移的な session tags
z63d
1
200
R-SCoRe: Revisiting Scene Coordinate Regression for Robust Large-Scale Visual Localization
takmin
0
430
あなたの知らない OneDrive
murachiakira
0
240
LLMエージェント時代に適応した開発フロー
hiragram
1
410
KiroでGameDay開催してみよう(準備編)
yuuuuuuu168
1
130
Webアクセシビリティ入門
recruitengineers
PRO
1
260
認知戦の理解と、市民としての対抗策
hogehuga
0
360
Featured
See All Featured
Thoughts on Productivity
jonyablonski
69
4.8k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
The Art of Programming - Codeland 2020
erikaheidi
55
13k
Unsuck your backbone
ammeep
671
58k
Become a Pro
speakerdeck
PRO
29
5.5k
A Modern Web Designer's Workflow
chriscoyier
695
190k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
31
2.2k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.6k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
[RailsConf 2023] Rails as a piece of cake
palkan
56
5.8k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Transcript
@wendigo Mateusz „Serafin” Gajewski • AWS UG Meetup Scaling infrastructure
beyond containers
@wendigo Agenda • Evolution of infrastructure at Allegro, • Why
Apache Mesos™? • Apache Mesos key concepts, • Future of datacenter and cloud computing?
@wendigo History of scaling infrastructure @ Allegro
@wendigo Infrastructure 1.0
@wendigo Job allocation problem
@wendigo Web Scale Resource management 100s dots 100s dots
@wendigo Infrastructure 2.0 1000s dots another 1000s dots
@wendigo Infrastructure 2.1 1000s dots another 1000s dots
@wendigo Challenges • cloud not used as cloud ;), •
high cost of virtualization, • effective resource utilization, • microservice architecture, • spread of new technologies, • heterogenous resources, • scalability, fault tolerance & HA, • performance isolation, • data processing at scale
@wendigo Beyond cloud computing
@wendigo Holly Grail of TCO
@wendigo Infrastructure 3.0 A Platform for Fine-Grained Resource Sharing in
the Data Center
@wendigo Scheduling
@wendigo Cluster scheduling
@wendigo Mesos architecture
@wendigo Mesos frameworks
@wendigo Offers
@wendigo Execution isolation
@wendigo External Containerizers
@wendigo Mesos HA • master election/failover with ZooKeeper, • master
maintains soft-state, • framework state reconciliation, • slave checkpointing, • slave recovery, • framework checkpointing
@wendigo Beyond offers • offer filters (constraints), • static (pre-startup)
reservations, • dynamic (post-startup) reservations, • oversubscription, • persistent volumes, • pluggable allocator scheduling policy (fair, priority based)
@wendigo Mesos frameworks
None
@wendigo Mesos recap • „programming against the datacenter", • distributed
datacenter kernel, • two-level multi-resource scheduler, • scalable, highly-available & fault-tolerant, • performance isolation with containers, • exposes homogeneous resources, • elastic, dynamic partitioning, • high resource utilization
@wendigo Future Datacenter
@wendigo IaaC
@wendigo Efficient utilization
@wendigo Google’s Omega source: Omega: flexible, scalable schedulers for large
compute clusters
None
Questions?
http://meetup.com/allegro.tech http://allegro.tech @AllegroTechBlog Work with us