Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Scaling infrastructure beyond containers
Mateusz Gajewski
September 24, 2015
Technology
1
81
Scaling infrastructure beyond containers
This is the talk about using Apache Mesos to effectively manage resources in modern infrastructure.
Mateusz Gajewski
September 24, 2015
Tweet
Share
More Decks by Mateusz Gajewski
See All by Mateusz Gajewski
3 kroki do tyłu, 2 lata w przód
wendigo
1
30
JDD - Dług techniczny
wendigo
2
740
WJUG 2017: 4 lata rewolucji mikrousługowej w Allegro
wendigo
0
240
4 lata rewolucji mikrousługowej w Allegro
wendigo
2
730
Project Rubicon
wendigo
0
320
RxJava - Reactive extensions for the JVM
wendigo
1
180
Building distributed systems with OSS
wendigo
1
98
Microservices architecture pitfalls
wendigo
1
160
RxJava - Introduction & Design
wendigo
2
130
Other Decks in Technology
See All in Technology
ソフトウェアテスト 2022 / Software Testing 2022
ak1210
1
1.6k
QiitaConference2022
fuwasegu
0
150
ラブグラフ紹介資料 〜プロダクト解体新書〜 / Lovegraph Product Deck
lovegraph
0
140
JDK Flight Recorder入門
chiroito
1
490
複数のスクラムチームをサポートするエンジニアリングマネジメントの話
okeicalm
0
1k
RDRA + JavaによるレジャーSaaSプロダクトの要件定義と実装のシームレスな接続
jjebejj
PRO
3
510
Istio入門
nutslove
15
4.9k
HoloLens2とMetaQuest2どちらも動くWebXRアプリをBabylon.jsで作る
iwaken71
0
180
ひとりでも安定して 組織を変える活動を続けていくための ストレスマネジメント
pastelinc
0
780
サイボウズの アジャイル・クオリティ / Agile Quality at Cybozu
cybozuinsideout
PRO
4
2k
組織の崩壊と再生、その中で何を考え、感じたのか。 そして本当に必要だったもの
kosako
8
3k
Retca Cloud
bau
0
160
Featured
See All Featured
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
19
1.4k
Typedesign – Prime Four
hannesfritz
33
1.3k
Fireside Chat
paigeccino
11
1.3k
Rebuilding a faster, lazier Slack
samanthasiow
62
7.2k
In The Pink: A Labor of Love
frogandcode
131
21k
Practical Orchestrator
shlominoach
178
8.6k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
269
11k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
349
27k
GitHub's CSS Performance
jonrohan
1020
420k
How GitHub Uses GitHub to Build GitHub
holman
465
280k
Put a Button on it: Removing Barriers to Going Fast.
kastner
56
2.3k
Build The Right Thing And Hit Your Dates
maggiecrowley
19
1.2k
Transcript
@wendigo Mateusz „Serafin” Gajewski • AWS UG Meetup Scaling infrastructure
beyond containers
@wendigo Agenda • Evolution of infrastructure at Allegro, • Why
Apache Mesos™? • Apache Mesos key concepts, • Future of datacenter and cloud computing?
@wendigo History of scaling infrastructure @ Allegro
@wendigo Infrastructure 1.0
@wendigo Job allocation problem
@wendigo Web Scale Resource management 100s dots 100s dots
@wendigo Infrastructure 2.0 1000s dots another 1000s dots
@wendigo Infrastructure 2.1 1000s dots another 1000s dots
@wendigo Challenges • cloud not used as cloud ;), •
high cost of virtualization, • effective resource utilization, • microservice architecture, • spread of new technologies, • heterogenous resources, • scalability, fault tolerance & HA, • performance isolation, • data processing at scale
@wendigo Beyond cloud computing
@wendigo Holly Grail of TCO
@wendigo Infrastructure 3.0 A Platform for Fine-Grained Resource Sharing in
the Data Center
@wendigo Scheduling
@wendigo Cluster scheduling
@wendigo Mesos architecture
@wendigo Mesos frameworks
@wendigo Offers
@wendigo Execution isolation
@wendigo External Containerizers
@wendigo Mesos HA • master election/failover with ZooKeeper, • master
maintains soft-state, • framework state reconciliation, • slave checkpointing, • slave recovery, • framework checkpointing
@wendigo Beyond offers • offer filters (constraints), • static (pre-startup)
reservations, • dynamic (post-startup) reservations, • oversubscription, • persistent volumes, • pluggable allocator scheduling policy (fair, priority based)
@wendigo Mesos frameworks
None
@wendigo Mesos recap • „programming against the datacenter", • distributed
datacenter kernel, • two-level multi-resource scheduler, • scalable, highly-available & fault-tolerant, • performance isolation with containers, • exposes homogeneous resources, • elastic, dynamic partitioning, • high resource utilization
@wendigo Future Datacenter
@wendigo IaaC
@wendigo Efficient utilization
@wendigo Google’s Omega source: Omega: flexible, scalable schedulers for large
compute clusters
None
Questions?
http://meetup.com/allegro.tech http://allegro.tech @AllegroTechBlog Work with us