Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling infrastructure beyond containers
Search
Mateusz Gajewski
September 24, 2015
Technology
1
170
Scaling infrastructure beyond containers
This is the talk about using Apache Mesos to effectively manage resources in modern infrastructure.
Mateusz Gajewski
September 24, 2015
Tweet
Share
More Decks by Mateusz Gajewski
See All by Mateusz Gajewski
Modern java for decade-old problems
wendigo
0
150
3 kroki do tyłu, 2 lata w przód
wendigo
2
170
JDD - Dług techniczny
wendigo
2
1.2k
WJUG 2017: 4 lata rewolucji mikrousługowej w Allegro
wendigo
0
340
4 lata rewolucji mikrousługowej w Allegro
wendigo
2
800
Project Rubicon
wendigo
0
370
RxJava - Reactive extensions for the JVM
wendigo
1
320
Building distributed systems with OSS
wendigo
1
170
Microservices architecture pitfalls
wendigo
1
220
Other Decks in Technology
See All in Technology
速習AGENTS.md:5分で精度を上げる "3ブロック" テンプレ
ismk
6
1.1k
LLM時代にデータエンジニアの役割はどう変わるか?
ikkimiyazaki
6
1.3k
ComposeではないコードをCompose化する case ビズリーチ / DroidKaigi 2025 koyasai
visional_engineering_and_design
0
110
やる気のない自分との向き合い方/How to Deal with Your Unmotivated Self
sanogemaru
0
500
20201008_ファインディ_品質意識を育てる役目は人かAIか___2_.pdf
findy_eventslides
2
620
リセラー企業のテクサポ担当が考える、生成 AI 時代のトラブルシュート 2025
kazzpapa3
1
160
Developer Advocate / Community Managerなるには?
tsho
0
140
社内お問い合わせBotの仕組みと学び
nish01
1
590
20251007: What happens when multi-agent systems become larger? (CyberAgent, Inc)
ornew
1
250
Git in Team
kawaguti
PRO
3
360
Geospatialの世界最前線を探る [2025年版]
dayjournal
1
220
能登半島地震で見えた災害対応の課題と組織変革の重要性
ditccsugii
0
690
Featured
See All Featured
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
20
1.2k
A better future with KSS
kneath
239
18k
Into the Great Unknown - MozCon
thekraken
40
2.1k
Building Flexible Design Systems
yeseniaperezcruz
329
39k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.2k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.7k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
115
20k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
620
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
359
30k
Art, The Web, and Tiny UX
lynnandtonic
303
21k
Transcript
@wendigo Mateusz „Serafin” Gajewski • AWS UG Meetup Scaling infrastructure
beyond containers
@wendigo Agenda • Evolution of infrastructure at Allegro, • Why
Apache Mesos™? • Apache Mesos key concepts, • Future of datacenter and cloud computing?
@wendigo History of scaling infrastructure @ Allegro
@wendigo Infrastructure 1.0
@wendigo Job allocation problem
@wendigo Web Scale Resource management 100s dots 100s dots
@wendigo Infrastructure 2.0 1000s dots another 1000s dots
@wendigo Infrastructure 2.1 1000s dots another 1000s dots
@wendigo Challenges • cloud not used as cloud ;), •
high cost of virtualization, • effective resource utilization, • microservice architecture, • spread of new technologies, • heterogenous resources, • scalability, fault tolerance & HA, • performance isolation, • data processing at scale
@wendigo Beyond cloud computing
@wendigo Holly Grail of TCO
@wendigo Infrastructure 3.0 A Platform for Fine-Grained Resource Sharing in
the Data Center
@wendigo Scheduling
@wendigo Cluster scheduling
@wendigo Mesos architecture
@wendigo Mesos frameworks
@wendigo Offers
@wendigo Execution isolation
@wendigo External Containerizers
@wendigo Mesos HA • master election/failover with ZooKeeper, • master
maintains soft-state, • framework state reconciliation, • slave checkpointing, • slave recovery, • framework checkpointing
@wendigo Beyond offers • offer filters (constraints), • static (pre-startup)
reservations, • dynamic (post-startup) reservations, • oversubscription, • persistent volumes, • pluggable allocator scheduling policy (fair, priority based)
@wendigo Mesos frameworks
None
@wendigo Mesos recap • „programming against the datacenter", • distributed
datacenter kernel, • two-level multi-resource scheduler, • scalable, highly-available & fault-tolerant, • performance isolation with containers, • exposes homogeneous resources, • elastic, dynamic partitioning, • high resource utilization
@wendigo Future Datacenter
@wendigo IaaC
@wendigo Efficient utilization
@wendigo Google’s Omega source: Omega: flexible, scalable schedulers for large
compute clusters
None
Questions?
http://meetup.com/allegro.tech http://allegro.tech @AllegroTechBlog Work with us