Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling infrastructure beyond containers
Search
Mateusz Gajewski
September 24, 2015
Technology
1
160
Scaling infrastructure beyond containers
This is the talk about using Apache Mesos to effectively manage resources in modern infrastructure.
Mateusz Gajewski
September 24, 2015
Tweet
Share
More Decks by Mateusz Gajewski
See All by Mateusz Gajewski
Modern java for decade-old problems
wendigo
0
150
3 kroki do tyłu, 2 lata w przód
wendigo
2
170
JDD - Dług techniczny
wendigo
2
1.2k
WJUG 2017: 4 lata rewolucji mikrousługowej w Allegro
wendigo
0
340
4 lata rewolucji mikrousługowej w Allegro
wendigo
2
800
Project Rubicon
wendigo
0
360
RxJava - Reactive extensions for the JVM
wendigo
1
310
Building distributed systems with OSS
wendigo
1
170
Microservices architecture pitfalls
wendigo
1
210
Other Decks in Technology
See All in Technology
2025-07-25 NOT A HOTEL TECH TALK ━ スマートホーム開発の最前線 ━ SOFTWARE
wakinchan
0
110
増え続ける脆弱性に立ち向かう: 事前対策と優先度づけによる 持続可能な脆弱性管理 / Confronting the Rise of Vulnerabilities: Sustainable Management Through Proactive Measures and Prioritization
nttcom
1
150
Building GoReleaser - from shell script to paid product
caarlos0
0
270
経験がないことを言い訳にしない、 AI時代の他領域への染み出し方
parayama0625
0
130
DATA+AI SummitとSnowflake Summit: ユーザから見た共通点と相違点 / DATA+AI Summit and Snowflake Summit
nttcom
0
210
20150719_Amazon Nova Canvas Virtual try-onアプリ 作成裏話
riz3f7
0
130
AI Ready API ─ AI時代に求められるAPI設計とは?/ AI-Ready API - Designing MCP and APIs in the AI Era
yokawasa
20
5.8k
AI駆動開発 with MixLeap Study【大阪支部 #3】
lycorptech_jp
PRO
0
200
Expertise as a Service via MCP
yodakeisuke
1
140
株式会社島津製作所_研究開発(集団協業と知的生産)の現場を支える、OSS知識基盤システムの導入
akahane92
1
1.2k
Shadow DOMとセキュリティ - 光と影の境界を探る / Shibuya.XSS techtalk #13
masatokinugawa
0
270
Wasmで社内ツールを作って配布しよう
askua
0
120
Featured
See All Featured
Building a Modern Day E-commerce SEO Strategy
aleyda
42
7.4k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
Typedesign – Prime Four
hannesfritz
42
2.7k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Measuring & Analyzing Core Web Vitals
bluesmoon
7
530
Docker and Python
trallard
45
3.5k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.3k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.4k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Facilitating Awesome Meetings
lara
54
6.5k
Testing 201, or: Great Expectations
jmmastey
43
7.6k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Transcript
@wendigo Mateusz „Serafin” Gajewski • AWS UG Meetup Scaling infrastructure
beyond containers
@wendigo Agenda • Evolution of infrastructure at Allegro, • Why
Apache Mesos™? • Apache Mesos key concepts, • Future of datacenter and cloud computing?
@wendigo History of scaling infrastructure @ Allegro
@wendigo Infrastructure 1.0
@wendigo Job allocation problem
@wendigo Web Scale Resource management 100s dots 100s dots
@wendigo Infrastructure 2.0 1000s dots another 1000s dots
@wendigo Infrastructure 2.1 1000s dots another 1000s dots
@wendigo Challenges • cloud not used as cloud ;), •
high cost of virtualization, • effective resource utilization, • microservice architecture, • spread of new technologies, • heterogenous resources, • scalability, fault tolerance & HA, • performance isolation, • data processing at scale
@wendigo Beyond cloud computing
@wendigo Holly Grail of TCO
@wendigo Infrastructure 3.0 A Platform for Fine-Grained Resource Sharing in
the Data Center
@wendigo Scheduling
@wendigo Cluster scheduling
@wendigo Mesos architecture
@wendigo Mesos frameworks
@wendigo Offers
@wendigo Execution isolation
@wendigo External Containerizers
@wendigo Mesos HA • master election/failover with ZooKeeper, • master
maintains soft-state, • framework state reconciliation, • slave checkpointing, • slave recovery, • framework checkpointing
@wendigo Beyond offers • offer filters (constraints), • static (pre-startup)
reservations, • dynamic (post-startup) reservations, • oversubscription, • persistent volumes, • pluggable allocator scheduling policy (fair, priority based)
@wendigo Mesos frameworks
None
@wendigo Mesos recap • „programming against the datacenter", • distributed
datacenter kernel, • two-level multi-resource scheduler, • scalable, highly-available & fault-tolerant, • performance isolation with containers, • exposes homogeneous resources, • elastic, dynamic partitioning, • high resource utilization
@wendigo Future Datacenter
@wendigo IaaC
@wendigo Efficient utilization
@wendigo Google’s Omega source: Omega: flexible, scalable schedulers for large
compute clusters
None
Questions?
http://meetup.com/allegro.tech http://allegro.tech @AllegroTechBlog Work with us