Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling infrastructure beyond containers
Search
Mateusz Gajewski
September 24, 2015
Technology
1
120
Scaling infrastructure beyond containers
This is the talk about using Apache Mesos to effectively manage resources in modern infrastructure.
Mateusz Gajewski
September 24, 2015
Tweet
Share
More Decks by Mateusz Gajewski
See All by Mateusz Gajewski
Modern java for decade-old problems
wendigo
0
88
3 kroki do tyłu, 2 lata w przód
wendigo
2
160
JDD - Dług techniczny
wendigo
2
1k
WJUG 2017: 4 lata rewolucji mikrousługowej w Allegro
wendigo
0
310
4 lata rewolucji mikrousługowej w Allegro
wendigo
2
760
Project Rubicon
wendigo
0
340
RxJava - Reactive extensions for the JVM
wendigo
1
240
Building distributed systems with OSS
wendigo
1
140
Microservices architecture pitfalls
wendigo
1
190
Other Decks in Technology
See All in Technology
プロンプトエンジニアリングでがんばらない-Agentic Workflow へ-近藤憲児
kenjikondobai
6
1.2k
ゼロから始めるVue.jsコミュニティ貢献 / first-vuejs-community-contribution-link-and-motivation
lmi
1
150
Android Target SDK 35 (Android 15) 対応の概要
akkie76
0
160
リテール金融(キャッシュレス・ネット銀行・ネット証券)の競争環境と経済圏
8maki
0
1.6k
本当のAWS基礎
toru_kubota
1
640
今日からできる!簡単 .NET 高速化 Tips -2024 edition-
xin9le
7
3.9k
よく聞くけど使ったことないソフトウェアNo.1 KafkaとSnowflake
foursue
4
510
R3のコードから見る実践LINQ実装最適化・コンカレントプログラミング実例
neuecc
3
2.6k
アクセス制御にまつわる改善 / Improving access control
itkq
0
590
M&A戦略を支えるデータマネジメント (MIDAS Tech Study #16 GENDA Komiyama)
kommy339
1
110
LangSmith入門―トレース/評価/プロンプト管理などを担うLLMアプリ開発プラットフォーム
os1ma
5
710
コードファーストの考え方。 Amplify Gen2から学ぶAWS次世代のWeb開発体験
yoshiitaka
2
370
Featured
See All Featured
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
21
1.4k
What's in a price? How to price your products and services
michaelherold
238
11k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
41
4.4k
10 Git Anti Patterns You Should be Aware of
lemiorhan
649
58k
Put a Button on it: Removing Barriers to Going Fast.
kastner
58
3.1k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
14
1.5k
Bash Introduction
62gerente
605
210k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
188
16k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
245
20k
A Philosophy of Restraint
colly
197
16k
Building Your Own Lightsaber
phodgson
100
5.7k
Web Components: a chance to create the future
zenorocha
306
41k
Transcript
@wendigo Mateusz „Serafin” Gajewski • AWS UG Meetup Scaling infrastructure
beyond containers
@wendigo Agenda • Evolution of infrastructure at Allegro, • Why
Apache Mesos™? • Apache Mesos key concepts, • Future of datacenter and cloud computing?
@wendigo History of scaling infrastructure @ Allegro
@wendigo Infrastructure 1.0
@wendigo Job allocation problem
@wendigo Web Scale Resource management 100s dots 100s dots
@wendigo Infrastructure 2.0 1000s dots another 1000s dots
@wendigo Infrastructure 2.1 1000s dots another 1000s dots
@wendigo Challenges • cloud not used as cloud ;), •
high cost of virtualization, • effective resource utilization, • microservice architecture, • spread of new technologies, • heterogenous resources, • scalability, fault tolerance & HA, • performance isolation, • data processing at scale
@wendigo Beyond cloud computing
@wendigo Holly Grail of TCO
@wendigo Infrastructure 3.0 A Platform for Fine-Grained Resource Sharing in
the Data Center
@wendigo Scheduling
@wendigo Cluster scheduling
@wendigo Mesos architecture
@wendigo Mesos frameworks
@wendigo Offers
@wendigo Execution isolation
@wendigo External Containerizers
@wendigo Mesos HA • master election/failover with ZooKeeper, • master
maintains soft-state, • framework state reconciliation, • slave checkpointing, • slave recovery, • framework checkpointing
@wendigo Beyond offers • offer filters (constraints), • static (pre-startup)
reservations, • dynamic (post-startup) reservations, • oversubscription, • persistent volumes, • pluggable allocator scheduling policy (fair, priority based)
@wendigo Mesos frameworks
None
@wendigo Mesos recap • „programming against the datacenter", • distributed
datacenter kernel, • two-level multi-resource scheduler, • scalable, highly-available & fault-tolerant, • performance isolation with containers, • exposes homogeneous resources, • elastic, dynamic partitioning, • high resource utilization
@wendigo Future Datacenter
@wendigo IaaC
@wendigo Efficient utilization
@wendigo Google’s Omega source: Omega: flexible, scalable schedulers for large
compute clusters
None
Questions?
http://meetup.com/allegro.tech http://allegro.tech @AllegroTechBlog Work with us