Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling infrastructure beyond containers
Search
AWS User Group Poland
September 24, 2015
Technology
0
110
Scaling infrastructure beyond containers
Presentation from AWS User Group (2015.09.24) in Warsaw by Mateusz "Serafin" Gajewski from Allegro.
AWS User Group Poland
September 24, 2015
Tweet
Share
More Decks by AWS User Group Poland
See All by AWS User Group Poland
Analytics System based on AWS Redshift and Kinesis
awsugpl
0
66
Formowanie Chmur AWS w TomTom
awsugpl
0
37
Hajs się musi zgadzać, czyli o sprzedaży na Amazon AW$
awsugpl
0
50
How to squeeze AWS costs
awsugpl
0
54
M****services on AWS ECS
awsugpl
1
68
Amazon Aurora - Let's Talk About Performance
awsugpl
1
60
Amazon API Gateway and AWS Lambda: Better Together
awsugpl
1
89
Other Decks in Technology
See All in Technology
マルチアカウント環境への発見的統制の導入
ch1aki
1
1.3k
"好き"との生活/Regularly update profile with GitHub Actions
judeeeee
0
150
反実仮想機械学習とは何か
usaito
PRO
6
1.6k
Garoon 開発チーム / Garoon development team
cybozuinsideout
PRO
1
2.9k
Discord とビルダー&チャットボットの使い方 / How to use Discord and Builder & Chatbots
ks91
PRO
0
130
社内勉強会運営のコツ
senoo
6
1.1k
PHP"オレ"カンファレンスの告知
ysknsid25
0
320
SIEMを用いて、セキュリティログ分析の可視化と分析を実現し、PDCAサイクルを回してみた
coconala_engineer
0
200
A (short) History of AI
harishpillay
0
110
Autonomous Database Cloud 技術詳細 / adb-s_technical_detail_jp
oracle4engineer
PRO
13
35k
AWS パートナー企業でテクニカルサポートに従事して2年経ったので思うところをまとめてみた
kazzpapa3
3
1.3k
Databricks における 『MLOps』
databricksjapan
2
130
Featured
See All Featured
Into the Great Unknown - MozCon
thekraken
10
980
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
1
1.3k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
6
990
10 Git Anti Patterns You Should be Aware of
lemiorhan
645
57k
Navigating Team Friction
lara
177
13k
Music & Morning Musume
bryan
40
5.6k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
243
20k
The Pragmatic Product Professional
lauravandoore
24
5.8k
ParisWeb 2013: Learning to Love: Crash Course in Emotional UX Design
dotmariusz
103
6.6k
Bash Introduction
62gerente
604
210k
VelocityConf: Rendering Performance Case Studies
addyosmani
319
23k
Building a Scalable Design System with Sketch
lauravandoore
455
32k
Transcript
@wendigo Mateusz „Serafin” Gajewski • AWS UG Meetup Scaling infrastructure
beyond containers
@wendigo Agenda • Evolution of infrastructure at Allegro, • Why
Apache Mesos™? • Apache Mesos key concepts, • Future of datacenter and cloud computing?
@wendigo History of scaling infrastructure @ Allegro
@wendigo Infrastructure 1.0
@wendigo Job allocation problem
@wendigo Web Scale Resource management 100s dots 100s dots
@wendigo Infrastructure 2.0 1000s dots another 1000s dots
@wendigo Infrastructure 2.1 1000s dots another 1000s dots
@wendigo Challenges • cloud not used as cloud ;), •
high cost of virtualization, • effective resource utilization, • microservice architecture, • spread of new technologies, • heterogenous resources, • scalability, fault tolerance & HA, • performance isolation, • data processing at scale
@wendigo Beyond cloud computing
@wendigo Holly Grail of TCO
@wendigo Infrastructure 3.0 A Platform for Fine-Grained Resource Sharing in
the Data Center
@wendigo Scheduling
@wendigo Cluster scheduling
@wendigo Mesos architecture
@wendigo Mesos frameworks
@wendigo Offers
@wendigo Execution isolation
@wendigo External Containerizers
@wendigo Mesos HA • master election/failover with ZooKeeper, • master
maintains soft-state, • framework state reconciliation, • slave checkpointing, • slave recovery, • framework checkpointing
@wendigo Beyond offers • offer filters (constraints), • static (pre-startup)
reservations, • dynamic (post-startup) reservations, • oversubscription, • persistent volumes, • pluggable allocator scheduling policy (fair, priority based)
@wendigo Mesos frameworks
None
@wendigo Mesos recap • „programming against the datacenter", • distributed
datacenter kernel, • two-level multi-resource scheduler, • scalable, highly-available & fault-tolerant, • performance isolation with containers, • exposes homogeneous resources, • elastic, dynamic partitioning, • high resource utilization
@wendigo Future Datacenter
@wendigo IaaC
@wendigo Efficient utilization
@wendigo Google’s Omega source: Omega: flexible, scalable schedulers for large
compute clusters
None
Questions?
http://meetup.com/allegro.tech http://allegro.tech @AllegroTechBlog Work with us