Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Wykorzystanie klastra Apache Mesos w deploymenc...
Search
Kamil Warguła
November 29, 2015
Technology
0
340
Wykorzystanie klastra Apache Mesos w deploymencie aplikacji pythonowych
Kamil Warguła
November 29, 2015
Tweet
Share
Other Decks in Technology
See All in Technology
AWS DMS で SQL Server を移行してみた/aws-dms-sql-server-migration
emiki
0
250
20251027_マルチエージェントとは
almondo_event
1
460
Dify on AWS 環境構築手順
yosse95ai
0
150
20251027_findyさん_音声エージェントLT
almondo_event
2
470
Open Table Format (OTF) が必要になった背景とその機能 (2025.10.28)
simosako
2
370
入院医療費算定業務をAIで支援する:包括医療費支払い制度とDPCコーディング (公開版)
hagino3000
0
110
オブザーバビリティが育むシステム理解と好奇心
maruloop
3
1.4k
IBC 2025 動画技術関連レポート / IBC 2025 Report
cyberagentdevelopers
PRO
2
210
「タコピーの原罪」から学ぶ間違った”支援” / the bad support of Takopii
piyonakajima
0
150
生成AI時代のPythonセキュリティとガバナンス
abenben
0
140
組織全員で向き合うAI Readyなデータ利活用
gappy50
3
1.2k
CLIPでマルチモーダル画像検索 →とても良い
wm3
0
190
Featured
See All Featured
Statistics for Hackers
jakevdp
799
220k
Reflections from 52 weeks, 52 projects
jeffersonlam
354
21k
The Cost Of JavaScript in 2023
addyosmani
55
9.1k
Code Review Best Practice
trishagee
72
19k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.7k
Stop Working from a Prison Cell
hatefulcrawdad
272
21k
Building a Modern Day E-commerce SEO Strategy
aleyda
44
7.8k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.7k
How to train your dragon (web standard)
notwaldorf
97
6.3k
Rails Girls Zürich Keynote
gr2m
95
14k
4 Signs Your Business is Dying
shpigford
185
22k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3.1k
Transcript
Wykorzystanie klastra Apache Mesos w deploymencie aplikacji pythonowych Kamil Warguła
1 github.com/quamilek
[email protected]
2
Apache Mesos 3
What is Mesos? Apache Mesos abstracts CPU, memory, storage, and
other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively. 4
Mesos architecture Mesos master Standby master Standby master ZooKeeper quorum
Mesos slave Framework executor task Mesos slave Framework executor task Mesos slave Marathon executor task Hadoop executor task Framework scheduler 5
Resource allocation 6
Mesos UI 7
Mesos frameworks 8
Marathon 9
Marathon Marathon is an Apache Mesos framework for long-running applications
10
Marathon features • Web UI • JSON/REST API • HA
• Constraints • Health Checks • Event Subscription • Basic Auth and SSL 11
Resource utilization 12
Resource utilization 13
Mesos saves $$$ 14 old way mesos way
simple python REST app import falcon import json class ExampleResource:
def on_get(self, req, resp): data = {'foo': 'bar'} resp.body = json.dumps(data) api = falcon.API() api.add_route('/', ExampleResource()) 15
integration with gunicorn from gunicorn.app.base import BaseApplication class StandaloneApplication(BaseApplication): ...
def _get_config(): ... def main(): api = falcon.API() api.add_route('/', Example Resource()) config = _get_config() StandaloneApplication(api, config).run() if __name__ == '__main__': main() 16
How to run virtualenv env pip install falcon==0.3.0 gunicorn==19.3.0 source
env/bin/activate python api.py -p 8099 17
Pack requirements with PEX pex -v -r falcon==0.3.0 -r gunicorn==19.3.0
-o app.pex 18
How to run app with pex ./app.pex api.py -p 8099
19
DEMO 20
Marathon UI 21
Marathon app definition { "id": "appname", "cmd": "python my_app.py", "cpus":
0.5, "instances": 1, "mem": 64, "uris": [ "http://artifacts.local/my_app_0.1.0.tar.gz" ] } 22
Marathon - deploy python app DEMO 23
app configuration • environment variables • distributed configuration service (ZooKeeper,
ETCD) 24
Service Discovery • Marathon-Consul • Mesos-DNS 25
Load Balancing • haproxy-marathon-bridge 26
Scale app - manual import json import requests url =
'http://marathon.local/v2/apps/myapp' data = {'instances': 5} requests.put(url, data=json.dumps(data)) 27
Scale app - manual DEMO 28
Autoscale app MIN_INSTANCES_COUNT = 2 MAX_INSTANCES_COUNT = 20 if overloaded_instances
>= 0.9 * total_instances: total_instances += 2 if total_instances <= MAX_INSTANCES_COUNT: scale_app(total_instances) elif overloaded_instances <= 0.5 * total_instances: total_instances -= 2 if total_instances >= MIN_INSTANCES_COUNT: scale_app(total_instances) 29
Autoscale app DEMO 30
Thank you! 31 goo.gl/qWivrx
Q/A? 32 goo.gl/qWivrx