Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Wykorzystanie klastra Apache Mesos w deploymencie aplikacji pythonowych
Search
Kamil Warguła
November 29, 2015
Technology
0
290
Wykorzystanie klastra Apache Mesos w deploymencie aplikacji pythonowych
Kamil Warguła
November 29, 2015
Tweet
Share
Other Decks in Technology
See All in Technology
AOAI Dev Day LLMシステム開発 Tips集
hirosatogamo
15
3.6k
[NIKKEI Tech Talk] KDDI/KAG Scrum & Community for Engineering Training
curanosuke
2
220
20240725 LLMによるDXのビジョンと、今何からやるべきか @Azure OpenAI Service Dev Day
nrryuya
3
1.1k
AWSサービスメニュー開発をしていてAWSを好きだ!と感じた瞬間
toru_kubota
0
130
「単なる OAuth 2.0 を認証に使うと、車が通れるほどのどでかいセキュリティー・ホールができる」のか検証してみた
terara
0
380
ここがすごいよ! AWS Systems Manager!
saichan11
0
1.8k
20240717_イケコパ代表Copilot_in_Teams会社でこう使ってます
ponponmikankan
2
430
【基調講演】変える、今ここから ― IoTとAIで紡ぐ未来
soracom
PRO
0
310
成長期に歩みを止めないための創業期の開発文化形成
mayah
6
420
[2024最新版]AWS Control Towerを使ったセキュアなマルチアカウント環境の作り方
hiashisan
0
270
データ分析基盤を作ってみよう~設計編~
nrinetcom
PRO
1
110
OSSコミットしてZennの課題を解決した話
dyoshikawa1993
0
150
Featured
See All Featured
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
29
2.5k
Practical Orchestrator
shlominoach
185
10k
The Power of CSS Pseudo Elements
geoffreycrofte
64
5.2k
How STYLIGHT went responsive
nonsquared
93
5k
Stop Working from a Prison Cell
hatefulcrawdad
266
20k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
360
22k
The Invisible Customer
myddelton
117
13k
Principles of Awesome APIs and How to Build Them.
keavy
124
16k
A Philosophy of Restraint
colly
200
16k
[RailsConf 2023] Rails as a piece of cake
palkan
35
4.4k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
24
1.8k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
189
16k
Transcript
Wykorzystanie klastra Apache Mesos w deploymencie aplikacji pythonowych Kamil Warguła
1 github.com/quamilek
[email protected]
2
Apache Mesos 3
What is Mesos? Apache Mesos abstracts CPU, memory, storage, and
other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively. 4
Mesos architecture Mesos master Standby master Standby master ZooKeeper quorum
Mesos slave Framework executor task Mesos slave Framework executor task Mesos slave Marathon executor task Hadoop executor task Framework scheduler 5
Resource allocation 6
Mesos UI 7
Mesos frameworks 8
Marathon 9
Marathon Marathon is an Apache Mesos framework for long-running applications
10
Marathon features • Web UI • JSON/REST API • HA
• Constraints • Health Checks • Event Subscription • Basic Auth and SSL 11
Resource utilization 12
Resource utilization 13
Mesos saves $$$ 14 old way mesos way
simple python REST app import falcon import json class ExampleResource:
def on_get(self, req, resp): data = {'foo': 'bar'} resp.body = json.dumps(data) api = falcon.API() api.add_route('/', ExampleResource()) 15
integration with gunicorn from gunicorn.app.base import BaseApplication class StandaloneApplication(BaseApplication): ...
def _get_config(): ... def main(): api = falcon.API() api.add_route('/', Example Resource()) config = _get_config() StandaloneApplication(api, config).run() if __name__ == '__main__': main() 16
How to run virtualenv env pip install falcon==0.3.0 gunicorn==19.3.0 source
env/bin/activate python api.py -p 8099 17
Pack requirements with PEX pex -v -r falcon==0.3.0 -r gunicorn==19.3.0
-o app.pex 18
How to run app with pex ./app.pex api.py -p 8099
19
DEMO 20
Marathon UI 21
Marathon app definition { "id": "appname", "cmd": "python my_app.py", "cpus":
0.5, "instances": 1, "mem": 64, "uris": [ "http://artifacts.local/my_app_0.1.0.tar.gz" ] } 22
Marathon - deploy python app DEMO 23
app configuration • environment variables • distributed configuration service (ZooKeeper,
ETCD) 24
Service Discovery • Marathon-Consul • Mesos-DNS 25
Load Balancing • haproxy-marathon-bridge 26
Scale app - manual import json import requests url =
'http://marathon.local/v2/apps/myapp' data = {'instances': 5} requests.put(url, data=json.dumps(data)) 27
Scale app - manual DEMO 28
Autoscale app MIN_INSTANCES_COUNT = 2 MAX_INSTANCES_COUNT = 20 if overloaded_instances
>= 0.9 * total_instances: total_instances += 2 if total_instances <= MAX_INSTANCES_COUNT: scale_app(total_instances) elif overloaded_instances <= 0.5 * total_instances: total_instances -= 2 if total_instances >= MIN_INSTANCES_COUNT: scale_app(total_instances) 29
Autoscale app DEMO 30
Thank you! 31 goo.gl/qWivrx
Q/A? 32 goo.gl/qWivrx