Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Wykorzystanie klastra Apache Mesos w deploymenc...
Search
Kamil Warguła
November 29, 2015
Technology
0
330
Wykorzystanie klastra Apache Mesos w deploymencie aplikacji pythonowych
Kamil Warguła
November 29, 2015
Tweet
Share
Other Decks in Technology
See All in Technology
事業価値と Engineering
recruitengineers
PRO
1
140
R-SCoRe: Revisiting Scene Coordinate Regression for Robust Large-Scale Visual Localization
takmin
0
430
実践アプリケーション設計 ②トランザクションスクリプトへの対応
recruitengineers
PRO
2
100
浸透しなさいRFC 5322&7208
hinono
0
110
ECS モニタリング手法大整理
yendoooo
1
120
人を動かすことについて考える
ichimichi
2
320
モバイルアプリ研修
recruitengineers
PRO
2
180
ABEMAにおける 生成AI活用の現在地 / The Current Status of Generative AI at ABEMA
dekatotoro
0
640
Observability for LLM Application lifecycle
ivry_presentationmaterials
1
240
Android Studio の 新しいAI機能を試してみよう / Try out the new AI features in Android Studio
yanzm
0
260
Backboneとしてのtimm2025
yu4u
4
1.4k
第64回コンピュータビジョン勉強会@関東(後編)
tsukamotokenji
0
220
Featured
See All Featured
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.9k
How to train your dragon (web standard)
notwaldorf
96
6.2k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.5k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
560
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.6k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
8
480
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Intergalactic Javascript Robots from Outer Space
tanoku
272
27k
Transcript
Wykorzystanie klastra Apache Mesos w deploymencie aplikacji pythonowych Kamil Warguła
1 github.com/quamilek
[email protected]
2
Apache Mesos 3
What is Mesos? Apache Mesos abstracts CPU, memory, storage, and
other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively. 4
Mesos architecture Mesos master Standby master Standby master ZooKeeper quorum
Mesos slave Framework executor task Mesos slave Framework executor task Mesos slave Marathon executor task Hadoop executor task Framework scheduler 5
Resource allocation 6
Mesos UI 7
Mesos frameworks 8
Marathon 9
Marathon Marathon is an Apache Mesos framework for long-running applications
10
Marathon features • Web UI • JSON/REST API • HA
• Constraints • Health Checks • Event Subscription • Basic Auth and SSL 11
Resource utilization 12
Resource utilization 13
Mesos saves $$$ 14 old way mesos way
simple python REST app import falcon import json class ExampleResource:
def on_get(self, req, resp): data = {'foo': 'bar'} resp.body = json.dumps(data) api = falcon.API() api.add_route('/', ExampleResource()) 15
integration with gunicorn from gunicorn.app.base import BaseApplication class StandaloneApplication(BaseApplication): ...
def _get_config(): ... def main(): api = falcon.API() api.add_route('/', Example Resource()) config = _get_config() StandaloneApplication(api, config).run() if __name__ == '__main__': main() 16
How to run virtualenv env pip install falcon==0.3.0 gunicorn==19.3.0 source
env/bin/activate python api.py -p 8099 17
Pack requirements with PEX pex -v -r falcon==0.3.0 -r gunicorn==19.3.0
-o app.pex 18
How to run app with pex ./app.pex api.py -p 8099
19
DEMO 20
Marathon UI 21
Marathon app definition { "id": "appname", "cmd": "python my_app.py", "cpus":
0.5, "instances": 1, "mem": 64, "uris": [ "http://artifacts.local/my_app_0.1.0.tar.gz" ] } 22
Marathon - deploy python app DEMO 23
app configuration • environment variables • distributed configuration service (ZooKeeper,
ETCD) 24
Service Discovery • Marathon-Consul • Mesos-DNS 25
Load Balancing • haproxy-marathon-bridge 26
Scale app - manual import json import requests url =
'http://marathon.local/v2/apps/myapp' data = {'instances': 5} requests.put(url, data=json.dumps(data)) 27
Scale app - manual DEMO 28
Autoscale app MIN_INSTANCES_COUNT = 2 MAX_INSTANCES_COUNT = 20 if overloaded_instances
>= 0.9 * total_instances: total_instances += 2 if total_instances <= MAX_INSTANCES_COUNT: scale_app(total_instances) elif overloaded_instances <= 0.5 * total_instances: total_instances -= 2 if total_instances >= MIN_INSTANCES_COUNT: scale_app(total_instances) 29
Autoscale app DEMO 30
Thank you! 31 goo.gl/qWivrx
Q/A? 32 goo.gl/qWivrx