Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
amazon-dsstne-2016-07
Search
ryo nakamaru
July 27, 2016
Programming
210
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
amazon-dsstne-2016-07
ryo nakamaru
July 27, 2016
Transcript
16 slides
DL for Recommendations OpenHyperScale ษڧձ 16.07.28
ͦͷલʹ 2
3 V1.0! http://mesos.apache.org/blog/mesos-1-0-0-released/ (Yesterday, Jul 27 2016)
4 GPU SUPPORT!! https://github.com/apache/mesos/blob/1.0.0/CHANGELOG
՝͋ͱগ͠ 5 https://issues.apache.org/jira/browse/MESOS-4424
6 WHAT ABOUT DC/OS? https://mesosphere.com/blog/2016/07/27/apache-mesos-reaches-1-0/
ଓ͖࣍ճ.. 7
ຊ 8
Ϩίϝϯυ 9
“͜ͷΛങͬͨਓ͜Μͳങ͍ͬͯ·͢” 10
11 http://aws.typepad.com/sajp/2016/07/generating-recommendations-at-amazon-scale-with-apache-spark-and-amazon-dsstne.html AMAZON ͷύʔιφϧϨίϝϯυΛ AWS Ͱ࣮
12 ֶशσʔλੜɿ Amazon EMR (cluster) + Apache Spark (CPU) ֶशɾਪɿ
ECS (cluster) + ECR (docker image) + Amazon DSSTNE (GPU) σʔλసૹɾڞ༗ɿ Amazon S3 Ϩίϝϯυੜڥͷߏ
13 ೖྗɾग़ྗϨΠϠʹ Amazon ͷेສΛஔ͠ɺ ݸਓ͝ͱʹ͜ͷϞσϧΛ༻͍ͯϨίϝϯυ͖͢Λਪଌɻ Ϩίϝϯσʔγϣϯੜ DNN Ϟσϧ ※ ࠨਤɺޡղΛট͘ֆͰ͢Έ·ͤΜɻ
࣮ࡍʹೖྗͱग़ྗಉ͡ϊʔυ
DSSTNE 14
15 εύʔεσʔλʹ࠷దԽ͠ɺͱεέʔϧΛॏࢹͨ͠ DNN with GPUs ͳϥΠϒϥϦɻAmazon ͕ OSS ͱͯ͠ެ։ɻ ࣗࣾͷ
DL ʹ·ͭΘΔ՝ղܾͷͨΊʹ࡞͞Εͨͷɻ AMAZON DSSTNE https://github.com/amznlabs/amazon-dsstne/
16 ɾͱͯ͑͠Δ͚ͩͷύϑΥʔϚϯεΛॏࢹ ɾϨίϝϯυੜࣗવݴޠཧղॲཧͰѻ͏εύʔεσʔλͰಛʹߴ ɹɹʢg2.8xlarge Πϯελϯε 1 Ͱ TensorFlow ͷ 2.1
ഒʣ ɾαʔό 1 × ෳ GPU Ͱͷԋࢉ͕͔ͳΓಘҙ ɹɹ-> GPU ͝ͱʹλεΫΛࣗಈࢄ ɹɹ-> GPU Λ૿ͤॲཧΛߴΊΒΕΔʢεέʔϧΞτʣ ɹɹɹ& ΑΓେ͖ͳϞσϧ͕ѻ͑Δ ɾωοτϫʔΫͷఆ͕ٛ Caffe ΑΓγϯϓϧ ɾΈࠐΈʹະରԠ ɾϦΧϨϯτݱࡏݶఆతαϙʔτɻ͍ͣΕϩʔυϚοϓʹ͋Δɻ ͦͷଞͷ DL ϥΠϒϥϦͱͷҧ͍ͳͲ https://github.com/amznlabs/amazon-dsstne/blob/master/FAQ.md
More Decks by ryo nakamaru
See All by ryo nakamaru
AWSで楽をするサービスメッシュ入門/appmesh-trial
pottava
1
1.7k
reinforce-2019-recap-lt
pottava
2
4.2k
ScaleShift-jp-2019-summer
pottava
1
240
Firecracker とは何か/what is Firecracker
pottava
12
5.7k
ハイブリッド並列 on Kubernetes/hybrid-parallel-program-on-kubernetes
pottava
1
480
AWS Fargate + Code 兄弟で始める継続的デリバリー / Continuous Delivery with AWS Fargate and Code brothers
pottava
12
3.4k
Singularity と NVIDIA GPU Cloud で作る ハイブリッド機械学習環境の構築 / Building a hybrid environment for Machine Learning with Singularity and NGC
pottava
3
1.6k
明日から始めるちょい足し λ / get-started-with-aws-lambda
pottava
4
2.6k
NGC と Singularity によるハイブリッド機械学習環境 / A hybrid environment for Machine Learning with NGC and Singularity
pottava
0
560
Other Decks in Programming
See All in Programming
우리의 Nitro Modules는 당신의 Turbo Modules보다 아름답다
l2hyunwoo
0
270
Spec-Driven Development - Why Vaadin Is the Perfect Fit
simas
PRO
0
260
AI駆動開発4年目の現在地(『あなたはFDEになるべきか?』イベント資料)
kumaiu
0
660
React Native에 현대적인 빌드 도구 더하기: Rollipop 구현기
leegeunhyeok
0
180
自宅サーバはどこまで再現できるのか - Proxmox VE上のVM環境をIaCとバックアップで戻してみた
youto
0
430
Enforcing Your Architecture with Unit Tests in the AI Era (Swift Connection X)
steliosf
PRO
1
190
そのデザイントークン、Flutterでも同じ見た目になりますか?
nakamizu
0
430
[DroidKnights 2026] Workflow AX 적용기 — 자동 정책 정리부터 E2E까지
dayeon
0
110
AIがコードを書いてくれる時代、我々は何を確かめるのか
yosuke_furukawa
PRO
13
6.1k
Kotlin Fest 2026: Kotlin コンパイラに Contribute してみよう! - Kotlin/Wasm で深い再帰を書けるようにした話
ternbusty
1
380
止まらない比較を止める — 余帰納法で無限ストリームを比べる
tarao
0
190
GraphQL の onError と Nullability のこれから
quramy
0
220
Featured
See All Featured
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
10k
State of Search Keynote: SEO is Dead Long Live SEO
ryanjones
0
350
世界の人気アプリ100個を分析して見えたペイウォール設計の心得
akihiro_kokubo
PRO
74
43k
Utilizing Notion as your number one productivity tool
mfonobong
4
650
Code Reviewing Like a Champion
maltzj
529
40k
HU Berlin: Industrial-Strength Natural Language Processing with spaCy and Prodigy
inesmontani
PRO
0
810
YesSQL, Process and Tooling at Scale
rocio
174
15k
16th Malabo Montpellier Forum Presentation
akademiya2063
PRO
0
470
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3.6k
Being A Developer After 40
akosma
PRO
92
590k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
287
14k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
18k