Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ScaleShift-jp-2019-summer
Search
ryo nakamaru
July 17, 2019
Research
1
210
ScaleShift-jp-2019-summer
ScaleShift の機能概要、Kubernetes クラスタとの連携について
ryo nakamaru
July 17, 2019
Tweet
Share
More Decks by ryo nakamaru
See All by ryo nakamaru
AWSで楽をするサービスメッシュ入門/appmesh-trial
pottava
1
1.5k
reinforce-2019-recap-lt
pottava
2
4.1k
Firecracker とは何か/what is Firecracker
pottava
12
5.4k
ハイブリッド並列 on Kubernetes/hybrid-parallel-program-on-kubernetes
pottava
1
430
AWS Fargate + Code 兄弟で始める継続的デリバリー / Continuous Delivery with AWS Fargate and Code brothers
pottava
12
3.2k
Singularity と NVIDIA GPU Cloud で作る ハイブリッド機械学習環境の構築 / Building a hybrid environment for Machine Learning with Singularity and NGC
pottava
3
1.3k
明日から始めるちょい足し λ / get-started-with-aws-lambda
pottava
4
2.5k
NGC と Singularity によるハイブリッド機械学習環境 / A hybrid environment for Machine Learning with NGC and Singularity
pottava
0
490
NGC × Singularity での機械学習環境/MachineLearning environment with NGC and Singularity
pottava
2
780
Other Decks in Research
See All in Research
NLP2025参加報告会 LT資料
hargon24
1
320
A multimodal data fusion model for accurate and interpretable urban land use mapping with uncertainty analysis
satai
3
230
SSII2025 [TS1] 光学・物理原理に基づく深層画像生成
ssii
PRO
4
3.7k
在庫管理のための機械学習と最適化の融合
mickey_kubo
3
1.1k
問いを起点に、社会と共鳴する知を育む場へ
matsumoto_r
PRO
0
390
Google Agent Development Kit (ADK) 入門 🚀
mickey_kubo
2
1.1k
MGDSS:慣性式モーションキャプチャを用いたジェスチャによるドローンの操作 / ec75-yamauchi
yumulab
0
250
ASSADS:ASMR動画に合わせて撫でられる感覚を提示するシステムの開発と評価 / ec75-shimizu
yumulab
1
400
Generative Models 2025
takahashihiroshi
21
12k
Agentic AIとMCPを利用したサービス作成入門
mickey_kubo
0
280
Adaptive fusion of multi-modal remote sensing data for optimal sub-field crop yield prediction
satai
3
220
(NULLCON Goa 2025)Windows Keylogger Detection: Targeting Past and Present Keylogging Techniques
asuna_jp
2
540
Featured
See All Featured
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Building an army of robots
kneath
306
45k
Why You Should Never Use an ORM
jnunemaker
PRO
58
9.4k
It's Worth the Effort
3n
185
28k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
Faster Mobile Websites
deanohume
307
31k
Optimising Largest Contentful Paint
csswizardry
37
3.3k
Six Lessons from altMBA
skipperchong
28
3.9k
Scaling GitHub
holman
460
140k
Site-Speed That Sticks
csswizardry
10
690
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Transcript
ScaleShift ΦϯϓϨϛε / Ϋϥυ Ͱ࣮ݱ͢Δػցֶशڥ June, 2019
!2 ScaleShift Docker ϕʔεɺΦʔϓϯιʔεͷ Web ΫϥΠΞϯτ ΞϓϦέʔγϣϯͰ͢ • ϞσϧߏஙϑΣʔζ -
NGC / ࣗࣾϦϙδτϦ͔Βػցֶश Docker ΠϝʔδΛϫϯΫϦοΫͰऔಘ - ͦͷҙͷ Docker ΠϝʔδΛ Jupyter notebook ίϯςφͱͯ͠ىಈ • ϞσϧֶशϑΣʔζ - ߏஙʹར༻ͨ͠ϥΠϒϥϦ͝ͱ Docker ΠϝʔδʹݻΊϦϙδτϦอଘ - ΫϦοΫ͚ͩͰ Kubernetes Ϋϥελ / Rescale େنܭࢉλεΫΛૹ৴
جຊతͳಈ͖ !3 How does it work?
!4 ScaleShift ͷىಈ ϩʔΧϧʹ Web αʔόʔ্ཱ͕͕ͪΓ·͢
!5 ػցֶशιϑτΣΞͷΠϯετʔϧ NGC / ϓϥΠϕʔτϨδετϦ ͔ΒϫϯΫϦοΫͰμϯϩʔυ
!6 Jupyter notebook ͰͷϞσϧߏங Jupyter Ͱϥοϓͨ͠ίϯςφ͕͔ΜͨΜʹىಈ ϙʔτ࡞ۀྖҬ ίϯςφ͝ͱʹ ͞Εͨ ΫϦʔϯͳڥ
ɹ.
!7 େنܭࢉͷͨΊͷϥοϐϯά ґଘϥΠϒϥϦιʔείʔυ܈Λ·ͱΊɺͻͱͭͷΠϝʔδʹݻΊ·͢
!8 ࣾΫϥελ / ΫϥυܭࢉλεΫೖ ೖઌʹԠͯ͡ඞཁͳ API ͕࣮ߦ͞Ε·͢ ར༻ϦιʔεྔΛܾΊ ΫϥελλεΫೖ
Kubernetes ࿈ܞ !9 Integration with a kubernetes cluster
!10 ػցֶश ͱ Kubernetes Web ք۾Λத৺ʹίϯςφΦʔέετϨʔγϣϯͷσϑΝΫτʹͳͬͨ k8sɻ ػցֶशͷจ຺Ͱίϯςφར༻͕ΜʹͳΓɺԠ༻ࣄྫ͕૿͍͑ͯ·͢ɻ - NVIDIA
͕ެࣜʹαϙʔτΛද໌ [ GTC 2018 Keynote, March 27 ] - Mercari ML Ops Night Vol.1 [ גࣜձࣾ ϝϧΧϦ / May 23, 2018 ] ɹhttps://mercari.connpass.com/event/85931/presentation/ - Jupyter ͚ͩͰػցֶशΛ࣮αʔϏεల։Ͱ͖Δج൫ [ גࣜձࣾϦΫϧʔτϥΠϑελΠϧ ] ɹhttps://engineer.recruit-lifestyle.co.jp/techblog/2018-10-04-ml-platform/ - KubernetesʹΑΔػցֶशج൫ͷઓ [ גࣜձࣾ Preferred Networks / Dec 4, 2018 ] ɹhttps://www.slideshare.net/pfi/kubernetes-125013757
!11 ScaleShift + Kubernetes ߏྫ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC
DockerHub ϓϥΠϕʔτ ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ ScaleShift ೖΓ ϩʔΧϧ
!12 1. ػցֶशιϑτΣΞͷબ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ
ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ GUI ͔ΒબͿ͚ͩͰ μϯϩʔυ͕࢝·Γ·͢
!13 2. Ϟσϧߏங ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ
ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ ScaleShift ͕ ϊʔτϒοΫΛىಈ͠·͢
!14 3. ࣮ߦڥɾೖྗσʔλͷసૹ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ
ϨδετϦ Kubernetes ScaleShift ͕෦తʹ ඞཁͳసૹΛߦ͍·͢ ݚڀ / ։ൃνʔϜ
!15 4. େنܭࢉͷ࣮ߦΛࢦࣔ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ
ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ Kubernetes ͷ Job ͱͯ͠ ܭࢉ݅Λૹ৴͠·͢
!16 5. େنܭࢉͷ࣮ߦ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub Kubernetes
ϓϥΠϕʔτ ϨδετϦ ݚڀ / ։ൃνʔϜ
!17 6. ܭࢉ݁Ռͷ֬ೝ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ
ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ
!18 Kubernetes ઃఆ / λεΫ࣮ߦը໘
ScaleShift ͷઃఆ !19 Configurations
!20 ֎෦࿈ܞ ࿈ܞػೳ ઃఆ NVIDIA GPU CLOUD • NVIDIA ࣾͷཧ͢Δػցֶश
Docker Πϝʔδͷ Ұཡ / ৄࡉใऔಘɺΠϝʔδͷμϯϩʔυ ɹAPI Ωʔ & Ϣʔβઃఆ ϓϥΠϕʔτϨδετϦ • ࣗࣾͰཧ͢Δػցֶश Docker Πϝʔδͷ ҰཡใऔಘɺΠϝʔδͷμϯϩʔυ ɹଓઌ & Ϣʔβઃఆ AWS • ػցֶश Docker Πϝʔδͷμϯϩʔυ • ϩʔΧϧϑΝΠϧγεςϜͱ S3 ؒͷσʔλ࿈ܞ ʢ࣮༧ఆʣ Kubernetes • ࣾΫϥελ / ΫϥυͰͷେنܭࢉ࣮ߦ ɹkubecfg Rescale • Rescale ϓϥοτϑΥʔϜͰͷେنܭࢉ࣮ߦ ɹҬࢦఆ & API Ωʔ
!21 ىಈΦϓγϣϯʢൈਮʣ ઃఆ֓ཁ ॳظ SS_JUPYTER_MINIMUM_PORT ίϯςφͷଓϙʔτಈతׂ։࢝൪߸ ɾ30000 SS_LOG_LEVEL ΞϓϦέʔγϣϯͷϩάग़ྗϨϕϧ ɹwarn
SS_WORKSPACE_HOST_DIR ϗετଆͷ࡞ۀσʔλอଘྖҬ ɹͳ͠ʢࢦఆඞਢʣ SS_NGC_REGISTRY_ENDPOINT NGC ଓઌ ɹhttps://registry.nvidia.com SS_NGC_REGISTRY_USER_NAME NGC Ϣʔβʔ໊ ɹ$oauthtoken SS_RESCALE_SINGULARITY_VERSION Rescale Ͱͷ Singularity ϥϯλΠϜόʔδϣϯ ɹ3.2.0 SS_RESCALE_JOB_WALLTIME Rescale ͰͷλεΫ࣮ߦ࠷େ࣌ؒ ɾ3600 docker-compose.yml ʹઃఆΛهࡌɺىಈͰ͖·͢