Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Rancherでつくる KubernetesのGPUクラスタ
sasaki
September 20, 2019
Technology
2
370
Rancherでつくる KubernetesのGPUクラスタ
sasaki
September 20, 2019
Tweet
Share
More Decks by sasaki
See All by sasaki
taishin
1
79
taishin
1
53
taishin
0
56
taishin
0
47
taishin
2
540
taishin
0
430
taishin
1
360
Other Decks in Technology
See All in Technology
ama_ch
0
3.7k
clustervr
0
200
chaspy
1
150
sugimomoto
1
240
neo_analytics
1
1.1k
kappa4
4
2.4k
yosshi_
3
340
stakaya
12
8k
iqbocchi
0
540
masakick
0
130
ocise
0
170
nkjzm
1
850
Featured
See All Featured
morganepeng
92
14k
chriscoyier
499
130k
pauljervisheath
196
15k
scottboms
251
11k
hannesfritz
27
930
bkeepers
321
53k
dougneiner
119
7.8k
holman
448
130k
garrettdimon
287
110k
3n
163
22k
mthomps
39
2.3k
bryan
30
3.3k
Transcript
3BODIFS.FFUVQJO,ZPUP 3BODIFSͰͭ͘Δ ,VCFSOFUFTͷ(16Ϋϥελ ࠤʑਅ
8IP Shinya Sasaki Head of Infrastructure Engineering at AlpacaJapan Co.,
Ltd. Osaka, Japan
Ҏલͷൃද
None
None
None
Ҏલͷൃද
None
ࠓ͜ͷลͷ
ઃஔڥ w σʔληϯλʔ ݿʁ w ϥοΫͱిݯɺΠϯλʔωοτଓΛΘͤͯΒ͍ͬͯ Δ w ػثͷઃஔɺ-"/݁ઢɺύʔπަݱͷํʹͬͯ
Β͑Δ
None
None
None
None
None
None
ϋʔυΣΞ w طଘͰ͍ͬͯΔ(16αʔόΛ࠶ηοτΞοϓ͠ɺॱ࣍Ϋ ϥελʹऩ༰ w ࠓޙ૿ઃ༧ఆ
ϚβʔϘʔυ w (16͕ຕૠ͞Δͷ͕΄͔ʹͳ͔ͬͨΒ͍͠
(16 w (F'PSDF(59 w (F'PSDF359
%JTL w 64#ଓͷ44% w ͠ΐͬͪΎ͏յΕΔͷͰ
ిݯཧ w *1.*ͱ͔J-0ͷΑ͏ͳϦϞʔτཧػೳͳ͍ w .FSPTTͰεϚϗͰిݯ0''0/
ඞཁͳλεΫ w 3BODIFSڥͷߏங w (16ϊʔυͷηοτΞοϓखॱͷཱ֬ w ηοτΞοϓͷࣗಈԽ
3BODIFSڥͷߏங
3BODIFSڥͷߏங ׂ Ѫ
(16ϊʔυͷηοτΞοϓखॱͷཱ֬
(16ϊʔυͷηοτΞοϓ w 04Πϯετʔϧ w $6%"Πϯετʔϧ w EPDLFSΠϯετʔϧ w OWJEJBEPDLFSΠϯετʔϧ w
3BODIFSΫϥελʹ+PJO
04Πϯετʔϧ w 6CVOUV
04Πϯετʔϧ w 6CVOUV-54 44%Λೝࣝ͠ͳ͍
04Πϯετʔϧ w 6CVOUV-54 w 6CVOUV-54
04Πϯετʔϧ w 6CVOUV-54 w 6CVOUV-54 Πϯετʔϧ్தͰΤϥʔ
04Πϯετʔϧ w 6CVOUV-54 w 6CVOUV-54 w 6CVOUV
04Πϯετʔϧ w 6CVOUV-54 w 6CVOUV-54 w 6CVOUV Πϯετʔϧޭ
04Πϯετʔϧ w 6CVOUV-54 w 6CVOUV-54 w 6CVOUV Πϯετʔϧޭ ͕ɺOWJEJBEPDLFS͕ʹର Ԡ͍ͯ͠ͳ͍ͷͰɺͷͷ
ΛΠϯετʔϧ͢Δͱ͔ɾɾɾ
04Πϯετʔϧ w 6CVOUV-54 w 6CVOUV-54 w 6CVOUV ͱΓ͋͑ͣɾɾɾ
͜͏͍͏ͷ͕͍ͰΫϥυΤϯδχΞʹ ͳͬͨͷͰͳ͔ͬͨͷ͔ɾɾɾ
͍࢝͟ΊΔͱɾɾɾ w w 6CVOUV-54ϦϦʔε
04Πϯετʔϧ w w 6CVOUV-54ϦϦʔε
͜͏͍͏ͷ͕͍Ͱ SZ
OWJEJBTNJ OWJEJBTNJ 5IV4FQ c/7*%*"4.*%SJWFS7FSTJPOc c
c(16/BNF1FSTJTUFODF.c#VT*E%JTQ"c7PMBUJMF6ODPSS&$$c c'BO5FNQ1FSG1XS6TBHF$BQc.FNPSZ6TBHFc(166UJM$PNQVUF.c c c c(F'PSDF(590⒎c0⒎c/"c c$188c.J#.J#c%FGBVMUc c(F'PSDF(590⒎c0⒎c/"c c$188c.J#.J#c%FGBVMUc c(F'PSDF(590⒎c0⒎c/"c c$188c.J#.J#c%FGBVMUc c(F'PSDF(590⒎c"0⒎c/"c c$188c.J#.J#c%FGBVMUc
ηοτΞοϓͷࣗಈԽ
͜Εʂ IUUQLZTNPIBUFOBCMPHKQFOUSZ
ཧ༻αʔό
None
.""4Α͔ͬͨ w ͦΜͳʹ͘͠ͳ͔ͬͨ w %)$1ͱ͔શ෦ཧͰ͖Δ w ϋʔυΣΞใݟΕΔ
͕ɺ͕ɾɾɾ w ηοτΞοϓதʹQPXFSP⒎͕Δ w ϚβʔϘʔυ͕෮ిʹରԠ͍ͯ͠ͳ͍ w ṖͷεΠονͷΈͰ͔͠෮چͰ͖ͳ͍ w ϦϞʔτͰରԠෆՄ
ṖͷεΠον
None
݁ہ͖͋ΒΊͯQSFTFFE w %)$1ɺ5'51ɺ1SFTFFEͦΕͧΕΠϯετʔϧ w ςΩετϕʔεͷઃఆϑΝΠϧ w ਓʹҾ͖ܧ͙͜ͱΛߟ͑ΔͱͰ͖ΕΓͨ͘ͳ͔ͬ ͨɾɾɾ
EIDQEDPOG IPTUOWYM\ IBSEXBSFFUIFSOFUFEFYYYYYY pYFEBEESFTT PQUJPOIPTUOBNFlTFSW ^ IPTUOWYM\ IBSEXBSFFUIFSOFUFEFYYYYZZ pYFEBEESFTT PQUJPOIPTUOBNFTFSW
^ *1ΞυϨεͷཧͱ͔ͨ͘͠ͳ͔͕ͬͨɺނোͨ͠ ͱ͖ͷަґཔͱ͔ߟ͑Δͱ."$ΞυϨεʹΑΔݻ ఆ*1ཧ͕ඞཁ
None
QSFTFFEDGH ɿ (OSΠϯετʔϧؔ࿈ུ) in-target /bin/mkdir /home/ubuntu/.ssh ;\ in-target /bin/chmod 700
/home/ubuntu/.ssh ;\ in-target /bin/sh -c 'echo "ssh-rsa xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" > /home/ ubuntu/.ssh/authorized_keys' ;\ in-target /bin/chown -R ubuntu:ubuntu /home/ubuntu/.ssh ;\ in-target /bin/sh -c 'echo "ubuntu ALL = NOPASSWD: ALL" > /etc/sudoers.d/ubuntu' ;\ in-target /bin/sh -c 'curl -fsSL https://xxxxxxxxxxxxxxxxxxxxxx/install-node.sh | bash' d-i finish-install/reboot_in_progress note
QSFTFFEDGH ɿ (OSΠϯετʔϧؔ࿈ུ) in-target /bin/mkdir /home/ubuntu/.ssh ;\ in-target /bin/chmod 700
/home/ubuntu/.ssh ;\ in-target /bin/sh -c 'echo "ssh-rsa xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" > /home/ ubuntu/.ssh/authorized_keys' ;\ in-target /bin/chown -R ubuntu:ubuntu /home/ubuntu/.ssh ;\ in-target /bin/sh -c 'echo "ubuntu ALL = NOPASSWD: ALL" > /etc/sudoers.d/ubuntu' ;\ in-target /bin/sh -c 'curl -fsSL https://xxxxxxxxxxxxxxxxxxxxxx/install-node.sh | bash' d-i finish-install/reboot_in_progress note 44)ϩάΠϯͰ͖ΔΑ͏ʹ
QSFTFFEDGH ɿ (OSΠϯετʔϧؔ࿈ུ) in-target /bin/mkdir /home/ubuntu/.ssh ;\ in-target /bin/chmod 700
/home/ubuntu/.ssh ;\ in-target /bin/sh -c 'echo "ssh-rsa xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" > /home/ ubuntu/.ssh/authorized_keys' ;\ in-target /bin/chown -R ubuntu:ubuntu /home/ubuntu/.ssh ;\ in-target /bin/sh -c 'echo "ubuntu ALL = NOPASSWD: ALL" > /etc/sudoers.d/ubuntu' ;\ in-target /bin/sh -c 'curl -fsSL https://xxxxxxxxxxxxxxxxxxxxxx/install-node.sh | bash' d-i finish-install/reboot_in_progress note ཧ͍͢͠Α͏ʹΠϯε τʔϧεΫϦϓτ֎෦ʹ
QSFTFFEDGH ɿ (OSΠϯετʔϧؔ࿈ུ) in-target /bin/mkdir /home/ubuntu/.ssh ;\ in-target /bin/chmod 700
/home/ubuntu/.ssh ;\ in-target /bin/sh -c 'echo "ssh-rsa xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" > /home/ ubuntu/.ssh/authorized_keys' ;\ in-target /bin/chown -R ubuntu:ubuntu /home/ubuntu/.ssh ;\ in-target /bin/sh -c 'echo "ubuntu ALL = NOPASSWD: ALL" > /etc/sudoers.d/ubuntu' ;\ in-target /bin/sh -c 'curl -fsSL https://xxxxxxxxxxxxxxxxxxxxxx/install-node.sh | bash' d-i finish-install/reboot_in_progress note ࠶ىಈ
JOTUBMMOPEFTI apt-get update hostnamectl set-hostname localhost apt-get -y install curl
apt-transport-https ca-certificates gnupg-agent software-properties-common open-iscsi curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable" add-apt-repository -y ppa:graphics-drivers curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu18.04/nvidia-docker.list | sudo tee /etc/apt/ sources.list.d/nvidia-docker.list apt-get update apt-get install -y vim nvidia-docker2 ubuntu-drivers-common nvidia-cuda-toolkit docker-ce docker-ca-cli containerd.io ubuntu-drivers autoinstall if ! grep -e "default-runtime" /etc/docker/daemon.json -e "default-runtime" /etc/docker/daemon.json >/dev/null; then sed -i -e "2i \ \ \ \ \"default-runtime\": \"nvidia\"," /etc/docker/daemon.json ; fi
ηοτΞοϓͷྲྀΕ w ݱͷํʹ-"/ʹଓ͠ɺىಈͯ͠Β͏ w ࣗಈ04Πϯετʔϧ։࢝ˠ࠶ىಈ w ϦϞʔτଓ֬ೝ͠ɺखಈͰΫϥελʹ+PJO͢ΔίϚϯυ ࣮ߦ
ΫϥελࢀՃ
ӡ༻։࢝ޙͷ(16ϊʔυͷׂ w ॳLTͷཧܥ݉Ͷ͍͕ͯͨɺΑ͘ίέΔͷͰཧܥ &$ʹͯ͠ɺ(16ϊʔυ8PSLFSઐ༻ʹͨ͠ w ཧܥίέΔͱ݁ߏ໘ɾɾɾ
͜ͷลͷ
ཧܥ&$
8PSLFSͱͯ͠ͷ&$
(16ϊʔυ
None
͜ͷลͷ ·ͨػձ͕͋Εʂ
͋Γ͕ͱ͏͍͟͝·ͨ͠