Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
rancher/system-toolsを試してみた
Search
nakayamam
November 19, 2018
Technology
0
330
rancher/system-toolsを試してみた
2018/11/19 Rancher Meetup #05 in Osaka LT資料
nakayamam
November 19, 2018
Tweet
Share
More Decks by nakayamam
See All by nakayamam
Rancherでkubeflow構築
nakayamam
3
19k
Other Decks in Technology
See All in Technology
MLOpsの「壁」を乗り越える、LINEヤフーの Data Quality as Code
lycorptech_jp
PRO
5
540
ServiceNow Knowledge Learning Rise up
manarobot
0
210
生産性向上チームの紹介
cybozuinsideout
PRO
1
870
よく聞くけど使ったことないソフトウェアNo.1 KafkaとSnowflake
foursue
4
360
元インフラエンジニアに成る / Human Resources to Human Relations
bobtani
4
930
【NW X Security JAWS#3】L3-4:AWS環境のIPv6移行に向けて知っておきたいこと
shotashiratori
0
380
レガシーをぶっ壊せ。AEONで始めるDevRelの話 / Qiita Night 2024-2-22
aeonpeople
3
1.3k
Grafana x PagerDuty Better Together
jacopen
0
100
20分で完全に理解するGrafanaダッシュボード
hamadakoji
3
680
現代CSSフレームワークの内部実装とその仕組み
poteboy
7
3.6k
Kernel MemoryでAzure OpenAI Serviceとお手軽データソース連携
mitsuzono
1
260
AOAI をきっかけに 社内の Azure 管理を見直した話
recruitengineers
PRO
1
300
Featured
See All Featured
Fontdeck: Realign not Redesign
paulrobertlloyd
76
4.9k
[RailsConf 2023] Rails as a piece of cake
palkan
23
4k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
121
39k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
25
2.3k
Navigating Team Friction
lara
178
13k
Scaling GitHub
holman
457
140k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
19
1.7k
Embracing the Ebb and Flow
colly
80
4.1k
Raft: Consensus for Rubyists
vanstee
132
6.3k
How GitHub (no longer) Works
holman
304
140k
Thoughts on Productivity
jonyablonski
58
3.8k
Done Done
chrislema
178
15k
Transcript
rancher/system-tools Λࢼͯ͠Έͨ Masaki-Nakayama
rancher/system-toolsͱʁ IUUQTHJUIVCDPNSBODIFSSBODIFSJTTVFT
rancher/system-toolsͱʁ IUUQTHJUIVCDPNSBODIFSTZTUFNUPPMT
rancher/system-toolsͱʁ
rancher/system-toolsͱʁ remove Ϋϥελ͔Βrancherؔ࿈ϦιʔεΛআ logs Ϋϥελ͔ΒϩάΛऩूͯ͠อଘ stats Ϋϥελ͔Β౷ܭใΛϦΞϧλΠϜͰऩू
લ४උ 1. DockerΠϯετʔϧࡁͷLinuxαʔόʔΛ༻ҙ 2. k8sͷconfigΛίϐʔʢkubeconfig.ymlͳͲͰอଘʣ 3. git͔Βsystem-toolsͷιʔεΛclone 4. makeίϚϯυͰϏϧυͯ͠system-toolsͷόΠφϦΛ࡞ ※releases͔ΒόΠφϦΛμϯϩʔυͯ͠ྑ͍
ڥ αʔόʔɿ rancherαʔόʔ x1ʢrancher v2.1.0ʣ Ϋϥελʔϊʔυ x1ʢetcd, control plane, workerʣ
※ࠓճKAGOYA CLOUD /2ͷVMΠϯελϯεΛ༻ OS: Ubuntu 16.04 ※Կ͔͋ΕιʔεΛ͍ͬͯ͡όΠφϦԽ͢ΔͨΊ ※makeͰͷόΠφϦԽmacͰಈ͖·ͤΜͰͨ͠ Docker: 18.06.1-ce
͜Μͳײ͡Ͱ࡞ͬͯ͋Γ·͢
system-tools remove # system-tools remove -c kubeconfig.yml Are you sure
you want to remove Rancher Management Plane in Namespace [cattle-system] [y/n]: y INFO[0002] Removing Rancher management plane in namespace: [cattle-system] INFO[0002] Getting conenction configuration INFO[0002] Removing Cattle deployment INFO[0002] Removed Cattle deployment succuessfully INFO[0002] Removing ClusterRoleBindings INFO[0002] deleting cluster role binding [cattle-admin-binding].. INFO[0002] deleting cluster role binding [clusterrolebinding-5c7rb].. INFO[0002] deleting cluster role binding [clusterrolebinding-5fwvp].. INFO[0004] Removing ClusterRoleBindings INFO[0005] deleting cluster role binding [clusterrolebinding-5fwvp].. INFO[0005] deleting cluster role binding [clusterrolebinding-cbz99].. INFO[0005] deleting cluster role binding [clusterrolebinding-sj6ps].. INFO[0005] deleting cluster role binding [clusterrolebinding-v8zwl].. INFO[0005] deleting cluster role binding [clusterrolebinding-vtflr].. INFO[0005] deleting cluster role binding [clusterrolebinding-zvdh6].. INFO[0005] deleting cluster role binding [globaladmin-u-p9d29].. INFO[0005] deleting cluster role binding [globaladmin-u-zz9xm].. INFO[0005] deleting cluster role binding [globaladmin-user-mbvd4].. INFO[0005] Successfully removed ClusterRoleBindings INFO[0005] Removing ClusterRoles INFO[0007] Removing ClusterRoles INFO[0009] Removing ClusterRoles FATA[0011] clusterroles.rbac.authorization.k8s.io is forbidden: User "user-mbvd4" cannot list clusterroles.rbac.authorization.k8s.io at the cluster scope
system-tools remove ྃͰ͖·ͤΜͰͨ͠ɾɾɾ ʢremove.goͷgetCattleClusterRoleBindingsList Ͱࣦഊ͍ͯ͠ΔͬΆ͍ɾɾɾʣ ଞʹࢼͨ͠ํ͋Εڭ͍͑ͯͩ͘͞ɾɾ
system-tools remove ຊདྷͰ͋Εɺk8sϊʔυ͔Βrancherؔ࿈ʢ໊લۭؒ: cattle-systemͳͲʣͷϦιʔεΛશͯআͯ͘͠ΕΔ • ClusterRoles • ClusterRoleBindings • Namespaces
• Projects • Users • Clusters • CRD
system-tools remove rancherΛར༻͍͕ͯͨ͠ࢭΊΔࡍɺ७ਮͳ k8sΫϥελʔͱͯͦ͠ͷ··ӡ༻Λଓ͚ͨ ͍߹ʹҠߦΛߟ͑ͳͯ͘ྑ͍ rancherΛ७ਮʹk8sΠϯετʔϥʔͱͯ͠ ͑ͦ͏
system-tools logs # system-tools logs -c kubeconfig.yml INFO[0000] deploying log
collection DaemonSet [log-collector].. INFO[0000] starting log collection.. INFO[0000] fetching logs from node [v133-18-205-207].. INFO[0002] fetching logs from node [v133-18-175-238].. INFO[0002] removing log collection DaemonSet [log-collector].. INFO[0002] log collection DaemonSet [log-collector] removed successfully.. # ls cluster-logs.tar system-tools # tar xvf cluster-logs.tar && ls cluster-logs.tar system-tools node1 node1# ls -l total 13720 -rw-r----- 1 root root 1537598 Nov 16 01:39 etcd.log -rw-r----- 1 root root 9966735 Nov 16 01:39 kube-apiserver.log -rw-r----- 1 root root 740657 Nov 16 01:39 kube-controller-manager.log -rw-r----- 1 root root 1746861 Nov 16 01:39 kubelet.log -rw-r----- 1 root root 21867 Nov 16 01:39 kube-proxy.log -rw-r----- 1 root root 23845 Nov 16 01:39 kube-scheduler.log
system-tools logs log-collectorͱ͍͏DaemonSetΛσϓϩΠͯ͠ɺϩάऩूͱग़ ྗ͕ऴΘΕআ͍ͯ͠ΔͷͰطଘڥΛԚ͞ͳ͍ ϩάͷதΛݟͨײ͡Ͱɺ/var/lib/docker/containers/{ίϯς φID}/{ίϯςφID}-json.logΛҾͬ͜ൈ͍͍ͯΔͬΆ͍ ૉͷdockerͷϩάͩͱԿͷϩά͔ͱͯΘ͔Γʹ͍͘ͷͰͦΕ ͚ͩͰॿ͔Δ ֎෦αʔόʔ͔ΒطଘΫϥελʔʹ΄΅ӨڹΛ༩͑ͣϊʔυ͝ͱ ͷlogΛऩू͢Δͱ͍ͬͨӡ༻Ίͦ͏
system-tools logs ͨͩɺԼهͷΑ͏ʹࣦഊ͢Δ͜ͱ͋Δ FATA[0000] error executing command on pod [cattle-system/log-
collector-q6rtt]: unable to upgrade connection: container not found ("log-collector") ͔ͨ͠͠ΒlogͷfetchΛͨͣͯ͠ DaemonSetΛআ͍ͯ͠Δͷ͔͠Εͳ͍…
system-tools stats # system-tools stats -c kubeconfig.yml ɾɾɾɾ INFO[0015] node
stats for [v133-18-205-207].. Linux 4.4.0-34-generic (stats-collector-mfgtn) 11/15/18 _x86_64_ (2 CPU) 14:32:31 CPU %user %nice %system %iowait %steal %idle 14:32:32 all 4.10 0.00 6.67 12.31 0.00 76.92 14:32:31 kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty 14:32:32 157740 233888 1890504 92.30 6136 239116 4688380 228.90 1621768 119652 808 14:32:31 MBfsfree MBfsused %fsused %ufsused Ifree Iused %Iused FILESYSTEM 14:32:32 22494 7615 25.29 30.11 1713379 252701 12.85 /dev/vda1 14:32:32 22494 7615 25.29 30.11 1713379 252701 12.85 /dev/vda1 14:32:32 22494 7615 25.29 30.11 1713379 252701 12.85 /dev/vda1 14:32:32 22494 7615 25.29 30.11 1713379 252701 12.85 /dev/vda1 Average: CPU %user %nice %system %iowait %steal %idle Average: all 4.10 0.00 6.67 12.31 0.00 76.92 Average: kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty Average: 157740 233888 1890504 92.30 6136 239116 4688380 228.90 1621768 119652 808 Summary: MBfsfree MBfsused %fsused %ufsused Ifree Iused %Iused FILESYSTEM Summary: 22494 7615 25.29 30.11 1713379 252701 12.85 /dev/vda1 Summary: 22494 7615 25.29 30.11 1713379 252701 12.85 /dev/vda1 Summary: 22494 7615 25.29 30.11 1713379 252701 12.85 /dev/vda1 Summary: 22494 7615 25.29 30.11 1713379 252701 12.85 /dev/vda1 ɾɾɾɾ
system-tools stats stats-collectorͱ͍͏DaemonSetΛσϓϩΠͯ͠statsΛग़ ྗ͠ଓ͚Δ →Ϣʔβʔૢ࡞ʹΑΔதஅͰDaemonSetআ ֎෦͔Βk8sΫϥελͷ౷ܭใͷऩू͕Ͱ͖Δ ग़ྗΛՃ͢ΕrancherͷϞχλʔ͕࡞Εͦ͏ͳՄೳੑΛײ͡ Δ
·ͱΊ ͔ͨ͠͠ΒRancherͷόʔδϣϯʹΑͬͯಈ͔͘ ͠Εͳ͍ʢ্ख͍ͬͨ͘Βͦͷ͏ͪQiitaʹͰॻ͖ ·͢ʣ ҆ఆ͢Εrancherӡ༻໘ͰͷϝϦοτ͕ظͰ͖ͦ͏ ίϚϯυ͕૿͍͖͑ͯͦ͏ͳظ