K3sマスターの道 #2 アーキテクチャ deep dive

Transcript

1. © Copyright 2020 Rancher Labs. All Rights Reserved. 1 ©

   Copyright 2020 Rancher Labs. All Rights Reserved. 1 K3sマスターの道 #2 アーキテクチャ deep dive 2020.10.8 Wenhan Shi Support Engineer

2. © Copyright 2020 Rancher Labs. All Rights Reserved. 2 前回のおさらい

   - k3s • CNCF Certificated Kubernetes distribution • シンプル＆軽量 • シングルプロセス • バイナリ50MB • 最低必要リソース(1core, 512MB) • 多様なCPUArchを対応 • x86_64 / Arm64 / Arm7 • コンポネートがカスタマイズできる • DB: SQLite(default) / PostgreSQL / MySQL / Dqlite / etcd • Container Runtime: contrainerd(default) / docker • CNI: Flannel(default) / Calico / Canal

3. © Copyright 2020 Rancher Labs. All Rights Reserved. 3 3

   k3sアーキテクチャー

4. © Copyright 2020 Rancher Labs. All Rights Reserved. 4 アーキテクチャー

   MariaDB PostgreSQL MySQL ETCD Canal Calico https://rancher.com/docs/k3s/latest/en/installation/datastore/ https://rancher.com/docs/k3s/latest/en/installation/network-options/ https://rancher.com/docs/k3s/latest/en/advanced/#using-docker-as-the-container-runtime Docker WebSocket

5. © Copyright 2020 Rancher Labs. All Rights Reserved. 5 軽量化のための施策

   - codeの増減

6. © Copyright 2020 Rancher Labs. All Rights Reserved. 6 軽量化のための施策

   - プロセスの削減

7. © Copyright 2020 Rancher Labs. All Rights Reserved. 7 ソースコードアーキテクチャ

   --disable-agent 単独プロセス

8. © Copyright 2020 Rancher Labs. All Rights Reserved. 8 demo

   - k3sのプロセス > multipass list Name State IPv4 Image k3s-agent Running 10.85.39.218 Ubuntu 20.04 LTS k3s-server Running 10.85.39.30 Ubuntu 20.04 LTS

9. © Copyright 2020 Rancher Labs. All Rights Reserved. 9 demo

   - k3sのプロセス - server # Launch a k3s server only ubuntu@k3s-server:~$ curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable-agent" sh - # Can’t see the node but the ns shows the install is done ubuntu@k3s-server:~$ sudo k3s kubectl get node No resources found in default namespace. ubuntu@k3s-server:~$ sudo k3s kubectl get ns NAME STATUS AGE default Active 35s kube-system Active 35s kube-public Active 35s kube-node-lease Active 35s

10. © Copyright 2020 Rancher Labs. All Rights Reserved. 10 demo

    - k3sのプロセス - server # k3s only launched 1 process ubuntu@k3s-server:~$ ps aux | grep k3s root 11378 12.5 22.5 631848 459388 ? Ssl 17:39 0:28 /usr/local/bin/k3s server --disable-agent # view the threads of k3s server ubuntu@k3s-server:~$ ps -T 11378 PID SPID TTY STAT TIME COMMAND 11378 11378 ? Ssl 0:00 /usr/local/bin/k3s server --disable-agent 11378 11385 ? Ssl 0:06 /usr/local/bin/k3s server --disable-agent 11378 11386 ? Ssl 0:08 /usr/local/bin/k3s server --disable-agent 11378 11387 ? Ssl 0:00 /usr/local/bin/k3s server --disable-agent 11378 11388 ? Ssl 0:00 /usr/local/bin/k3s server --disable-agent 11378 11389 ? Ssl 0:02 /usr/local/bin/k3s server --disable-agent 11378 11390 ? Ssl 0:04 /usr/local/bin/k3s server --disable-agent 11378 11391 ? Ssl 0:06 /usr/local/bin/k3s server --disable-agent 11378 11392 ? Ssl 0:02 /usr/local/bin/k3s server --disable-agent 11378 11393 ? Ssl 0:06 /usr/local/bin/k3s server --disable-agent 11378 11398 ? Ssl 0:00 /usr/local/bin/k3s server --disable-agent 11378 11399 ? Ssl 0:00 /usr/local/bin/k3s server --disable-agent 11378 11400 ? Ssl 0:00 /usr/local/bin/k3s server --disable-agent

11. © Copyright 2020 Rancher Labs. All Rights Reserved. 11 demo

    - k3sのプロセス - client # add agent to server ubuntu@k3s-agent:~$ curl -sfL https://get.k3s.io | K3S_URL=https://k3s-server:6443 K3S_TOKEN=XXX sh - # now we can see this node ubuntu@k3s-server:~$ sudo k3s kubectl get node NAME STATUS ROLES AGE VERSION k3s-agent Ready <none> 2m15s v1.18.9+k3s1

12. © Copyright 2020 Rancher Labs. All Rights Reserved. 12 demo

    - k3sのプロセス - client • containerd は独⽴のプロセスで、k3sの⼦プロセスとして存在している。 # check the thread of agent process ubuntu@k3s-agent:~$ ps aux | grep k3s root 12605 5.7 5.7 215600 117300 ? Ssl 00:40 0:03 /usr/local/bin/k3s agent root 12626 8.0 6.3 217108 129900 ? Sl 00:40 0:04 containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd # containerd is the child process of k3s agent ubuntu@k3s-agent:~$ ps --ppid 12605 PID TTY TIME CMD 12626 ? 00:00:10 containerd

13. © Copyright 2020 Rancher Labs. All Rights Reserved. 13 demo

    - k3sのプロセス - client # check the thread of agent process, implemented by goroutines ubuntu@k3s-agent:~$ ps -T 12605 PID SPID TTY STAT TIME COMMAND 12605 12605 ? Ssl 0:00 /usr/local/bin/k3s agent 12605 12619 ? Ssl 0:02 /usr/local/bin/k3s agent 12605 12620 ? Ssl 0:01 /usr/local/bin/k3s agent 12605 12621 ? Ssl 0:00 /usr/local/bin/k3s agent 12605 12622 ? Ssl 0:00 /usr/local/bin/k3s agent …

14. © Copyright 2020 Rancher Labs. All Rights Reserved. 14 demo

    - k3sのプロセス - client • runtimeサポートを提供するために、単独のcontainerd-shimプロセスを起動する必要がある • Pod内のコンテナがこのcontainerd-shimの⼦プロセスである。 # check the thread of this process, implemented by goroutines ubuntu@k3s-agent:~$ ps aux | grep shim … root 12943 0.0 0.4 112504 8608 ? Sl 00:40 0:00 /var/lib/rancher/k3s/data/688c8ca42a6cd0c042322efea271d6f3849d3de17c850739b0da2461f6c69ee8/bin/contain erd-shim-runc-v2 -namespace k8s.io -id b2ae4f9f7f2ea77cf2199717137f2b0aa1a86a69f4107fe788e74bfa70073f1b -address /run/k3s/containerd/containerd.sock … ubuntu@k3s-agent:~$ pstree -p -aT 12943 containerd-shim,12943 -namespace k8s.io -id b2ae… -address/run/k3 ├─coredns,13240 -conf /etc/coredns/Corefile └─pause,13040

15. © Copyright 2020 Rancher Labs. All Rights Reserved. 17 17

    Containerd

16. © Copyright 2020 Rancher Labs. All Rights Reserved. 18 Containerdを操作するcmd

    K8sの観点から Podとコンテナーを管理 コンテナーを管理のみ

17. © Copyright 2020 Rancher Labs. All Rights Reserved. 19 Containerdを操作するcmd

    - image イメージ Docker ContainerD リスト docker images crictl images ctr images list ダウンロード docker pull crictl pull ctr (images) pull アップロード docker push ctr (images) push 削除 docker rmi crictl rmi ctr images remove タグ docker tag ctr (images) tag 詳細情報 docker inspect crictl inspecti

18. © Copyright 2020 Rancher Labs. All Rights Reserved. 20 Containerdを操作するcmd

    - container Container Docker ContainerD リスト docker ps crictl ps ctr containers list 作成 docker create crictl create ctr containers create 実⾏ docker start crictl start ctr (tasks) start 停⽌ docker stop crictl stop ctr (tasks) pause 削除 docker rm crictl rm ctr (tasks) rm 詳細情報 docker inspect crictl inspect 接続 docker attach crictl attach ctr (tasks) attach 内部コマンド docker exec crictl exec ctr (tasks) exec ログ docker logs crictl logs ステータス docker stats crictl stats

19. © Copyright 2020 Rancher Labs. All Rights Reserved. 21 Containerdを操作するcmd

    - pod POD Docker ContainerD リスト crictl pods 詳細情報 crictl inspectp 実⾏ crictl runp 停⽌ crictl stopp 削除 crictl rmp Port forward crictl port-foward

20. © Copyright 2020 Rancher Labs. All Rights Reserved. 22 demo

    - containerdのオペレーション • k3s をインストールした時に、crictlとctrはデフォルトでインストールされる ubuntu@k3s-agent:~$ ls -al /usr/local/bin/ total 52224 drwxr-xr-x 2 root root 4096 Oct 8 00:40 . drwxr-xr-x 10 root root 4096 Sep 22 06:38 .. lrwxrwxrwx 1 root root 3 Oct 8 00:40 crictl -> k3s lrwxrwxrwx 1 root root 3 Oct 8 00:40 ctr -> k3s -rwxr-xr-x 1 root root 53460992 Oct 8 00:40 k3s -rwxr-xr-x 1 root root 953 Oct 8 00:40 k3s-agent-uninstall.sh -rwxr-xr-x 1 root root 1521 Oct 8 00:40 k3s-killall.sh lrwxrwxrwx 1 root root 3 Oct 8 00:40 kubectl -> k3s

21. © Copyright 2020 Rancher Labs. All Rights Reserved. 23 demo

    - containerdのオペレーション • imageのリスト処理では、crictlの⽅がより読みやすい ubuntu@k3s-agent:~$ sudo crictl images IMAGE TAG IMAGE ID SIZE docker.io/rancher/coredns-coredns 1.6.9 4e797b3234604 13.4MB docker.io/rancher/klipper-helm v0.2.7 1087ccbd2ab61 50MB docker.io/rancher/klipper-lb v0.1.2 897ce3c5fc8ff 2.71MB docker.io/rancher/library-traefik 1.7.19 aa764f7db3051 24MB docker.io/rancher/local-path-provisioner v0.0.11 9d12f9848b99f 12MB docker.io/rancher/metrics-server v0.3.6 9dd718864ce61 10.5MB docker.io/rancher/pause 3.1 da86e6ba6ca19 327k ubuntu@k3s-agent:~$ sudo ctr images list -q docker.io/rancher/coredns-coredns:1.6.9 docker.io/rancher/coredns-coredns@sha256:e70c936deab8efed89db66f04847fec137dbb81d5b456e8068b6e71cb770f6c0 docker.io/rancher/klipper-helm:v0.2.7 docker.io/rancher/klipper-helm@sha256:70c671e0b5bf5a9dffb90dabf59ee83fbcb27971d2e69f43a1c4021234d179a1 docker.io/rancher/klipper-lb:v0.1.2 docker.io/rancher/klipper-lb@sha256:2fb97818f5d64096d635bc72501a6cb2c8b88d5d16bc031cf71b5b6460925e4a docker.io/rancher/library-traefik:1.7.19 docker.io/rancher/library-traefik@sha256:3ba3ed48c4632f2b02671923950b30b5b7f1b556e559ce15446d1f5d648a037d docker.io/rancher/local-path-provisioner:v0.0.11 docker.io/rancher/local-path-provisioner@sha256:0d60b97b101e432606035ab955c623604493e8956484af1cfa207753329bdf81 docker.io/rancher/metrics-server:v0.3.6 docker.io/rancher/metrics-server@sha256:b85628b103169d7db52a32a48b46d8942accb7bde3709c0a4888a23d035f9f1e docker.io/rancher/pause:3.1 …

22. © Copyright 2020 Rancher Labs. All Rights Reserved. 24 demo

    - containerdのオペレーション • コンテナー表⽰では、crictlの⽅にPod IDがあるが、ctrの⽅はない ubuntu@k3s-agent:~$ sudo crictl ps CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID 85cb81ac22a93 aa764f7db3051 11 hours ago Running traefik 0 fa62b7735c1bc 34dea7676e1e2 897ce3c5fc8ff 11 hours ago Running lb-port-443 0 4846ae9e043e0 80e18fbf0a472 897ce3c5fc8ff 11 hours ago Running lb-port-80 0 4846ae9e043e0 bfcb3a82b1657 4e797b3234604 11 hours ago Running coredns 0 b2ae4f9f7f2ea 2e118fc093e16 9dd718864ce61 11 hours ago Running metrics-server 0 f6b41bda738ab 314c23ec540da 9d12f9848b99f 11 hours ago Running local-path-provisioner 0 692c06fcd2bea ubuntu@k3s-agent:~$ sudo ctr containers list CONTAINER IMAGE RUNTIME 04195dfe50a0c1cb9eb1734b7f252771aaa880c8c76dd1dbaa66e4f3b429a567 docker.io/rancher/klipper-helm:v0.2.7 io.containerd.runc.v2 2e118fc093e169e85ea105db9abcb34ba2e3c285fd3194e13f78589ec88750ca docker.io/rancher/metrics-server:v0.3.6 io.containerd.runc.v2 314c23ec540da5364e845afd7fc1072eb0502e58089101dedc1c433c7e405e0c docker.io/rancher/local-path-provisioner:v0.0.11 io.containerd.runc.v2 34dea7676e1e2ebd51c0648e23584bd659c6786b431bd32fcad332efb0657483 docker.io/rancher/klipper-lb:v0.1.2 io.containerd.runc.v2 4846ae9e043e09fc52afbf16993565b673b0ac9d784b35db8a96d2465bf5f80a docker.io/rancher/pause:3.1 io.containerd.runc.v2 692c06fcd2beaf73ced23dccdb1c3235dc8d8b6865aebdacb2eca22e38ba54ab docker.io/rancher/pause:3.1 io.containerd.runc.v2 80e18fbf0a472c704580314d9d78220b0d7fbc3e714b1b94c26e8ce826306fd1 docker.io/rancher/klipper-lb:v0.1.2 io.containerd.runc.v2 8154e17c8c4f036f931f9ae9f2adf79ff7204bca681190cbf0e502276986990e docker.io/rancher/pause:3.1 io.containerd.runc.v2 85cb81ac22a93d313036969af166533fa7fc2e31f3f5be0f787dbb7ca4a0e830 docker.io/rancher/library-traefik:1.7.19 io.containerd.runc.v2 b2ae4f9f7f2ea77cf2199717137f2b0aa1a86a69f4107fe788e74bfa70073f1b docker.io/rancher/pause:3.1 io.containerd.runc.v2 bfcb3a82b165797c2f2924e41115b47c87b03ebf8f7b66db9ffcfc92ed23b594 docker.io/rancher/coredns-coredns:1.6.9 io.containerd.runc.v2 e1aeb789aa5ea2f3fa21cc36ba1189ae426b8fb60ff85da4aaa06abfa2ae7ba0 docker.io/rancher/pause:3.1 io.containerd.runc.v2 f6b41bda738abe21da774d8d41b9c2efefbe6a3c8dcadfe964aa8968d686e014 docker.io/rancher/pause:3.1 io.containerd.runc.v2 fa62b7735c1bc89218d57236c820b5e279abdd59bc6ccb6d9f194673c01513e1 docker.io/rancher/pause:3.1 io.containerd.runc.v2

23. © Copyright 2020 Rancher Labs. All Rights Reserved. 25 demo

    - containerdのオペレーション • コンテナー内実⾏、ctrの⽅ならプロセスIDも必要 ubuntu@k3s-agent:~$ sudo crictl ps CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID 85cb81ac22a93 aa764f7db3051 11 hours ago Running traefik 0 fa62b7735c1bc 34dea7676e1e2 897ce3c5fc8ff 11 hours ago Running lb-port-443 0 4846ae9e043e0 80e18fbf0a472 897ce3c5fc8ff 11 hours ago Running lb-port-80 0 4846ae9e043e0 bfcb3a82b1657 4e797b3234604 11 hours ago Running coredns 0 b2ae4f9f7f2ea 2e118fc093e16 9dd718864ce61 11 hours ago Running metrics-server 0 f6b41bda738ab 314c23ec540da 9d12f9848b99f 11 hours ago Running local-path-provisioner 0 692c06fcd2bea ubuntu@k3s-agent:~$ sudo crictl exec -it 314c23ec540da sh / # exit ubuntu@k3s-agent:~$ sudo ctr task list TASK PID STATUS 80e18fbf0a472c704580314d9d78220b0d7fbc3e714b1b94c26e8ce826306fd1 13877 RUNNING fa62b7735c1bc89218d57236c820b5e279abdd59bc6ccb6d9f194673c01513e1 13773 RUNNING 34dea7676e1e2ebd51c0648e23584bd659c6786b431bd32fcad332efb0657483 13909 RUNNING 692c06fcd2beaf73ced23dccdb1c3235dc8d8b6865aebdacb2eca22e38ba54ab 13012 RUNNING b2ae4f9f7f2ea77cf2199717137f2b0aa1a86a69f4107fe788e74bfa70073f1b 13040 RUNNING 314c23ec540da5364e845afd7fc1072eb0502e58089101dedc1c433c7e405e0c 13143 RUNNING 2e118fc093e169e85ea105db9abcb34ba2e3c285fd3194e13f78589ec88750ca 13212 RUNNING f6b41bda738abe21da774d8d41b9c2efefbe6a3c8dcadfe964aa8968d686e014 13055 RUNNING 4846ae9e043e09fc52afbf16993565b673b0ac9d784b35db8a96d2465bf5f80a 13592 RUNNING bfcb3a82b165797c2f2924e41115b47c87b03ebf8f7b66db9ffcfc92ed23b594 13240 RUNNING 85cb81ac22a93d313036969af166533fa7fc2e31f3f5be0f787dbb7ca4a0e830 14004 RUNNING ubuntu@k3s-agent:~$ sudo ctr task exec -t --exec-id 13143 314c23ec540da5364e845afd7fc1072eb0502e58089101dedc1c433c7e405e0c sh / # exit

24. © Copyright 2020 Rancher Labs. All Rights Reserved. 26 demo

    - containerdのオペレーション • Tag処理はcrictlのみ可能 ubuntu@k3s-agent:~$ sudo crictl images IMAGE TAG IMAGE ID SIZE docker.io/rancher/coredns-coredns 1.6.9 4e797b3234604 13.4MB docker.io/rancher/klipper-helm v0.2.7 1087ccbd2ab61 50MB docker.io/rancher/klipper-lb v0.1.2 897ce3c5fc8ff 2.71MB docker.io/rancher/library-traefik 1.7.19 aa764f7db3051 24MB docker.io/rancher/local-path-provisioner v0.0.11 9d12f9848b99f 12MB docker.io/rancher/metrics-server v0.3.6 9dd718864ce61 10.5MB docker.io/rancher/pause 3.1 da86e6ba6ca19 327kB ubuntu@k3s-agent:~$ sudo ctr images tag docker.io/rancher/coredns-coredns:1.6.9 wenhan/coredns:1.6.9 wenhan/coredns:1.6.9 ubuntu@k3s-agent:~$ sudo crictl images IMAGE TAG IMAGE ID SIZE docker.io/rancher/coredns-coredns 1.6.9 4e797b3234604 13.4MB docker.io/wenhan/coredns 1.6.9 4e797b3234604 13.4MB docker.io/rancher/klipper-helm v0.2.7 1087ccbd2ab61 50MB docker.io/rancher/klipper-lb v0.1.2 897ce3c5fc8ff 2.71MB docker.io/rancher/library-traefik 1.7.19 aa764f7db3051 24MB docker.io/rancher/local-path-provisioner v0.0.11 9d12f9848b99f 12MB docker.io/rancher/metrics-server v0.3.6 9dd718864ce61 10.5MB docker.io/rancher/pause 3.1 da86e6ba6ca19 327kB

25. © Copyright 2020 Rancher Labs. All Rights Reserved. 27 demo

    - containerdのオペレーション • containerdのlogはk3sのフォルダ以下に存在 ubuntu@k3s-agent:~$ sudo tail /var/lib/rancher/k3s/agent/containerd/containerd.log time="2020-10-08T00:41:18.368390455+09:00" level=info msg="CreateContainer within sandbox \"fa62b7735c1bc89218d57236c820b5e279abdd59bc6ccb6d9f194673c01513e1\" for container &ContainerMetadata{Name:traefik,Attempt:0,}" time="2020-10-08T00:41:19.265805266+09:00" level=info msg="CreateContainer within sandbox \"fa62b7735c1bc89218d57236c820b5e279abdd59bc6ccb6d9f194673c01513e1\" for &ContainerMetadata{Name:traefik,Attempt:0,} returns container id \"85cb81ac22a93d313036969af166533fa7fc2e31f3f5be0f787dbb7ca4a0e830\"" time="2020-10-08T00:41:19.266393179+09:00" level=info msg="StartContainer for \"85cb81ac22a93d313036969af166533fa7fc2e31f3f5be0f787dbb7ca4a0e830\"" time="2020-10-08T00:41:19.342298540+09:00" level=info msg="StartContainer for \"85cb81ac22a93d313036969af166533fa7fc2e31f3f5be0f787dbb7ca4a0e830\" returns successfully" time="2020-10-08T11:31:32.316393354+09:00" level=info msg="Exec for \"314c\" with command [sh], tty true and stdin true" time="2020-10-08T11:31:32.316465861+09:00" level=info msg="Exec for \"314c\" returns URL \"http://127.0.0.1:10010/exec/SvSBzlmg\"" time="2020-10-08T11:31:34.310365484+09:00" level=info msg="Exec process \"0a3dfc129bc732267fdde08d816e894c82a591d402d85cad705e1ec705a9f0f9\" exits with exit code 0 and error <nil>" time="2020-10-08T11:31:34.313760096+09:00" level=info msg="Finish piping \"stdout\" of container exec \"0a3dfc129bc732267fdde08d816e894c82a591d402d85cad705e1ec705a9f0f9\"" time="2020-10-08T11:31:34.314519873+09:00" level=info msg="Container exec \"0a3dfc129bc732267fdde08d816e894c82a591d402d85cad705e1ec705a9f0f9\" stdin closed" time="2020-10-08T11:42:57.585883696+09:00" level=info msg="ImageCreate event &ImageCreate{Name:wenhan/coredns:1.6.9,Labels:map[string]string{io.cri-containerd.image: managed,},XXX_unrecognized:[],}"

26. © Copyright 2020 Rancher Labs. All Rights Reserved. 28 demo

    - containerdの設定 • containerdの設定ファイルは /var/lib/rancher/k3s/agent/etc/containerd/config.toml ubuntu@k3s-agent:~$ sudo cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml [plugins.opt] path = "/var/lib/rancher/k3s/agent/containerd" [plugins.cri] stream_server_address = "127.0.0.1" stream_server_port = "10010" enable_selinux = false sandbox_image = "docker.io/rancher/pause:3.1" [plugins.cri.cni] bin_dir = "/var/lib/rancher/k3s/data/688c8ca42a6cd0c042322efea271d6f3849d3de17c850739b0da2461f6c69ee8/bin" conf_dir = "/var/lib/rancher/k3s/agent/etc/cni/net.d" [plugins.cri.containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2"

27. © Copyright 2020 Rancher Labs. All Rights Reserved. 29 demo

    - containerdの設定 • 設定変更は、まずテンプレートを作成し、設定ファイルがこれに基づいて⽣成される。 ubuntu@k3s-agent:~$ sudo cp /var/lib/rancher/k3s/agent/etc/containerd/config.toml /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl ubuntu@k3s-agent:~$ sudo ls /var/lib/rancher/k3s/agent/etc/containerd/ config.toml config.toml.tmpl

28. © Copyright 2020 Rancher Labs. All Rights Reserved. 30 demo

    - containerdの設定 • テンプレートファイルを変更 root@k3s-agent:~# vim /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl root@k3s-agent:~# cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl [plugins.opt] path = "/var/lib/rancher/k3s/agent/containerd" [plugins.cri] stream_server_address = "127.0.0.1" stream_server_port = "10010" enable_selinux = false sandbox_image = "docker.io/rancher/pause:3.1" [plugins.cri.cni] bin_dir = "/var/lib/rancher/k3s/data/688c8ca42a6cd0c042322efea271d6f3849d3de17c850739b0da2461f6c69ee8/bin" conf_dir = "/var/lib/rancher/k3s/agent/etc/cni/net.d" [plugins.cri.containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2" # metrics configuration [metrics] # tcp address! address = "127.0.0.1:1234"

29. © Copyright 2020 Rancher Labs. All Rights Reserved. 31 demo

    - containerdの設定 • agentを再起動し、config.tomlが変更されたことを確認 root@k3s-agent:~# systemctl restart k3s-agent root@k3s-agent:~# cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml [plugins.opt] path = "/var/lib/rancher/k3s/agent/containerd" [plugins.cri] stream_server_address = "127.0.0.1" stream_server_port = "10010" enable_selinux = false sandbox_image = "docker.io/rancher/pause:3.1" [plugins.cri.cni] bin_dir = "/var/lib/rancher/k3s/data/688c8ca42a6cd0c042322efea271d6f3849d3de17c850739b0da2461f6c69ee8/bin" conf_dir = "/var/lib/rancher/k3s/agent/etc/cni/net.d" [plugins.cri.containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2" # metrics configuration [metrics] # tcp address! address = "127.0.0.1:1234"

30. © Copyright 2020 Rancher Labs. All Rights Reserved. 32 Containerdのリポジトリの変更

    • プライベート環境のk3sを利⽤する時、プライベートのRegistryの設定が必要 • https://github.com/containerd/cri/blob/master/docs/registry.md

31. © Copyright 2020 Rancher Labs. All Rights Reserved. 33 Containerdのリポジトリの変更

    • Internal Networkのみの利⽤で、tls certを使わずHTTPでリポジトリが構築したい場合、 • /etc/rancher/k3s/registries.yamlを新規作成すれば、簡単に設定できる root@k3s-agent:~# cat /etc/rancher/k3s/registries.yaml mirrors: "172.31.7.129:5000": endpoint: - "http://172.31.7.129:5000" root@k3s-agent:~# systemctl restart k3s-agent root@k3s-agent:~# cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml <snip> [plugins.cri.containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2" # metrics configuration [metrics] # tcp address! address = "127.0.0.1:1234" [plugins.cri.registry.mirrors] [plugins.cri.registry.mirrors."172.31.7.129:5000"] endpoint = ["http://172.31.7.129:5000"]

32. © Copyright 2020 Rancher Labs. All Rights Reserved. 34 34

    次回予告 - k３s HA

33. © Copyright 2020 Rancher Labs. All Rights Reserved. 35 シングルノード構成

34. © Copyright 2020 Rancher Labs. All Rights Reserved. 36 マルチノード構成（HA構成）

35. © Copyright 2020 Rancher Labs. All Rights Reserved. 37 HA構成

    - ServerのIPアドレス • L4 LoadBalancer • Round-robin DNS • VIP or Elastic IP Address

36. © Copyright 2020 Rancher Labs. All Rights Reserved. 38 Thank

    you, Questions?