BYOK - Build your Own Kubernetes Cluster with Raspberry Pis, k3s, and k3sup

Slide 1

Slide 1 text

BYOK - Build Your Own Kubernetes Cluster with Raspberry Pis, k3s, and k3sup Michael Irwin Jan 9, 2020

Slide 2

Slide 2 text

@mikesir87 About Michael Irwin ● Full-time at VT since 2011 (℅ 2011!) ○ Developer, but I wear lots of hats ○ Oﬃce of the CTO, working on several development and skunkwork projects ○ Adjunct Faculty Instructor in CS Dept ● Recognized Docker Captain and Community Leader ● Follow me at @mikesir87

Slide 3

Slide 3 text

@mikesir87 Raspberry Pi

Slide 4

Slide 4 text

@mikesir87 Raspberry Pi 4

Slide 5

Slide 5 text

@mikesir87 What can you do with them? ● Replace desktop ● Print Server ● Home automation ● Media center ● DNS Filter ● Gaming console ● Run a webserver ● Motion capture security system ● Digital photo frames ● Smart mirrors ● NAS Box

Slide 6

Slide 6 text

@mikesir87 And run a Kubernetes cluster!

Slide 7

Slide 7 text

@mikesir87 My Cluster’s Parts ● 4x - Raspberry Pi 4 - 4GB ($55 each) ● 4x - PoE Hats ($20 each) ● 4x - Samsung EVO Plus microSD - 32GB ($11 each) ● 1x - TP-Link 5-Port Gigabit Ethernet Switch w/ PoE ($50) ● 1x - 6 Pack of 1-ft Cat6 Ethernet Cables ($9) ● 1x - Yahboom Stackable Cluster case ($19) Total Cost: ~$422

Slide 8

Slide 8 text

@mikesir87 Setting up each Pi - Before turning it on ● Flashed Raspian onto each SD card ○ The Etcher tool made this super easy (balena.io/etcher/) ● Gave each a static IP (ip=192.168.212.101 in /boot/cmdline.txt) ● Enable SSH by default (create /boot/ssh ﬁle)

Slide 9

Slide 9 text

@mikesir87 Setting up each Pi - After boot ● Install updates (which updates ﬁrmware too) ● Conﬁgure SSH key authorization ● Change default password for pi user ● Setup Prometheus-based monitoring ○ Install node_exporter and systemd service ○ Install rpi_exporter and service to get temperature readings

Slide 10

Slide 10 text

@mikesir87 Network Setup 192.168.212.0/24 pi0 pi1 pi2 pi3 Ethernet Switch Upstream Network wlan

Slide 11

Slide 11 text

@mikesir87 Setting up the NAT (pi0) ● Enable packet forwarding ○ Uncomment net.ipv4.ip_forward=1 in /etc/sysctl.conf ● Install dnsmasq (apt-get install dnsmasq) ● Conﬁgure wireless connections in /etc/wpa_supplicant/wpa_supplicant.conf network={ ssid="IrwinsAreUs" psk="***" id_str="IrwinsAreUs" }

Slide 12

Slide 12 text

@mikesir87 More pi0 Setup ● Install Docker ● Start a few services ○ Prometheus - scrape and store metrics for all nodes ○ Proxy - HAProxy to forward 80/443 to pi1-pi3

Slide 13

Slide 13 text

@mikesir87 Setting up pi1-pi3 ● Set eth0 interface conﬁg in /etc/dhcpcd.conf interface eth0 static ip_address=192.168.212.101 static routers=192.168.212.1

Slide 14

Slide 14 text

@mikesir87 Now the fun part!

Slide 15

Slide 15 text

@mikesir87 Kubernetes Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications. Comes from 15 years of Google experience running workloads

Slide 16

Slide 16 text

@mikesir87

Slide 17

Slide 17 text

@mikesir87

Slide 18

Slide 18 text

@mikesir87 k3s ● Lightweight certiﬁed Kubernetes distro made by Rancher ● Designed and optimized for IoT/Edge and ARM ● Packaged into a single < 40MB binary (can work air-gapped) ● Bundled with a few components (Flannel, Traeﬁk ingress, Local Path Provisioner)

Slide 19

Slide 19 text

@mikesir87 k3s Architecture

Slide 20

Slide 20 text

@mikesir87 k3sup ● Tool written by Alex Ellis that gives the UX of swarm cluster creation/joining ● “k3sup is a light-weight utility to get from zero to KUBECONFIG with k3s on any local or remote VM. All you need is ssh access and the k3sup binary to get kubectl access immediately” - https://github.com/alexellis/k3sup

Slide 21

Slide 21 text

@mikesir87 Setting up cluster master k3sup install --ip $IP --user pi \ --local-path ~/.kube/config --merge \ --context=pi-cluster ● Login to $IP as user pi and install k3s ● Gets the kube conﬁg and merges into ~/.kube/conﬁg with the context pi-cluster

Slide 22

Slide 22 text

@mikesir87 Joining Nodes to the Cluster k3sup join --ip $IP --user pi \ --server-ip $SERVER_IP ● Login to machine $IP with user pi and install k3s ● Join the node as an agent to the cluster with master at $SERVER_IP

Slide 23

Slide 23 text

@mikesir87 Kubectl ● The default CLI used to manage/interact with a Kubernetes cluster ● Works much like the Docker CLI… it’s merely a CLI-based client that interacts with the cluster API

Slide 24

Slide 24 text

@mikesir87 Let’s build our cluster!

Slide 25

Slide 25 text

@mikesir87 We have a running k3s cluster!

Slide 26

Slide 26 text

@mikesir87 Some Kubernetes terminology...

Slide 27

Slide 27 text

@mikesir87 Pods ● Pods are the smallest deployable units of computing that can be created and managed in Kubernetes ● Deﬁnes one or containers that are run together ○ Scheduled to run on the same node ○ Share namespaces and network space ○ Each pod gets its own IP address ● Can start one using the kubectl run command

Slide 28

Slide 28 text

@mikesir87 ReplicaSets ● Maintains a stable set of replica Pods running at any given time ● Often used to guarantee the availability of a speciﬁed number of identical Pods ● If you say you want two running, it ensures two are running

Slide 29

Slide 29 text

@mikesir87 Deployments ● Provides declarative updates for Pods and ReplicaSets ● Allows you to set a desired state and it changes the actual state to the desired state at a controlled rate

Slide 30

Slide 30 text

@mikesir87 Services ● Provides the ability to register pods for service discovery/management ● Services select the pods they attach to use selectors (think tags) ● All services are discoverable via DNS within the same namespace

Slide 31

Slide 31 text

@mikesir87 Custom Resources (CRDs) ● Custom resources/extensions of the Kubernetes API ● Examples might include ○ Ingress route handling ○ Message queues/topics ○ Database instances ○ Whatever else you can think of!

Slide 32

Slide 32 text

@mikesir87 Traefik Ingress Controller ● An Ingress controller that provides ability to route requests as they come into the cluster ● Traefik listens on 80/443 and then uses config from the cluster to route the request ● Define routes using IngressRoute objects (a CRD)

Slide 33

Slide 33 text

@mikesir87 Applying Changes ● All changes can be done directly via the CLI ● Preferred route is via conﬁguration ﬁles kubectl apply -f my-file.yml

Slide 34

Slide 34 text

@mikesir87 Let’s play!

Slide 35

Slide 35 text

@mikesir87 OpenFaaS

Slide 36

Slide 36 text

@mikesir87 Installing OpenFaaS With k3sup, it’s as simple as: k3sup app install openfaas