BYOK - Build your Own Kubernetes Cluster with Raspberry Pis, k3s, and k3sup

BYOK - Build Your Own Kubernetes Cluster with Raspberry Pis,
k3s, and k3sup Michael Irwin Jan 9, 2020

@mikesir87 About Michael Irwin • Full-time at VT since 2011
(℅ 2011!) ◦ Developer, but I wear lots of hats ◦ Oﬃce of the CTO, working on several development and skunkwork projects ◦ Adjunct Faculty Instructor in CS Dept • Recognized Docker Captain and Community Leader • Follow me at @mikesir87

@mikesir87 Raspberry Pi

@mikesir87 Raspberry Pi 4

@mikesir87 What can you do with them? • Replace desktop
• Print Server • Home automation • Media center • DNS Filter • Gaming console • Run a webserver • Motion capture security system • Digital photo frames • Smart mirrors • NAS Box

@mikesir87 And run a Kubernetes cluster!

@mikesir87 My Cluster’s Parts • 4x - Raspberry Pi 4
- 4GB ($55 each) • 4x - PoE Hats ($20 each) • 4x - Samsung EVO Plus microSD - 32GB ($11 each) • 1x - TP-Link 5-Port Gigabit Ethernet Switch w/ PoE ($50) • 1x - 6 Pack of 1-ft Cat6 Ethernet Cables ($9) • 1x - Yahboom Stackable Cluster case ($19) Total Cost: ~$422

@mikesir87 Setting up each Pi - Before turning it on
• Flashed Raspian onto each SD card ◦ The Etcher tool made this super easy (balena.io/etcher/) • Gave each a static IP (ip=192.168.212.101 in /boot/cmdline.txt) • Enable SSH by default (create /boot/ssh ﬁle)

@mikesir87 Setting up each Pi - After boot • Install
updates (which updates ﬁrmware too) • Conﬁgure SSH key authorization • Change default password for pi user • Setup Prometheus-based monitoring ◦ Install node_exporter and systemd service ◦ Install rpi_exporter and service to get temperature readings

@mikesir87 Network Setup 192.168.212.0/24 pi0 pi1 pi2 pi3 Ethernet Switch
Upstream Network wlan

@mikesir87 Setting up the NAT (pi0) • Enable packet forwarding
◦ Uncomment net.ipv4.ip_forward=1 in /etc/sysctl.conf • Install dnsmasq (apt-get install dnsmasq) • Conﬁgure wireless connections in /etc/wpa_supplicant/wpa_supplicant.conf network={ ssid="IrwinsAreUs" psk="***" id_str="IrwinsAreUs" }

@mikesir87 More pi0 Setup • Install Docker • Start a
few services ◦ Prometheus - scrape and store metrics for all nodes ◦ Proxy - HAProxy to forward 80/443 to pi1-pi3

@mikesir87 Setting up pi1-pi3 • Set eth0 interface conﬁg in
/etc/dhcpcd.conf interface eth0 static ip_address=192.168.212.101 static routers=192.168.212.1

@mikesir87 Now the fun part!

@mikesir87 Kubernetes Kubernetes (K8s) is an open-source system for automating
deployment, scaling, and management of containerized applications. Comes from 15 years of Google experience running workloads

@mikesir87

@mikesir87 k3s • Lightweight certiﬁed Kubernetes distro made by Rancher
• Designed and optimized for IoT/Edge and ARM • Packaged into a single < 40MB binary (can work air-gapped) • Bundled with a few components (Flannel, Traeﬁk ingress, Local Path Provisioner)

@mikesir87 k3s Architecture

@mikesir87 k3sup • Tool written by Alex Ellis that gives
the UX of swarm cluster creation/joining • “k3sup is a light-weight utility to get from zero to KUBECONFIG with k3s on any local or remote VM. All you need is ssh access and the k3sup binary to get kubectl access immediately” - https://github.com/alexellis/k3sup

@mikesir87 Setting up cluster master k3sup install --ip $IP --user
pi \ --local-path ~/.kube/config --merge \ --context=pi-cluster • Login to $IP as user pi and install k3s • Gets the kube conﬁg and merges into ~/.kube/conﬁg with the context pi-cluster

@mikesir87 Joining Nodes to the Cluster k3sup join --ip $IP
--user pi \ --server-ip $SERVER_IP • Login to machine $IP with user pi and install k3s • Join the node as an agent to the cluster with master at $SERVER_IP

@mikesir87 Kubectl • The default CLI used to manage/interact with
a Kubernetes cluster • Works much like the Docker CLI… it’s merely a CLI-based client that interacts with the cluster API

@mikesir87 Let’s build our cluster!

@mikesir87 We have a running k3s cluster!

@mikesir87 Some Kubernetes terminology...

@mikesir87 Pods • Pods are the smallest deployable units of
computing that can be created and managed in Kubernetes • Deﬁnes one or containers that are run together ◦ Scheduled to run on the same node ◦ Share namespaces and network space ◦ Each pod gets its own IP address • Can start one using the kubectl run command

@mikesir87 ReplicaSets • Maintains a stable set of replica Pods
running at any given time • Often used to guarantee the availability of a speciﬁed number of identical Pods • If you say you want two running, it ensures two are running

@mikesir87 Deployments • Provides declarative updates for Pods and ReplicaSets
• Allows you to set a desired state and it changes the actual state to the desired state at a controlled rate

@mikesir87 Services • Provides the ability to register pods for
service discovery/management • Services select the pods they attach to use selectors (think tags) • All services are discoverable via DNS within the same namespace

@mikesir87 Custom Resources (CRDs) • Custom resources/extensions of the Kubernetes
API • Examples might include ◦ Ingress route handling ◦ Message queues/topics ◦ Database instances ◦ Whatever else you can think of!

@mikesir87 Traefik Ingress Controller • An Ingress controller that provides
ability to route requests as they come into the cluster • Traefik listens on 80/443 and then uses config from the cluster to route the request • Define routes using IngressRoute objects (a CRD)

@mikesir87 Applying Changes • All changes can be done directly
via the CLI • Preferred route is via conﬁguration ﬁles kubectl apply -f my-file.yml

@mikesir87 Let’s play!

@mikesir87 OpenFaaS

@mikesir87 Installing OpenFaaS With k3sup, it’s as simple as: k3sup
app install openfaas

@mikesir87 Let’s try it out!

@mikesir87 Thanks! Questions?

BYOK - Build your Own Kubernetes Cluster with Raspberry Pis, k3s, and k3sup

BYOK - Build your Own Kubernetes Cluster with Raspberry Pis, k3s, and k3sup

More Decks by Michael Irwin

Other Decks in Technology

Featured

Transcript