$30 off During Our Annual Pro Sale. View Details »

BYOK - Build your Own Kubernetes Cluster with Raspberry Pis, k3s, and k3sup

BYOK - Build your Own Kubernetes Cluster with Raspberry Pis, k3s, and k3sup

Rancher’s k3s was designed to be a lightweight distro of Kubernetes to support IoT and edge type devices. However, it’s gained a lot of traction everywhere! We’ll talk about it, but see how easy it is to install and setup a Raspberry Pi cluster using k3sup, a utility created by Alex Ellis (creator of OpenFaaS). We’ll talk about Kubernetes primitives and deploy a few demo apps. We'll then see how it easy it is to install other apps using k3sup, including OpenFaaS! Be prepared for lots of fun and demos!

Michael Irwin

January 09, 2020

More Decks by Michael Irwin

Other Decks in Technology


  1. BYOK - Build Your Own Kubernetes Cluster with Raspberry Pis,

    k3s, and k3sup Michael Irwin Jan 9, 2020
  2. @mikesir87 About Michael Irwin • Full-time at VT since 2011

    (℅ 2011!) ◦ Developer, but I wear lots of hats ◦ Office of the CTO, working on several development and skunkwork projects ◦ Adjunct Faculty Instructor in CS Dept • Recognized Docker Captain and Community Leader • Follow me at @mikesir87
  3. @mikesir87 Raspberry Pi

  4. @mikesir87 Raspberry Pi 4

  5. @mikesir87 What can you do with them? • Replace desktop

    • Print Server • Home automation • Media center • DNS Filter • Gaming console • Run a webserver • Motion capture security system • Digital photo frames • Smart mirrors • NAS Box
  6. @mikesir87 And run a Kubernetes cluster!

  7. @mikesir87 My Cluster’s Parts • 4x - Raspberry Pi 4

    - 4GB ($55 each) • 4x - PoE Hats ($20 each) • 4x - Samsung EVO Plus microSD - 32GB ($11 each) • 1x - TP-Link 5-Port Gigabit Ethernet Switch w/ PoE ($50) • 1x - 6 Pack of 1-ft Cat6 Ethernet Cables ($9) • 1x - Yahboom Stackable Cluster case ($19) Total Cost: ~$422
  8. @mikesir87 Setting up each Pi - Before turning it on

    • Flashed Raspian onto each SD card ◦ The Etcher tool made this super easy (balena.io/etcher/) • Gave each a static IP (ip= in /boot/cmdline.txt) • Enable SSH by default (create /boot/ssh file)
  9. @mikesir87 Setting up each Pi - After boot • Install

    updates (which updates firmware too) • Configure SSH key authorization • Change default password for pi user • Setup Prometheus-based monitoring ◦ Install node_exporter and systemd service ◦ Install rpi_exporter and service to get temperature readings
  10. @mikesir87 Network Setup pi0 pi1 pi2 pi3 Ethernet Switch

    Upstream Network wlan
  11. @mikesir87 Setting up the NAT (pi0) • Enable packet forwarding

    ◦ Uncomment net.ipv4.ip_forward=1 in /etc/sysctl.conf • Install dnsmasq (apt-get install dnsmasq) • Configure wireless connections in /etc/wpa_supplicant/wpa_supplicant.conf network={ ssid="IrwinsAreUs" psk="***" id_str="IrwinsAreUs" }
  12. @mikesir87 More pi0 Setup • Install Docker • Start a

    few services ◦ Prometheus - scrape and store metrics for all nodes ◦ Proxy - HAProxy to forward 80/443 to pi1-pi3
  13. @mikesir87 Setting up pi1-pi3 • Set eth0 interface config in

    /etc/dhcpcd.conf interface eth0 static ip_address= static routers=
  14. @mikesir87 Now the fun part!

  15. @mikesir87 Kubernetes Kubernetes (K8s) is an open-source system for automating

    deployment, scaling, and management of containerized applications. Comes from 15 years of Google experience running workloads
  16. @mikesir87

  17. @mikesir87

  18. @mikesir87 k3s • Lightweight certified Kubernetes distro made by Rancher

    • Designed and optimized for IoT/Edge and ARM • Packaged into a single < 40MB binary (can work air-gapped) • Bundled with a few components (Flannel, Traefik ingress, Local Path Provisioner)
  19. @mikesir87 k3s Architecture

  20. @mikesir87 k3sup • Tool written by Alex Ellis that gives

    the UX of swarm cluster creation/joining • “k3sup is a light-weight utility to get from zero to KUBECONFIG with k3s on any local or remote VM. All you need is ssh access and the k3sup binary to get kubectl access immediately” - https://github.com/alexellis/k3sup
  21. @mikesir87 Setting up cluster master k3sup install --ip $IP --user

    pi \ --local-path ~/.kube/config --merge \ --context=pi-cluster • Login to $IP as user pi and install k3s • Gets the kube config and merges into ~/.kube/config with the context pi-cluster
  22. @mikesir87 Joining Nodes to the Cluster k3sup join --ip $IP

    --user pi \ --server-ip $SERVER_IP • Login to machine $IP with user pi and install k3s • Join the node as an agent to the cluster with master at $SERVER_IP
  23. @mikesir87 Kubectl • The default CLI used to manage/interact with

    a Kubernetes cluster • Works much like the Docker CLI… it’s merely a CLI-based client that interacts with the cluster API
  24. @mikesir87 Let’s build our cluster!

  25. @mikesir87 We have a running k3s cluster!

  26. @mikesir87 Some Kubernetes terminology...

  27. @mikesir87 Pods • Pods are the smallest deployable units of

    computing that can be created and managed in Kubernetes • Defines one or containers that are run together ◦ Scheduled to run on the same node ◦ Share namespaces and network space ◦ Each pod gets its own IP address • Can start one using the kubectl run command
  28. @mikesir87 ReplicaSets • Maintains a stable set of replica Pods

    running at any given time • Often used to guarantee the availability of a specified number of identical Pods • If you say you want two running, it ensures two are running
  29. @mikesir87 Deployments • Provides declarative updates for Pods and ReplicaSets

    • Allows you to set a desired state and it changes the actual state to the desired state at a controlled rate
  30. @mikesir87 Services • Provides the ability to register pods for

    service discovery/management • Services select the pods they attach to use selectors (think tags) • All services are discoverable via DNS within the same namespace
  31. @mikesir87 Custom Resources (CRDs) • Custom resources/extensions of the Kubernetes

    API • Examples might include ◦ Ingress route handling ◦ Message queues/topics ◦ Database instances ◦ Whatever else you can think of!
  32. @mikesir87 Traefik Ingress Controller • An Ingress controller that provides

    ability to route requests as they come into the cluster • Traefik listens on 80/443 and then uses config from the cluster to route the request • Define routes using IngressRoute objects (a CRD)
  33. @mikesir87 Applying Changes • All changes can be done directly

    via the CLI • Preferred route is via configuration files kubectl apply -f my-file.yml
  34. @mikesir87 Let’s play!

  35. @mikesir87 OpenFaaS

  36. @mikesir87 Installing OpenFaaS With k3sup, it’s as simple as: k3sup

    app install openfaas
  37. @mikesir87 Let’s try it out!

  38. @mikesir87 Thanks! Questions?