Tweet
Tweet

Slide 1

Slide 1 text

Legacy Applications on Kubernetes Joshua Timberman @jtimberman

Slide 2

Slide 2 text

https://commons.wikimedia.org/wiki/File:BalticServers_data_center.jpg % whoami

Slide 3

Slide 3 text

Why are we here?

Slide 4

Slide 4 text

Running Legacy Applications on Kubernetes

Slide 5

Slide 5 text

Running Legacy Heritage Applications on Kubernetes

Slide 6

Slide 6 text

What is a Legacy Application? In computing, a legacy application program of, relating to, or being a previous or outdated computer system, yet still in use. Often referencing a system as "legacy" means that it paved the way for the standards that would follow it. This can also imply that the system is out of date or in need of replacement.

Slide 7

Slide 7 text

Chef Ops runs many applications

Slide 8

Slide 8 text

Applications, sample set • Hosted Chef (api.chef.io) • Downloads (downloads.chef.io) • Omnitruck (omnitruck.chef.io) • Learn Chef (learn.chef.io) • Chef Community Cookbooks (supermarket.chef.io) • Internal and External utilities • "Gateway" and "glue" services • Kubernetes itself ;) • etc...

Slide 9

Slide 9 text

Workflow Matters

Slide 10

Slide 10 text

Why Kubernetes?

Slide 11

Slide 11 text

Cloud Native

Slide 12

Slide 12 text

Platform Consistency

Slide 13

Slide 13 text

Cohesion

Slide 14

Slide 14 text

Cost-savings https://en.wikipedia.org/wiki/Denomination_effect#/media/File:Exchange_Money_Conversion_to_Foreign_Currency.jpg

Slide 15

Slide 15 text

https://commons.wikimedia.org/wiki/File:Frozen_peas.JPG

Slide 16

Slide 16 text

https://commons.wikimedia.org/wiki/File:Mashed_potatoes_(1).jpg

Slide 17

Slide 17 text

What about Chef?

Slide 18

Slide 18 text

As it turns out... https://www.flickr.com/photos/tofu_mugwump/22420266811 https://creativecommons.org/licenses/by/2.0/

Slide 19

Slide 19 text

We still have an OS.

Slide 20

Slide 20 text

But we use K8SaaS! https://www.jpl.nasa.gov/news/news.php?feature=7113

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

Policyfiles https://en.wikipedia.org/wiki/Ledger#/media/File:Ledger.png

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

Test Kitchen (for cookbooks) https://commons.wikimedia.org/wiki/File:Conference_House_-_Rustic_Kitchen.jpg

Slide 25

Slide 25 text

DevSec Linux Baseline https://dev-sec.io/baselines/linux/

Slide 26

Slide 26 text

CIS AWS Benchmark https://commons.wikimedia.org/wiki/File:Benchmark_at_former_Abbey_Cinema.jpg

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

Ruby Scaffolding https://commons.wikimedia.org/wiki/File:Seattle_- _construction_at_SE_corner_of_11th_%26_E_Pine_01.jpg

Slide 29

Slide 29 text

core-plans/scaffolding-base core-plans/scaffolding-chef core-plans/scaffolding-go core-plans/scaffolding-go17 core-plans/scaffolding-gradle core-plans/scaffolding-inspec core-plans/scaffolding-node core-plans/scaffolding-ruby scaffolding plans

Slide 30

Slide 30 text

Effortless Config

Slide 31

Slide 31 text

How did we get here?

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

https://flic.kr/p/bFe9Bs

Slide 34

Slide 34 text

Kubernetes is a portable, extensible open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation.

Slide 35

Slide 35 text

Tl;dr: Docker as a Service https://libreshot.com/container-ship-freighter/

Slide 36

Slide 36 text

YAML* * The "A" is optional, though.

Slide 37

Slide 37 text

kops - "Kubernetes Operations"

Slide 38

Slide 38 text

apiVersion: kops/v1alpha2 kind: InstanceGroup metadata: labels: kops.k8s.io/cluster: kubernetes.chef-internal name: nodes spec: additionalUserData: - content: | #!/bin/sh - installs, configures, and runs chef name: chefbootstrap.sh type: text/x-shellscript cloudLabels: X-Application: kubernetes X-Contact: jtimberman X-Dept: Operations X-Environment: production image: kope.io/k8s-1.8-debian-jessie-amd64-hvm-ebs-2018-02-08 machineType: m4.xlarge maxSize: 6 minSize: 4 nodeLabels: kops.k8s.io/instancegroup: nodes role: Node subnets: - us-west-2a Kops nodes and masters configuration

Slide 39

Slide 39 text

#!/bin/bash pkg_origin=$1 pkg_name=$2 if [ ! -e "/bin/hab" ]; then curl https://raw.githubusercontent.com/habitat-sh/habitat/master/components/hab/install.sh | bash fi [[ `grep "^hab:" /etc/passwd` ]] || useradd hab [[ `grep "^hab:" /etc/group `]] || groupadd hab hab pkg install $pkg_origin/$pkg_name pkg_prefix=$(find /hab/pkgs/$pkg_origin/$pkg_name -maxdepth 2 -mindepth 2 | sort | tail -n 1) hab pkg exec $pkg_origin/$pkg_name chef-client -z -c $pkg_prefix/config/bootstrap-config.rb chefbootstrap.sh - Effortless Config!

Slide 40

Slide 40 text

k8s-node Chef Infra Policyfile • User management (LDAP, SSH Keys) • Centralized Logging • Send InSpec profile data to Chef Automate • Keeps Chef Infra package updated • Ensure Chef Infra is running as a service

Slide 41

Slide 41 text

CI / CD

Slide 42

Slide 42 text

== CI SaaS...

Slide 43

Slide 43 text

Expeditor

Slide 44

Slide 44 text

Expeditor listens to webhooks and takes actions

Slide 45

Slide 45 text

pipelines: - verify: description: Pull Request validation tests - habitat/build: description: Build Habitat package - deploy/acceptance: description: Deploy changes to Kubernetes definition: .expeditor/deploy.pipeline.yml env: - ENVIRONMENT: acceptance - APP: APPNAME - deploy/production: description: Deploy changes to Kubernetes definition: .expeditor/deploy.pipeline.yml env: - ENVIRONMENT: production - APP: APPNAME .expeditor/config.yml: pipelines

Slide 46

Slide 46 text

YAML INTENSIFIES

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

Build Habitat Packages

Slide 49

Slide 49 text

jtimberman@localhost% hab studio enter ... [1][default:/src:0]# build ... learnchef-backend: learnchef-backend: I love it when a plan.sh comes together. learnchef-backend: learnchef-backend: Build time: 2m24s Build Habitat Packages

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Publish Docker Images

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

pipelines: - verify: description: Pull Request validation tests - deploy/acceptance: description: Deploy changes to kubernetes definition: .expeditor/deploy.pipeline.yml env: - ENVIRONMENT: acceptance - APP: APPNAME - deploy/production: description: Deploy changes to kubernetes definition: .expeditor/deploy.pipeline.yml env: - ENVIRONMENT: production - APP: APPNAME Buildkite Deploys to Kubernetes

Slide 54

Slide 54 text

SECRETS??!! https://commons.wikimedia.org/wiki/File:WinonaSavingsBankVault.JPG

Slide 55

Slide 55 text

Secrets Management • SSL certificates • API tokens • SSH keys • Configuration files • Repositories • AWS Accounts • Kubernetes

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

K8s Secrets / ConfigMap https://commons.wikimedia.org/wiki/File:World_map_.jpg

Slide 58

Slide 58 text

Application Migration https://www.maxpixel.net/Fly-Flock-Of-Birds-Crane-Bird-Migration-2147788

Slide 59

Slide 59 text

Habitat Package

Slide 60

Slide 60 text

pkg_name=APPNAME pkg_origin=chefops pkg_version=0.1.0 pkg_maintainer="Chef Operations " pkg_license=('Apache-2.0') pkg_deps=( core/coreutils ) pkg_build_deps=( core/make core/gcc core/tar ) pkg_scaffolding=core/scaffolding-ruby scaffolding_ruby_pkg=core/ruby24/$(cat "$SRC_PATH/.ruby-version") declare -A scaffolding_env scaffolding_env[APPNAME_YAML]="/hab/svc/$pkg_name/config/config.yml" habitat/plan.sh

Slide 61

Slide 61 text

Expeditor & Buildkite

Slide 62

Slide 62 text

Expeditor & Buildkite Configuration • Pipeline definitions • Expeditor's config.yml • Scripts run by pipelines in Buildkite • Kubernetes deployment

Slide 63

Slide 63 text

Kubernetes Configuration

Slide 64

Slide 64 text

#!/bin/bash set -euo pipefail if [[ ! -z ${CI+x} ]]; then mkdir -p ~/.kube aws --profile chef-cd s3 cp s3://chef-cd-citadel/kubernetes.chef.co.config ~/.kube/config else echo "WARN: Not running in Buildkite, assuming local manual deployment" echo "WARN: This requires that ~/.kube/config exists with the proper content" fi export ENVIRONMENT=${ENVIRONMENT:-dev} export APP=${APP} DEBUG=${DEBUG:-false} # This block translates the "environment" into the appropriate Habitat # channel from which to deploy the packages if [ "$ENVIRONMENT" == "acceptance" ]; then export CHANNEL=acceptance elif [ "$ENVIRONMENT" == "production" ]; then export CHANNEL=stable elif [ "$ENVIRONMENT" == "dev" ] || [ "$ENVIRONMENT" == "test" ]; then export CHANNEL=unstable else echo "We do not currently support deploying to $ENVIRONMENT" exit 1 fi # We need the HAB_AUTH_TOKEN set (via Buildkite pipeline) for private packages get_image_tag() { results=$(curl --silent -H "Authorization: Bearer $HAB_AUTH_TOKEN" https://willem.habitat.sh/v1/depot/channels/chefops/${CHANNEL}/pkgs/${APP}/latest | jq '.ident') pkg_version=$(echo "$results" | jq -r .version) pkg_release=$(echo "$results" | jq -r .release) echo "${pkg_version}-${pkg_release}" } # Retrieves the ELB's public DNS name get_elb_hostname() { kubectl get services ${APP}-${ENVIRONMENT} --namespace=${APP} -o json 2>/dev/null | \ jq '.status.loadBalancer.ingress[].hostname' -r } # The ELB isn't ready until the hostname is set, so wait until it's ready wait_for_elb() { attempts=0 max_attempts=10 elb_host="" while [[ $attempts -lt $max_attempts ]]; do elb_host=$(get_elb_hostname || echo) if [[ ! -n $elb_host ]]; then echo "Did not find ELB yet... sleeping 5s" attempts=$[$attempts + 1] sleep 5 else echo "Found ELB: $elb_host" break fi done } # Used for debugging on a local workstation if [[ $DEBUG == "true" ]]; then echo "--- DEBUG: Environment" echo "Application: ${APP}" echo "Channel: ${CHANNEL}" echo "Environment: ${ENVIRONMENT}" echo "Version: $(get_image_tag)" fi echo "--- Applying kubernetes configuration for ${ENVIRONMENT} to cluster" IMAGE_TAG=$(get_image_tag) erb -T- kubernetes/deployment.yml | kubectl apply -f - if [[ `grep -c "^kind: Service$" kubernetes/deployment.yml` -gt 0 ]]; then echo "+++ Waiting for Load Balancer..." wait_for_elb fi kubernetes.sh

Slide 65

Slide 65 text

.expeditor/buildkite/kubernetes.sh • Deploy pipeline in Buildkite runs the script • Copies credentials used for kubectl • Retrieves the Habitat package information from Builder • Applies the kubernetes/deployment.yml manifest • Waits until the ELB is available (if there is one) • Success!

Slide 66

Slide 66 text

Habitat package information • Version • Release • Channel(s) • `get_image_tag`

Slide 67

Slide 67 text

export IMAGE_TAG=$(get_image_tag) erb -T- kubernetes/deployment.yml | kubectl apply -f - kubernetes.sh

Slide 68

Slide 68 text

Kubernetes Resources • Pods • StatefulSets & Deployments • Services • Namespaces • Custom Resource Definitions (CRDs) • (e.g., Habitat operator, Prometheus operator)

Slide 69

Slide 69 text

--- apiVersion: v1 kind: Namespace metadata: name: nginx-demo --- apiVersion: habitat.sh/v1beta1 kind: Habitat metadata: name: nginx-demo-<%= ENV['ENVIRONMENT'] %> namespace: nginx-demo labels: app: nginx-demo-<%= ENV['ENVIRONMENT'] %> customVersion: v1beta2 spec: v1beta2: image: chefops/nginx-demo:<%= ENV['IMAGE_TAG'] %> count: 2 service: name: nginx-demo-<%= ENV['ENVIRONMENT'] %> topology: standalone --- apiVersion: v1 kind: Service metadata: name: nginx-demo-<%= ENV['ENVIRONMENT'] %> namespace: nginx-demo spec: selector: habitat-name: nginx-demo-<%= ENV['ENVIRONMENT'] %> ports: - name: http port: 80 targetPort: 80 protocol: TCP type: LoadBalancer Kubernetes: A Primer via Configuration

Slide 70

Slide 70 text

--- apiVersion: v1 kind: Namespace metadata: name: nginx-demo kubernetes/deployment.yml - Namespace • Namespaces isolate resources in "virtual clusters" • Each application gets a namespace

Slide 71

Slide 71 text

apiVersion: habitat.sh/v1beta1 kind: Habitat metadata: name: nginx-demo-<%= ENV['ENVIRONMENT'] %> namespace: nginx-demo labels: app: nginx-demo-<%= ENV['ENVIRONMENT'] %> customVersion: v1beta2 spec: v1beta2: image: chefops/nginx-demo:<%= ENV['IMAGE_TAG'] %> count: 2 service: name: nginx-demo-<%= ENV['ENVIRONMENT'] %> topology: standalone kubernetes/deployment.yml - Habitat • Applications are deployed using the Habitat Operator • This creates a StatefulSet with two pods • The pods will use the Docker image specified

Slide 72

Slide 72 text

apiVersion: v1 kind: Service metadata: name: nginx-demo-<%= ENV['ENVIRONMENT'] %> namespace: nginx-demo spec: selector: habitat-name: nginx-demo-<%= ENV['ENVIRONMENT'] %> ports: - name: http port: 80 targetPort: 80 protocol: TCP type: LoadBalancer kubernetes/deployment.yml - Service

Slide 73

Slide 73 text

% kubectl get pods -n omnitruck NAME READY STATUS RESTARTS AGE omnitruck-acceptance-0 1/1 Running 0 5d9h omnitruck-acceptance-1 1/1 Running 0 5d9h omnitruck-poller-acceptance-1558471800-hvh45 0/1 Completed 0 31m omnitruck-poller-acceptance-1558472400-d4pps 0/1 Completed 0 21m omnitruck-poller-acceptance-1558473000-dsmcn 0/1 Completed 0 11m omnitruck-poller-acceptance-1558473600-b72lg 1/1 Running 0 112s omnitruck-poller-production-1558471800-rbt7n 0/1 Completed 0 31m omnitruck-poller-production-1558472400-92qkv 0/1 Completed 0 21m omnitruck-poller-production-1558473000-fb6lh 0/1 Completed 0 11m omnitruck-poller-production-1558473600-lw868 1/1 Running 0 111s omnitruck-production-0 1/1 Running 0 5d9h omnitruck-production-1 1/1 Running 0 5d9h omnitruck-production-2 1/1 Running 0 5d9h pods

Slide 74

Slide 74 text

% kubectl get services -n omnitruck NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE omnitruck-acceptance LoadBalancer 100.69.37.56 a260b98a3fcab... 443:32433/TCP 162d omnitruck-production LoadBalancer 100.70.244.59 a92a69a04fcc2... 443:31683/TCP 161d services

Slide 75

Slide 75 text

Stateless Services Only

Slide 76

Slide 76 text

Monitoring

Slide 77

Slide 77 text

Monitoring Observability

Slide 78

Slide 78 text

Prometheus + Grafana

Slide 79

Slide 79 text

https://github.com/coreos/prometheus-operator

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

% uptime

Slide 82

Slide 82 text

Workflow Steps

Slide 83

Slide 83 text

No Live Demo Today! https://flic.kr/p/RJ5xEs

Slide 84

Slide 84 text

Procedure 1.Make changes to the Repository, open a pull request 2.Wait for Buildkite verification, peer approval, then merge 3.Wait for Expeditor to complete merge actions 4.Verify acceptance with stakeholders 5.Promote to Production

Slide 85

Slide 85 text

Previous Procedure • Let's just say... • It was more than 5 steps • And many of them were manual • And the procedure differed slightly between applications...

Slide 86

Slide 86 text

Make changes to the repository, open a pull request git checkout -b $USERNAME/my-branch # make some changes with your favorite editor git add . git commit -m 'updating the site with new features or fixes' git push origin $USERNAME/my-branch

Slide 87

Slide 87 text

Wait for Expeditor to complete merge actions

Slide 88

Slide 88 text

Verify Acceptance with stakeholders

Slide 89

Slide 89 text

Promote to Production

Slide 90

Slide 90 text

Coworker: "Wait... that's all?"

Slide 91

Slide 91 text

*Spoiler: That's not all... https://commons.wikimedia.org/wiki/File:Bugatti_veyron2.jpg

Slide 92

Slide 92 text

Conclusion

Slide 93

Slide 93 text

Automation has shifted

Slide 94

Slide 94 text

Workflow > Infrastructure

Slide 95

Slide 95 text

Business benefit

Slide 96

Slide 96 text

Questions?

Slide 97

Slide 97 text

Thank you! Joshua Timberman @jtimberman All images used in this presentation are public domain, created by me, or licensed under the Creative Commons Attribution-Share Alike 4.0 license: https://creativecommons.org/licenses/by-sa/4.0/deed.en The only modification made to any images is resizing to fit into the slides