Scale Jenkins with Kubernetes on Google Container Engine Guillaume Laforge Developer Advocate Google Cloud Platform @glaforge

Agenda What are all those cool tools? Why would we want to use these for our CI/CD? How are we going to implement this?

What?

No content

Source: https://www.cloudbees.com/jenkins/about

No content

Source: https://kubernetes.io/

@glaforge Kubernetes concepts Pod Volume Service Deployment Cluster Label Selector Secret Ingress Liveness Probe Readiness Probe Job Liveness Probe Readiness Probe ReplicaSet DaemonSet StatefulSet ConfigMap

No content

Source: https://cloud.google.com/container-engine/

Why?

@glaforge One developer

@glaforge A few developers

@glaforge Tons of developers!

@glaforge One project

@glaforge Tons of projects

@glaforge Many projects & developers, tons of commits...

With more slaves, you’d build a pyramid!

@glaforge Our solution! + +

How? How?

DEMO / LAB https://github.com/GoogleCloudPlatform /continuous-deployment-on-kubernetes

@glaforge Creating a Kubernetes cluster for Jenkins // set a compute zone to run our environment $ gcloud config set compute/zone us-east1-d // create a cluster with 3 minimum nodes $ gcloud container clusters create jenkins-cd --num-nodes 3 --scopes "https://www.googleapis.com/auth/projecthosting,storage-rw" // retrieve the credentials for our cluster $ gcloud container clusters get-credentials jenkins-cd

@glaforge Creating a Kubernetes cluster for Jenkins // create a dedicated namespace $ kubectl create ns jenkins // create the Jenkins home volume $ gcloud compute images create jenkins-home-image --source-uri https://storage.googleapis.com/solutions-public-assets/jenkins-cd/jenkins-home-v3.tar.gz $ gcloud compute disks create jenkins-home --image jenkins-home-image

@glaforge Creating a Kubernetes cluster for Jenkins ● Jenkins Kubernetes Plugin Start Jenkins build executor containers in the Kubernetes cluster when builds are requested, terminate those containers when builds complete, freeing resources up for the rest of the cluster ● Jenkins Pipelines Define our build pipeline declaratively and keep it checked into source code management alongside our application code ● Google OAuth Plugin Allows you to add your google OAuth credentials to Jenkins

@glaforge Creating a Kubernetes cluster for Jenkins apiVersion: extensions/v1beta1 kind: Deployment metadata: name: jenkins namespace: jenkins spec: replicas: 1 template: metadata: labels: app: master spec: containers: - name: master image: jenkins:2.7.2 ports: - containerPort: 8080 - containerPort: 50000 volumeMounts: - mountPath: /var/jenkins_home name: jenkins-home resources: limits: cpu: 500m memory: 1500Mi requests: cpu: 500m memory: 1500Mi volumes: - name: jenkins-home gcePersistentDisk: pdName: jenkins-home fsType: ext4 partition: 1 readinessProbe: httpGet: path: /login port: 8080 periodSeconds: 10 timeoutSeconds: 5 successThreshold: 2 failureThreshold: 5 env: - name: JENKINS_OPTS valueFrom: secretKeyRef: name: jenkins key: options jenkins.yaml

@glaforge Creating a Kubernetes cluster for Jenkins service_jenkins.yaml # jenkins_service_ui --- kind: Service apiVersion: v1 metadata: name: jenkins-ui namespace: jenkins spec: type: NodePort selector: app: master ports: - protocol: TCP port: 8080 targetPort: 8080 name: ui # jenkins_service_discovery --- kind: Service apiVersion: v1 metadata: name: jenkins-discovery namespace: jenkins spec: selector: app: master ports: - protocol: TCP port: 50000 targetPort: 50000 name: slaves

@glaforge The app to build and deploy

@glaforge // create a new cluster for our app $ kubectl create ns production // setup the different environments $ kubectl --namespace=production apply -f k8s/production $ kubectl --namespace=production apply -f k8s/canary $ kubectl --namespace=production apply -f k8s/services // scale the production service $ kubectl --namespace=production scale deployment gceme-frontend-production --replicas=4 Building and deploying

@glaforge Building and deploying kind: Deployment apiVersion: extensions/v1beta1 metadata: name: gceme-frontend-production spec: replicas: template: metadata: name: frontend labels: app: gceme role: frontend env: production spec: containers: - name: frontend image: gcr.io/cloud-solutions-images/gceme:1.0.0 resources: limits: memory: "500Mi" cpu: "100m" imagePullPolicy: Always readinessProbe: httpGet: path: /healthz port: 80 command: ["sh", "-c", "app -frontend=true -backend-service=http://gceme-backend:8080 - ports: - name: frontend containerPort: 80 frontend_production.yaml

@glaforge Building and deploying FROM jenkinsci/jnlp-slave ENV CLOUDSDK_CORE_DISABLE_PROMPTS 1 ENV PATH /opt/google-cloud-sdk/bin:$PATH USER root RUN apt-get update -y RUN apt-get install -y jq RUN curl https://sdk.cloud.google.com | bash && mv google-cloud-sdk /opt RUN gcloud components install kubectl Dockerfile

@glaforge Building and deploying node { def project = 'jenkins-gke-scaling' def appName = 'gceme' def feSvcName = "${appName}-frontend" def imageTag = "gcr.io/${project}/${appName}:${env.BRANCH_NAME}.${env.BUILD_NUMBER}" checkout scm stage 'Build image' sh("docker build -t ${imageTag} .") stage 'Run Go tests' sh("docker run ${imageTag} go test") stage 'Push image to registry' sh("gcloud docker -- push ${imageTag}") Jenkinsfile

@glaforge Building and deploying stage "Deploy Application" switch (env.BRANCH_NAME) { // Roll out to canary environment case "canary": // Change deployed image in canary to the one we just built sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/canary/*.yaml") sh("kubectl --namespace=production apply -f k8s/services/") sh("kubectl --namespace=production apply -f k8s/canary/") sh("echo http://`kubectl --namespace=production get service/${feSvcName} --output=json | " + "jq -r '.status.loadBalancer.ingress[0].ip'` > ${feSvcName}") break Jenkinsfile

@glaforge Building and deploying // Roll out to production case "master": // Change deployed image in canary to the one we just built sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/production/*.yaml") sh("kubectl --namespace=production apply -f k8s/services/") sh("kubectl --namespace=production apply -f k8s/production/") sh("echo http://`kubectl --namespace=production get service/${feSvcName} --output=json | " + "jq -r '.status.loadBalancer.ingress[0].ip'` > ${feSvcName}") break Jenkinsfile

@glaforge Building and deploying // Roll out a dev environment default: // Create namespace if it doesn't exist sh("kubectl get ns ${env.BRANCH_NAME} || kubectl create ns ${env.BRANCH_NAME}") // Don't use public load balancing for development branches sh("sed -i.bak 's#LoadBalancer#ClusterIP#' ./k8s/services/frontend.yaml") sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/dev/*.yaml") sh("kubectl --namespace=${env.BRANCH_NAME} apply -f k8s/services/") sh("kubectl --namespace=${env.BRANCH_NAME} apply -f k8s/dev/") echo 'To access your environment run `kubectl proxy`' echo "Then access your service via " + "http://localhost:8001/api/v1/proxy/namespaces/${env.BRANCH_NAME}/services/${feSvcName}:80/" } } Jenkinsfile

Summary & resources

@glaforge Summary

@glaforge Summary

@glaforge Resources Overview of Jenkins on Container Engine https://cloud.google.com/solutions/jenkins-on-container-engine Setting up Jenkins on Container Engine https://cloud.google.com/solutions/jenkins-on-container-engine-tutorial Configuring Jenkins for Container Engine https://cloud.google.com/solutions/configuring-jenkins-container-engine Continuous Deployment to Container Engine using Jenkins https://cloud.google.com/solutions/continuous-delivery-jenkins-container-engine Lab: Build a Continuous Deployment Pipeline with Jenkins and Kubernetes https://github.com/GoogleCloudPlatform/continuous-deployment-on-kubernetes

Questions?

KTHXBAI And come to our booth & say hi!