Scaling Jenkins with Kubernetes on Google Container Engine

Transcript

1. Scale Jenkins with Kubernetes on Google Container Engine Guillaume Laforge Developer Advocate Google Cloud Platform @glaforge
2. Agenda What are all those cool tools? Why would we want to use these for our CI/CD? How are we going to implement this?
3. What?
5. Source: https://www.cloudbees.com/jenkins/about
7. Source: https://kubernetes.io/
8. @glaforge Kubernetes concepts Pod Volume Service Deployment Cluster Label Selector Secret Ingress Liveness Probe Readiness Probe Job Liveness Probe Readiness Probe ReplicaSet DaemonSet StatefulSet ConfigMap
10. Source: https://cloud.google.com/container-engine/
11. Why?
12. @glaforge One developer
13. @glaforge A few developers
14. @glaforge Tons of developers!
15. @glaforge One project
16. @glaforge Tons of projects
17. @glaforge Many projects & developers, tons of commits...
18. With more slaves, you’d build a pyramid!
19. @glaforge Our solution! + +
20. How? How?
21. DEMO / LAB https://github.com/GoogleCloudPlatform /continuous-deployment-on-kubernetes
22. @glaforge Creating a Kubernetes cluster for Jenkins // set a compute zone to run our environment $ gcloud config set compute/zone us-east1-d // create a cluster with 3 minimum nodes $ gcloud container clusters create jenkins-cd --num-nodes 3 --scopes "https://www.googleapis.com/auth/projecthosting,storage-rw" // retrieve the credentials for our cluster $ gcloud container clusters get-credentials jenkins-cd
23. @glaforge Creating a Kubernetes cluster for Jenkins // create a dedicated namespace $ kubectl create ns jenkins // create the Jenkins home volume $ gcloud compute images create jenkins-home-image --source-uri https://storage.googleapis.com/solutions-public-assets/jenkins-cd/jenkins-home-v3.tar.gz $ gcloud compute disks create jenkins-home --image jenkins-home-image
24. @glaforge Creating a Kubernetes cluster for Jenkins • Jenkins Kubernetes Plugin Start Jenkins build executor containers in the Kubernetes cluster when builds are requested, terminate those containers when builds complete, freeing resources up for the rest of the cluster • Jenkins Pipelines Define our build pipeline declaratively and keep it checked into source code management alongside our application code • Google OAuth Plugin Allows you to add your google OAuth credentials to Jenkins
25. @glaforge Creating a Kubernetes cluster for Jenkins apiVersion: extensions/v1beta1 kind: Deployment metadata: name: jenkins namespace: jenkins spec: replicas: 1 template: metadata: labels: app: master spec: containers: - name: master image: jenkins:2.7.2 ports: - containerPort: 8080 - containerPort: 50000 volumeMounts: - mountPath: /var/jenkins_home name: jenkins-home resources: limits: cpu: 500m memory: 1500Mi requests: cpu: 500m memory: 1500Mi volumes: - name: jenkins-home gcePersistentDisk: pdName: jenkins-home fsType: ext4 partition: 1 readinessProbe: httpGet: path: /login port: 8080 periodSeconds: 10 timeoutSeconds: 5 successThreshold: 2 failureThreshold: 5 env: - name: JENKINS_OPTS valueFrom: secretKeyRef: name: jenkins key: options jenkins.yaml
26. @glaforge Creating a Kubernetes cluster for Jenkins service_jenkins.yaml # jenkins_service_ui --- kind: Service apiVersion: v1 metadata: name: jenkins-ui namespace: jenkins spec: type: NodePort selector: app: master ports: - protocol: TCP port: 8080 targetPort: 8080 name: ui # jenkins_service_discovery --- kind: Service apiVersion: v1 metadata: name: jenkins-discovery namespace: jenkins spec: selector: app: master ports: - protocol: TCP port: 50000 targetPort: 50000 name: slaves
27. @glaforge The app to build and deploy
28. @glaforge // create a new cluster for our app $ kubectl create ns production // setup the different environments $ kubectl --namespace=production apply -f k8s/production $ kubectl --namespace=production apply -f k8s/canary $ kubectl --namespace=production apply -f k8s/services // scale the production service $ kubectl --namespace=production scale deployment gceme-frontend-production --replicas=4 Building and deploying
29. @glaforge Building and deploying kind: Deployment apiVersion: extensions/v1beta1 metadata: name: gceme-frontend-production spec: replicas: template: metadata: name: frontend labels: app: gceme role: frontend env: production spec: containers: - name: frontend image: gcr.io/cloud-solutions-images/gceme:1.0.0 resources: limits: memory: "500Mi" cpu: "100m" imagePullPolicy: Always readinessProbe: httpGet: path: /healthz port: 80 command: ["sh", "-c", "app -frontend=true -backend-service=http://gceme-backend:8080 - ports: - name: frontend containerPort: 80 frontend_production.yaml
30. @glaforge Building and deploying FROM jenkinsci/jnlp-slave ENV CLOUDSDK_CORE_DISABLE_PROMPTS 1 ENV PATH /opt/google-cloud-sdk/bin:$PATH USER root RUN apt-get update -y RUN apt-get install -y jq RUN curl https://sdk.cloud.google.com | bash && mv google-cloud-sdk /opt RUN gcloud components install kubectl Dockerfile
31. @glaforge Building and deploying node { def project = 'jenkins-gke-scaling' def appName = 'gceme' def feSvcName = "${appName}-frontend" def imageTag = "gcr.io/${project}/${appName}:${env.BRANCH_NAME}.${env.BUILD_NUMBER}" checkout scm stage 'Build image' sh("docker build -t ${imageTag} .") stage 'Run Go tests' sh("docker run ${imageTag} go test") stage 'Push image to registry' sh("gcloud docker -- push ${imageTag}") Jenkinsfile
32. @glaforge Building and deploying stage "Deploy Application" switch (env.BRANCH_NAME) { // Roll out to canary environment case "canary": // Change deployed image in canary to the one we just built sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/canary/*.yaml") sh("kubectl --namespace=production apply -f k8s/services/") sh("kubectl --namespace=production apply -f k8s/canary/") sh("echo http://`kubectl --namespace=production get service/${feSvcName} --output=json | " + "jq -r '.status.loadBalancer.ingress[0].ip'` > ${feSvcName}") break Jenkinsfile
33. @glaforge Building and deploying // Roll out to production case "master": // Change deployed image in canary to the one we just built sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/production/*.yaml") sh("kubectl --namespace=production apply -f k8s/services/") sh("kubectl --namespace=production apply -f k8s/production/") sh("echo http://`kubectl --namespace=production get service/${feSvcName} --output=json | " + "jq -r '.status.loadBalancer.ingress[0].ip'` > ${feSvcName}") break Jenkinsfile
34. @glaforge Building and deploying // Roll out a dev environment default: // Create namespace if it doesn't exist sh("kubectl get ns ${env.BRANCH_NAME} || kubectl create ns ${env.BRANCH_NAME}") // Don't use public load balancing for development branches sh("sed -i.bak 's#LoadBalancer#ClusterIP#' ./k8s/services/frontend.yaml") sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/dev/*.yaml") sh("kubectl --namespace=${env.BRANCH_NAME} apply -f k8s/services/") sh("kubectl --namespace=${env.BRANCH_NAME} apply -f k8s/dev/") echo 'To access your environment run `kubectl proxy`' echo "Then access your service via " + "http://localhost:8001/api/v1/proxy/namespaces/${env.BRANCH_NAME}/services/${feSvcName}:80/" } } Jenkinsfile
35. Summary & resources
36. @glaforge Summary
37. @glaforge Summary
38. @glaforge Resources Overview of Jenkins on Container Engine https://cloud.google.com/solutions/jenkins-on-container-engine Setting up Jenkins on Container Engine https://cloud.google.com/solutions/jenkins-on-container-engine-tutorial Configuring Jenkins for Container Engine https://cloud.google.com/solutions/configuring-jenkins-container-engine Continuous Deployment to Container Engine using Jenkins https://cloud.google.com/solutions/continuous-delivery-jenkins-container-engine Lab: Build a Continuous Deployment Pipeline with Jenkins and Kubernetes https://github.com/GoogleCloudPlatform/continuous-deployment-on-kubernetes
39. Questions?
40. KTHXBAI And come to our booth & say hi!