Scaling Jenkins with Kubernetes on Google Container Engine

Scaling Jenkins with Kubernetes on Google Container Engine

137d3908243acfc30e126615d59d4e6d?s=128

Guillaume Laforge

July 11, 2017
Tweet

Transcript

  1. Scale Jenkins with Kubernetes on Google Container Engine Guillaume Laforge

    Developer Advocate Google Cloud Platform @glaforge
  2. Agenda What are all those cool tools? Why would we

    want to use these for our CI/CD? How are we going to implement this?
  3. What?

  4. None
  5. Source: https://www.cloudbees.com/jenkins/about

  6. None
  7. Source: https://kubernetes.io/

  8. @glaforge Kubernetes concepts Pod Volume Service Deployment Cluster Label Selector

    Secret Ingress Liveness Probe Readiness Probe Job Liveness Probe Readiness Probe ReplicaSet DaemonSet StatefulSet ConfigMap
  9. None
  10. Source: https://cloud.google.com/container-engine/

  11. Why?

  12. @glaforge One developer

  13. @glaforge A few developers

  14. @glaforge Tons of developers!

  15. @glaforge One project

  16. @glaforge Tons of projects

  17. @glaforge Many projects & developers, tons of commits...

  18. With more slaves, you’d build a pyramid!

  19. @glaforge Our solution! + +

  20. How? How?

  21. DEMO / LAB https://github.com/GoogleCloudPlatform /continuous-deployment-on-kubernetes

  22. @glaforge Creating a Kubernetes cluster for Jenkins // set a

    compute zone to run our environment $ gcloud config set compute/zone us-east1-d // create a cluster with 3 minimum nodes $ gcloud container clusters create jenkins-cd --num-nodes 3 --scopes "https://www.googleapis.com/auth/projecthosting,storage-rw" // retrieve the credentials for our cluster $ gcloud container clusters get-credentials jenkins-cd
  23. @glaforge Creating a Kubernetes cluster for Jenkins // create a

    dedicated namespace $ kubectl create ns jenkins // create the Jenkins home volume $ gcloud compute images create jenkins-home-image --source-uri https://storage.googleapis.com/solutions-public-assets/jenkins-cd/jenkins-home-v3.tar.gz $ gcloud compute disks create jenkins-home --image jenkins-home-image
  24. @glaforge Creating a Kubernetes cluster for Jenkins • Jenkins Kubernetes

    Plugin Start Jenkins build executor containers in the Kubernetes cluster when builds are requested, terminate those containers when builds complete, freeing resources up for the rest of the cluster • Jenkins Pipelines Define our build pipeline declaratively and keep it checked into source code management alongside our application code • Google OAuth Plugin Allows you to add your google OAuth credentials to Jenkins
  25. @glaforge Creating a Kubernetes cluster for Jenkins apiVersion: extensions/v1beta1 kind:

    Deployment metadata: name: jenkins namespace: jenkins spec: replicas: 1 template: metadata: labels: app: master spec: containers: - name: master image: jenkins:2.7.2 ports: - containerPort: 8080 - containerPort: 50000 volumeMounts: - mountPath: /var/jenkins_home name: jenkins-home resources: limits: cpu: 500m memory: 1500Mi requests: cpu: 500m memory: 1500Mi volumes: - name: jenkins-home gcePersistentDisk: pdName: jenkins-home fsType: ext4 partition: 1 readinessProbe: httpGet: path: /login port: 8080 periodSeconds: 10 timeoutSeconds: 5 successThreshold: 2 failureThreshold: 5 env: - name: JENKINS_OPTS valueFrom: secretKeyRef: name: jenkins key: options jenkins.yaml
  26. @glaforge Creating a Kubernetes cluster for Jenkins service_jenkins.yaml # jenkins_service_ui

    --- kind: Service apiVersion: v1 metadata: name: jenkins-ui namespace: jenkins spec: type: NodePort selector: app: master ports: - protocol: TCP port: 8080 targetPort: 8080 name: ui # jenkins_service_discovery --- kind: Service apiVersion: v1 metadata: name: jenkins-discovery namespace: jenkins spec: selector: app: master ports: - protocol: TCP port: 50000 targetPort: 50000 name: slaves
  27. @glaforge The app to build and deploy

  28. @glaforge // create a new cluster for our app $

    kubectl create ns production // setup the different environments $ kubectl --namespace=production apply -f k8s/production $ kubectl --namespace=production apply -f k8s/canary $ kubectl --namespace=production apply -f k8s/services // scale the production service $ kubectl --namespace=production scale deployment gceme-frontend-production --replicas=4 Building and deploying
  29. @glaforge Building and deploying kind: Deployment apiVersion: extensions/v1beta1 metadata: name:

    gceme-frontend-production spec: replicas: template: metadata: name: frontend labels: app: gceme role: frontend env: production spec: containers: - name: frontend image: gcr.io/cloud-solutions-images/gceme:1.0.0 resources: limits: memory: "500Mi" cpu: "100m" imagePullPolicy: Always readinessProbe: httpGet: path: /healthz port: 80 command: ["sh", "-c", "app -frontend=true -backend-service=http://gceme-backend:8080 - ports: - name: frontend containerPort: 80 frontend_production.yaml
  30. @glaforge Building and deploying FROM jenkinsci/jnlp-slave ENV CLOUDSDK_CORE_DISABLE_PROMPTS 1 ENV

    PATH /opt/google-cloud-sdk/bin:$PATH USER root RUN apt-get update -y RUN apt-get install -y jq RUN curl https://sdk.cloud.google.com | bash && mv google-cloud-sdk /opt RUN gcloud components install kubectl Dockerfile
  31. @glaforge Building and deploying node { def project = 'jenkins-gke-scaling'

    def appName = 'gceme' def feSvcName = "${appName}-frontend" def imageTag = "gcr.io/${project}/${appName}:${env.BRANCH_NAME}.${env.BUILD_NUMBER}" checkout scm stage 'Build image' sh("docker build -t ${imageTag} .") stage 'Run Go tests' sh("docker run ${imageTag} go test") stage 'Push image to registry' sh("gcloud docker -- push ${imageTag}") Jenkinsfile
  32. @glaforge Building and deploying stage "Deploy Application" switch (env.BRANCH_NAME) {

    // Roll out to canary environment case "canary": // Change deployed image in canary to the one we just built sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/canary/*.yaml") sh("kubectl --namespace=production apply -f k8s/services/") sh("kubectl --namespace=production apply -f k8s/canary/") sh("echo http://`kubectl --namespace=production get service/${feSvcName} --output=json | " + "jq -r '.status.loadBalancer.ingress[0].ip'` > ${feSvcName}") break Jenkinsfile
  33. @glaforge Building and deploying // Roll out to production case

    "master": // Change deployed image in canary to the one we just built sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/production/*.yaml") sh("kubectl --namespace=production apply -f k8s/services/") sh("kubectl --namespace=production apply -f k8s/production/") sh("echo http://`kubectl --namespace=production get service/${feSvcName} --output=json | " + "jq -r '.status.loadBalancer.ingress[0].ip'` > ${feSvcName}") break Jenkinsfile
  34. @glaforge Building and deploying // Roll out a dev environment

    default: // Create namespace if it doesn't exist sh("kubectl get ns ${env.BRANCH_NAME} || kubectl create ns ${env.BRANCH_NAME}") // Don't use public load balancing for development branches sh("sed -i.bak 's#LoadBalancer#ClusterIP#' ./k8s/services/frontend.yaml") sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/dev/*.yaml") sh("kubectl --namespace=${env.BRANCH_NAME} apply -f k8s/services/") sh("kubectl --namespace=${env.BRANCH_NAME} apply -f k8s/dev/") echo 'To access your environment run `kubectl proxy`' echo "Then access your service via " + "http://localhost:8001/api/v1/proxy/namespaces/${env.BRANCH_NAME}/services/${feSvcName}:80/" } } Jenkinsfile
  35. Summary & resources

  36. @glaforge Summary

  37. @glaforge Summary

  38. @glaforge Resources Overview of Jenkins on Container Engine https://cloud.google.com/solutions/jenkins-on-container-engine Setting

    up Jenkins on Container Engine https://cloud.google.com/solutions/jenkins-on-container-engine-tutorial Configuring Jenkins for Container Engine https://cloud.google.com/solutions/configuring-jenkins-container-engine Continuous Deployment to Container Engine using Jenkins https://cloud.google.com/solutions/continuous-delivery-jenkins-container-engine Lab: Build a Continuous Deployment Pipeline with Jenkins and Kubernetes https://github.com/GoogleCloudPlatform/continuous-deployment-on-kubernetes
  39. Questions?

  40. KTHXBAI And come to our booth & say hi!