Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling Jenkins with Kubernetes on Google Container Engine

Scaling Jenkins with Kubernetes on Google Container Engine

Guillaume Laforge

July 11, 2017
Tweet

More Decks by Guillaume Laforge

Other Decks in Technology

Transcript

  1. Scale Jenkins with Kubernetes on Google Container Engine Guillaume Laforge

    Developer Advocate Google Cloud Platform @glaforge
  2. Agenda What are all those cool tools? Why would we

    want to use these for our CI/CD? How are we going to implement this?
  3. @glaforge Kubernetes concepts Pod Volume Service Deployment Cluster Label Selector

    Secret Ingress Liveness Probe Readiness Probe Job Liveness Probe Readiness Probe ReplicaSet DaemonSet StatefulSet ConfigMap
  4. @glaforge Creating a Kubernetes cluster for Jenkins // set a

    compute zone to run our environment $ gcloud config set compute/zone us-east1-d // create a cluster with 3 minimum nodes $ gcloud container clusters create jenkins-cd --num-nodes 3 --scopes "https://www.googleapis.com/auth/projecthosting,storage-rw" // retrieve the credentials for our cluster $ gcloud container clusters get-credentials jenkins-cd
  5. @glaforge Creating a Kubernetes cluster for Jenkins // create a

    dedicated namespace $ kubectl create ns jenkins // create the Jenkins home volume $ gcloud compute images create jenkins-home-image --source-uri https://storage.googleapis.com/solutions-public-assets/jenkins-cd/jenkins-home-v3.tar.gz $ gcloud compute disks create jenkins-home --image jenkins-home-image
  6. @glaforge Creating a Kubernetes cluster for Jenkins • Jenkins Kubernetes

    Plugin Start Jenkins build executor containers in the Kubernetes cluster when builds are requested, terminate those containers when builds complete, freeing resources up for the rest of the cluster • Jenkins Pipelines Define our build pipeline declaratively and keep it checked into source code management alongside our application code • Google OAuth Plugin Allows you to add your google OAuth credentials to Jenkins
  7. @glaforge Creating a Kubernetes cluster for Jenkins apiVersion: extensions/v1beta1 kind:

    Deployment metadata: name: jenkins namespace: jenkins spec: replicas: 1 template: metadata: labels: app: master spec: containers: - name: master image: jenkins:2.7.2 ports: - containerPort: 8080 - containerPort: 50000 volumeMounts: - mountPath: /var/jenkins_home name: jenkins-home resources: limits: cpu: 500m memory: 1500Mi requests: cpu: 500m memory: 1500Mi volumes: - name: jenkins-home gcePersistentDisk: pdName: jenkins-home fsType: ext4 partition: 1 readinessProbe: httpGet: path: /login port: 8080 periodSeconds: 10 timeoutSeconds: 5 successThreshold: 2 failureThreshold: 5 env: - name: JENKINS_OPTS valueFrom: secretKeyRef: name: jenkins key: options jenkins.yaml
  8. @glaforge Creating a Kubernetes cluster for Jenkins service_jenkins.yaml # jenkins_service_ui

    --- kind: Service apiVersion: v1 metadata: name: jenkins-ui namespace: jenkins spec: type: NodePort selector: app: master ports: - protocol: TCP port: 8080 targetPort: 8080 name: ui # jenkins_service_discovery --- kind: Service apiVersion: v1 metadata: name: jenkins-discovery namespace: jenkins spec: selector: app: master ports: - protocol: TCP port: 50000 targetPort: 50000 name: slaves
  9. @glaforge // create a new cluster for our app $

    kubectl create ns production // setup the different environments $ kubectl --namespace=production apply -f k8s/production $ kubectl --namespace=production apply -f k8s/canary $ kubectl --namespace=production apply -f k8s/services // scale the production service $ kubectl --namespace=production scale deployment gceme-frontend-production --replicas=4 Building and deploying
  10. @glaforge Building and deploying kind: Deployment apiVersion: extensions/v1beta1 metadata: name:

    gceme-frontend-production spec: replicas: template: metadata: name: frontend labels: app: gceme role: frontend env: production spec: containers: - name: frontend image: gcr.io/cloud-solutions-images/gceme:1.0.0 resources: limits: memory: "500Mi" cpu: "100m" imagePullPolicy: Always readinessProbe: httpGet: path: /healthz port: 80 command: ["sh", "-c", "app -frontend=true -backend-service=http://gceme-backend:8080 - ports: - name: frontend containerPort: 80 frontend_production.yaml
  11. @glaforge Building and deploying FROM jenkinsci/jnlp-slave ENV CLOUDSDK_CORE_DISABLE_PROMPTS 1 ENV

    PATH /opt/google-cloud-sdk/bin:$PATH USER root RUN apt-get update -y RUN apt-get install -y jq RUN curl https://sdk.cloud.google.com | bash && mv google-cloud-sdk /opt RUN gcloud components install kubectl Dockerfile
  12. @glaforge Building and deploying node { def project = 'jenkins-gke-scaling'

    def appName = 'gceme' def feSvcName = "${appName}-frontend" def imageTag = "gcr.io/${project}/${appName}:${env.BRANCH_NAME}.${env.BUILD_NUMBER}" checkout scm stage 'Build image' sh("docker build -t ${imageTag} .") stage 'Run Go tests' sh("docker run ${imageTag} go test") stage 'Push image to registry' sh("gcloud docker -- push ${imageTag}") Jenkinsfile
  13. @glaforge Building and deploying stage "Deploy Application" switch (env.BRANCH_NAME) {

    // Roll out to canary environment case "canary": // Change deployed image in canary to the one we just built sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/canary/*.yaml") sh("kubectl --namespace=production apply -f k8s/services/") sh("kubectl --namespace=production apply -f k8s/canary/") sh("echo http://`kubectl --namespace=production get service/${feSvcName} --output=json | " + "jq -r '.status.loadBalancer.ingress[0].ip'` > ${feSvcName}") break Jenkinsfile
  14. @glaforge Building and deploying // Roll out to production case

    "master": // Change deployed image in canary to the one we just built sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/production/*.yaml") sh("kubectl --namespace=production apply -f k8s/services/") sh("kubectl --namespace=production apply -f k8s/production/") sh("echo http://`kubectl --namespace=production get service/${feSvcName} --output=json | " + "jq -r '.status.loadBalancer.ingress[0].ip'` > ${feSvcName}") break Jenkinsfile
  15. @glaforge Building and deploying // Roll out a dev environment

    default: // Create namespace if it doesn't exist sh("kubectl get ns ${env.BRANCH_NAME} || kubectl create ns ${env.BRANCH_NAME}") // Don't use public load balancing for development branches sh("sed -i.bak 's#LoadBalancer#ClusterIP#' ./k8s/services/frontend.yaml") sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${imageTag}#' ./k8s/dev/*.yaml") sh("kubectl --namespace=${env.BRANCH_NAME} apply -f k8s/services/") sh("kubectl --namespace=${env.BRANCH_NAME} apply -f k8s/dev/") echo 'To access your environment run `kubectl proxy`' echo "Then access your service via " + "http://localhost:8001/api/v1/proxy/namespaces/${env.BRANCH_NAME}/services/${feSvcName}:80/" } } Jenkinsfile
  16. @glaforge Resources Overview of Jenkins on Container Engine https://cloud.google.com/solutions/jenkins-on-container-engine Setting

    up Jenkins on Container Engine https://cloud.google.com/solutions/jenkins-on-container-engine-tutorial Configuring Jenkins for Container Engine https://cloud.google.com/solutions/configuring-jenkins-container-engine Continuous Deployment to Container Engine using Jenkins https://cloud.google.com/solutions/continuous-delivery-jenkins-container-engine Lab: Build a Continuous Deployment Pipeline with Jenkins and Kubernetes https://github.com/GoogleCloudPlatform/continuous-deployment-on-kubernetes