Hands-on with Cloud-Native CI/CD

07ff350ffa93e4e69f3e7f0e5795befb?s=47 Dave Stanke
December 03, 2019

Hands-on with Cloud-Native CI/CD

Continuous Integration and Continuous Delivery (CI/CD) have taken root in software teams of all sizes: the process of delivering applications from dev to prod is increasingly automated, tested, and measured. As applications migrate to cloud infrastructure, so can CI/CD systems. Cloud-native principles like immutability and configuration as code are giving rise to a new generation of automation tools, supporting faster, more reliable software delivery. In this workshop, participants will learn cloud-native fundamentals, and craft a pipeline definition that continuously builds and deploys a sample microservice application using Google’s serverless Cloud Build CI/CD platform. Through hands-on tutorials and challenges, we will gain experience with containerized toolchains, versioned configuration, and other best practices of modern developer tooling.

07ff350ffa93e4e69f3e7f0e5795befb?s=128

Dave Stanke

December 03, 2019
Tweet

Transcript

  1. Hands-on with Cloud-Native CI/CD Dave Stanke • Craig Barber DevOpsWorld

    Lisbon 2019-12-03
  2. DevOps Advocate Software Engineer

  3. 1. Intro: Cloud-Native Computing and Cloud-native CI/CD 2. Container basics

    3. Containerize an application 4. Continuous* Integration & Delivery 5. Serverless runtime 6. Continuous* Deployment 7. Triggering 8. Continuous!!! Integration, Delivery, & Deployment 9. Advanced Cloud-Native CI/CD Agenda
  4. What is cloud computing?

  5. On-demand self-service Broad network access Resource pooling Rapid elasticity Measured

    service NIST: Cloud computing is...
  6. Elite performers are 24x more likely to have met all

    cloud characteristics get the report @ cloud.google.com/devops
  7. What is cloud-native computing?

  8. Cloud-Native Computing Cloud native technologies empower organizations to build and

    run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil.
  9. Cloud-Native Computing Cloud native technologies empower organizations to build and

    run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil. 1 3 4 7 2 5 6 Containerization Immutable Infrastructure Declarative APIs Observability Loose Coupling Automation Predictability
  10. Runtime infra vs. Tooling infra bare metal VMs cloud instances

    kubernetes serverless
  11. Runtime infra vs. Tooling infra bare metal VMs cloud instances

    kubernetes serverless Application platform? CI/CD platform?
  12. Sheepdog

  13. So what? So… • Security • Scalability • Portability •

    Adaptability • Economy
  14. Cloud-Native CI/CD • Use Cloud-Native tooling to build and deploy

    cloud-native [or not] applications • Benefit from: ◦ Security ◦ Scalability ◦ Portability ◦ Adaptability ◦ Economy
  15. SETUP 1. Go to github.com/davidstanke/cloudbuild-demo 2. Click Use this template

    3. Name your repo cloudbuild-demo 4. Make it Private
  16. 6. From the repo’s README, click Open in Cloud Shell

    Click accept/proceed when prompted 7. Follow the instructions under “Preparing Google Cloud to run this demo” 1. Enable the needed APIs 2. Grant Cloud Build permission to deploy to Cloud Run 3. configure Git 8. Run git push in the console 1. At the prompt, log in with your GitHub credentials 2. TIP: if you use two-factor authentication with GitHub, you’ll need to use a personal access token as your password. Visit github.com/settings/tokens to get one. Challenge: make yourself the “author” of this application.
  17. Our application github.com/davidstanke/cloudbuild-demo

  18. Exercise: install and run the app “locally” Install the app:

    npm install Run the app: npm start Preview on port 8080
  19. Cloud-Native: Containerization

  20. What’s a container? ≠

  21. A container is... • A packaging mechanism • With layers

    • It’s basically a decomposable TAR file
  22. H A R D W A R E

  23. A container provides isolation from its host environment • Only

    the files inside the container are visible (by default) • Processes inside and outside the container can’t communicate (by default)
  24. What’s a container? ≈

  25. Containerization: Dockerfile FROM ubuntu COPY ./dir . CMD server.sh EXPOSE

    80, 443 Docker build ubuntu copied files server.sh
  26. 1. If the application is still running, press CTRL-C to

    terminate it. 2. Use the skeleton file provided at ./Dockerfile 3. Replace the “???”s so the build will: a. Pull an appropriate base image b. Copy source files to the image c. Install the application d. Add a command to start the application when the container is run e. Expose the application port Docker build -t gcr.io/$PROJECT_ID/hello . Exercise: write a Dockerfile Hint: the container is going to run the same application that you ran “locally.” How did you prepare it to run? What’s the command you used to run it?
  27. 1. docker build -t gcr.io/$PROJECT_ID/hello . Exercise: build a container

    Don’t forget the dot!
  28. Containerization: Registry • A repository of container images ◦ ~artifact

    registry (e.g. artifactory) • Stores multiple images, multiple tagged versions of each • Layer-aware • Public registry: Dockerhub • Private registry: GCP (AWS, Azure…)
  29. 1. docker push gcr.io/$PROJECT_ID/hello Exercise: push a container

  30. Exercise: install and run the app “locally” in a container

    1. docker run -d -p 8080:8080 \ gcr.io/$PROJECT_ID/hello 2. Preview on port 8080 BREAK: start again at 14:40
  31. Serverless • Automatic, on-demand provisioning and deprovisioning • User has

    no need (or ability!) to manage the platform • User is billed only while requests are actively being processed • Well-defined boundary between application code and platform ◦ → e.g. inside a container vs. outside
  32. A serverless runtime: Cloud Run • Serve a container as

    an HTTP service • Autoscaling • HTTPS termination ◦ Request on port 443 → container port 8080
  33. Demo: deploying to Cloud Run

  34. Exercise: Deploy a serverless service 1. Deploy your application container

    to Cloud Run 2. View your running application
  35. 1. git add . 2. git commit -am "add Dockerfile"

    3. git push Exercise: Push updated code to GitHub
  36. Container ENTRYPOINT • Containers can be run as executables, with

    arguments • The program to be run inside the container is specified with ENTRYPOINT in the Dockerfile • A container’s entrypoint can be overridden at run time > docker run weather “Lisbon, Portugal” 16º and sunny! ENTRYPOINT [“forecast”] > docker run weather --entrypoint=geocode “Lisbon, Portugal” 38.7223° N, 9.1393° W
  37. GCP’s Serverless CI/CD: Cloud Build CI/CD automation service for executing

    builds on GCP • Trigger builds on source events • Run tests and build artifacts • Builds are executed as a series of containerized tasks • All resources provisioned on-demand: scales from and to zero
  38. Cloud Build execution flow Build step Build step Build step

    Build step /workspace Source code Artifact
  39. Google Cloud Builders docker gcr.io/cloud-builders/docker docker example go gcr.io/cloud-builders/go go

    example gcloud gcr.io/cloud-builders/gcloud gcloud example gradle gcr.io/cloud-builders/gradle gradle example maven gcr.io/cloud-builders/mvn maven example kubectl gcr.io/cloud-builders/kubectl kubectl example npm gcr.io/cloud-builders/npm npm example >75 containers with common languages and tools installed in them that you can use in build steps (or bring your own)
  40. Cloud Build config: cloudbuild.yaml The cloudbuild file defines the work

    to be done by Cloud Build steps: - name: CONTAINER_TO_RUN entrypoint: COMMAND_TO_RUN args: ['arg1','arg2','arg3'] - name: CONTAINER_TO_RUN entrypoint: args: options: - optionA - optionB
  41. Demo: Use Cloud Build to containerize an application Docker build

    Docker push /workspace GitHub Google Container Registry
  42. Exercise: write a Cloud Build config file 1. Complete cloudbuild.yaml

    to build and push a Docker image 2. Run gcloud builds submit 3. View the container registry and see the updated container image 4. Commit the latest changes and push to GitHub Docker build Docker push /workspace GitHub Google Container Registry
  43. Demo: automated deployment

  44. Exercise: add automated deployment 1. Make an application code change

    (e.g. make a text change to line 18 of app.js) 2. Add a deploy step to cloud build config (see: cloudbuild_deploy_snippet.html) Deploy: 3. gcloud builds submit 4. Refresh the application in your browser and verify the change 5. Commit your changes and push them to GitHub Challenge: modify your Dockerfile to produce a smaller container image
  45. Automated build triggers CI/CD System Source Repository Build branch Build

    PR Push code change to branch notify via webhook Open a new pull request notify via webhook post results to PR Developer
  46. Demo: configure triggering

  47. Exercise: set up triggering 1. Add a Cloud Build trigger

    a. Choose first option for trigger: “GitHub <BETA>” b. Say yes to all the things 2. Make a code change → commit → push to GitHub 3. View the running build in Google Cloud Console > Cloud Build History 4. Reload the application to see your change
  48. More CI: multi-stage pipelines

  49. More CI: Build step sources steps: - name: gcr.io/cloud-builders/<image> GCP-provided

    builders steps: - name: <image> Dockerhub 1. Clone builder repo (or write your own) 2. Docker build 3. Push to your Google Container Registry Community builders, or write-your-own steps: - name: gcr.io/$PROJECT_ID/<image>
  50. Demo: install and test application

  51. Exercise: add `npm` build steps 1. Add an ‘install’ step

    using npm (from Dockerhub) a. Hint: your docker image knows how to pull the node image from dockerhub b. Hint: remember that the command to install is ‘npm install’ 2. Add a ‘test’ step using npm (from Dockerhub) a. Hint: The command to test is ‘npm test’ 3. Git commit and push 4. View build output in Cloud Build > History Got errors? Fix and re-submit until the build is green! CHALLENGE: add a linter step DOUBLE CHALLENGE: run the linter in parallel with the test step
  52. Teardown (if desired) delete the repo from your GitHub account

    Your Google Cloud project and all the resources it contains will be automatically deleted within 1-2 days.
  53. Thank you Feedback please! Using the DWJW app... 1. Select

    "Schedule" on the lower menu bar. 2. Select the workshop title in the mobile app. 3. Scroll to Post-Session Survey within the workshop details. 4. Complete the survey. Stay in touch! @davidstanke craigdbarber Questions?