Hands-on with Cloud-Native CI/CD

Continuous Integration and Continuous Delivery (CI/CD) have taken root in software teams of all sizes: the process of delivering applications from dev to prod is increasingly automated, tested, and measured. As applications migrate to cloud infrastructure, so can CI/CD systems. Cloud-native principles like immutability and configuration as code are giving rise to a new generation of automation tools, supporting faster, more reliable software delivery. In this workshop, participants will learn cloud-native fundamentals, and craft a pipeline definition that continuously builds and deploys a sample microservice application using Google’s serverless Cloud Build CI/CD platform. Through hands-on tutorials and challenges, we will gain experience with containerized toolchains, versioned configuration, and other best practices of modern developer tooling.

Transcript

1. Hands-on with Cloud-Native CI/CD Dave Stanke • Craig Barber DevOpsWorld Lisbon 2019-12-03
2. DevOps Advocate Software Engineer
3. 1. Intro: Cloud-Native Computing and Cloud-native CI/CD 2. Container basics 3. Containerize an application 4. Continuous* Integration & Delivery 5. Serverless runtime 6. Continuous* Deployment 7. Triggering 8. Continuous!!! Integration, Delivery, & Deployment 9. Advanced Cloud-Native CI/CD Agenda
4. What is cloud computing?
5. On-demand self-service Broad network access Resource pooling Rapid elasticity Measured service NIST: Cloud computing is...
6. Elite performers are 24x more likely to have met all cloud characteristics get the report @ cloud.google.com/devops
7. What is cloud-native computing?
8. Cloud-Native Computing Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil.
9. Cloud-Native Computing Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil. 1 3 4 7 2 5 6 Containerization Immutable Infrastructure Declarative APIs Observability Loose Coupling Automation Predictability
10. Runtime infra vs. Tooling infra bare metal VMs cloud instances kubernetes serverless
11. Runtime infra vs. Tooling infra bare metal VMs cloud instances kubernetes serverless Application platform? CI/CD platform?
12. Sheepdog
13. So what? So… • Security • Scalability • Portability • Adaptability • Economy
14. Cloud-Native CI/CD • Use Cloud-Native tooling to build and deploy cloud-native [or not] applications • Benefit from: ◦ Security ◦ Scalability ◦ Portability ◦ Adaptability ◦ Economy
15. SETUP 1. Go to github.com/davidstanke/cloudbuild-demo 2. Click Use this template 3. Name your repo cloudbuild-demo 4. Make it Private
16. 6. From the repo’s README, click Open in Cloud Shell Click accept/proceed when prompted 7. Follow the instructions under “Preparing Google Cloud to run this demo” 1. Enable the needed APIs 2. Grant Cloud Build permission to deploy to Cloud Run 3. configure Git 8. Run git push in the console 1. At the prompt, log in with your GitHub credentials 2. TIP: if you use two-factor authentication with GitHub, you’ll need to use a personal access token as your password. Visit github.com/settings/tokens to get one. Challenge: make yourself the “author” of this application.
17. Our application github.com/davidstanke/cloudbuild-demo
18. Exercise: install and run the app “locally” Install the app: npm install Run the app: npm start Preview on port 8080
19. Cloud-Native: Containerization
20. What’s a container? ≠
21. A container is... • A packaging mechanism • With layers • It’s basically a decomposable TAR file
22. H A R D W A R E
23. A container provides isolation from its host environment • Only the files inside the container are visible (by default) • Processes inside and outside the container can’t communicate (by default)
24. What’s a container? ≈
25. Containerization: Dockerfile FROM ubuntu COPY ./dir . CMD server.sh EXPOSE 80, 443 Docker build ubuntu copied files server.sh
26. 1. If the application is still running, press CTRL-C to terminate it. 2. Use the skeleton file provided at ./Dockerfile 3. Replace the “???”s so the build will: a. Pull an appropriate base image b. Copy source files to the image c. Install the application d. Add a command to start the application when the container is run e. Expose the application port Docker build -t gcr.io/$PROJECT_ID/hello . Exercise: write a Dockerfile Hint: the container is going to run the same application that you ran “locally.” How did you prepare it to run? What’s the command you used to run it?
27. 1. docker build -t gcr.io/$PROJECT_ID/hello . Exercise: build a container Don’t forget the dot!
28. Containerization: Registry • A repository of container images ◦ ~artifact registry (e.g. artifactory) • Stores multiple images, multiple tagged versions of each • Layer-aware • Public registry: Dockerhub • Private registry: GCP (AWS, Azure…)
29. 1. docker push gcr.io/$PROJECT_ID/hello Exercise: push a container
30. Exercise: install and run the app “locally” in a container 1. docker run -d -p 8080:8080 \ gcr.io/$PROJECT_ID/hello 2. Preview on port 8080 BREAK: start again at 14:40
31. Serverless • Automatic, on-demand provisioning and deprovisioning • User has no need (or ability!) to manage the platform • User is billed only while requests are actively being processed • Well-defined boundary between application code and platform ◦ → e.g. inside a container vs. outside
32. A serverless runtime: Cloud Run • Serve a container as an HTTP service • Autoscaling • HTTPS termination ◦ Request on port 443 → container port 8080
33. Demo: deploying to Cloud Run
34. Exercise: Deploy a serverless service 1. Deploy your application container to Cloud Run 2. View your running application
35. 1. git add . 2. git commit -am "add Dockerfile" 3. git push Exercise: Push updated code to GitHub
36. Container ENTRYPOINT • Containers can be run as executables, with arguments • The program to be run inside the container is specified with ENTRYPOINT in the Dockerfile • A container’s entrypoint can be overridden at run time > docker run weather “Lisbon, Portugal” 16º and sunny! ENTRYPOINT [“forecast”] > docker run weather --entrypoint=geocode “Lisbon, Portugal” 38.7223° N, 9.1393° W
37. GCP’s Serverless CI/CD: Cloud Build CI/CD automation service for executing builds on GCP • Trigger builds on source events • Run tests and build artifacts • Builds are executed as a series of containerized tasks • All resources provisioned on-demand: scales from and to zero
38. Cloud Build execution flow Build step Build step Build step Build step /workspace Source code Artifact
39. Google Cloud Builders docker gcr.io/cloud-builders/docker docker example go gcr.io/cloud-builders/go go example gcloud gcr.io/cloud-builders/gcloud gcloud example gradle gcr.io/cloud-builders/gradle gradle example maven gcr.io/cloud-builders/mvn maven example kubectl gcr.io/cloud-builders/kubectl kubectl example npm gcr.io/cloud-builders/npm npm example >75 containers with common languages and tools installed in them that you can use in build steps (or bring your own)
40. Cloud Build config: cloudbuild.yaml The cloudbuild file defines the work to be done by Cloud Build steps: - name: CONTAINER_TO_RUN entrypoint: COMMAND_TO_RUN args: ['arg1','arg2','arg3'] - name: CONTAINER_TO_RUN entrypoint: args: options: - optionA - optionB
41. Demo: Use Cloud Build to containerize an application Docker build Docker push /workspace GitHub Google Container Registry
42. Exercise: write a Cloud Build config file 1. Complete cloudbuild.yaml to build and push a Docker image 2. Run gcloud builds submit 3. View the container registry and see the updated container image 4. Commit the latest changes and push to GitHub Docker build Docker push /workspace GitHub Google Container Registry
43. Demo: automated deployment
44. Exercise: add automated deployment 1. Make an application code change (e.g. make a text change to line 18 of app.js) 2. Add a deploy step to cloud build config (see: cloudbuild_deploy_snippet.html) Deploy: 3. gcloud builds submit 4. Refresh the application in your browser and verify the change 5. Commit your changes and push them to GitHub Challenge: modify your Dockerfile to produce a smaller container image
45. Automated build triggers CI/CD System Source Repository Build branch Build PR Push code change to branch notify via webhook Open a new pull request notify via webhook post results to PR Developer
46. Demo: configure triggering
47. Exercise: set up triggering 1. Add a Cloud Build trigger a. Choose first option for trigger: “GitHub <BETA>” b. Say yes to all the things 2. Make a code change → commit → push to GitHub 3. View the running build in Google Cloud Console > Cloud Build History 4. Reload the application to see your change
48. More CI: multi-stage pipelines
49. More CI: Build step sources steps: - name: gcr.io/cloud-builders/<image> GCP-provided builders steps: - name: <image> Dockerhub 1. Clone builder repo (or write your own) 2. Docker build 3. Push to your Google Container Registry Community builders, or write-your-own steps: - name: gcr.io/$PROJECT_ID/<image>
50. Demo: install and test application
51. Exercise: add `npm` build steps 1. Add an ‘install’ step using npm (from Dockerhub) a. Hint: your docker image knows how to pull the node image from dockerhub b. Hint: remember that the command to install is ‘npm install’ 2. Add a ‘test’ step using npm (from Dockerhub) a. Hint: The command to test is ‘npm test’ 3. Git commit and push 4. View build output in Cloud Build > History Got errors? Fix and re-submit until the build is green! CHALLENGE: add a linter step DOUBLE CHALLENGE: run the linter in parallel with the test step
52. Teardown (if desired) delete the repo from your GitHub account Your Google Cloud project and all the resources it contains will be automatically deleted within 1-2 days.
53. Thank you Feedback please! Using the DWJW app... 1. Select "Schedule" on the lower menu bar. 2. Select the workshop title in the mobile app. 3. Scroll to Post-Session Survey within the workshop details. 4. Complete the survey. Stay in touch! @davidstanke craigdbarber Questions?