MAD · NOV 23-24 · 2018 Create your CI/CD environment with Google Cloud Build Carlos Coloma (@ccescribano) Developer, otaku, addicted to sports. MAD · NOV 23-24 · 2018 

MAD · NOV 23-24 · 2018 Who am I? ■ Google Certified Cloud Architect ■ Google Cloud Platform and Spring Framework trainer ■ Mr Wolf at Extrema Sistemas ∘ Other people call me “full stack” and stuff ■ I am NOT considered an expert on CI/CD ∘ I know things, and I share things. Just that. 

MAD · NOV 23-24 · 2018 What this talk is NOT about ■ Google Cloud marketing ■ Advanced tips for CI/CD architectures ■ A containers talk ∘ We will rather focus on Cloud Build itself ■ A Happy API/Documentation Reading Session 

MAD · NOV 23-24 · 2018 What this talk IS about ■ Basic concepts to start a real CI/CD system ■ Strengths of the Cloud Build service ■ Things to consider when designing a CI/CD system And in general, my own experience 

MAD · NOV 23-24 · 2018 Life with CI/CD is clean and simple 

MAD · NOV 23-24 · 2018 Theory (a.k.a. The World of Unicorns and Rainbows) ■ Build ■ Test ■ Deploy (optional) ■ Profit! 

MAD · NOV 23-24 · 2018 Real world ■ Different programming languages in your pipeline ■ Runtime versions (node, JVM, python) ■ Source code hosting ■ Deployment target: On-prem, IaaS, Containers, PaaS, FaaS ■ Environments: test, qa, prod ■ Security: Authentication and authorization ■ CI/CD error notifications 

MAD · NOV 23-24 · 2018 Building a pipeline is like trying to drink lemonade through your eyes It doesn’t matter the CI/CD system you use Photo by Nery Montenegro on Unsplash 

MAD · NOV 23-24 · 2018 Our goal is to look for the way that hurts less* *Also known as the Less Shitty Way 

MAD · NOV 23-24 · 2018 Let’s start Hello World 

MAD · NOV 23-24 · 2018 Hello world project 1. Source on GitHub 2. Google Cloud Build config ■ Dockerfile ■ cloudbuild.yaml (see next slides) 3. A trigger listening to changes in master 

MAD · NOV 23-24 · 2018 Google Cloud Build YAML 

MAD · NOV 23-24 · 2018 Steps ■ Independent containers ■ Share code using /workspace ■ Basic attributes ∘ Name ⇒ Cloud Builder ⇒ Container image to use ∘ Args ∘ Env ∘ Entrypoint ■ Example steps: - name: ‘gcr.io/cloud-builders/npm’ args: [‘install’] 

MAD · NOV 23-24 · 2018 Additional config ■ Timeout ■ Disk size ■ Machine type ■ Substitutions ■ Images ■ Secrets ■ Artifacts 

MAD · NOV 23-24 · 2018 Additional config ■ Timeout ■ Disk size ■ Machine type ■ Substitutions ■ Images ■ Secrets ■ Artifacts 

MAD · NOV 23-24 · 2018 Real world requirements 

MAD · NOV 23-24 · 2018 Real world requirements ■ Security ■ Notifications ■ Environments ■ Secrets ■ Custom Cloud Builders 

MAD · NOV 23-24 · 2018 Security ■ Authentication ∘ Cloud Build uses a Service Account ∘ @cloudbuild.gserviceaccount.com · It rotates keys automagically ∘ Maybe we don’t have to configure anything ■ Authorization ∘ Google Cloud IAM ∘ Granular control 

MAD · NOV 23-24 · 2018 Real world requirements ■ Security ■ Notifications ■ Environments ■ Secrets ■ Custom Cloud Builders 

MAD · NOV 23-24 · 2018 Notifications ■ There is a Pub/Sub topic for Cloud Build ■ Dispatches multiple types of notifications ■ A subscriber can listen to these events ∘ You can use Cloud Functions to implement it 

MAD · NOV 23-24 · 2018 Real world requirements ■ Security ■ Notifications ■ Environments ■ Secrets ■ Custom Cloud Builders 

MAD · NOV 23-24 · 2018 ■ Development, CI, sandbox, production ■ The difference is in the config files and the target infrastructure ■ We can use ∘ Environment variables ∘ Substitutions Environments 

MAD · NOV 23-24 · 2018 Environment variables ■ steps: - name: ‘ubuntu’ entrypoint: ‘bash’ args: [‘./my-build-script’] env: [‘MY_ENVIRONMENT_VARIABLE=helloworld’] ■ Later, the script will resolve the environment variable 

MAD · NOV 23-24 · 2018 Substitutions ■ steps: - name: ‘gcr.io/cloud-builders/gcloud’ args: [‘functions’, ‘deploy’, ‘${_NAME}’, ‘--runtime’, ‘nodejs8’, ‘--trigger-http’] substitutions: _NAME: “helloworld-sandbox” #default value ■ gcloud builds submit --config=cloudbuild.yaml --substitutions=_NAME=helloworld-prod 

MAD · NOV 23-24 · 2018 Real world requirements ■ Security ■ Notifications ■ Environments ■ Secrets ■ Custom Cloud Builders 

MAD · NOV 23-24 · 2018 Secrets ■ Store sensitive info: certificates, user/pass pairs, etc ∘ Do it on your own, from scratch ∘ Third party systems like Vault ∘ KMS: A system on Google Cloud designed to keep secrets 

MAD · NOV 23-24 · 2018 Secrets 

MAD · NOV 23-24 · 2018 Secrets 

MAD · NOV 23-24 · 2018 Using Secrets steps: - name: ‘gcr.io/cloud-builders/gradle’ entrypoint: ‘bash’ args: [‘-c’, ‘gradle install -Duser=foo -Dpassword=$$PASSWORD’] secretEnv: [‘PASSWORD’] secrets: kmsKeyName: secretEnv: PASSWORD: 

MAD · NOV 23-24 · 2018 Using Secrets steps: - name: ‘gcr.io/cloud-builders/gradle’ entrypoint: ‘bash’ args: [‘-c’, ‘gradle install -Duser=foo -Dpassword=$$PASSWORD’] secretEnv: [‘PASSWORD’] secrets: kmsKeyName: secretEnv: PASSWORD: 

MAD · NOV 23-24 · 2018 

MAD · NOV 23-24 · 2018 Real world requirements ■ Security ■ Notifications ■ Environments ■ Secrets ■ Custom Cloud Builders 

MAD · NOV 23-24 · 2018 Custom Cloud Builders ■ steps: - name: ‘gcr.io/cloud-builders/npm’ args: [‘install’] ■ Tons of Cloud Builders available ∘ 16 provided by Google Cloud: go, npm, gradle, yarn... ∘ +35 provided by the community: terraform, skaffold, scala, firebase... ∘ You can use any container image 

MAD · NOV 23-24 · 2018 Custom Cloud Builders ■ If no standard builder fits the bill, you can create your own. ■ Create a Docker container image with everything you want ∘ That’s just a Dockerfile. Google doesn’t do anything here. ■ Upload it to any registry ∘ gcloud builds submit . --tag ‘gcr.io//’ ■ Use it in your cloudbuild.yaml! 

MAD · NOV 23-24 · 2018 How is Cloud Build different from managing my own IaaS CI/CD environment? Remember, this is SaaS 

MAD · NOV 23-24 · 2018 What if my CI/CD environment is down? ■ Travis, Jenkins, even Google Cloud Build can be down when you need them ■ You should be able to deploy, even when the CI system is unavailable ■ Use Cloud Build without the Cloud Build Service ∘ That was a serious sentence. In all seriousness. 

MAD · NOV 23-24 · 2018 What if my CI/CD environment is down? ■ Thanks to docker you can launch the CI pipeline in a local machine ∘ Remember, steps are containers ■ What you will need: ∘ Docker ∘ The Google Cloud SDK installed, incl. the cloud-build-local component ∘ A user with the same permissions of the Cloud Build Service Account ■ To launch your pipeline in your local environment: ∘ cloud-build-local --dryrun=false --substitutions=__ENV=sandbox . 

MAD · NOV 23-24 · 2018 Cloud Build can take a long time to build “My CI/CD pipeline is fast” - said noone, ever 

MAD · NOV 23-24 · 2018 Speeding up your builds ■ Check your machine size ∘ You can use machineType and diskSizeGb in cloudbuild.yaml to scale your system up ■ Create custom, more efficient builders ■ Run steps concurrently (see next slide) ■ Cache containers (not covered) ■ Cache directories (not covered) 

MAD · NOV 23-24 · 2018 Run steps concurrently Sequential steps steps: - name: foo ... - name: bar ... - name: baz ... Concurrent steps steps: - name: foo id: A ... - name: bar waitFor: [‘A’] ... - name: baz waitFor: [‘A’] ... 

MAD · NOV 23-24 · 2018 Wrap-up 

MAD · NOV 23-24 · 2018 Google Cloud Build ■ Google Cloud Build is a powerful tool, but starting a CI pipeline from scratch is hard ■ SaaS means that you don’t maintain any server ■ Every step is a container, even if the final artifact does not need to be a container image ■ Can be run locally ■ Security can be configured through IAM and KMS ■ Free tier: 120 minutes per day, free of cost 

MAD · NOV 23-24 · 2018 Create your CI/CD system with Google Cloud Build Carlos Coloma (@ccescribano) Developer, otaku, addicted to sports. MAD · NOV 23-24 · 2018 Thanks