Slide 1

Slide 1 text

DevOps 環境の構築と運用 @tnir GCPUG DevOps Day Sep 2017, Google Japan

Slide 2

Slide 2 text

@tnir: Takuya Noguchi / tnir.gitlab.io

Slide 3

Slide 3 text

3

Slide 4

Slide 4 text

Agenda Our situation How to build our general CI/CD infrastructure DevOps on GAE Firebase CI/CD 4

Slide 5

Slide 5 text

Our situation 2012: < 10 engineers + 0 partner (for 10 PJs) All engineers for application development No infra/ops => 2017: 20 engineers + 200 partner engineers (for 700 PJs) Not many devs want to move to GCP Few devs know GCP 5

Slide 6

Slide 6 text

Our CI/CD infra ~ 5 yrs ago: Subversion + Jenkins (with jobs broken) + build/deploy locally Started to build our CI/CD infrastructure 6

Slide 7

Slide 7 text

1st try SaaS: wercker etc. There was no CI SaaS Problems: Price GitHub Dif cult to force all devs to use SaaS 7

Slide 8

Slide 8 text

Current state Almost introduce work ow with GitLab GitLab on GCE one of Top 30 open source projects 2017 (Linux Foundation) = GitHub (Enterprise?) + CircleCI Enterprise + Docker Hub Private (Quay Private) See my JTF2017 talk 「GitLab を軸とした、 ビジネスを支える高速 デリバリを実現するアー キテクチャ」 cf. gitlab-org/gitlab-ce#19423 8

Slide 9

Slide 9 text

GAE We use: Python Go SE: Python 2.7 only (while go1.8 for Go 1.8) Java 8 GA today 9

Slide 10

Slide 10 text

GAE makes us ease to introduce CD 10

Slide 11

Slide 11 text

CI/CD on GAE Create .gitlab-ci.yml for GitLab CI/CD: image: google/cloud-sdk stages: - deploy deploy: stage: deploy before_script: - echo "$GCP_CREDENTIAL" | base64 -d | gzip -dc > client-secre - gcloud auth activate-service-account --key-file=client-secre script: - gcloud -q --project $PROJ app deploy app.yaml environment: name: prod only: - master 11

Slide 12

Slide 12 text

cf. similar with Continuous Delivery with Travis CI 12

Slide 13

Slide 13 text

GAE Managed SSL The easiest way to secure system ever 13

Slide 14

Slide 14 text

GAE Managed SSL The easiest way to secure system ever con rmed with newly-created GAE project. 14

Slide 15

Slide 15 text

GAE Managed SSL (2) 15

Slide 16

Slide 16 text

Failed to enable Managed SSL on an old GAE project (last deployed in 2012) Changing CNAME for custom domain: ghs.google.com (old style) -> ghs.googlehosted.com (current style) Succeeded 16

Slide 17

Slide 17 text

Firebase 17

Slide 18

Slide 18 text

CI/CD of Firebase with rebase-tools (ex. rebase-cli) https://github.com/ rebase/ rebase-tools 18

Slide 19

Slide 19 text

Prepare CI settings Starts with: docker run -it -p 9005:9005 node:8.5.0-alpine sh yarn init --yes yarn add firebase-tools yarn add v1.0.2 info No lockfile found. [1/4] Resolving packages... [...] ├ ─ [email protected] └ ─ [email protected] Done in 42.64s. 19

Slide 20

Slide 20 text

❌ The way I noted on Qiita (2016) $(yarn bin)/firebase login -> creates les: ~/.config/configstore/firebase-tools.json ~/.config/configstore/update-notifier-firebase-tools.json Not good for CI 20

Slide 21

Slide 21 text

✔ The recommended way (AFAIK) $(yarn bin)/firebase login:ci Visit this URL on any device to log in: https://accounts.google.com/o/oauth2/auth?client_id=XXXXXXXXX. apps.googleusercontent.com&scope=email%20openid%20https%3A%2F% 2Fwww.googleapis.com%2Fauth%2Fcloudplatformprojects.readonly%2 0https%3A%2F%2Fwww.googleapis.com%2Fauth%2Ffirebase%20https%3A %2F%2Fwww.googleapis.com%2Fauth%2Fcloudplatform&response_type= code&state=XXXXXXXX&redirect_uri=http%3A%2F%2Flocalhost%3A9005 Waiting for authentication... ✔ Success! Use this token to login on a CI server: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Example: firebase deploy --token "$FIREBASE_TOKEN" 21

Slide 22

Slide 22 text

Set up CI Create .gitlab-ci.yml for GitLab CI/CD: image: google/cloud-sdk test: stage: test script: - echo 'test test test' deploy: stage: deploy script: - firebase deploy --token "$FIREBASE_TOKEN" 22

Slide 23

Slide 23 text

Use full image ✔ full image (650MB) image: google/cloud-sdk ❌ Alpine-based image (40MB) image: google/cloud-sdk:alpine (slim (202 MB) may work while I didn't try it yet) 23

Slide 24

Slide 24 text

Pipelining CI/CD stage: test [...] stage: deploy Pipelining stabilizes CI/CD. 24

Slide 25

Slide 25 text

Summary We built our CI/CD and introduced dev work ow on GCP for GCP. We operates our system with less efforts. I personally want to integrate Stackdriver with GItLab expand use cases of Cloud Functions for Firebase. 25

Slide 26

Slide 26 text

More information for DevOps GitLab.JP on Connpass Prometheus Tokyo on Connpass pydata-dev.connpass.com 26

Slide 27

Slide 27 text

GKE Workload monitoring 27