Development and Operations of DevOps

DevOps環境の構築と運用 GCPUG Tokyo DevOps Day September 2017

Transcript

1. DevOps 環境の構築と運用 @tnir GCPUG DevOps Day Sep 2017, Google Japan
2. @tnir: Takuya Noguchi / tnir.gitlab.io
4. Agenda Our situation How to build our general CI/CD infrastructure DevOps on GAE Firebase CI/CD 4
5. Our situation 2012: < 10 engineers + 0 partner (for 10 PJs) All engineers for application development No infra/ops => 2017: 20 engineers + 200 partner engineers (for 700 PJs) Not many devs want to move to GCP Few devs know GCP 5
6. Our CI/CD infra ~ 5 yrs ago: Subversion + Jenkins (with jobs broken) + build/deploy locally Started to build our CI/CD infrastructure 6
7. 1st try SaaS: wercker etc. There was no CI SaaS Problems: Price GitHub Dif cult to force all devs to use SaaS 7
8. Current state Almost introduce work ow with GitLab GitLab on GCE one of Top 30 open source projects 2017 (Linux Foundation) = GitHub (Enterprise?) + CircleCI Enterprise + Docker Hub Private (Quay Private) See my JTF2017 talk 「GitLab を軸とした、 ビジネスを支える高速 デリバリを実現するアー キテクチャ」 cf. gitlab-org/gitlab-ce#19423 8
9. GAE We use: Python Go SE: Python 2.7 only (while go1.8 for Go 1.8) Java 8 GA today 9
10. GAE makes us ease to introduce CD 10
11. CI/CD on GAE Create .gitlab-ci.yml for GitLab CI/CD: image: google/cloud-sdk stages: - deploy deploy: stage: deploy before_script: - echo "$GCP_CREDENTIAL" | base64 -d | gzip -dc > client-secre - gcloud auth activate-service-account --key-file=client-secre script: - gcloud -q --project $PROJ app deploy app.yaml environment: name: prod only: - master 11
12. cf. similar with Continuous Delivery with Travis CI 12
13. GAE Managed SSL The easiest way to secure system ever 13
14. GAE Managed SSL The easiest way to secure system ever con rmed with newly-created GAE project. 14
15. GAE Managed SSL (2) 15
16. Failed to enable Managed SSL on an old GAE project (last deployed in 2012) Changing CNAME for custom domain: ghs.google.com (old style) -> ghs.googlehosted.com (current style) Succeeded 16
17. Firebase 17
18. CI/CD of Firebase with rebase-tools (ex. rebase-cli) https://github.com/ rebase/ rebase-tools 18
19. Prepare CI settings Starts with: docker run -it -p 9005:9005 node:8.5.0-alpine sh yarn init --yes yarn add firebase-tools yarn add v1.0.2 info No lockfile found. [1/4] Resolving packages... [...] ├ ─ yargs@8.0.2 └ ─ zip-stream@0.6.0 Done in 42.64s. 19
20. ❌ The way I noted on Qiita (2016) $(yarn bin)/firebase login -> creates les: ~/.config/configstore/firebase-tools.json ~/.config/configstore/update-notifier-firebase-tools.json Not good for CI 20
21. ✔ The recommended way (AFAIK) $(yarn bin)/firebase login:ci Visit this URL on any device to log in: https://accounts.google.com/o/oauth2/auth?client_id=XXXXXXXXX. apps.googleusercontent.com&scope=email%20openid%20https%3A%2F% 2Fwww.googleapis.com%2Fauth%2Fcloudplatformprojects.readonly%2 0https%3A%2F%2Fwww.googleapis.com%2Fauth%2Ffirebase%20https%3A %2F%2Fwww.googleapis.com%2Fauth%2Fcloudplatform&response_type= code&state=XXXXXXXX&redirect_uri=http%3A%2F%2Flocalhost%3A9005 Waiting for authentication... ✔ Success! Use this token to login on a CI server: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Example: firebase deploy --token "$FIREBASE_TOKEN" 21
22. Set up CI Create .gitlab-ci.yml for GitLab CI/CD: image: google/cloud-sdk test: stage: test script: - echo 'test test test' deploy: stage: deploy script: - firebase deploy --token "$FIREBASE_TOKEN" 22
23. Use full image ✔ full image (650MB) image: google/cloud-sdk ❌ Alpine-based image (40MB) image: google/cloud-sdk:alpine (slim (202 MB) may work while I didn't try it yet) 23
24. Pipelining CI/CD stage: test [...] stage: deploy Pipelining stabilizes CI/CD. 24
25. Summary We built our CI/CD and introduced dev work ow on GCP for GCP. We operates our system with less efforts. I personally want to integrate Stackdriver with GItLab expand use cases of Cloud Functions for Firebase. 25
26. More information for DevOps GitLab.JP on Connpass Prometheus Tokyo on Connpass pydata-dev.connpass.com 26
27. GKE Workload monitoring 27