GCP+GKE Deep Dive Part 1: Initial App Development

Slide 1

Slide 1 text

PART 1 13:00~ Initial App Deployment

Slide 2

Slide 2 text

팖뼣켆푢 !

Slide 3

Slide 3 text

PART 1 Initial App Deployment GitLab CEܳ GCP+GKEী ߓನ೧ࠇद׮ Advanced Cluster Management ৈ۞ ௿۞झఠ ਍৔ ప௼ץਸ ࣗѐ೤פ׮ PART 2 13:00-13:50 14:00-14:50

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Google Kubernetes Engine펞 얺큲캫컿쭎컪찒큲짾밚힎

Slide 8

Slide 8 text

kubernetes

Slide 9

Slide 9 text

핂뻖큲흂잏 Container Scheduling 핞솧핺쫃묺 Auto-healing 컪찒큲싢큲쩒읺 Service Discovery 컲헣뫎읺 Conﬁg Management 쭎쭒칾  Load Balancing 슿슿˘ kubernetes

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

Kubernetes Engine

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

픒짾쫓킪삲

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

Open Source Ruby on Rails엖핒풚 PostgreSQL섾핂쩮핂큲 Redis핟펓짝킪섾핂쩮핂큲 Community Edition

Slide 17

Slide 17 text

Cloud SQL FOR POSTGRESQL Cloud Memorystore FOR REDIS

Slide 18

Slide 18 text

✅ 훎옪힎풞 ✅ 잲삖힎슪컪찒큲 ✅ 큲핊잏힎풞 ✅ High Availability묺컿힎풞 Cloud SQL Cloud Memorystore

Slide 19

Slide 19 text

GitLab Deployment Multiple Pods GitLab Deployment Multiple Pods GitLab Service GitLab Ingress GLBC Cloud HTTP(S) Load Balancer Postgres Cloud SQL Instance Redis Cloud Memorystore Instance Kubernetes Engine Google Cloud Platform

Slide 20

Slide 20 text

킪핟믾헒 • 핂뻖펞샎핂퐎캏헏핆팮읺핂켦짾뫊헣펞샎 핂읊헒헪옪삖삲 ˖ 잶슪않핆솒묺픦칺푷픒믾쫆픊옪삖삲 • Google Cloud SDK (gcloud)짝  Kubernetes CLI (kubectl) 픒칺푷삖삲

Slide 21

Slide 21 text

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

$ gcloud container clusters create hello-gke \ --project=shakr-openinfra-demo \ --zone=asia-northeast1-b \ --cluster-version=1.10.4-gke.2 \ --machine-type=n1-standard-1 \ --num-nodes=3 \ --enable-ip-alias \ --enable-autorepair

Slide 24

Slide 24 text

--enable-autorepair 쫃묺많푢Node많핖픒쌚핞솧픊옪쫃묺쿦 ˖ Node Health Check많킲쁢몋푾 ˖ Node많status쫂몮읊힎팘쁢몋푾 ˖ Node픦쭎싢큲핢펺푷얗핂펔쁢몋푾

Slide 25

Slide 25 text

--enable-ip-alias VPC뻲풚픦CIDR쯢옫픒칺푷펺Pod IP읊샇 ˖ GKE퐎VPC많컪옪맧픎CIDR쯢옫픒칺푷쿦핖픚 ˖ GKE Cluster 짤펞컪솒VPC 뺂않졂 Pod IP펞헟믊많쁳 ˖ Proxy 펔핂Cloud Memorystore헟믊많쁳

Slide 26

Slide 26 text

$ gcloud container clusters create hello-gke \ --project=shakr-openinfra-demo \ --zone=asia-east1-b \ --cluster-version=1.10.4-gke.2 \ --machine-type=n1-standard-1 \ --num-nodes=3 \ --enable-ip-alias \ --enable-autorepair

Slide 27

Slide 27 text

--project=shakr-openinfra-demo \ --zone=asia-east1-b \ --cluster-version=1.10.4-gke.2 \ --machine-type=n1-standard-1 \ --num-nodes=3 \ --enable-ip-alias \ --enable-autorepair Creating cluster hello-gke...done. Created [https://container.googleapis.com/v1/projects/shakr- openinfra-demo/zones/asia-northeast1-b/clusters/hello-gke]. NAME LOCATION MASTER_VERSION MASTER_IP hello-gke asia-northeast1-b 1.10.4-gke.2 35.200.25.152

Slide 28

Slide 28 text

Slide 29

Slide 29 text

$ gcloud sql instances create gitlab-postgresql \ --availability-type=regional \ --cpu=1 --memory=4GiB \ --database-version=POSTGRES_9_6 \ --region=asia-east1 \ --storage-size=10GB \ --storage-type=SSD \ --storage-auto-increase $ gcloud sql users create gitlab % \ --instance=gitlab-postgresql --password=mySecurePassword! Cloud SQL핆큲큲캫컿

Slide 30

Slide 30 text

$ gcloud alpha redis instances create gitlab-redis \ --region=asia-east1 \ --size=2 Cloud Memorystore핆큲큲캫컿

Slide 31

Slide 31 text

Service Account

Slide 32

Slide 32 text

Service Account • GCP컪찒큲펞헟믊쁢맏팮읺핂켦픦킮풞픒샎 • 읺콚큲쪒뭚쭎펺많쁳 • JSON킫픦읊슫펺칺푷 • GKE Pod펞컪Cloud SQL헟믊킪칺푷

Slide 33

Slide 33 text

$ gcloud iam service-accounts create gitlab \ --display-name="GitLab Service Account" Service Account캫컿

Slide 34

Slide 34 text

$ gcloud projects add-iam-policy-binding $PROJECT \ --member="serviceAccount:$EMAIL"\ --role="roles/cloudsql.client" # Service Accountী Cloud Storage Admin Roleਸ ೡ׼ $ gcloud projects add-iam-policy-binding $PROJECT \ --member="serviceAccount:$EMAIL"\ --role="roles/storage.admin" 뭚쭎펺

Slide 35

Slide 35 text

$ gcloud iam service-accounts keys create \ ./artifacts/serviceaccount.json \ --iam-account $EMAIL 슫

Slide 36

Slide 36 text

{ "type": "service_account", "project_id": "shakr-openinfra-demo", "private_key_id": "1234567890abcdef1234567890", "private_key": "-----BEGIN PRIVATE KEY-----\n....\n-----END PRIVATE KEY-----\n", "client_email": "gitlab@shakr-openinfra- demo.iam.gserviceaccount.com", "client_id": "12345678901234567890", "auth_uri": "https://accounts.google.com/o/oauth2/auth", "token_uri": "https://accounts.google.com/o/oauth2/token", JSON

Slide 37

Slide 37 text

Pod & Deployment

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Pod • Kubernetes뺂많핳핟픎픦삶퓒 • 빦픎펺얺맪픦핂뻖옪묺컿 • Pod팖픦핂뻖쁢헎핳뫃맒슿픒뫃퓮 • 믾쫆헏픊옪홓욚킪졶슮섾핂많칻헪쇶 • 푢킪PersistentVolume슿픒칺푷펺핊퓮힎많쁳

Slide 40

Slide 40 text

Deployment • Instance Group(Auto-Scaling Group)뫊맧픎맪뼞 • Pod픒짾몮뫎읺훚 • replica읊힎헣훊졂믆쿦잚Pod픒많픎칻헪 • Rolling update 짝 rollback 힎풞

Slide 41

Slide 41 text

apiVersion: apps/v1 kind: Deployment metadata: name: gitlab labels: app: gitlab spec: replicas: 1 selector: matchLabels: app: gitlab template: metadata: labels: apps: gitlab spec: containers: - name: gitlab image: gitlab/gitlab-ce:latest resources: requests: cpu: "0.5" memory: 1Gi env: - name: GITLAB_OMNIBUS_CONFIG value: ... deployment.yml Deployment spec Pod spec

Slide 42

Slide 42 text

spec: containers: - name: gitlab deployment.yml (pod spec) - name: cloudsql-proxy

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

Cloud SQL Proxy • Cloud SQL픎Memorystore퐎삲읂멚VPC IP옪짢옪헟믊쿦 펔몮몮헣*1샎펻픒whitelisting쁢짷킫핂않GKE얺큲펞컪 칺푷핂삲콚쭖 • Cloud SQL Proxy읊핂푷졂Service Account픦Cloud SQL Client Role옪옪옫킪읊폂쿦핖삲 • Google펞컪맪짪펺짢핂뻖읺짝Docker핂짆힎짾

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

spec: containers: - name: gitlab image: gitlab/gitlab-ce:latest deployment.yml (pod spec) - name: cloudsql-proxy image: gcr.io/cloudsql-docker/gce-proxy command: ["/cloud_sql_proxy", "-instances=..."]

Slide 47

Slide 47 text

spec: containers: - name: gitlab image: gitlab/gitlab-ce:latest env: - name: GITLAB_OMNIBUS_CONFIG value: ... deployment.yml (pod spec) - name: cloudsql-proxy image: gcr.io/cloudsql-docker/gce-proxy command: ["/cloud_sql_proxy", "-instances=..."] env: - name: GOOGLE_APPLICATION_CREDENTIALS value: ... #

Slide 48

Slide 48 text

Slide 49

Slide 49 text

Secret

Slide 50

Slide 50 text

Secret • Kubernetes얺큲펞컪짊맞헣쫂읊헎핳쌚칺푷 • GKE샎킪쫂슪펞컪쿶멶힞읺 • 짊맞힎팘픎몋쪎쿦슿픒헎핳쌚쁢찒킅묺혾핆ConﬁgMap 픒칺푷

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

apiVersion: v1 kind: Secret metadata: name: my-secrets type: Opaque data: GOOGLE_CLOUD_KEYFILE_JSON: SENTRY_DSN: secret.example.yml

Slide 53

Slide 53 text

apiVersion: v1 kind: Secret metadata: name: my-secrets type: Opaque data: GOOGLE_CLOUD_KEYFILE_JSON: eyJseXJpY3MiOiAiV2UncmUgbm8gc3RyYW5nZXJzIHRvIGxvdmUNCllvdSBr  bm93IHRoZSBydWxlcyBhbmQgc28gZG8gSQ0KQSBmdWxsIGNvbW1pdG1lbnQncyB3aGF0IEknbSB0aGlua2luZyBvZ g0KWW91IHdvdWxkbid0IGdldCB0aGlzIGZyb20gYW55IG90aGVyIGd1eQ0KSSBqdXN0IHdhbm5hIHRlbGwgeW91IG hvdyBJJ20gZmVlbGluZw0KR290dGEgbWFrZSB5b3UgdW5kZXJzdGFuZA0KTmV2ZXIgZ29ubmEgZ2l2ZSB5b3UgdXA NCk5ldmVyIGdvbm5hIGxldCB5b3UgZG93bg0KTmV2ZXIgZ29ubmEgcnVuIGFyb3VuZCBhbmQgZGVzZXJ0IHlvdQ0K TmV2ZXIgZ29ubmEgbWFrZSB5b3UgY3J5DQpOZXZlciBnb25uYSBzYXkgZ29vZGJ5ZQ0KTmV2ZXIgZ29ubmEgdGVsb CBhIGxpZSBhbmQgaHVydCB5b3UifQ== SENTRY_DSN: aHR0cHM6Ly9yaWNrOmFzaGxleUBuZXZlcmdvbm5hZ2l2ZXlvdS51cDo1MzIxNA== secret.example.yml

Slide 54

Slide 54 text

$ kubectl create secret generic gitlab-config \ --from-literal=redis_host=10.0.0.3 \ --from-file=./artifacts/gitlab.rb \ --from-file=./artifacts/serviceaccount.json kubectl픒핂푷Secret캫컿

Slide 55

Slide 55 text

spec: containers: - name: gitlab image: gitlab/gitlab-ce:latest env: - name: GITLAB_OMNIBUS_CONFIG valueFrom: secretKeyRef: name: gitlab-config key: gitlab.rb deployment.yml (pod spec) - name: cloudsql-proxy image: gcr.io/cloudsql-docker/gce-proxy command: ["/cloud_sql_proxy", "-instances=..."] env: - name: GOOGLE_APPLICATION_CREDENTIALS value: /mnt/config/serviceaccount.json volumeMounts: - name: config mountPath: /mnt/config readOnly: true volumes: - name: config secret: secretName: gitlab-config #

Slide 56

Slide 56 text

Slide 57

Slide 57 text

"GitLab픦 Git 섾핂쁢펂싢펞헎핳쇦힎?"

Slide 58

Slide 58 text

RTFM

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Volume 잖풂픦홓윦 • Secret • ConﬁgMap • gcePersistentDisk • hostPath • emptyDir • persistentVolumeClaim • …

Slide 63

Slide 63 text

$ gcloud compute disks create my-disk \ --size=10GB \ --type=pd-ssd \ --zone asia-east1-a \ ... Compute Engine Persistent Disk 캫컿 &

Slide 64

Slide 64 text

Volume 잖풂픦홓윦 • Secret • ConﬁgMap • gcePersistentDisk • hostPath • emptyDir • persistentVolumeClaim • …

Slide 65

Slide 65 text

PersistentVolumeClaim

Slide 66

Slide 66 text

PersistentVolumeClaim • 섾핂많쫂홂쇦쁢싢큲읊 Kubernetes펞푢 • GCE Persistent Disk읊핞솧픊옪캫컿몮믆펞캏픟쁢 PersistentVolume픒캫컿 • StorageClass옪Persistent Disk픦홓윦읊힎헣

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ssd provisioner: kubernetes.io/gce-pd parameters: type: pd-ssd storageclass.yml

Slide 71

Slide 71 text

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: gitlab-data spec: accessModes: - ReadWriteOnce storageClassName: ssd resources: requests: storage: 50Gi pvc.yml

Slide 72

Slide 72 text

spec: containers: - name: gitlab image: gitlab/gitlab-ce:latest env: - name: GITLAB_OMNIBUS_CONFIG valueFrom: secretKeyRef: name: gitlab-config key: gitlab.rb volumeMounts: - name: gitlab-data mountPath: /var/opt/gitlab volumes: - name: gitlab-data persistentVolumeClaim: claimName: gitlab-data deployment-with-pvc.yml (pod spec) - name: cloudsql-proxy image: gcr.io/cloudsql-docker/gce-proxy command: ["/cloud_sql_proxy", "-instances=..."] env: - name: GOOGLE_APPLICATION_CREDENTIALS value: /mnt/config/serviceaccount.json volumeMounts: - name: config mountPath: /mnt/config readOnly: true volumes: - name: config secret: secretName: gitlab-config

Slide 73

Slide 73 text

짾

Slide 74

Slide 74 text

$ kubectl apply -f deployment.yml kubectl옪Deployment캫컿

Slide 75

Slide 75 text

Demo

Slide 76

Slide 76 text

$ kubectl port-forward POD_NAME 8080:80 옪Port forwarding proxy 캫컿 #

Slide 77

Slide 77 text

"칺푷핞많GitLab펞펂쎉멚헟콛힎

Slide 78

Slide 78 text

Service & Ingress

Slide 79

Slide 79 text

Service • ౠ੿ Pod ٜਸ Kubernetes 얺큲뺂펞컪펂쎉멚헟믊퍊쁢힎 헣픦쁢짷쩣 • NodePort핓Worker Node픦읊샇 • Loadbalancer핓TCP Load Balancer 캫컿(GCP) • Internal/External 졶숞힎풞

Slide 80

Slide 80 text

$ kubectl expose deployment gitlab \ --port=80 --target-port=80 --type=NodePort kubectl옪Service캫컿

Slide 81

Slide 81 text

apiVersion: v1 kind: Service metadata: name: gitlab spec: selector: app: gitlab ports: - port: 80 protocol: TCP name: http type: NodePort service.yml

Slide 82

Slide 82 text

apiVersion: v1 kind: Service metadata: name: gitlab spec: selector: app: gitlab ports: - port: 80 protocol: TCP name: http type: NodePort service.yml

Slide 83

Slide 83 text

apiVersion: v1 kind: Service metadata: name: gitlab spec: selector: app: gitlab ports: - port: 80 protocol: TCP name: http - port: 443 protocol: TCP name: https type: NodePort service-multiport.yml

Slide 84

Slide 84 text

Ingress • Service৬ Public internet픒펾멾훊쁢읺콚큲 • Service펞type=LoadBalancer읊칺푷졂짦슪킪푢힎쁢팘픚 • 옲얺많않푾픒샂샇(NGINX, Traeﬁk, …) • GKE펞컪쁢Cloud HTTP(S) Load Balancer읊칺푷 • IPv6/SSL Termination, CDN, HTTP->HTTPS Redirect슿  삲퍟믾쁳힎풞 BETA

Slide 85

Slide 85 text

apiVersion: extensions/v1beta1 kind: Ingress metadata: name: gitlab namespace: default spec: backend: serviceName: gitlab servicePort: 80 ingress.yml

Slide 86

Slide 86 text

apiVersion: extensions/v1beta1 kind: Ingress metadata: name: gitlab namespace: default spec: tls: - secretName: tls-gitlab backend: serviceName: gitlab servicePort: 80 ingress.yml

Slide 87

Slide 87 text

$ kubectl create secret tls tls-gitlab \ --cert=./artifacts/tls/cert.crt \ --key=./artifacts/tls/key.key kubectl옪TLS Secret캫컿

Slide 88

Slide 88 text

apiVersion: extensions/v1beta1 kind: Ingress metadata: name: gitlab namespace: default spec: tls: - secretName: tls-gitlab backend: serviceName: gitlab servicePort: 80 ingress.yml

Slide 89

Slide 89 text

apiVersion: extensions/v1beta1 kind: Ingress metadata: name: gitlab namespace: default annotations: kubernetes.io/ingress.allow-http: "false" spec: tls: - secretName: tls-gitlab backend: serviceName: gitlab servicePort: 80 ingress.yml '

Slide 90

Slide 90 text

apiVersion: extensions/v1beta1 kind: Ingress metadata: name: gitlab namespace: default annotations: kubernetes.io/ingress.allow-http: "false" kubernetes.io/ingress.global-static-ip-name: "gitlab" spec: tls: - secretName: tls-gitlab backend: serviceName: gitlab servicePort: 80 ingress.yml

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

$ gcloud compute addresses create gitlab --global gcloud옪Regional IP캫컿

Slide 93

Slide 93 text

Demo

Slide 94

Slide 94 text

No content

Slide 95

Slide 95 text

apiVersion: extensions/v1beta1 kind: Ingress metadata: name: gitlab namespace: default spec: tls: - secretName: tls-a - secretName: tls-b rules: - host: a.exmaple.com http: paths: - backend: serviceName: a servicePort: 80 - host: b.example.com http: paths: - backend: serviceName: b servicePort: 80 ingress-advanced.yml

Slide 96

Slide 96 text

Recap

Slide 97

Slide 97 text

https://twitter.com/tenderlove/status/988887936128040960

Slide 98

Slide 98 text

Slide 99

Slide 99 text

No content

Slide 100

Slide 100 text

No content

Slide 101

Slide 101 text

No content

Slide 102

Slide 102 text

No content

Slide 103

Slide 103 text

No content

Slide 104

Slide 104 text

No content

Slide 105

Slide 105 text

No content

Slide 106

Slide 106 text

맞칺삖삲 (

Slide 107

Slide 107 text

GCP+GKE Deep Dive Minku Lee CTO Shakr Shakr펞컪쁳엳핖쁢펢힎삖펂읊졶킻삖삲 careers.shakr.com