Drone CI/CD 自動化測試及部署

Slide 1

Slide 1 text

Drone CI/CD ⾃動化測試及部署

Slide 2

Slide 2 text

About me • Software Engineer in Mediatek • Member of Drone CI/CD Platform • Member of Gitea Platform • Member of Gin Golang Framework • Teacher of Udemy Platform: Golang + Drone https://blog.wu-boy.com

Slide 3

Slide 3 text

課程⼤綱 • Docker 基礎介紹 • CI/CD 服務簡介 • Drone 架構介紹 • Drone 基礎使⽤ • Drone 外掛撰寫

Slide 4

Slide 4 text

Docker 容器介紹

Slide 5

Slide 5 text

ၚ౷7.㐘ٖػՍߏ

Slide 6

Slide 6 text

VM 虛擬機好處 • 不同的作業系統環境 • 善⽤資源分配 (CPU, RAM, Storage) • 透過 Hypervisor 管理 VM 快照

Slide 7

Slide 7 text

$POUBJOFS༰ثՍߏ

Slide 8

Slide 8 text

Container 容器好處 • 容器檔案非常⼩ (Ubuntu 才⼗幾 MB) • 快速開啟 (不到⼀秒就可以起⼀個 Ubuntu) • 透過 Docker fi le 客製化容器內容 • 快速實踐 Deployment + Operation

Slide 9

Slide 9 text

docker 基礎指令 • docker ps • 列出正在跑的容器 • docker images • 列出已經下載的 Images • docker image prune -a -f • 清除沒在使⽤的 Images (空間如果不夠)

Slide 10

Slide 10 text

• docker container prune -f • 清除沒在跑的容器 • docker login -u xxxx • 登入 docker registry (GitLab 內建, 預設 DockerHub) • docker run -ti ubuntu /bin/bash • 快速建立 ubuntu 容器並且登入 Bash Shell • ⽤於測試及除錯

Slide 11

Slide 11 text

建立 Docker 服務 • docker run -d -p 8081:80 nginx • docker run -d -p 8082:80 nginx • docker run -d -p 8083:80 nginx • -p port 對應關係 (host port : container port) • -d 丟到背景執⾏ • —name 容器命名 • -e 環境變數 • -v 掛載硬碟

Slide 12

Slide 12 text

進入已存在容器 • docker exec -ti name /bin/bash • docker ps 可以知道正在跑的容器 • 除錯常⽤

Slide 13

Slide 13 text

練習 • 在本機端架設底下服務 • redis (port 6380:6379) • mysql (port 3307:3306) • postgres (port 5433:5432) • ⽤ Go 或 Node 測試連線

Slide 14

Slide 14 text

練習 • 架設 Go Module Proxy • https://github.com/gomods/athens • export GOPROXY=http://localhost:3010 • 掛載 local 端空間到 athens

Slide 15

Slide 15 text

建立容器

Slide 16

Slide 16 text

使⽤ Docker fi le

Slide 17

Slide 17 text

FROM alpine:3.15 LABEL maintainer="Bo-Yi Wu " RUN apk add - - no - cache ca - certif i cates & & \ rm - rf /var/cache/apk / * COPY release/linux/amd64/agent /bin/ ENTRYPOINT [ "/bin/agent" ]

Slide 18

Slide 18 text

編譯容器並上傳 • docker build -t appleboy/server:tag -f path . • -t 組織名稱/容器名稱 • -f Docker fi le 檔案路徑 • tag: 版本 (⽤來 deploy production) • docker push host:port/appleboy/server • host: docker registry host (預設上傳到 docker hub) • port: docker registry port

Slide 19

Slide 19 text

多個服務串接

Slide 20

Slide 20 text

使⽤ docker-compose.yml

Slide 21

Slide 21 text

version: '3.1' services: prometheus: image: prom/prometheus:v2.31.1 volumes: - ./prometheus/:/etc/prometheus/ - ./data/prometheus:/prometheus command: - ' - - conf i g.f i le=/etc/prometheus/prometheus.yml' - ' - - storage.tsdb.path=/prometheus' - ' - - web.console.libraries=/usr/share/prometheus/console_libraries' - ' - - web.console.templates=/usr/share/prometheus/consoles' - ' - - storage.tsdb.retention.time=365d' restart: always grafana: image: grafana/grafana:8.2.6 depends_on: - prometheus volumes: - ./data/grafana:/var/lib/grafana - ./grafana/provisioning/:/etc/grafana/provisioning/ env_f i le: - ./grafana/conf i g.monitoring restart: always

Slide 22

Slide 22 text

docker-compose 指令 • docker-compose up -d [service_name] • -d 在背景執⾏ • docker-compose down -v • -v 移除 volume 資料 • docker-compose logs -f [service_name] • 看單⼀服務 Logs • docker-compose pull • 更新全部 Image (每次 deploy 都需要做此步驟) • docker-compose exec db /bin/bash • 進入容器看 DB 資料

Slide 23

Slide 23 text

練習 • 將 Go 服務搭配 redis + (mysql or postgres) • redis • mysql • postgres • ⽤ docker-compose 串起上⾯服務

Slide 24

Slide 24 text

部署流程 GitHub Flow vs Git Flow IUUQTCMPHXVCPZDPNHJUIVC fl PXWTHJU fl PX

Slide 25

Slide 25 text

(JU)VC'MPX 14 Develop Git Push Git Tag Develop Git Push Git Tag Testing Deploy Deploy Deploy Production Staging Production Testing Deploy Staging

Slide 26

Slide 26 text

GitHub Flow + Git Flow in opensource IUUQTHJUIVCDPNHPHJUFBHJUFB

Slide 27

Slide 27 text

如何⾃動化上⾯的步驟?

Slide 28

Slide 28 text

為什麼我不選 Jenkins?

Slide 29

Slide 29 text

Why? • Complicated project setting • Write the plugin (Java language) • Maintenance? • Learning Curve? • Grow your team?

Slide 30

Slide 30 text

為什麼我不選 GitLab CI?

Slide 31

Slide 31 text

Why? • GitLab Only • Not Extensible

Slide 32

Slide 32 text

Drone CI/CD 介紹

Slide 33

Slide 33 text

Drone CI • Container native CI/CD platform • Easy to install & maintain • Isolate builds • Simple YAML Con fi guration • Integrates with several VCS Providers • Rich set of of fi cial plugins (any container can be a plugin) • Execute locally with simple command (drone exec) • open source (https://github.com/harness/drone)

Slide 34

Slide 34 text

Drone CI/CD 平台真的免費嗎 IUUQTCMPHXVCPZDPNESPOFMJDFOTF

Slide 35

Slide 35 text

Everything is a Docker Container

Slide 36

Slide 36 text

Project CI/CD Flow HJUDMPOF UFTUJOH EFQMPZ OPUJGZ

Slide 37

Slide 37 text

Drone CI Infrastructure Runner Server Step 1 git clone Step 2 make build Step 3 deploy app work space extra service Runner

Slide 38

Slide 38 text

如何安裝 Drone + GitHub

Slide 39

Slide 39 text

申請 GitHub Application

Slide 40

Slide 40 text

Apply GitHub Application https: / / github.com/settings/developers

Slide 41

Slide 41 text

https: / / github.com/settings/applications/new

Slide 42

Slide 42 text

Drone Server and Runner In docker-compose.yml

Slide 43

Slide 43 text

version: '3' services: drone - server: image: drone/drone:2 ports: - 8081 : 80 volumes: - ./:/data restart: always environment: - DRONE_SERVER_HOST=${DRONE_SERVER_HOST} - DRONE_SERVER_PROTO=${DRONE_SERVER_PROTO} - DRONE_RPC_SECRET=${DRONE_RPC_SECRET} # GitHub Conf i g - DRONE_GITHUB_SERVER=https: / / github.com - DRONE_GITHUB_CLIENT_ID=${DRONE_GITHUB_CLIENT_ID} - DRONE_GITHUB_CLIENT_SECRET=${DRONE_GITHUB_CLIENT_SECRET} - DRONE_LOGS_PRETTY=true - DRONE_LOGS_COLOR=true $SFBUFEPDLFSDPNQPTFZNM

Slide 44

Slide 44 text

DRONE_SERVER_HOST= DRONE_SERVER_PROTO= DRONE_RPC_SECRET= DRONE_GITHUB_CLIENT_ID= DRONE_GITHUB_CLIENT_SECRET= $SFBUFFOW fi MF

Slide 45

Slide 45 text

drone - runner: image: drone/drone - runner - docker:1 restart: always depends_on: - drone - server ports: - 3000 : 3000 volumes: - /var/run/docker.sock:/var/run/docker.sock environment: - DRONE_RPC_HOST=${DRONE_SERVER_HOST} - DRONE_RPC_PROTO=${DRONE_SERVER_PROTO} - DRONE_RPC_SECRET=${DRONE_RPC_SECRET} - DRONE_RUNNER_CAPACITY=2 - DRONE_RUNNER_NAME=my - f i rst - runner

Slide 46

Slide 46 text

Drone Web UI Introduction

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

撰寫 .drone.yml 檔案

Slide 52

Slide 52 text

kind: pipeline type: docker name: default steps: - name: greeting image: alpine commands: - echo hello - echo world

Slide 53

Slide 53 text

kind: pipeline type: docker name: linux_amd64 steps: - name: build image: golang:1.17 commands: - make build_linux_amd64

Slide 54

Slide 54 text

build_linux_amd64 : CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \ - v - a - o release/linux/amd64/helloworld

Slide 55

Slide 55 text

build_linux_amd64 : CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \ - v - a - o release/linux/amd64/helloworld

Slide 56

Slide 56 text

- - - kind: pipeline type: docker name: linux_amd64 steps: - name: build image: golang:1.17 environment: GOOS : linux GOARCH : amd64 CGO_ENABLED : 0 commands: - make build_linux_amd64

Slide 57

Slide 57 text

build: go build - v - a - o \ release/${GOOS}/${GOARCH}/helloworld

Slide 58

Slide 58 text

kind: pipeline type: docker name: linux_amd64 steps: - name: build image: golang:1.17 commands: - make build_linux_amd64 - - - kind: pipeline type: docker name: linux_i386 steps: - name: build image: golang:1.17 commands: - make build_linux_i386 .VMUJQMF1JQFMJOF

Slide 59

Slide 59 text

- - - kind: pipeline type: docker name: linux_i386 steps: - name: build image: golang:1.17 environment: GOOS : linux GOARCH : 386 CGO_ENABLED : 0 commands: - make build_linux_i386 $VTUPN&OWJSPONFOU

Slide 60

Slide 60 text

- - - kind: pipeline type: docker name: linux_i386 environment: GOOS : linux GOARCH : 386 CGO_ENABLED : 0 steps: - name: build image: golang:1.17 commands: - make build_linux_i386 (MPCBM&OWJSPONFOU

Slide 61

Slide 61 text

Routing 將特定任務丟到特定主機 (GPU or High Memory)

Slide 62

Slide 62 text

kind: pipeline type: docker name: default steps: - name: build image: golang commands: - go build - go test node: keyA : valueA keyB : valueB %30/&@36//&3@-"#&-4LFZ"WBMVF" LFZ#WBMVF#

Slide 63

Slide 63 text

Parallelism 平⾏處理

Slide 64

Slide 64 text

kind: pipeline type: docker name: default steps: - name: backend image: golang commands: - go build - go test - name: frontend image: node commands: - npm install - npm test - name: notify image: plugins/slack depends_on: - frontend - backend

Slide 65

Slide 65 text

Conditions and Triggers

Slide 66

Slide 66 text

kind: pipeline type: docker name: default steps: - name: build image: golang commands: - go build - go test when: branch: - master - feature / * kind: pipeline type: docker name: default steps: - name: build image: golang commands: - go build - go test trigger: branch: - master

Slide 67

Slide 67 text

Volumes

Slide 68

Slide 68 text

kind: pipeline type: docker name: default steps: - name: test image: golang volumes: - name: cache path: /go commands: - go get - go test - name: build image: golang volumes: - name: cache path: /go commands: - go build volumes: - name: cache temp: {}

Slide 69

Slide 69 text

練習 • 任何程式語⾔進⾏ Lint + Testing + Build • Go 語⾔使⽤ golangci-lint • 測試 go test -v -cover • 編譯 go build -o release/linux/amd64/out

Slide 70

Slide 70 text

Plugin IUUQQMVHJOTESPOFJP

Slide 71

Slide 71 text

- name: publish pull: always image: plugins/docker:linux - amd64 settings: auto_tag: true auto_tag_suff i x: linux - amd64 cache_from: appleboy/gorush daemon_off: false dockerf i le: docker/Dockerf i le.linux.amd64 password: from_secret: docker_password repo: appleboy/gorush username: from_secret: docker_username when: event: exclude: - pull_request

Slide 72

Slide 72 text

Slide 73

Slide 73 text

No content

Slide 74

Slide 74 text

Plugin Input

Slide 75

Slide 75 text

Slide 76

Slide 76 text

PLUGIN_USERNAME=appleboy PLUGIN_PASSWORD=password PLUGIN_REPO=appleboy/gorush PLUGIN_TAGS=1.0.0,1.0

Slide 77

Slide 77 text

PLUGIN_USERNAME=appleboy PLUGIN_PASSWORD=password PLUGIN_REPO=appleboy/gorush PLUGIN_TAGS=1.0.0,1.0

Slide 78

Slide 78 text

如何製作 Plugins

Slide 79

Slide 79 text

Go Plugin vs Bash Plugin (Any language you want)

Slide 80

Slide 80 text

Bash Plugin

Slide 81

Slide 81 text

kind: pipeline type: docker name: default steps: - name: webhook image: acme/webhook settings: url: http: / / hook.acme.com method: post body: | hello world

Slide 82

Slide 82 text

# ! /bin/sh curl \ -X ${PLUGIN_METHOD} \ - d ${PLUGIN_BODY} \ ${PLUGIN_URL}

Slide 83

Slide 83 text

FROM alpine:3.14 ADD script.sh /bin/ RUN chmod + x /bin/script.sh RUN apk -Uuv add curl ca - certif i cates ENTRYPOINT /bin/script.sh

Slide 84

Slide 84 text

$ docker build -t acme/webhook . $ docker push acme/webhook $ docker run --rm \ -e PLUGIN_METHOD=post \ -e PLUGIN_URL=http://hook.acme.com \ -e PLUGIN_BODY=hello \ acme/webhook ฤᩄ

Slide 85

Slide 85 text

$ docker build -t acme/webhook . $ docker push acme/webhook $ docker run --rm \ -e PLUGIN_METHOD=post \ -e PLUGIN_URL=http://hook.acme.com \ -e PLUGIN_BODY=hello \ acme/webhook ଌࢼ

Slide 86

Slide 86 text

Go Plugin

Slide 87

Slide 87 text

package main import ( "net/http" "os" "strings" ) func main() { body : = strings.NewReader( os.Getenv("PLUGIN_BODY"), ) _, err : = http.NewRequest( os.Getenv("PLUGIN_METHOD"), os.Getenv("PLUGIN_URL"), body, ) if err ! = nil { os.Exit(1) } } GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o webhook

Slide 88

Slide 88 text

FROM alpine:3.14 ADD webhook /bin/ RUN apk -Uuv add ca - certif i cates ENTRYPOINT /bin/webhook

Slide 89

Slide 89 text

練習 • 任何程式語⾔進⾏ Plugin 撰寫 • 申請 dockerhub 帳號密碼 • 透過 Drone ⾃動打包 Image 並上傳

Slide 90

Slide 90 text

Deploy to Production Server via SSH Command

Slide 91

Slide 91 text

- name: ssh commands image: appleboy/drone - ssh settings: host: foo.com username: root key: from_secret: ssh_key passphrase: 1234 port: 22 script: - mkdir abc/def/efg - echo "you can't see the steps."

Slide 92

Slide 92 text

Promotions ⼿動部署

Slide 93

Slide 93 text

kind: pipeline type: docker name: default steps: - name: test image: node commands: - npm install - npm run test - npm run bundle - name: deploy image: appleboy/drone - ssh settings: . . . trigger: event: - promote target: - production kind: pipeline type: docker name: deploy steps: - name: test image: node commands: - npm install - npm run test - npm run bundle - name: deploy image: appleboy/drone - ssh settings: . . . when: event: - promote target: - production

Slide 94

Slide 94 text

$ drone build promote octocat/hello-world 42 staging $ drone build promote $ drone build promote octocat/hello-world 42 production

Slide 95

Slide 95 text

Thanks