Drone CI/CD 自動化測試及部署

Drone CI/CD

⾃動化測試及部署

View Slide

About me
• Software Engineer in Mediatek

• Member of Drone CI/CD Platform

• Member of Gitea Platform

• Member of Gin Golang Framework

• Teacher of Udemy Platform: Golang + Drone
https://blog.wu-boy.com

View Slide

課程⼤綱
• Docker 基礎介紹

• CI/CD 服務簡介

• Drone 架構介紹

• Drone 基礎使⽤

• Drone 外掛撰寫

View Slide

Docker 容器介紹

View Slide

ၚ౷7.㐘ٖػՍߏ

View Slide

VM 虛擬機好處
• 不同的作業系統環境

• 善⽤資源分配 (CPU, RAM, Storage)

• 透過 Hypervisor 管理 VM 快照

View Slide

$POUBJOFS༰ثՍߏ

View Slide

Container 容器好處
• 容器檔案非常⼩ (Ubuntu 才⼗幾 MB)

• 快速開啟 (不到⼀秒就可以起⼀個 Ubuntu)

• 透過 Docker
fi
le 客製化容器內容

• 快速實踐 Deployment + Operation

View Slide

docker 基礎指令
• docker ps

• 列出正在跑的容器

• docker images

• 列出已經下載的 Images

• docker image prune -a -f

• 清除沒在使⽤的 Images (空間如果不夠)

View Slide

• docker container prune -f

• 清除沒在跑的容器

• docker login -u xxxx

• 登入 docker registry (GitLab 內建, 預設 DockerHub)

• docker run -ti ubuntu /bin/bash

• 快速建立 ubuntu 容器並且登入 Bash Shell

• ⽤於測試及除錯

View Slide

建立 Docker 服務
• docker run -d -p 8081:80 nginx



• -p port 對應關係 (host port : container port)

• -d 丟到背景執⾏

• —name 容器命名

• -e 環境變數

• -v 掛載硬碟

View Slide

進入已存在容器
• docker exec -ti name /bin/bash

• docker ps 可以知道正在跑的容器

• 除錯常⽤

View Slide

練習
• 在本機端架設底下服務

• redis (port 6380:6379)

• mysql (port 3307:3306)

• postgres (port 5433:5432)

• ⽤ Go 或 Node 測試連線

View Slide

練習
• 架設 Go Module Proxy

• https://github.com/gomods/athens

• export GOPROXY=http://localhost:3010

• 掛載 local 端空間到 athens

View Slide

建立容器

View Slide

使⽤ Docker
fi
le

View Slide

FROM alpine:3.15

LABEL maintainer="Bo-Yi Wu "

RUN apk add
- -
no
-
cache ca
-
certif
i
cates
& &
\

rm
-
rf /var/cache/apk
/ *

COPY release/linux/amd64/agent /bin/

ENTRYPOINT [ "/bin/agent" ]

View Slide

編譯容器並上傳
• docker build -t appleboy/server:tag -f path .

• -t 組織名稱/容器名稱

• -f Docker
fi
le 檔案路徑

• tag: 版本 (⽤來 deploy production)

• docker push host:port/appleboy/server

• host: docker registry host (預設上傳到 docker hub)

• port: docker registry port

View Slide

多個服務串接

View Slide

使⽤

docker-compose.yml

View Slide

version: '3.1'

services:

prometheus:

image: prom/prometheus:v2.31.1

volumes:

- ./prometheus/:/etc/prometheus/

- ./data/prometheus:/prometheus

command:

- '
- -
conf
i
g.f
i
le=/etc/prometheus/prometheus.yml'

- '
- -
storage.tsdb.path=/prometheus'

- '
- -
web.console.libraries=/usr/share/prometheus/console_libraries'

- '
- -
web.console.templates=/usr/share/prometheus/consoles'

- '
- -
storage.tsdb.retention.time=365d'

restart: always

grafana:

image: grafana/grafana:8.2.6

depends_on:

- prometheus

volumes:

- ./data/grafana:/var/lib/grafana

- ./grafana/provisioning/:/etc/grafana/provisioning/

env_f
i
le:

- ./grafana/conf
i
g.monitoring

restart: always

View Slide

docker-compose 指令
• docker-compose up -d [service_name]

• -d 在背景執⾏

• docker-compose down -v

• -v 移除 volume 資料

• docker-compose logs -f [service_name]

• 看單⼀服務 Logs

• docker-compose pull

• 更新全部 Image (每次 deploy 都需要做此步驟)

• docker-compose exec db /bin/bash

• 進入容器看 DB 資料

View Slide

練習
• 將 Go 服務搭配 redis + (mysql or postgres)

• redis

• mysql

• postgres

• ⽤ docker-compose 串起上⾯服務

View Slide

部署流程

GitHub Flow vs Git Flow
IUUQTCMPHXVCPZDPNHJUIVC
fl
PXWTHJU
fl
PX

View Slide

(JU)VC'MPX
14
Develop
Git Push
Git Tag
Develop
Git Push
Git Tag
Testing
Deploy
Deploy
Deploy
Production
Staging
Production
Testing
Deploy
Staging

View Slide

GitHub Flow + Git Flow

in opensource
IUUQTHJUIVCDPNHPHJUFBHJUFB

View Slide

如何⾃動化上⾯的步驟?

View Slide

為什麼我不選 Jenkins?

View Slide

Why?
• Complicated project setting

• Write the plugin (Java language)

• Maintenance?

• Learning Curve?

• Grow your team?

View Slide

為什麼我不選

GitLab CI?

View Slide

Why?
• GitLab Only

• Not Extensible

View Slide

Drone CI/CD 介紹

View Slide

Drone CI
• Container native CI/CD platform

• Easy to install & maintain

• Isolate builds

• Simple YAML Con
fi
guration

• Integrates with several VCS Providers

• Rich set of of
fi
cial plugins (any container can be a plugin)

• Execute locally with simple command (drone exec)

• open source (https://github.com/harness/drone)

View Slide

Drone CI/CD 平台真的免費嗎
IUUQTCMPHXVCPZDPNESPOFMJDFOTF

View Slide

Everything is a

Docker Container

View Slide

Project CI/CD Flow
HJUDMPOF UFTUJOH EFQMPZ OPUJGZ

View Slide

Drone CI Infrastructure
Runner
Server
Step 1

git clone
Step 2

make build
Step 3

deploy app
work space
extra service
Runner

View Slide

如何安裝

Drone + GitHub

View Slide

申請

GitHub Application

View Slide

Apply GitHub Application
https:
/ /
github.com/settings/developers

View Slide

https:
/ /
github.com/settings/applications/new

View Slide

Drone Server and Runner

In docker-compose.yml

View Slide

version: '3'

services:

drone
-
server:

image: drone/drone:2

ports:

- 8081
:
80

volumes:

- ./:/data

restart: always

environment:

- DRONE_SERVER_HOST=${DRONE_SERVER_HOST}

- DRONE_SERVER_PROTO=${DRONE_SERVER_PROTO}

- DRONE_RPC_SECRET=${DRONE_RPC_SECRET}

# GitHub Conf
i
g

- DRONE_GITHUB_SERVER=https:
/ /
github.com

- DRONE_GITHUB_CLIENT_ID=${DRONE_GITHUB_CLIENT_ID}

- DRONE_GITHUB_CLIENT_SECRET=${DRONE_GITHUB_CLIENT_SECRET}

- DRONE_LOGS_PRETTY=true

- DRONE_LOGS_COLOR=true
$SFBUFEPDLFSDPNQPTFZNM

View Slide

DRONE_SERVER_HOST=

DRONE_SERVER_PROTO=

DRONE_RPC_SECRET=

DRONE_GITHUB_CLIENT_ID=

DRONE_GITHUB_CLIENT_SECRET=
$SFBUFFOW
fi
MF

View Slide

drone
-
runner:

image: drone/drone
-
runner
-
docker:1

restart: always

depends_on:

- drone
-
server

ports:

- 3000
:
3000

volumes:

- /var/run/docker.sock:/var/run/docker.sock

environment:

- DRONE_RPC_HOST=${DRONE_SERVER_HOST}

- DRONE_RPC_PROTO=${DRONE_SERVER_PROTO}

- DRONE_RPC_SECRET=${DRONE_RPC_SECRET}

- DRONE_RUNNER_CAPACITY=2

- DRONE_RUNNER_NAME=my
-
f
i
rst
-
runner

View Slide

Drone Web UI Introduction

View Slide

View Slide

撰寫 .drone.yml 檔案

View Slide

kind: pipeline

type: docker

name: default

steps:

- name: greeting

image: alpine

commands:

- echo hello

- echo world

View Slide

kind: pipeline

type: docker

name: linux_amd64

steps:

- name: build

image: golang:1.17

commands:

- make build_linux_amd64

View Slide

build_linux_amd64
:

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \

-
v
-
a
-
o release/linux/amd64/helloworld

View Slide

- - -

kind: pipeline

type: docker

name: linux_amd64

steps:

- name: build

image: golang:1.17

environment:

GOOS
:
linux

GOARCH
:
amd64

CGO_ENABLED
:
0

commands:


View Slide

build:

go build
-
v
-
a
-
o \

release/${GOOS}/${GOARCH}/helloworld

View Slide

kind: pipeline

type: docker

name: linux_amd64

steps:

- name: build

image: golang:1.17

commands:


- - -

kind: pipeline

type: docker

name: linux_i386

steps:

- name: build

image: golang:1.17

commands:

- make build_linux_i386
.VMUJQMF1JQFMJOF

View Slide

- - -

kind: pipeline

type: docker

name: linux_i386

steps:

- name: build

image: golang:1.17

environment:

GOOS
:
linux

GOARCH
:
386

CGO_ENABLED
:
0

commands:

$VTUPN&OWJSPONFOU

View Slide

- - -

kind: pipeline

type: docker

name: linux_i386

environment:

GOOS
:
linux

GOARCH
:
386

CGO_ENABLED
:
0

steps:

- name: build

image: golang:1.17

commands:

(MPCBM&OWJSPONFOU

View Slide

Routing

將特定任務丟到特定主機

(GPU or High Memory)

View Slide

kind: pipeline

type: docker

name: default

steps:

- name: build

image: golang

commands:

- go build

- go test

node:

keyA
:
valueA

keyB
:
valueB

%30/&@36//&[email protected]"#&-4LFZ"WBMVF" LFZ#WBMVF#

View Slide

Parallelism

平⾏處理

View Slide

kind: pipeline

type: docker

name: default

steps:

- name: backend

image: golang

commands:

- go build

- go test

- name: frontend

image: node

commands:

- npm install

- npm test

- name: notify

image: plugins/slack

depends_on:

- frontend

- backend

View Slide

Conditions and Triggers

View Slide

kind: pipeline

type: docker

name: default

steps:

- name: build

image: golang

commands:

- go build

- go test

when:

branch:

- master

- feature
/ *

kind: pipeline

type: docker

name: default

steps:

- name: build

image: golang

commands:

- go build

- go test

trigger:

branch:

- master

View Slide

Volumes

View Slide

kind: pipeline

type: docker

name: default

steps:

- name: test

image: golang

volumes:

- name: cache

path: /go

commands:

- go get

- go test

- name: build

image: golang

volumes:

- name: cache

path: /go

commands:

- go build

volumes:

- name: cache

temp: {}

View Slide

練習
• 任何程式語⾔進⾏ Lint + Testing + Build

• Go 語⾔使⽤ golangci-lint

• 測試 go test -v -cover

• 編譯 go build -o release/linux/amd64/out

View Slide

Plugin
IUUQQMVHJOTESPOFJP

View Slide

- name: publish

pull: always

image: plugins/docker:linux
-
amd64

settings:

auto_tag: true

auto_tag_suff
i
x: linux
-
amd64

cache_from: appleboy/gorush

daemon_off: false

dockerf
i
le: docker/Dockerf
i
le.linux.amd64

password:

from_secret: docker_password

repo: appleboy/gorush

username:

from_secret: docker_username

when:

event:

exclude:

- pull_request

View Slide

View Slide

Plugin Input

View Slide

- name: publish

pull: always

image: plugins/docker:linux
-
amd64

settings:

auto_tag: true

auto_tag_suff
i
x: linux
-
amd64

cache_from: appleboy/gorush

daemon_off: false

dockerf
i
le: docker/Dockerf
i
le.linux.amd64

password:

from_secret: docker_password

repo: appleboy/gorush

username:

from_secret: docker_username

when:

event:

exclude:

- pull_request

View Slide

PLUGIN_USERNAME=appleboy
PLUGIN_PASSWORD=password
PLUGIN_REPO=appleboy/gorush
PLUGIN_TAGS=1.0.0,1.0

View Slide

如何製作

Plugins

View Slide

Go Plugin vs Bash Plugin

(Any language you want)

View Slide

Bash Plugin

View Slide

kind: pipeline

type: docker

name: default

steps:

- name: webhook

image: acme/webhook

settings:

url: http:
/ /
hook.acme.com

method: post

body: |

hello world

View Slide

# !
/bin/sh

curl \

-X ${PLUGIN_METHOD} \

-
d ${PLUGIN_BODY} \

${PLUGIN_URL}

View Slide

FROM alpine:3.14

ADD script.sh /bin/

RUN chmod
+
x /bin/script.sh

RUN apk -Uuv add curl ca
-
certif
i
cates

ENTRYPOINT /bin/script.sh

View Slide

$ docker build -t acme/webhook .
$ docker push acme/webhook
$ docker run --rm \
-e PLUGIN_METHOD=post \
-e PLUGIN_URL=http://hook.acme.com \
-e PLUGIN_BODY=hello \
acme/webhook
ฤᩄ

View Slide

$ docker build -t acme/webhook .
$ docker push acme/webhook
$ docker run --rm \
-e PLUGIN_METHOD=post \
-e PLUGIN_URL=http://hook.acme.com \
-e PLUGIN_BODY=hello \
acme/webhook
ଌࢼ

View Slide

Go Plugin

View Slide

package main

import (

"net/http"

"os"

"strings"

)

func main() {

body
: =
strings.NewReader(

os.Getenv("PLUGIN_BODY"),

)

_, err
: =
http.NewRequest(

os.Getenv("PLUGIN_METHOD"),

os.Getenv("PLUGIN_URL"),

body,

)

if err
! =
nil {

os.Exit(1)

}

} GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o webhook

View Slide

FROM alpine:3.14

ADD webhook /bin/

RUN apk -Uuv add ca
-
certif
i
cates

ENTRYPOINT /bin/webhook

View Slide

練習
• 任何程式語⾔進⾏ Plugin 撰寫

• 申請 dockerhub 帳號密碼

• 透過 Drone ⾃動打包 Image 並上傳

View Slide

Deploy to Production Server

via SSH Command

View Slide

- name: ssh commands

image: appleboy/drone
-
ssh

settings:

host: foo.com

username: root

key:

from_secret: ssh_key

passphrase: 1234

port: 22

script:

- mkdir abc/def/efg

- echo "you can't see the steps."

View Slide

Promotions

⼿動部署

View Slide

kind: pipeline

type: docker

name: default

steps:

- name: test

image: node

commands:

- npm install

- npm run test

- npm run bundle

- name: deploy

-
ssh

settings:
. . .

trigger:

event:

- promote

target:

- production

kind: pipeline

type: docker

name: deploy

steps:

- name: test

image: node

commands:

- npm install

- npm run test

- npm run bundle

- name: deploy

-
ssh

settings:
. . .

when:

event:

- promote

target:

- production

View Slide

$ drone build promote octocat/hello-world 42 staging
$ drone build promote
$ drone build promote octocat/hello-world 42 production

View Slide

Thanks

View Slide

Drone CI/CD 自動化測試及部署

Drone CI/CD 自動化測試及部署

Bo-Yi Wu

More Decks by Bo-Yi Wu

Other Decks in Technology

Featured

Transcript