Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Drone CI/CD 自動化測試及部署

265bcbb56e831266de7a9f9281aab57a?s=47 Bo-Yi Wu
January 15, 2022

Drone CI/CD 自動化測試及部署

## 課程大綱

1. Drone 基礎認識
2. Drone 安裝方式
3. Drone 基本用法與指令介紹
4. Drone 外掛介紹與外掛撰寫

## 課程簡述

1. 用 Go 語言所打造的 Drone 輕量級容器持續交付平台,Drone 是一套基於 Docker 容器技術的持續交付平台
2. 每個 Build 都在一個短暫的 Docker 容器中執行,讓開發人員可以完全控制他們的建置環境並保證隔離互不干擾
3. Drone 非常容易安裝及維護,並且提供強大的插件(plugin)系統,也可以讓開發者輕易完成個人插件
4. 如果您正在煩惱該選擇 Jenkins 或 GitLab CI 等工具,建議您來嘗試看看本次課程介紹之輕量級的 Drone
5. 本課程會帶您深入了解由 Go 語言所撰寫的 Drone 架構及如何輕易整合原有的 Git 服務(像是 Github、Bitbucket、Gitea 或 GitLab)來快速部署及測試軟體品質

265bcbb56e831266de7a9f9281aab57a?s=128

Bo-Yi Wu

January 15, 2022
Tweet

Transcript

  1. Drone CI/CD ⾃動化測試及部署

  2. About me • Software Engineer in Mediatek • Member of

    Drone CI/CD Platform • Member of Gitea Platform • Member of Gin Golang Framework • Teacher of Udemy Platform: Golang + Drone https://blog.wu-boy.com
  3. 課程⼤綱 • Docker 基礎介紹 • CI/CD 服務簡介 • Drone 架構介紹

    • Drone 基礎使⽤ • Drone 外掛撰寫
  4. Docker 容器介紹

  5. ၚ౷7.㐘ٖػՍߏ

  6. VM 虛擬機好處 • 不同的作業系統環境 • 善⽤資源分配 (CPU, RAM, Storage) •

    透過 Hypervisor 管理 VM 快照
  7. $POUBJOFS༰ثՍߏ

  8. Container 容器好處 • 容器檔案非常⼩ (Ubuntu 才⼗幾 MB) • 快速開啟 (不到⼀秒就可以起⼀個

    Ubuntu) • 透過 Docker fi le 客製化容器內容 • 快速實踐 Deployment + Operation
  9. docker 基礎指令 • docker ps • 列出正在跑的容器 • docker images

    • 列出已經下載的 Images • docker image prune -a -f • 清除沒在使⽤的 Images (空間如果不夠)
  10. • docker container prune -f • 清除沒在跑的容器 • docker login

    -u xxxx • 登入 docker registry (GitLab 內建, 預設 DockerHub) • docker run -ti ubuntu /bin/bash • 快速建立 ubuntu 容器並且登入 Bash Shell • ⽤於測試及除錯
  11. 建立 Docker 服務 • docker run -d -p 8081:80 nginx

    • docker run -d -p 8082:80 nginx • docker run -d -p 8083:80 nginx • -p port 對應關係 (host port : container port) • -d 丟到背景執⾏ • —name 容器命名 • -e 環境變數 • -v 掛載硬碟
  12. 進入已存在容器 • docker exec -ti name /bin/bash • docker ps

    可以知道正在跑的容器 • 除錯常⽤
  13. 練習 • 在本機端架設底下服務 • redis (port 6380:6379) • mysql (port

    3307:3306) • postgres (port 5433:5432) • ⽤ Go 或 Node 測試連線
  14. 練習 • 架設 Go Module Proxy • https://github.com/gomods/athens • export

    GOPROXY=http://localhost:3010 • 掛載 local 端空間到 athens
  15. 建立容器

  16. 使⽤ Docker fi le

  17. FROM alpine:3.15 LABEL maintainer="Bo-Yi Wu <appleboy.tw@gmail.com>" RUN apk add -

    - no - cache ca - certif i cates & & \ rm - rf /var/cache/apk / * COPY release/linux/amd64/agent /bin/ ENTRYPOINT [ "/bin/agent" ]
  18. 編譯容器並上傳 • docker build -t appleboy/server:tag -f path . •

    -t 組織名稱/容器名稱 • -f Docker fi le 檔案路徑 • tag: 版本 (⽤來 deploy production) • docker push host:port/appleboy/server • host: docker registry host (預設上傳到 docker hub) • port: docker registry port
  19. 多個服務串接

  20. 使⽤ docker-compose.yml

  21. version: '3.1' services: prometheus: image: prom/prometheus:v2.31.1 volumes: - ./prometheus/:/etc/prometheus/ -

    ./data/prometheus:/prometheus command: - ' - - conf i g.f i le=/etc/prometheus/prometheus.yml' - ' - - storage.tsdb.path=/prometheus' - ' - - web.console.libraries=/usr/share/prometheus/console_libraries' - ' - - web.console.templates=/usr/share/prometheus/consoles' - ' - - storage.tsdb.retention.time=365d' restart: always grafana: image: grafana/grafana:8.2.6 depends_on: - prometheus volumes: - ./data/grafana:/var/lib/grafana - ./grafana/provisioning/:/etc/grafana/provisioning/ env_f i le: - ./grafana/conf i g.monitoring restart: always
  22. docker-compose 指令 • docker-compose up -d [service_name] • -d 在背景執⾏

    • docker-compose down -v • -v 移除 volume 資料 • docker-compose logs -f [service_name] • 看單⼀服務 Logs • docker-compose pull • 更新全部 Image (每次 deploy 都需要做此步驟) • docker-compose exec db /bin/bash • 進入容器看 DB 資料
  23. 練習 • 將 Go 服務搭配 redis + (mysql or postgres)

    • redis • mysql • postgres • ⽤ docker-compose 串起上⾯服務
  24. 部署流程 GitHub Flow vs Git Flow IUUQTCMPHXVCPZDPNHJUIVC fl PXWTHJU fl

    PX
  25. (JU)VC'MPX 14 Develop Git Push Git Tag Develop Git Push

    Git Tag Testing Deploy Deploy Deploy Production Staging Production Testing Deploy Staging
  26. GitHub Flow + Git Flow in opensource IUUQTHJUIVCDPNHPHJUFBHJUFB

  27. 如何⾃動化上⾯的步驟?

  28. 為什麼我不選 Jenkins?

  29. Why? • Complicated project setting • Write the plugin (Java

    language) • Maintenance? • Learning Curve? • Grow your team?
  30. 為什麼我不選 GitLab CI?

  31. Why? • GitLab Only • Not Extensible

  32. Drone CI/CD 介紹

  33. Drone CI • Container native CI/CD platform • Easy to

    install & maintain • Isolate builds • Simple YAML Con fi guration • Integrates with several VCS Providers • Rich set of of fi cial plugins (any container can be a plugin) • Execute locally with simple command (drone exec) • open source (https://github.com/harness/drone)
  34. Drone CI/CD 平台真的免費嗎 IUUQTCMPHXVCPZDPNESPOFMJDFOTF

  35. Everything is a Docker Container

  36. Project CI/CD Flow HJUDMPOF UFTUJOH EFQMPZ OPUJGZ

  37. Drone CI Infrastructure Runner Server Step 1 git clone Step

    2 make build Step 3 deploy app work space extra service Runner
  38. 如何安裝 Drone + GitHub

  39. 申請 GitHub Application

  40. Apply GitHub Application https: / / github.com/settings/developers

  41. https: / / github.com/settings/applications/new

  42. Drone Server and Runner In docker-compose.yml

  43. version: '3' services: drone - server: image: drone/drone:2 ports: -

    8081 : 80 volumes: - ./:/data restart: always environment: - DRONE_SERVER_HOST=${DRONE_SERVER_HOST} - DRONE_SERVER_PROTO=${DRONE_SERVER_PROTO} - DRONE_RPC_SECRET=${DRONE_RPC_SECRET} # GitHub Conf i g - DRONE_GITHUB_SERVER=https: / / github.com - DRONE_GITHUB_CLIENT_ID=${DRONE_GITHUB_CLIENT_ID} - DRONE_GITHUB_CLIENT_SECRET=${DRONE_GITHUB_CLIENT_SECRET} - DRONE_LOGS_PRETTY=true - DRONE_LOGS_COLOR=true $SFBUFEPDLFSDPNQPTFZNM
  44. DRONE_SERVER_HOST= DRONE_SERVER_PROTO= DRONE_RPC_SECRET= DRONE_GITHUB_CLIENT_ID= DRONE_GITHUB_CLIENT_SECRET= $SFBUFFOW fi MF

  45. drone - runner: image: drone/drone - runner - docker:1 restart:

    always depends_on: - drone - server ports: - 3000 : 3000 volumes: - /var/run/docker.sock:/var/run/docker.sock environment: - DRONE_RPC_HOST=${DRONE_SERVER_HOST} - DRONE_RPC_PROTO=${DRONE_SERVER_PROTO} - DRONE_RPC_SECRET=${DRONE_RPC_SECRET} - DRONE_RUNNER_CAPACITY=2 - DRONE_RUNNER_NAME=my - f i rst - runner
  46. Drone Web UI Introduction

  47. None
  48. None
  49. None
  50. None
  51. 撰寫 .drone.yml 檔案

  52. kind: pipeline type: docker name: default steps: - name: greeting

    image: alpine commands: - echo hello - echo world
  53. kind: pipeline type: docker name: linux_amd64 steps: - name: build

    image: golang:1.17 commands: - make build_linux_amd64
  54. build_linux_amd64 : CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \ - v

    - a - o release/linux/amd64/helloworld
  55. build_linux_amd64 : CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \ - v

    - a - o release/linux/amd64/helloworld
  56. - - - kind: pipeline type: docker name: linux_amd64 steps:

    - name: build image: golang:1.17 environment: GOOS : linux GOARCH : amd64 CGO_ENABLED : 0 commands: - make build_linux_amd64
  57. build: go build - v - a - o \

    release/${GOOS}/${GOARCH}/helloworld
  58. kind: pipeline type: docker name: linux_amd64 steps: - name: build

    image: golang:1.17 commands: - make build_linux_amd64 - - - kind: pipeline type: docker name: linux_i386 steps: - name: build image: golang:1.17 commands: - make build_linux_i386 .VMUJQMF1JQFMJOF
  59. - - - kind: pipeline type: docker name: linux_i386 steps:

    - name: build image: golang:1.17 environment: GOOS : linux GOARCH : 386 CGO_ENABLED : 0 commands: - make build_linux_i386 $VTUPN&OWJSPONFOU
  60. - - - kind: pipeline type: docker name: linux_i386 environment:

    GOOS : linux GOARCH : 386 CGO_ENABLED : 0 steps: - name: build image: golang:1.17 commands: - make build_linux_i386 (MPCBM&OWJSPONFOU
  61. Routing 將特定任務丟到特定主機 (GPU or High Memory)

  62. kind: pipeline type: docker name: default steps: - name: build

    image: golang commands: - go build - go test node: keyA : valueA keyB : valueB %30/&@36//&3@-"#&-4LFZ"WBMVF" LFZ#WBMVF#
  63. Parallelism 平⾏處理

  64. kind: pipeline type: docker name: default steps: - name: backend

    image: golang commands: - go build - go test - name: frontend image: node commands: - npm install - npm test - name: notify image: plugins/slack depends_on: - frontend - backend
  65. Conditions and Triggers

  66. kind: pipeline type: docker name: default steps: - name: build

    image: golang commands: - go build - go test when: branch: - master - feature / * kind: pipeline type: docker name: default steps: - name: build image: golang commands: - go build - go test trigger: branch: - master
  67. Volumes

  68. kind: pipeline type: docker name: default steps: - name: test

    image: golang volumes: - name: cache path: /go commands: - go get - go test - name: build image: golang volumes: - name: cache path: /go commands: - go build volumes: - name: cache temp: {}
  69. 練習 • 任何程式語⾔進⾏ Lint + Testing + Build • Go

    語⾔使⽤ golangci-lint • 測試 go test -v -cover • 編譯 go build -o release/linux/amd64/out
  70. Plugin IUUQQMVHJOTESPOFJP

  71. - name: publish pull: always image: plugins/docker:linux - amd64 settings:

    auto_tag: true auto_tag_suff i x: linux - amd64 cache_from: appleboy/gorush daemon_off: false dockerf i le: docker/Dockerf i le.linux.amd64 password: from_secret: docker_password repo: appleboy/gorush username: from_secret: docker_username when: event: exclude: - pull_request
  72. - name: publish pull: always image: plugins/docker:linux - amd64 settings:

    auto_tag: true auto_tag_suff i x: linux - amd64 cache_from: appleboy/gorush daemon_off: false dockerf i le: docker/Dockerf i le.linux.amd64 password: from_secret: docker_password repo: appleboy/gorush username: from_secret: docker_username when: event: exclude: - pull_request
  73. None
  74. Plugin Input

  75. - name: publish pull: always image: plugins/docker:linux - amd64 settings:

    auto_tag: true auto_tag_suff i x: linux - amd64 cache_from: appleboy/gorush daemon_off: false dockerf i le: docker/Dockerf i le.linux.amd64 password: from_secret: docker_password repo: appleboy/gorush username: from_secret: docker_username when: event: exclude: - pull_request
  76. PLUGIN_USERNAME=appleboy PLUGIN_PASSWORD=password PLUGIN_REPO=appleboy/gorush PLUGIN_TAGS=1.0.0,1.0

  77. PLUGIN_USERNAME=appleboy PLUGIN_PASSWORD=password PLUGIN_REPO=appleboy/gorush PLUGIN_TAGS=1.0.0,1.0

  78. 如何製作 Plugins

  79. Go Plugin vs Bash Plugin (Any language you want)

  80. Bash Plugin

  81. kind: pipeline type: docker name: default steps: - name: webhook

    image: acme/webhook settings: url: http: / / hook.acme.com method: post body: | hello world
  82. # ! /bin/sh curl \ -X ${PLUGIN_METHOD} \ - d

    ${PLUGIN_BODY} \ ${PLUGIN_URL}
  83. FROM alpine:3.14 ADD script.sh /bin/ RUN chmod + x /bin/script.sh

    RUN apk -Uuv add curl ca - certif i cates ENTRYPOINT /bin/script.sh
  84. $ docker build -t acme/webhook . $ docker push acme/webhook

    $ docker run --rm \ -e PLUGIN_METHOD=post \ -e PLUGIN_URL=http://hook.acme.com \ -e PLUGIN_BODY=hello \ acme/webhook ฤᩄ
  85. $ docker build -t acme/webhook . $ docker push acme/webhook

    $ docker run --rm \ -e PLUGIN_METHOD=post \ -e PLUGIN_URL=http://hook.acme.com \ -e PLUGIN_BODY=hello \ acme/webhook ଌࢼ
  86. Go Plugin

  87. package main import ( "net/http" "os" "strings" ) func main()

    { body : = strings.NewReader( os.Getenv("PLUGIN_BODY"), ) _, err : = http.NewRequest( os.Getenv("PLUGIN_METHOD"), os.Getenv("PLUGIN_URL"), body, ) if err ! = nil { os.Exit(1) } } GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o webhook
  88. FROM alpine:3.14 ADD webhook /bin/ RUN apk -Uuv add ca

    - certif i cates ENTRYPOINT /bin/webhook
  89. 練習 • 任何程式語⾔進⾏ Plugin 撰寫 • 申請 dockerhub 帳號密碼 •

    透過 Drone ⾃動打包 Image 並上傳
  90. Deploy to Production Server via SSH Command

  91. - name: ssh commands image: appleboy/drone - ssh settings: host:

    foo.com username: root key: from_secret: ssh_key passphrase: 1234 port: 22 script: - mkdir abc/def/efg - echo "you can't see the steps."
  92. Promotions ⼿動部署

  93. kind: pipeline type: docker name: default steps: - name: test

    image: node commands: - npm install - npm run test - npm run bundle - name: deploy image: appleboy/drone - ssh settings: . . . trigger: event: - promote target: - production kind: pipeline type: docker name: deploy steps: - name: test image: node commands: - npm install - npm run test - npm run bundle - name: deploy image: appleboy/drone - ssh settings: . . . when: event: - promote target: - production
  94. $ drone build promote octocat/hello-world 42 staging $ drone build

    promote <repo> <number> <environment> $ drone build promote octocat/hello-world 42 production
  95. Thanks