Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Drone CI/CD 自動化測試及部署

Bo-Yi Wu
January 15, 2022

Drone CI/CD 自動化測試及部署

## 課程大綱

1. Drone 基礎認識
2. Drone 安裝方式
3. Drone 基本用法與指令介紹
4. Drone 外掛介紹與外掛撰寫

## 課程簡述

1. 用 Go 語言所打造的 Drone 輕量級容器持續交付平台,Drone 是一套基於 Docker 容器技術的持續交付平台
2. 每個 Build 都在一個短暫的 Docker 容器中執行,讓開發人員可以完全控制他們的建置環境並保證隔離互不干擾
3. Drone 非常容易安裝及維護,並且提供強大的插件(plugin)系統,也可以讓開發者輕易完成個人插件
4. 如果您正在煩惱該選擇 Jenkins 或 GitLab CI 等工具,建議您來嘗試看看本次課程介紹之輕量級的 Drone
5. 本課程會帶您深入了解由 Go 語言所撰寫的 Drone 架構及如何輕易整合原有的 Git 服務(像是 Github、Bitbucket、Gitea 或 GitLab)來快速部署及測試軟體品質

Bo-Yi Wu

January 15, 2022
Tweet

More Decks by Bo-Yi Wu

Other Decks in Technology

Transcript

  1. Drone CI/CD


    ⾃動化測試及部署

    View full-size slide

  2. About me
    • Software Engineer in Mediatek


    • Member of Drone CI/CD Platform


    • Member of Gitea Platform


    • Member of Gin Golang Framework


    • Teacher of Udemy Platform: Golang + Drone
    https://blog.wu-boy.com

    View full-size slide

  3. 課程⼤綱
    • Docker 基礎介紹


    • CI/CD 服務簡介


    • Drone 架構介紹


    • Drone 基礎使⽤


    • Drone 外掛撰寫

    View full-size slide

  4. Docker 容器介紹

    View full-size slide

  5. ၚ౷7.㐘ٖػՍߏ

    View full-size slide

  6. VM 虛擬機好處
    • 不同的作業系統環境


    • 善⽤資源分配 (CPU, RAM, Storage)


    • 透過 Hypervisor 管理 VM 快照

    View full-size slide

  7. $POUBJOFS༰ثՍߏ

    View full-size slide

  8. Container 容器好處
    • 容器檔案非常⼩ (Ubuntu 才⼗幾 MB)


    • 快速開啟 (不到⼀秒就可以起⼀個 Ubuntu)


    • 透過 Docker
    fi
    le 客製化容器內容


    • 快速實踐 Deployment + Operation

    View full-size slide

  9. docker 基礎指令
    • docker ps


    • 列出正在跑的容器


    • docker images


    • 列出已經下載的 Images


    • docker image prune -a -f


    • 清除沒在使⽤的 Images (空間如果不夠)

    View full-size slide

  10. • docker container prune -f


    • 清除沒在跑的容器


    • docker login -u xxxx


    • 登入 docker registry (GitLab 內建, 預設 DockerHub)


    • docker run -ti ubuntu /bin/bash


    • 快速建立 ubuntu 容器並且登入 Bash Shell


    • ⽤於測試及除錯

    View full-size slide

  11. 建立 Docker 服務
    • docker run -d -p 8081:80 nginx


    • docker run -d -p 8082:80 nginx


    • docker run -d -p 8083:80 nginx


    • -p port 對應關係 (host port : container port)


    • -d 丟到背景執⾏


    • —name 容器命名


    • -e 環境變數


    • -v 掛載硬碟

    View full-size slide

  12. 進入已存在容器
    • docker exec -ti name /bin/bash


    • docker ps 可以知道正在跑的容器


    • 除錯常⽤

    View full-size slide

  13. 練習
    • 在本機端架設底下服務


    • redis (port 6380:6379)


    • mysql (port 3307:3306)


    • postgres (port 5433:5432)


    • ⽤ Go 或 Node 測試連線

    View full-size slide

  14. 練習
    • 架設 Go Module Proxy


    • https://github.com/gomods/athens


    • export GOPROXY=http://localhost:3010


    • 掛載 local 端空間到 athens

    View full-size slide

  15. 建立容器

    View full-size slide

  16. 使⽤ Docker
    fi
    le

    View full-size slide

  17. FROM alpine:3.15


    LABEL maintainer="Bo-Yi Wu "


    RUN apk add
    - -
    no
    -
    cache ca
    -
    certif
    i
    cates
    & &
    \


    rm
    -
    rf /var/cache/apk
    / *

    COPY release/linux/amd64/agent /bin/


    ENTRYPOINT [ "/bin/agent" ]

    View full-size slide

  18. 編譯容器並上傳
    • docker build -t appleboy/server:tag -f path .


    • -t 組織名稱/容器名稱


    • -f Docker
    fi
    le 檔案路徑


    • tag: 版本 (⽤來 deploy production)


    • docker push host:port/appleboy/server


    • host: docker registry host (預設上傳到 docker hub)


    • port: docker registry port

    View full-size slide

  19. 多個服務串接

    View full-size slide

  20. 使⽤


    docker-compose.yml

    View full-size slide

  21. version: '3.1'


    services:


    prometheus:


    image: prom/prometheus:v2.31.1


    volumes:


    - ./prometheus/:/etc/prometheus/


    - ./data/prometheus:/prometheus


    command:


    - '
    - -
    conf
    i
    g.f
    i
    le=/etc/prometheus/prometheus.yml'


    - '
    - -
    storage.tsdb.path=/prometheus'


    - '
    - -
    web.console.libraries=/usr/share/prometheus/console_libraries'


    - '
    - -
    web.console.templates=/usr/share/prometheus/consoles'


    - '
    - -
    storage.tsdb.retention.time=365d'


    restart: always


    grafana:


    image: grafana/grafana:8.2.6


    depends_on:


    - prometheus


    volumes:


    - ./data/grafana:/var/lib/grafana


    - ./grafana/provisioning/:/etc/grafana/provisioning/


    env_f
    i
    le:


    - ./grafana/conf
    i
    g.monitoring


    restart: always

    View full-size slide

  22. docker-compose 指令
    • docker-compose up -d [service_name]


    • -d 在背景執⾏


    • docker-compose down -v


    • -v 移除 volume 資料


    • docker-compose logs -f [service_name]


    • 看單⼀服務 Logs


    • docker-compose pull


    • 更新全部 Image (每次 deploy 都需要做此步驟)


    • docker-compose exec db /bin/bash


    • 進入容器看 DB 資料

    View full-size slide

  23. 練習
    • 將 Go 服務搭配 redis + (mysql or postgres)


    • redis


    • mysql


    • postgres


    • ⽤ docker-compose 串起上⾯服務

    View full-size slide

  24. 部署流程


    GitHub Flow vs Git Flow
    IUUQTCMPHXVCPZDPNHJUIVC
    fl
    PXWTHJU
    fl
    PX

    View full-size slide

  25. (JU)VC'MPX
    14
    Develop
    Git Push
    Git Tag
    Develop
    Git Push
    Git Tag
    Testing
    Deploy
    Deploy
    Deploy
    Production
    Staging
    Production
    Testing
    Deploy
    Staging

    View full-size slide

  26. GitHub Flow + Git Flow


    in opensource
    IUUQTHJUIVCDPNHPHJUFBHJUFB

    View full-size slide

  27. 如何⾃動化上⾯的步驟?

    View full-size slide

  28. 為什麼我不選 Jenkins?

    View full-size slide

  29. Why?
    • Complicated project setting


    • Write the plugin (Java language)


    • Maintenance?


    • Learning Curve?


    • Grow your team?

    View full-size slide

  30. 為什麼我不選


    GitLab CI?

    View full-size slide

  31. Why?
    • GitLab Only


    • Not Extensible

    View full-size slide

  32. Drone CI/CD 介紹

    View full-size slide

  33. Drone CI
    • Container native CI/CD platform


    • Easy to install & maintain


    • Isolate builds


    • Simple YAML Con
    fi
    guration


    • Integrates with several VCS Providers


    • Rich set of of
    fi
    cial plugins (any container can be a plugin)


    • Execute locally with simple command (drone exec)


    • open source (https://github.com/harness/drone)

    View full-size slide

  34. Drone CI/CD 平台真的免費嗎
    IUUQTCMPHXVCPZDPNESPOFMJDFOTF

    View full-size slide

  35. Everything is a


    Docker Container

    View full-size slide

  36. Project CI/CD Flow
    HJUDMPOF UFTUJOH EFQMPZ OPUJGZ

    View full-size slide

  37. Drone CI Infrastructure
    Runner
    Server
    Step 1


    git clone
    Step 2


    make build
    Step 3


    deploy app
    work space
    extra service
    Runner

    View full-size slide

  38. 如何安裝


    Drone + GitHub


    View full-size slide

  39. 申請


    GitHub Application

    View full-size slide

  40. Apply GitHub Application
    https:
    / /
    github.com/settings/developers

    View full-size slide

  41. https:
    / /
    github.com/settings/applications/new

    View full-size slide

  42. Drone Server and Runner


    In docker-compose.yml


    View full-size slide

  43. version: '3'


    services:


    drone
    -
    server:


    image: drone/drone:2


    ports:


    - 8081
    :
    80


    volumes:


    - ./:/data


    restart: always


    environment:


    - DRONE_SERVER_HOST=${DRONE_SERVER_HOST}


    - DRONE_SERVER_PROTO=${DRONE_SERVER_PROTO}


    - DRONE_RPC_SECRET=${DRONE_RPC_SECRET}


    # GitHub Conf
    i
    g


    - DRONE_GITHUB_SERVER=https:
    / /
    github.com


    - DRONE_GITHUB_CLIENT_ID=${DRONE_GITHUB_CLIENT_ID}


    - DRONE_GITHUB_CLIENT_SECRET=${DRONE_GITHUB_CLIENT_SECRET}


    - DRONE_LOGS_PRETTY=true


    - DRONE_LOGS_COLOR=true
    $SFBUFEPDLFSDPNQPTFZNM

    View full-size slide

  44. DRONE_SERVER_HOST=


    DRONE_SERVER_PROTO=


    DRONE_RPC_SECRET=


    DRONE_GITHUB_CLIENT_ID=


    DRONE_GITHUB_CLIENT_SECRET=
    $SFBUFFOW
    fi
    MF

    View full-size slide

  45. drone
    -
    runner:


    image: drone/drone
    -
    runner
    -
    docker:1


    restart: always


    depends_on:


    - drone
    -
    server


    ports:


    - 3000
    :
    3000


    volumes:


    - /var/run/docker.sock:/var/run/docker.sock


    environment:


    - DRONE_RPC_HOST=${DRONE_SERVER_HOST}


    - DRONE_RPC_PROTO=${DRONE_SERVER_PROTO}


    - DRONE_RPC_SECRET=${DRONE_RPC_SECRET}


    - DRONE_RUNNER_CAPACITY=2


    - DRONE_RUNNER_NAME=my
    -
    f
    i
    rst
    -
    runner

    View full-size slide

  46. Drone Web UI Introduction

    View full-size slide

  47. 撰寫 .drone.yml 檔案

    View full-size slide

  48. kind: pipeline


    type: docker


    name: default


    steps:


    - name: greeting


    image: alpine


    commands:


    - echo hello


    - echo world

    View full-size slide

  49. kind: pipeline


    type: docker


    name: linux_amd64


    steps:


    - name: build


    image: golang:1.17


    commands:


    - make build_linux_amd64

    View full-size slide

  50. build_linux_amd64
    :

    CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \


    -
    v
    -
    a
    -
    o release/linux/amd64/helloworld


    View full-size slide

  51. build_linux_amd64
    :

    CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \


    -
    v
    -
    a
    -
    o release/linux/amd64/helloworld


    View full-size slide

  52. - - -

    kind: pipeline


    type: docker


    name: linux_amd64


    steps:


    - name: build


    image: golang:1.17


    environment:


    GOOS
    :
    linux


    GOARCH
    :
    amd64


    CGO_ENABLED
    :
    0


    commands:


    - make build_linux_amd64

    View full-size slide

  53. build:


    go build
    -
    v
    -
    a
    -
    o \


    release/${GOOS}/${GOARCH}/helloworld


    View full-size slide

  54. kind: pipeline


    type: docker


    name: linux_amd64


    steps:


    - name: build


    image: golang:1.17


    commands:


    - make build_linux_amd64


    - - -

    kind: pipeline


    type: docker


    name: linux_i386


    steps:


    - name: build


    image: golang:1.17


    commands:


    - make build_linux_i386
    .VMUJQMF1JQFMJOF

    View full-size slide

  55. - - -

    kind: pipeline


    type: docker


    name: linux_i386


    steps:


    - name: build


    image: golang:1.17


    environment:


    GOOS
    :
    linux


    GOARCH
    :
    386


    CGO_ENABLED
    :
    0


    commands:


    - make build_linux_i386
    $VTUPN&OWJSPONFOU

    View full-size slide

  56. - - -

    kind: pipeline


    type: docker


    name: linux_i386


    environment:


    GOOS
    :
    linux


    GOARCH
    :
    386


    CGO_ENABLED
    :
    0


    steps:


    - name: build


    image: golang:1.17


    commands:


    - make build_linux_i386
    (MPCBM&OWJSPONFOU

    View full-size slide

  57. Routing


    將特定任務丟到特定主機


    (GPU or High Memory)

    View full-size slide

  58. kind: pipeline


    type: docker


    name: default


    steps:


    - name: build


    image: golang


    commands:


    - go build


    - go test


    node:


    keyA
    :
    valueA


    keyB
    :
    valueB


    %30/&@36//&3@-"#&-4LFZ"WBMVF" LFZ#WBMVF#

    View full-size slide

  59. Parallelism


    平⾏處理

    View full-size slide

  60. kind: pipeline


    type: docker


    name: default


    steps:


    - name: backend


    image: golang


    commands:


    - go build


    - go test


    - name: frontend


    image: node


    commands:


    - npm install


    - npm test


    - name: notify


    image: plugins/slack


    depends_on:


    - frontend


    - backend


    View full-size slide

  61. Conditions and Triggers

    View full-size slide

  62. kind: pipeline


    type: docker


    name: default


    steps:


    - name: build


    image: golang


    commands:


    - go build


    - go test


    when:


    branch:


    - master


    - feature
    / *

    kind: pipeline


    type: docker


    name: default


    steps:


    - name: build


    image: golang


    commands:


    - go build


    - go test


    trigger:


    branch:


    - master


    View full-size slide

  63. kind: pipeline


    type: docker


    name: default


    steps:


    - name: test


    image: golang


    volumes:


    - name: cache


    path: /go


    commands:


    - go get


    - go test


    - name: build


    image: golang


    volumes:


    - name: cache


    path: /go


    commands:


    - go build


    volumes:


    - name: cache


    temp: {}


    View full-size slide

  64. 練習
    • 任何程式語⾔進⾏ Lint + Testing + Build


    • Go 語⾔使⽤ golangci-lint


    • 測試 go test -v -cover


    • 編譯 go build -o release/linux/amd64/out

    View full-size slide

  65. Plugin
    IUUQQMVHJOTESPOFJP

    View full-size slide

  66. - name: publish


    pull: always


    image: plugins/docker:linux
    -
    amd64


    settings:


    auto_tag: true


    auto_tag_suff
    i
    x: linux
    -
    amd64


    cache_from: appleboy/gorush


    daemon_off: false


    dockerf
    i
    le: docker/Dockerf
    i
    le.linux.amd64


    password:


    from_secret: docker_password


    repo: appleboy/gorush


    username:


    from_secret: docker_username


    when:


    event:


    exclude:


    - pull_request


    View full-size slide

  67. - name: publish


    pull: always


    image: plugins/docker:linux
    -
    amd64


    settings:


    auto_tag: true


    auto_tag_suff
    i
    x: linux
    -
    amd64


    cache_from: appleboy/gorush


    daemon_off: false


    dockerf
    i
    le: docker/Dockerf
    i
    le.linux.amd64


    password:


    from_secret: docker_password


    repo: appleboy/gorush


    username:


    from_secret: docker_username


    when:


    event:


    exclude:


    - pull_request


    View full-size slide

  68. Plugin Input

    View full-size slide

  69. - name: publish


    pull: always


    image: plugins/docker:linux
    -
    amd64


    settings:


    auto_tag: true


    auto_tag_suff
    i
    x: linux
    -
    amd64


    cache_from: appleboy/gorush


    daemon_off: false


    dockerf
    i
    le: docker/Dockerf
    i
    le.linux.amd64


    password:


    from_secret: docker_password


    repo: appleboy/gorush


    username:


    from_secret: docker_username


    when:


    event:


    exclude:


    - pull_request


    View full-size slide

  70. PLUGIN_USERNAME=appleboy
    PLUGIN_PASSWORD=password
    PLUGIN_REPO=appleboy/gorush
    PLUGIN_TAGS=1.0.0,1.0

    View full-size slide

  71. PLUGIN_USERNAME=appleboy
    PLUGIN_PASSWORD=password
    PLUGIN_REPO=appleboy/gorush
    PLUGIN_TAGS=1.0.0,1.0

    View full-size slide

  72. 如何製作


    Plugins

    View full-size slide

  73. Go Plugin vs Bash Plugin


    (Any language you want)

    View full-size slide

  74. kind: pipeline


    type: docker


    name: default


    steps:


    - name: webhook


    image: acme/webhook


    settings:


    url: http:
    / /
    hook.acme.com


    method: post


    body: |


    hello world

    View full-size slide

  75. # !
    /bin/sh


    curl \


    -X ${PLUGIN_METHOD} \


    -
    d ${PLUGIN_BODY} \


    ${PLUGIN_URL}


    View full-size slide

  76. FROM alpine:3.14


    ADD script.sh /bin/


    RUN chmod
    +
    x /bin/script.sh


    RUN apk -Uuv add curl ca
    -
    certif
    i
    cates


    ENTRYPOINT /bin/script.sh


    View full-size slide

  77. $ docker build -t acme/webhook .
    $ docker push acme/webhook
    $ docker run --rm \
    -e PLUGIN_METHOD=post \
    -e PLUGIN_URL=http://hook.acme.com \
    -e PLUGIN_BODY=hello \
    acme/webhook
    ฤᩄ

    View full-size slide

  78. $ docker build -t acme/webhook .
    $ docker push acme/webhook
    $ docker run --rm \
    -e PLUGIN_METHOD=post \
    -e PLUGIN_URL=http://hook.acme.com \
    -e PLUGIN_BODY=hello \
    acme/webhook
    ଌࢼ

    View full-size slide

  79. package main


    import (


    "net/http"


    "os"


    "strings"


    )


    func main() {


    body
    : =
    strings.NewReader(


    os.Getenv("PLUGIN_BODY"),


    )


    _, err
    : =
    http.NewRequest(


    os.Getenv("PLUGIN_METHOD"),


    os.Getenv("PLUGIN_URL"),


    body,


    )


    if err
    ! =
    nil {


    os.Exit(1)


    }


    } GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o webhook

    View full-size slide

  80. FROM alpine:3.14


    ADD webhook /bin/


    RUN apk -Uuv add ca
    -
    certif
    i
    cates


    ENTRYPOINT /bin/webhook

    View full-size slide

  81. 練習
    • 任何程式語⾔進⾏ Plugin 撰寫


    • 申請 dockerhub 帳號密碼


    • 透過 Drone ⾃動打包 Image 並上傳

    View full-size slide

  82. Deploy to Production Server


    via SSH Command

    View full-size slide

  83. - name: ssh commands


    image: appleboy/drone
    -
    ssh


    settings:


    host: foo.com


    username: root


    key:


    from_secret: ssh_key


    passphrase: 1234


    port: 22


    script:


    - mkdir abc/def/efg


    - echo "you can't see the steps."


    View full-size slide

  84. Promotions


    ⼿動部署

    View full-size slide

  85. kind: pipeline


    type: docker


    name: default


    steps:


    - name: test


    image: node


    commands:


    - npm install


    - npm run test


    - npm run bundle


    - name: deploy


    image: appleboy/drone
    -
    ssh


    settings:
    . . .

    trigger:


    event:


    - promote


    target:


    - production


    kind: pipeline


    type: docker


    name: deploy


    steps:


    - name: test


    image: node


    commands:


    - npm install


    - npm run test


    - npm run bundle


    - name: deploy


    image: appleboy/drone
    -
    ssh


    settings:
    . . .

    when:


    event:


    - promote


    target:


    - production


    View full-size slide

  86. $ drone build promote octocat/hello-world 42 staging
    $ drone build promote
    $ drone build promote octocat/hello-world 42 production

    View full-size slide