Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
GolangでDockerベースのCIを作る
Search
Shunsuke Maeda
May 18, 2019
Technology
3
3.4k
GolangでDockerベースのCIを作る
Golang を使って Dockerベース のCIを作るお話です。
Shunsuke Maeda
May 18, 2019
Tweet
Share
More Decks by Shunsuke Maeda
See All by Shunsuke Maeda
静的解析ツール detekt で任意の条件で警告させる
duck8823
1
1.3k
GitHub と連携する CI を作る
duck8823
3
2.4k
Other Decks in Technology
See All in Technology
Figma Dev Modeで進化するデザインとエンジニアリングの協働 / figma-with-engineering
cyberagentdevelopers
PRO
1
430
顧客が本当に必要だったもの - パフォーマンス改善編 / Make what is needed
soudai
24
6.8k
Product Engineer Night #6プロダクトエンジニアを育む仕組み・施策
hacomono
PRO
1
470
スプリントゴールにチームの状態も設定する背景とその効果 / Team state in sprint goals why and impact
kakehashi
2
100
【若手エンジニア応援LT会】AWS Security Hubの活用に苦労した話
kazushi_ohata
0
170
来年もre:Invent2024 に行きたいあなたへ - “集中”と“つながり”で楽しむ -
ny7760
0
470
AWS CDKでデータリストアの運用、どのように設計する?~Aurora・EFSの実践事例を紹介~/aws-cdk-data-restore-aurora-efs
mhrtech
4
650
Shift-from-React-to-Vue
calm1205
3
1.3k
AWS CodePipelineでコンテナアプリをデプロイした際に、古いイメージを自動で削除する
smt7174
1
100
リンクアンドモチベーション ソフトウェアエンジニア向け紹介資料 / Introduction to Link and Motivation for Software Engineers
lmi
4
290k
Java x Spring Boot Warm up
kazu_kichi_67
2
490
カメラを用いた店内計測におけるオプトインの仕組みの実現 / ai-optin-camera
cyberagentdevelopers
PRO
1
120
Featured
See All Featured
BBQ
matthewcrist
85
9.3k
Product Roadmaps are Hard
iamctodd
PRO
48
10k
Designing on Purpose - Digital PM Summit 2013
jponch
115
6.9k
Designing for humans not robots
tammielis
249
25k
Building Your Own Lightsaber
phodgson
102
6.1k
Rails Girls Zürich Keynote
gr2m
93
13k
GraphQLとの向き合い方2022年版
quramy
43
13k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
26
2.1k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
41
2.1k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9.1k
How To Stay Up To Date on Web Technology
chriscoyier
788
250k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
107
49k
Transcript
Golang Ͱ Docker ϕʔεͷ CI Λ࡞Δ ɹ GoConference 2019 Spring
2019.05.18 [SAT] Shunsuke Maeda (@duck8823)
ࣗݾհ ✓ Shunsuke Maeda ✓ @duck8823 ✓ ॴଐ ✓גࣜձࣾΤεɾΤϜɾΤε ✓EarthCampusגࣜձࣾʢ෭ʣ
2/43
CI͠ΜͲ͍ 3/43
CI͠ΜͲ͍ ! ✓CIΛߏங͍ͨ͠ ci := ci.New() for ci.IsBroken { //
CI͕յΕͯͨΒ fix(ci) // ϩʔΧϧͰ͚͢Ͳಈ࡞֬ೝͰ͖ͳ͍ͷͰ git.Commit() // ίϛοτ ͯ͠ git.Push() // ϓογϡ ͯ͠ ci.Run() // CI ಈ͔ͯ͠ΈΔ } 4/43
Golang Ͱ Docker ϕʔεͷ CI Λ࡞Δ duck8823/duci 5/43
duci ͷಛ ✓DockerfileͰఆٛ ✓ ֶशίετ͕͍ ✓ ϩʔΧϧͰ࣮ߦ ✓GitHub ʹରԠ ✓
Push / PR্ͷίϝϯτ Ͱ࣮ߦ ✓ ίϛοτεςʔλεΛ࡞ 6/43
ֶशίετ ✓Dockerfile ͷΈ FROM golang:1.12.4-alpine # ϕʔεΠϝʔδ RUN apk --update
add --no-cache ... # ඞཁͳύοέʔδͷΠϯετʔϧ WORKDIR /workdir COPY . . ENTRYPOINT ["make"] # λεΫϥϯφʔͷར༻Λਪ CMD ["test"] # σϑΥϧτͷλεΫ 7/43
ֶशίετ ✓DockerͷࣝͰΩϟογϡԽ FROM golang:1.12.4-alpine RUN apk --update add --no-cache ...
WORKDIR /workdir COPY go.mod . COPY go.sum . RUN go mod download COPY . . ENTRYPOINT ["make"] CMD ["test"] 8/43
ϩʔΧϧͰ࠶ݱ ✓ίϚϯυΛ༻ҙ ✓ϩʔΧϧͰࢼ͔ͯ͠Β commit & push Ͱ͖Δ duci run INFO[14/May/2019
09:25:57.353] Step 1/9 : FROM openjdk:11 as Build INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> 0aa10063a184 INFO[14/May/2019 09:25:57.353] Step 2/9 : WORKDIR /workdir INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> Using cache INFO[14/May/2019 09:25:57.353] ---> 2dcc968d8994 ... 9/43
CIΛ࡞Δ 10/43
Continuous Integration 11/43
Continuous Integration ✓ఆظతʹ ✓ϏϧυςετΛ࣮ߦ ͢Δ͜ͱͰ ✓ૣظʹ ϑΟʔυόοΫ Λಘͯ όά͕ຊ൪ڥʹࠞೖ͢ΔͷΛ͙ 12/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
13/43
duci ʹ͓͚Δߏ ✓࣮ߦλΠϛϯά ✓ GitHub Webhooks ✓࣮ߦڥ ✓ Dockerίϯςφ ✓࣮ߦ݁Ռ
✓ ϩάग़ྗ / GitHub ίϛοτεςʔλε 14/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
15/43
࣮ߦλΠϛϯά ✓GitHub Webhooks https://github.com/<owner>/<repo>/settings/hooks ✓ ҙͷλΠϛϯάͰ Payload Λඈ͢ 16/43
PayloadΛड͚औΔ ✓GitHub ͔ΒඈΜͰ͘Δ Payload JSON Λड͚औΔඞཁ͕͋Δ import "net/http" func JobHandler(w
http.ResponseWriter, r *http.Request) { // ֤ΠϕϯτʹରԠͨ͠PayloadͷύʔεॲཧͱCIδϣϒͷ࣮ߦ } func main() { http.HandleFunc("/", JobHandler) http.ListenAndServe(":8080", nil) } 17/43
GitHub ͷ Webhooks ✓Event ͱ Action ͷΈ߹ΘͤͰλΠϛϯά੍ޚ ✓Event ✓ Webhooksͷछྨ
(Header: X-GitHub-Event) ྫ. push, issue_comment, pull_request ✓ GitHubͰઃఆՄೳ ✓Action ✓ Eventຖͷࡉ͔͍छྨ (JSON payload) ྫ. created, opened, synchronize 18/43
Eventͷऔಘ ✓ϦΫΤετͷϔομʔ X-GitHub-Event ͔ΒΠϕϯτΛऔಘ ✓Πϕϯτຖʹ Payload ͕ҟͳΔ func JobHandler(w http.ResponseWriter,
r *http.Request) { event := r.Header.Get("X-GitHub-Event") switch event { case "push": // https://developer.github.com/v3/activity/events/types/#pushevent case "issue_comment": // https://developer.github.com/v3/activity/events/types/#issuecommentevent case "pull_request": // https://developer.github.com/v3/activity/events/types/#pullrequestevent default: http.Error(w, fmt.Sprintf("Bad event type: %s", event), http.StatusBadRequest) } } 19/43
Payload JSON ͷऔΓѻ͍ ✓encoding/json ͰσίʔυͰ͖Δ ✓Payloadͷܕ github.com/google/go-github import ( "encoding/json"
"github.com/google/go-github/github" ) func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "push": event := &github.PushEvent{} err := json.NewDecoder(r.Body).Decode(event) ... } } 20/43
Forkͨ͠ϦϙδτϦ͔Βͷ Pull Request ✓push ΠϕϯτඈΜͰ͜ͳ͍ ✓ pull_request Πϕϯτͷ synchronize ΞΫγϣϯ
func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "pull_request": event := &github.PullRequestEvent{} json.NewDecoder(r.Body).Decode(event) switch event.GetAction() { case "synchronize", "open": ... } } } 21/43
Pull Request ͷίϝϯτ import ( "github.com/google/go-github/github" "golang.org/x/oauth2" ) func JobHandler(w
http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "issue_comment": event := &github.IssueComment{} json.NewDecoder(r.Body).Decode(event) ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "access token"}) tc := oauth2.NewClient(context.Background(), ts) cli := github.NewClient(tc) pr := cli.PullRequests.Get( context.Background(), ownerName, repoName, event.GetIssue().GetNumber(), ) ... } } 22/43
ඇಉظͰϨεϙϯεΛฦ͢ ✓δϣϒ͕͔͔࣌ؒΔ => ඇಉظʹ࣮ߦͯ͠ϨεϙϯεΛฦ͢ func JobHandler(w http.ResponseWriter, r *http.Request) {
if (invalidRequest(r)) { http.Error(w, r.Error(), http.StatusBadRequest) return } go func() { // CIδϣϒͷ࣮ߦ }() w.WriteHeader(http.StatusOK) } 23/43
࣮ߦλΠϛϯά (·ͱΊ) ✓ϦΫΤετड => net/http ✓Payload ͷܕ => ϥΠϒϥϦͷར༻ ྫ.
GitHub Webhooks: github.com/google/go-github ✓ GitHub Event ͱ Action ͰλΠϛϯάΛ੍ޚ ✓ඇಉظͰϨεϙϯεΛฦ͢ => goroutine 24/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
25/43
࣮ߦڥ ✓Dockerίϯςφ ✓ ϗετΛԚ͞ͳ͍ ✓ ϩʔΧϧ / CI Ͱͷಈ࡞ʹ࠶ݱੑΛͱΓ͍͢ ✓github.com/docker/docker
Λར༻ 26/43
github.com/docker/docker ✓ϦϙδτϦ moby/moby ✓Docker Daemon ͷରԠAPIόʔδϣϯ ɹ- Docker for Mac
1.39 ✓ Moby ͷ master 1.41 docker version Server: Docker Engine - Community Engine: Version: 18.09.2 API version: 1.39 (minimum version 1.12) ... 27/43
DockerΠϝʔδ/ίϯςφΛѻ͏ import "github.com/docker/docker/client" cli, _ := client.NewClientWithOpts(client.FromEnv) ✓ײతͳAPI ✓ cli#ImageBuild
= docker build ✓ cli#ContainerCreate = docker create ✓ cli#ContainerStart = docker start 28/43
DockerΠϝʔδͷϏϧυ ✓ίϯςΩετʢσΟϨΫτϦʣΛ tarball ͷܗࣜʹ͢Δඞཁ͕͋Δ ✓cli#ImageBuild ✓ ඇಉظʹ࣮ߦ͞ΕΔ ✓ resp.Body Λ
EOF ·ͰಡΜͰऴྃͪ opts := types.ImageBuildOptions{ Tags: []string{"tag"} } resp, _ := cli.ImageBuild(ctx, tarball("/workdir"), opts) ioutil.ReadAll(resp.Body) 29/43
ίϯςφͷ࣮ߦ ✓ίϯςφͷ࡞ cli#ContainerCreate ✓cli#ContainerStart ඇಉظ ✓ cli#ContainerWait cli#ContainerLogs ͰδϣϒͷใΛऔಘ
sopts := types.ContainerStartOptions{} cli.ContainerStart(context.Background(), "containerId", sopts) code, _ := cli.ContainerWait(context.Background(), "containerId") if code != 0 { // exit code ͕ 0Ҏ֎ => δϣϒͷࣦഊ } lopts := types.ContainerLogsOptions{ ShowStdout: true, ShowStderr: true } log _ := cli.ContainerLogs(context.Background(), "containerId", lopts) 30/43
δϣϒͷฒྻͱλΠϜΞτ ✓ϦΫΤετຖʹ goroutine Ͱ࣮ߦ ✓ δϣϒΛ͍࣮ͭ͘ߦͯ͠͠·͏ ✓ϗετͷϦιʔε༗ݶ => ฒྻͷ੍ޚ ✓
͕͔͔࣌ؒΔॲཧ͕͋Δͱ࣍ͷδϣϒ͕࣮ߦ͞Εͳ͍ => λΠϜΞτͷઃఆ 31/43
ฒྻͷ੍ޚ ✓channel Λར༻ var sem = make(chan struct{}, 2) //
2ͭΛ༻ҙ func execute() { sem <- struct{}{} // ΛҰͭ֬อ͢Δ(ۭ͖͕ͳ͚Ε͜͜ͰॲཧΛͭ worker.execute() // CIδϣϒͷ࣮ߦ <-sem // ΛҰͭղ์͢Δ } func main() { go execute() go execute() go execute() } 32/43
λΠϜΞτͷઃఆ ✓context#WithTimeout ͰλΠϜΞτΛઃఆͰ͖Δ func execute() { err := make(chan error)
timeout, cancel := context.WithTimeout(context.Background(), 10 * time.Second) defer cancel() go func() { err <- worker.execute() // CIδϣϒͷ࣮ߦ }() select { case <-timeout.Done(): println("timeout") case e := <-err: println("done") } } 33/43
࣮ߦڥ (·ͱΊ) ✓DockerΫϥΠΞϯτ ✓ github.com/docker/docker ✓ ײతͳAPI ✓ ಉظॲཧ image
build ͱ container start ͰҟͳΔ ✓ฒྻͷ੍ޚ ✓ channel ✓λΠϜΞτͷઃఆ ✓ context#WithTimeout 34/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
35/43
࣮ߦ݁Ռ ✓GitHub ͷίϛοτεςʔλε 36/43
ίϛοτεςʔλε ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "ΞΫηετʔΫϯ"}) tc := oauth2.NewClient(context.Background(), ts) cli
:= github.NewClient(tc) stat := &github.Status{ Context: github.String("ίϛοτεςʔλεͷ໊લ"), Description: github.String("ৄࡉઆ໌ͳͲ(จࣈ੍ݶ͋Γ)"), State: github.String("εςʔλεʢྫ. failureʣ"), TargetURL: github.String("֎෦ϖʔδͷϦϯΫ"), } cli.Repositories.CreateStatus( context.Background(), "ownerName", "repoName", "ref (commit hash)", stat, ) 37/43
ϩά / Ϩεϙϯεͷදࣔ ✓http.ResponseWriter#Write ͰϨεϙϯεʹॻ͖ࠐΈ func LogHandler(w http.ResponseWriter, r *http.Request)
{ w.WriteHeader(http.StatusOK) w.Write([]byte("log")) } 38/43
࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓ૄͳઃܭʹ͓ͯ͘͠ͱࠩͰػೳΛ࣮͍͢͠ ✓݁ՌΛग़ྗ͍ͨ͠λΠϛϯάδϣϒ࣮ߦத(લޙ)ʹ ✓ δϣϒͷొ࣌ ✓ δϣϒͷ࣮ߦ։࢝࣌ ✓ ϩά ✓
δϣϒͷऴྃ࣌ʢޭ/ࣦഊ/Τϥʔ) 39/43
࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓࣮ߦڥͰϑΟʔυόοΫ༻ؔΛઃఆͰ͖ΔΑ͏ʹ͢Δ type worker struct { startFun func() } func
(w *worker) run() { err := w.startFun() ... } func localRun() { w := &worker{ startFun: printLog } w.run() } func serverRun() { w := &worker{ startFun: printLogAndCommitStatus } w.run() } 40/43
࣮ߦ݁Ռ (·ͱΊ) ✓GitHubͷίϛοτεςʔλε ✓ github.com/google/go-github ✓Ϩεϙϯε ✓ net/http ͷ http.ResponseWriter#Write
✓࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓ ϑΟʔυόοΫ༻ؔΛઃఆͰ͖ΔΑ͏ʹ͢Δ 41/43
·ͱΊ ✓ඪ४ػೳ/ඪ४ϥΠϒϥϦ ✓ ඇಉظλΠϜΞτ ✓ HTTPαʔόʔ ✓GitHub Docker ͳͲͷΫϥΠΞϯτϥΠϒϥϦΛར༻
✓ҰͭҰͭͷʢهड़ʣ؆ܿ ✓ Έ߹ΘͤͯCIΛ࡞Δ͜ͱ͕Ͱ͖Δ 42/43
Let's make CI server! 43/43