Upgrade to Pro — share decks privately, control downloads, hide ads and more …

GolangでDockerベースのCIを作る

 GolangでDockerベースのCIを作る

Golang を使って Dockerベース のCIを作るお話です。

Shunsuke Maeda

May 18, 2019
Tweet

More Decks by Shunsuke Maeda

Other Decks in Technology

Transcript

  1. Golang Ͱ Docker ϕʔεͷ CI Λ࡞Δ ɹ GoConference 2019 Spring

    2019.05.18 [SAT] Shunsuke Maeda (@duck8823)
  2. CI͠ΜͲ͍ ! ✓CIΛߏங͍ͨ͠ ci := ci.New() for ci.IsBroken { //

    CI͕յΕͯͨΒ fix(ci) // ϩʔΧϧͰ௚͚͢Ͳಈ࡞֬ೝͰ͖ͳ͍ͷͰ git.Commit() // ίϛοτ ͯ͠ git.Push() // ϓογϡ ͯ͠ ci.Run() // CI ಈ͔ͯ͠ΈΔ } 4/43
  3. duci ͷಛ௃ ✓DockerfileͰఆٛ ✓ ֶशίετ͕௿͍ ✓ ϩʔΧϧͰ΋࣮ߦ ✓GitHub ʹରԠ ✓

    Push / PR্ͷίϝϯτ Ͱ࣮ߦ ✓ ίϛοτεςʔλεΛ࡞੒ 6/43
  4. ֶशίετ ✓Dockerfile ͷΈ FROM golang:1.12.4-alpine # ϕʔεΠϝʔδ RUN apk --update

    add --no-cache ... # ඞཁͳύοέʔδͷΠϯετʔϧ WORKDIR /workdir COPY . . ENTRYPOINT ["make"] # λεΫϥϯφʔͷར༻Λਪ঑ CMD ["test"] # σϑΥϧτͷλεΫ 7/43
  5. ֶशίετ ✓Dockerͷ஌ࣝͰΩϟογϡԽ FROM golang:1.12.4-alpine RUN apk --update add --no-cache ...

    WORKDIR /workdir COPY go.mod . COPY go.sum . RUN go mod download COPY . . ENTRYPOINT ["make"] CMD ["test"] 8/43
  6. ϩʔΧϧͰ࠶ݱ ✓ίϚϯυΛ༻ҙ ✓ϩʔΧϧͰࢼ͔ͯ͠Β commit & push Ͱ͖Δ duci run INFO[14/May/2019

    09:25:57.353] Step 1/9 : FROM openjdk:11 as Build INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> 0aa10063a184 INFO[14/May/2019 09:25:57.353] Step 2/9 : WORKDIR /workdir INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> Using cache INFO[14/May/2019 09:25:57.353] ---> 2dcc968d8994 ... 9/43
  7. PayloadΛड͚औΔ ✓GitHub ͔ΒඈΜͰ͘Δ Payload JSON Λड͚औΔඞཁ͕͋Δ import "net/http" func JobHandler(w

    http.ResponseWriter, r *http.Request) { // ֤ΠϕϯτʹରԠͨ͠PayloadͷύʔεॲཧͱCIδϣϒͷ࣮ߦ } func main() { http.HandleFunc("/", JobHandler) http.ListenAndServe(":8080", nil) } 17/43
  8. GitHub ͷ Webhooks ✓Event ͱ Action ͷ૊Έ߹ΘͤͰλΠϛϯά੍ޚ ✓Event ✓ Webhooksͷछྨ

    (Header: X-GitHub-Event) ྫ. push, issue_comment, pull_request ✓ GitHubͰઃఆՄೳ ✓Action ✓ Eventຖͷࡉ͔͍छྨ (JSON payload಺) ྫ. created, opened, synchronize 18/43
  9. Eventͷऔಘ ✓ϦΫΤετͷϔομʔ X-GitHub-Event ͔ΒΠϕϯτΛऔಘ ✓Πϕϯτຖʹ Payload ͕ҟͳΔ func JobHandler(w http.ResponseWriter,

    r *http.Request) { event := r.Header.Get("X-GitHub-Event") switch event { case "push": // https://developer.github.com/v3/activity/events/types/#pushevent case "issue_comment": // https://developer.github.com/v3/activity/events/types/#issuecommentevent case "pull_request": // https://developer.github.com/v3/activity/events/types/#pullrequestevent default: http.Error(w, fmt.Sprintf("Bad event type: %s", event), http.StatusBadRequest) } } 19/43
  10. Payload JSON ͷऔΓѻ͍ ✓encoding/json ͰσίʔυͰ͖Δ ✓Payloadͷܕ github.com/google/go-github import ( "encoding/json"

    "github.com/google/go-github/github" ) func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "push": event := &github.PushEvent{} err := json.NewDecoder(r.Body).Decode(event) ... } } 20/43
  11. Forkͨ͠ϦϙδτϦ͔Βͷ Pull Request ✓push Πϕϯτ͸ඈΜͰ͜ͳ͍ ✓ pull_request Πϕϯτͷ synchronize ΞΫγϣϯ

    func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "pull_request": event := &github.PullRequestEvent{} json.NewDecoder(r.Body).Decode(event) switch event.GetAction() { case "synchronize", "open": ... } } } 21/43
  12. Pull Request ͷίϝϯτ import ( "github.com/google/go-github/github" "golang.org/x/oauth2" ) func JobHandler(w

    http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "issue_comment": event := &github.IssueComment{} json.NewDecoder(r.Body).Decode(event) ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "access token"}) tc := oauth2.NewClient(context.Background(), ts) cli := github.NewClient(tc) pr := cli.PullRequests.Get( context.Background(), ownerName, repoName, event.GetIssue().GetNumber(), ) ... } } 22/43
  13. ඇಉظͰϨεϙϯεΛฦ͢ ✓δϣϒ͸͕͔͔࣌ؒΔ => ඇಉظʹ࣮ߦͯ͠ϨεϙϯεΛฦ͢ func JobHandler(w http.ResponseWriter, r *http.Request) {

    if (invalidRequest(r)) { http.Error(w, r.Error(), http.StatusBadRequest) return } go func() { // CIδϣϒͷ࣮ߦ }() w.WriteHeader(http.StatusOK) } 23/43
  14. ࣮ߦλΠϛϯά (·ͱΊ) ✓ϦΫΤετड෇ => net/http ✓Payload ͷܕ => ϥΠϒϥϦͷར༻ ྫ.

    GitHub Webhooks: github.com/google/go-github ✓ GitHub ͸ Event ͱ Action ͰλΠϛϯάΛ੍ޚ ✓ඇಉظͰϨεϙϯεΛฦ͢ => goroutine 24/43
  15. github.com/docker/docker ✓ϦϙδτϦ͸ moby/moby ✓Docker Daemon ͷରԠAPIόʔδϣϯ ɹ- Docker for Mac

    ͸ 1.39 ✓ Moby ͷ master ͸ 1.41 docker version Server: Docker Engine - Community Engine: Version: 18.09.2 API version: 1.39 (minimum version 1.12) ... 27/43
  16. DockerΠϝʔδͷϏϧυ ✓ίϯςΩετʢσΟϨΫτϦʣΛ tarball ͷܗࣜʹ͢Δඞཁ͕͋Δ ✓cli#ImageBuild ✓ ඇಉظʹ࣮ߦ͞ΕΔ ✓ resp.Body Λ

    EOF ·ͰಡΜͰऴྃ଴ͪ opts := types.ImageBuildOptions{ Tags: []string{"tag"} } resp, _ := cli.ImageBuild(ctx, tarball("/workdir"), opts) ioutil.ReadAll(resp.Body) 29/43
  17. ίϯςφͷ࣮ߦ ✓ίϯςφͷ࡞੒͸ cli#ContainerCreate ✓cli#ContainerStart ͸ඇಉظ ✓ cli#ContainerWait ΍ cli#ContainerLogs Ͱδϣϒͷ৘ใΛऔಘ

    sopts := types.ContainerStartOptions{} cli.ContainerStart(context.Background(), "containerId", sopts) code, _ := cli.ContainerWait(context.Background(), "containerId") if code != 0 { // exit code ͕ 0Ҏ֎ => δϣϒͷࣦഊ } lopts := types.ContainerLogsOptions{ ShowStdout: true, ShowStderr: true } log _ := cli.ContainerLogs(context.Background(), "containerId", lopts) 30/43
  18. ฒྻ਺ͷ੍ޚ ✓channel Λར༻ var sem = make(chan struct{}, 2) //

    2ͭ࿮Λ༻ҙ func execute() { sem <- struct{}{} // ࿮ΛҰͭ֬อ͢Δ(ۭ͖͕ͳ͚Ε͹͜͜ͰॲཧΛ଴ͭ worker.execute() // CIδϣϒͷ࣮ߦ <-sem // ࿮ΛҰͭղ์͢Δ } func main() { go execute() go execute() go execute() } 32/43
  19. λΠϜΞ΢τͷઃఆ ✓context#WithTimeout ͰλΠϜΞ΢τΛઃఆͰ͖Δ func execute() { err := make(chan error)

    timeout, cancel := context.WithTimeout(context.Background(), 10 * time.Second) defer cancel() go func() { err <- worker.execute() // CIδϣϒͷ࣮ߦ }() select { case <-timeout.Done(): println("timeout") case e := <-err: println("done") } } 33/43
  20. ࣮ߦ؀ڥ (·ͱΊ) ✓DockerΫϥΠΞϯτ ✓ github.com/docker/docker ✓ ௚ײతͳAPI ✓ ಉظॲཧ͸ image

    build ͱ container start ͰҟͳΔ ✓ฒྻ਺ͷ੍ޚ ✓ channel ✓λΠϜΞ΢τͷઃఆ ✓ context#WithTimeout 34/43
  21. ίϛοτεςʔλε ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "ΞΫηετʔΫϯ"}) tc := oauth2.NewClient(context.Background(), ts) cli

    := github.NewClient(tc) stat := &github.Status{ Context: github.String("ίϛοτεςʔλεͷ໊લ"), Description: github.String("ৄࡉઆ໌ͳͲ(จࣈ਺੍ݶ͋Γ)"), State: github.String("εςʔλεʢྫ. failureʣ"), TargetURL: github.String("֎෦ϖʔδ΁ͷϦϯΫ"), } cli.Repositories.CreateStatus( context.Background(), "ownerName", "repoName", "ref (commit hash)", stat, ) 37/43
  22. ࣮ߦ؀ڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓࣮ߦ؀ڥͰϑΟʔυόοΫ༻ؔ਺ΛઃఆͰ͖ΔΑ͏ʹ͢Δ type worker struct { startFun func() } func

    (w *worker) run() { err := w.startFun() ... } func localRun() { w := &worker{ startFun: printLog } w.run() } func serverRun() { w := &worker{ startFun: printLogAndCommitStatus } w.run() } 40/43
  23. ࣮ߦ݁Ռ (·ͱΊ) ✓GitHubͷίϛοτεςʔλε ✓ github.com/google/go-github ✓Ϩεϙϯε ✓ net/http ͷ http.ResponseWriter#Write

    ✓࣮ߦ؀ڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓ ϑΟʔυόοΫ༻ؔ਺ΛઃఆͰ͖ΔΑ͏ʹ͢Δ 41/43