GitHub上で構築するコードメトリクス計測基盤 / TECH STAND #9 GitHub

by Ken’ichiro Oyama

Slide 1

Slide 1 text

GitHub্Ͱߏங͢Δ ίʔυϝτϦΫεܭଌج൫ খࢁ݈Ұ࿠ / GMO PEPABO inc. 2022.8.31 TECH STAND #9 GitHub 1

Slide 2

Slide 2 text

ࣗݾ঺հ ٕज़෦ɹٕज़ج൫νʔϜ 2018೥ த్ೖࣾ খࢁ ݈Ұ࿠ Ken’ichiro Oyama গ࣮͠༻తͰখ͞ͳOSSΛॻ͘ͷ͕झຯɻ ● GitHub : k1LoW ● Twitter : @k1LoW 2

Slide 3

Slide 3 text

GMOϖύϘʹ͍ͭͯ 3 https://pepabo.com

Slide 4

Slide 4 text

4 ίʔυϝτϦΫε

Slide 5

Slide 5 text

• ιϑτ΢ΣΞͷιʔείʔυʹ·ͭΘΔ͞·͟·ͳϝτϦΫε • ࢀߟʮίʔυϝτϦοΫ஋ʯ https://docs.microsoft.com/ja-jp/visualstudio/code- quality/code-metrics-values?view=vs-2022 • ʢ͜ͷηογϣϯʹ͓͚ΔʮίʔυϝτϦΫεʯʣͷྫͱͯ͠ • ίʔυΧόϨοδʢςετΧόϨοδʣ • Code to Test Ratio • ςετ࣮ߦ࣌ؒ • อक༰қੑࢦ਺ • αΠΫϩϚςΟοΫෳࡶ౓ 5 ίʔυϝτϦΫε ίʔυϝτϦΫεͱ͸

Slide 6

Slide 6 text

• ίʔυΧόϨοδ͸֤ϓϩάϥϛϯάݴޠͷςεςΟϯάϑϨʔϜϫʔΫ͸େ఍ܭ ଌɾग़ྗͰ͖Δ • ඪ४ग़ྗ • HTML • ͭ·ΓɺGitHub Actions্ͰCI؀ڥΛߏஙͯ͠ςετΛ࣮ߦ͍ͯ͠Διϑτ΢ΣΞͰ ͋Ε͹େମऔಘͰ͖Δ • ςετΛ࣮ߦ͍ͯ͠ΔςεςΟϯάϑϨʔϜϫʔΫͷΦϓγϣϯΛม͑Δ͚ͩ 6 ίʔυϝτϦΫε ྫ͑͹ίʔυΧόϨοδʢςετΧόϨοδʣ

Slide 7

Slide 7 text

✓ CIͷඪ४ग़ྗʹग़ྗ͍ͯ͠Δ ✓ ΧόϨοδʹᮢ஋ΛܾΊͯᮢ஋ΛԼճͬͨ৔߹͸CIΛFAIL͍ͤͯ͞Δ ✓ ΧόϨοδܭଌ݁ՌΛPull Requestʹίϝϯτ͍ͯ͠Δ ✓ લճͷܭଌ݁Ռͱൺֱͨ݁͠Ռ΋ද͍ࣔͯ͠Δ ✓ ܭଌ݁ՌΛ஝ੵ͍ͯ͠Δ 7 ίʔυϝτϦΫε ܭଌͨ͠ΧόϨοδΛͲ͏͍ͯ͠Δʁ

Slide 8

Slide 8 text

8 ͜ΕΒΛGitHub্Ͱߏங͢Δ

Slide 9

Slide 9 text

octocov 9

Slide 10

Slide 10 text

• https://github.com/k1LoW/octocov • ίʔυϝτϦΫεπʔϧ…ίʔυΧόϨοδɺCode to Test Ratioɺςετ࣮ߦ࣌ؒ • ଟ਺ͷϓϩάϥϛϯάݴޠͱɺओཁͳΧόϨοδϨϙʔτϑΥʔϚοτʹରԠ • GitHub Actionsͷactionͱͯ͠ίʔυϝτϦΫεΛूܭɺP/RίϝϯτɺϨϙʔτ • CLIπʔϧͱͯ͠ίʔυΧόϨοδΛूܭɻදࣔ • ΧόϨοδόοδੜ੒ octocov 10 10 octocov

Slide 11

Slide 11 text

octocov • ༷ʑͳछྨͷΧόϨοδϑΥʔϚοτʹରԠ • Go coverage … Go • LCOV … ੲ͔Β͋ΔϑΥʔϚοτ • SimpleCov … Ruby • Clover … PHPUnit etc • Cobertura … Coverage.py etc • JaCoCo … Java Kotlin etc αϙʔτ͍ͯ͠ΔΧόϨοδϑΥʔϚοτ 11

Slide 12

Slide 12 text

octocov • ίʔυϝτϦΫεͷܭଌ݁ՌΛࢦఆͷσʔλετΞʹه࿥͢Δ͜ͱ͕Ͱ͖Δ • octocov͸ҟͳΔΧόϨοδϑΥʔϚοτ΋ڞ௨ͷߏ଄Խσʔλʹม׵͢Δ • ༷ʑͳछྨͷσʔλετΞʹରԠ • ϩʔΧϧϑΝΠϧγεςϜ • GitHubϦϙδτϦ • GitHub Artifact • Amazon S3 • Google Cloud Storage(GCS) • BigQuery ίʔυϝτϦΫεσʔλͷه࿥ 12

Slide 13

Slide 13 text

13 ίʔυϝτϦΫεͷܭଌ

Slide 14

Slide 14 text

ίʔυϝτϦΫεͷܭଌ • ͨͬͨ3εςοϓ 1. ςεςΟϯάϑϨʔϜϫʔΫͰΧόϨοδϨϙʔτΛग़ྗ͢ΔΑ͏ʹWorkflowΛमਖ਼ • ֤ςεςΟϯάϑϨʔϜϫʔΫͷΦϓγϣϯΛมߋ͢Δ͚ͩ ɹɹgo test ./... -coverprofile=coverage.out -covermode=count 2. .octocov.yml ΛϓϩδΣΫτϧʔτʹઃஔ octocov init 3. k1LoW/octocov-action ΛΧόϨοδϨϙʔτग़ྗޙͷStepʹ௥Ճ octocovͷઃఆʢ୯ҰϦϙδτϦͰͷίʔυϝτϦΫεͷܭଌʣ 14

Slide 15

Slide 15 text

ίʔυϝτϦΫεͷܭଌ 15 .octocov.yml

Slide 16

Slide 16 text

ίʔυϝτϦΫεͷܭଌ 16 .github/workflows/ci.yml

Slide 17

Slide 17 text

ίʔυϝτϦΫεͷܭଌ 17

Slide 18

Slide 18 text

ίʔυϝτϦΫεͷܭଌ ✓ CIͷඪ४ग़ྗʹग़ྗ͍ͯ͠Δ ✓ ΧόϨοδʹᮢ஋ΛܾΊͯᮢ஋ΛԼճͬͨ৔߹͸CIΛFAIL͍ͤͯ͞Δ • શͯͷϝτϦΫεͰᮢ஋ΛઃఆՄೳ ✓ ΧόϨοδܭଌ݁ՌΛPull Requestʹίϝϯτ͍ͯ͠Δ ✓ લճͷܭଌ݁Ռͱൺֱͨ݁͠Ռ΋ද͍ࣔͯ͠Δ • ϦϙδτϦͷArtifactʹܭଌ݁ՌΛอଘ͢Δ͜ͱͰલճܭଌ݁ՌΛऔಘՄೳ ✓ ܭଌ݁ՌΛ஝ੵ͍ͯ͠Δ • Artifactʹܭଌ݁ՌΛ஝ੵͰ͖Δʢͨͩɺ࠶ར༻͸͠ʹ͍͘ʣ GitHubͷ୯ҰϦϙδτϦ಺ʹดͨ͡ܗͰίʔυϝτϦΫεͷܭଌ͕Մೳ 18

Slide 19

Slide 19 text

ίʔυϝτϦΫεͷܭଌ • monorepoʹରԠ • ͦΕͧΕͷίϯϙʔωϯτͰίʔυϝτϦΫεܭଌ͕Մೳ • ΧόϨοδϨϙʔτͷϚʔδ͕Մೳ • ରԠ͍ͯ͠ΔશͯͷΧόϨοδϑΥʔϚοτͷܭଌ݁ՌΛڞ௨ͷߏ଄Խσʔλʹม׵ՄೳͳͨΊ • ฒྻ࣮ߦͨ͠ςετΧόϨοδͷϚʔδ΋Մೳ • ҟͳΔϓϩάϥϛϯάݴޠͷςετΧόϨοδͷϚʔδ΋Մೳ • ಠཱͨ͠CLIπʔϧͱͯ͠ར༻Մೳ • ରԠ͍ͯ͠ΔΧόϨοδϑΥʔϚοτͰ͋Ε͹खݩͰίʔυΧόϨοδΛ֬ೝͰ͖Δ • ιʔείʔυͷΧόʔ͍ͯ͠Δߦͷ֬ೝ΋Մೳ • https://k1low.hatenablog.com/entry/2022/05/10/083000 ଞʹ΋octocovͰ͖Δ͜ͱ 19

Slide 20

Slide 20 text

20 ίʔυϝτϦΫεͷ஝ੵ

Slide 21

Slide 21 text

ίʔυϝτϦΫεͷ஝ੵ • ίʔυϝτϦΫε͸ͦͷίʔυͷϦϙδτϦʹϓϩμΫτͷϝτϦΫεͱͳΓ͏Δ • Four Keys΋ϓϩμΫτͷϝτϦΫε • Four KeysΛGitHub͔Βऔಘ͢ΔऔΓ૊Έʹ͍ͭͯ͸ https://tech.pepabo.com/ 2022/01/06/four-keys-dashboard/ Λੋඇ • ୯ҰϦϙδτϦͰͷܭଌͰ΋ྑ͍͕ɺϓϩδΣΫτ୯Ґɺ૊৫୯ҐͰΈΔͱݟ͑ͯ͘Δ ΋ͷ͕͋Δ • ࣌ܥྻʹϓϩοτ͢Δ͜ͱͰݟ͑ͯ͘Δ΋ͷ͕͋Δ ૊৫શମͰͷίʔυϝτϦΫεͷ஝ੵ 21

Slide 22

Slide 22 text

ίʔυϝτϦΫεͷ஝ੵ • ίʔυϝτϦΫεͷܭଌ݁ՌΛࢦఆͷσʔλετΞʹه࿥͢Δ͜ͱ͕Ͱ͖Δ • ରԠ͍ͯ͠ΔΧόϨοδϑΥʔϚοτͷܭଌ݁Ռ͸ڞ௨ͷߏ଄Խσʔλʹม׵ ͢Δ • ༷ʑͳछྨͷσʔλετΞʹରԠ • ϩʔΧϧϑΝΠϧγεςϜ • GitHubϦϙδτϦ • GitHub Artifact • Amazon S3 • Google Cloud Storage(GCS) • BigQuery ίʔυϝτϦΫεσʔλͷه࿥ʢ࠶ܝʣ 22

Slide 23

Slide 23 text

ίʔυϝτϦΫεͷ஝ੵ • octocov͸ .octocov.yml ͷઃఆΛมߋ͢Δ͜ͱͰίʔυϝτϦΫεΛܭଌ͢ΔϞʔυ͔Βɺίʔ υϝτϦΫεσʔλΛऩू͢ΔϞʔυʢCentral modeʣʹಈ࡞Λ੾Γସ͑Δ͜ͱ͕Ͱ͖Δɻ • GitHub ActionsͰಈ͔͢૝ఆͰ࡞ΒΕ͍ͯΔ • Central modeͰͰ͖Δ͜ͱ • σʔλετΞ͔Β֤ϦϙδτϦͷ࠷৽ͷίʔυϝτϦΫεσʔλΛऔಘͯ͠ϩʔΧϧʹอ ଘ͢Δ • ඞཁͰ͋Ε͹ git push ΋͢Δ • ίʔυϝτϦΫεσʔλ͔ΒόοδΛੜ੒͢Δ • ࠷৽ͷίʔυϝτϦΫεͷ஋΍όοδURLΛऔಘͰ͖ΔMarkdownϖʔδʢΠϯσοΫ εʣΛ࡞੒͢Δ Central mode 23

Slide 24

Slide 24 text

ίʔυϝτϦΫεͷ஝ੵ 24

Slide 25

Slide 25 text

ίʔυϝτϦΫεͷ஝ੵ • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹCentral RepoͷGitHubϦϙ δτϦ΋ࢦఆ • Central RepoͰ͸ϦϙδτϦ͔Βͷίϛοτͷon.pushΠϕϯτΛτϦΨʔʹόοδ ΍ΠϯσοΫεΛੜ੒ ίʔυϝτϦΫεܭଌج൫ on GitHub 25

Slide 26

Slide 26 text

ίʔυϝτϦΫεͷ஝ੵ • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹS3όέοτΛࢦఆ • Central RepoͰ͸on.scheduleΛτϦΨʔʹS3όέοτͷϝτϦΫεσʔλ͔Βόο δ΍ΠϯσοΫεΛੜ੒ ίʔυϝτϦΫεܭଌج൫ using Amazon S3 bucket 26

Slide 27

Slide 27 text

ίʔυϝτϦΫεͷ஝ੵ • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹGCSόέοτΛࢦఆ • Central RepoͰ͸on.scheduleΛτϦΨʔʹGCSόέοτͷϝτϦΫεσʔλ͔Β όοδ΍ΠϯσοΫεΛੜ੒ ίʔυϝτϦΫεܭଌج൫ using Google Cloud Storage 27

Slide 28

Slide 28 text

ίʔυϝτϦΫεͷ஝ੵ • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹBigQueryςʔϒϧΛࢦఆ • ςʔϒϧ͸͋Β͔͡ΊࢦఆͷεΩʔϚͰ࡞੒͓ͯ͘͠ octocov migrate-bq-table • Central RepoͰ͸on.scheduleΛτϦΨʔʹBigQueryͷςʔϒϧͷϝτϦΫεσʔλ ͔Βόοδ΍ΠϯσοΫεΛੜ੒ ίʔυϝτϦΫεܭଌج൫ using BigQuery 28

Slide 29

Slide 29 text

ίʔυϝτϦΫεͷ஝ੵ • BigQueryʹ஝ੵ֤ͨ͠Ϧϙδτ ϦͷίʔυϝτϦΫεͷσʔλΛ ͔ͭͬͯϝτϦΫεμογϡϘʔ υΛ࡞੒͠ɺՄࢹԽ • ৄ͘͠͸ https:// tech.pepabo.com/2022/04/25/ code-metrics-dashboard/ Λੋඇ ίʔυϝτϦΫεܭଌج൫ at ϖύϘ 29

Slide 30

Slide 30 text

ίʔυϝτϦΫεͷ஝ੵ 30

Slide 31

Slide 31 text

ίʔυϝτϦΫεܭଌج൫Λ ࣮ݱ͢ΔͨΊʹ࢖༻ͨ͠ GitHub΍GitHub Actionsͷػೳ 31

Slide 32

Slide 32 text

GitHub API 32

Slide 33

Slide 33 text

GitHub API • ਎΋֖΋ͳ͍Ͱ͕͢ɺGitHub্ʹίʔυϝτϦΫεܭଌج൫Λߏங͢Δͱ͍͏ ʮGitHubΛ֦ு͢Δʯʹ͋ͨͬͯඞਢͰ͔ͭศརͳػೳ͸API • REST APIɺGraphQL APIͱ΋ʹ׆༻ • ͔͜͜ΒʮGitHub্ʹίʔυϝτϦΫεܭଌج൫Λߏங͢Δʹ͋ͨͬͯศརͳAPI ΤϯυϙΠϯτʯΛ঺հ GitHub API 33

Slide 34

Slide 34 text

34 Deep Dive

Slide 35

Slide 35 text

GitHub API • List repository languages • octocov init Ͱݴޠ༻ͷ .octocov.yml Λग़ྗ͢ΔͨΊͷϦϙδτϦͷओͱͳΔϓϩά ϥϛϯάݴޠಛఆͷͨΊػೳͷϑΥʔϧόοΫઌͱͯ͠࢖༻ • ݴޠಛఆػೳࣗମ͸·ͩGoͷಛఆ͔͠࡞͍ͬͯͳ͍ͷͰɺ࣮࣭͜ͷAPI͕ୟ͔ΕΔ GET /repos/{owner}/{repo}/languages 35

Slide 36

Slide 36 text

GitHub API • List jobs for a workflow run • ςετ࣮ߦ࣌ؒͷऔಘͷͨΊʹ࢖༻ • ΧόϨοδϨϙʔτϑΝΠϧͷλΠϜελϯϓ͕ൣғ಺ʹؚ·ΕΔStepͷܦա࣌ؒ=ςετ ࣮ߦ࣌ؒͱ͍ͯ͠Δ • StepΛಛఆ͢ΔͨΊʹɺ·ͣ͸ݱࡏͷJob ID͕ඞཁ • ͔͠͠APIͰ࢖͏Job ID͸؀ڥม਺GTIHUB_JOBʹؚ·Ε͍ͯΔจࣈྻͰ͸ͳ͍ɻ਺ ࣈɻ • Job IDΛಛఆ͢ΔͨΊʹؤுͬͯୟ͍͍ͯΔAPI͕ίϨ • ؀ڥม਺GITHUB_JOB_ID͕ཉ͍͠!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! • https://github.com/actions/runner/issues/324 GET /repos/{owner}/{repo}/actions/runs/{run_id}/jobs 36

Slide 37

Slide 37 text

GitHub API • Minimizes a comment on an Issue, Commit, Pull Request, or Gist. • ΧόϨοδϨϙʔτͷPull Request΁ͷίϝϯτ࣌ʹҎલͷίϝϯτΛӅͨ͢Ίʹ࢖ ༻ • ࣮͸REST API (v3)ʹ͸ͳ͍ػೳ • GraphQL APIʹ͋Δ • https://docs.github.com/en/graphql/reference/mutations#minimizecomment minimizeComment 37

Slide 38

Slide 38 text

GitHub API • Download an artifact • σʔλετΞͱͯ͠ͷར༻ • ͔͠͠Upload an artifactͷAPI͸ެࣜϦϑΝϨϯεʹ͸هࡌ͕ͳ͍ • ͳͥʁ → Actions؀ڥҎ֎͔Β͸ୟ͚ͳ͍API͔ͩΒ 1. Actions؀ڥͰ͔͠ಘΒΕͳ͍؀ڥม਺ACTIONS_RUNTIME_URL΍GITHUB_RUN_ID Λ࢖ͬͯߏஙͨ͠URLʹArtifactͷΞοϓϩʔυ༻Ұ࣌URLΛϦΫΤετͯ͠ɺ 2. Actions؀ڥͰ͔͠ಘΒΕͳ͍؀ڥม਺ACTIONS_RUNTIME_TOKENΛೝূΩʔͱ͠ ͯɺҰ࣌URLʹରͯ͠Ξοϓϩʔυ • https://github.com/actions/toolkit ͷίʔυΛݩʹGoͰ࣮૷ • https://github.com/k1LoW/go-github-actions GET /repos/{owner}/{repo}/actions/artifacts/{artifact_id}/{archive_format} 38

Slide 39

Slide 39 text

͓ΘΓʹ 39

Slide 40

Slide 40 text

• GitHub্Ͱߏங͢ΔίʔυϝτϦΫεܭଌج൫ͷίΞͱͳΔπʔϧͱͯ͠octocovΛ ঺հ • ૊৫ͱͯ͠ܭଌͨ͠ίʔυϝτϦΫεΛ׆༻͍ͯ͘͠࢓૊Έͱͯ͠octocovͷ Central mode΍ɺԠ༻ࣄྫͱͯ͠BigQueryʹऩूͨ͠ίʔυϝτϦΫεΛՄࢹԽ͠ ͨϖύϘͷࣄྫΛ঺հ • GitHub API͸ศར 40 ͓ΘΓʹ ͓ΘΓʹ

Slide 41

Slide 41 text

Thank You! Thank You! 41

Slide 42

Slide 42 text

42 GitHub΁Feature Request!!!!!!!!!

Slide 43

Slide 43 text

GitHub API • ؀ڥม਺GITHUB_JOB_ID͕ཉ͍͠!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! • https://github.com/actions/runner/issues/324 • ೖΕͯ͸͍͚ͳ͍ཧ༝͕ͳͦ͞͏ͳͷͰੋඇݕ౼ͯ͠ཉ͍͠ GITHUB_JOB_ID͕ཉ͍͠!!!!!!!!!!!!!! 43

Slide 44

Slide 44 text

GitHub API • ྫ͑͹forkઌϦϙδτϦ͔ΒͷPull RequestΛτϦΨʔʹίʔυΧόϨοδΛܭଌ͠ ͯίϝϯτ͍͕ͨ͠Actionsͷݖݶ͕଍Γͳͯ͘ͰͰ͖ͳ͍ • 403 Resource not accessible by integration • ࣗ਎ʢಉҰuserʣͷଞͷϦϙδτϦ΁ͷwriteݖݶͱ͔… • Personal Access TokenΛSecretsʹొ࿥͢ΔҎ֎ͷํ๏Ͱ… GitHub Actions͕΋͍ͬͯΔ secrets.GITHUB_TOKEN ͷݖݶΛ͍͡Γ͍ͨ!!!!!!!! 44