$30 off During Our Annual Pro Sale. View Details »

GitHub上で構築するコードメトリクス計測基盤 / TECH STAND #9 GitHub

GitHub上で構築するコードメトリクス計測基盤 / TECH STAND #9 GitHub

https://standfm.connpass.com/event/256786/

GMOペパボではGitHub Enterprise Server を利用しています。
また、CI/CD基盤としてGitHub Actionsを活用してしています。
本発表ではGitHub上に構築したコードメトリクス計測基盤と、それを実現するために使用したGitHubやGitHub Actionsの機能について紹介します。

Links:
https://github.com/k1LoW/octocov
https://tech.pepabo.com/2022/01/06/four-keys-dashboard/
https://tech.pepabo.com/2022/04/25/code-metrics-dashboard/
https://recruit.pepabo.com/

Ken’ichiro Oyama

August 31, 2022
Tweet

More Decks by Ken’ichiro Oyama

Other Decks in Technology

Transcript

  1. GitHub্Ͱߏங͢Δ
    ίʔυϝτϦΫεܭଌج൫
    খࢁ݈Ұ࿠ / GMO PEPABO inc.
    2022.8.31 TECH STAND #9 GitHub
    1

    View Slide

  2. ࣗݾ঺հ
    ٕज़෦ɹٕज़ج൫νʔϜ
    2018೥ த్ೖࣾ
    খࢁ ݈Ұ࿠ Ken’ichiro Oyama
    গ࣮͠༻తͰখ͞ͳOSSΛॻ͘ͷ͕झຯɻ
    ● GitHub : k1LoW
    ● Twitter : @k1LoW
    2

    View Slide

  3. GMOϖύϘʹ͍ͭͯ
    3
    https://pepabo.com

    View Slide

  4. 4
    ίʔυϝτϦΫε

    View Slide

  5. • ιϑτ΢ΣΞͷιʔείʔυʹ·ͭΘΔ͞·͟·ͳϝτϦΫε
    • ࢀߟʮίʔυϝτϦοΫ஋ʯ https://docs.microsoft.com/ja-jp/visualstudio/code-
    quality/code-metrics-values?view=vs-2022
    • ʢ͜ͷηογϣϯʹ͓͚ΔʮίʔυϝτϦΫεʯʣͷྫͱͯ͠
    • ίʔυΧόϨοδʢςετΧόϨοδʣ
    • Code to Test Ratio
    • ςετ࣮ߦ࣌ؒ
    • อक༰қੑࢦ਺
    • αΠΫϩϚςΟοΫෳࡶ౓
    5
    ίʔυϝτϦΫε
    ίʔυϝτϦΫεͱ͸

    View Slide

  6. • ίʔυΧόϨοδ͸֤ϓϩάϥϛϯάݴޠͷςεςΟϯάϑϨʔϜϫʔΫ͸େ఍ܭ
    ଌɾग़ྗͰ͖Δ
    • ඪ४ग़ྗ
    • HTML
    • ͭ·ΓɺGitHub Actions্ͰCI؀ڥΛߏஙͯ͠ςετΛ࣮ߦ͍ͯ͠Διϑτ΢ΣΞͰ
    ͋Ε͹େମऔಘͰ͖Δ
    • ςετΛ࣮ߦ͍ͯ͠ΔςεςΟϯάϑϨʔϜϫʔΫͷΦϓγϣϯΛม͑Δ͚ͩ
    6
    ίʔυϝτϦΫε
    ྫ͑͹ίʔυΧόϨοδʢςετΧόϨοδʣ

    View Slide

  7. ✓ CIͷඪ४ग़ྗʹग़ྗ͍ͯ͠Δ
    ✓ ΧόϨοδʹᮢ஋ΛܾΊͯᮢ஋ΛԼճͬͨ৔߹͸CIΛFAIL͍ͤͯ͞Δ
    ✓ ΧόϨοδܭଌ݁ՌΛPull Requestʹίϝϯτ͍ͯ͠Δ
    ✓ લճͷܭଌ݁Ռͱൺֱͨ݁͠Ռ΋ද͍ࣔͯ͠Δ
    ✓ ܭଌ݁ՌΛ஝ੵ͍ͯ͠Δ
    7
    ίʔυϝτϦΫε
    ܭଌͨ͠ΧόϨοδΛͲ͏͍ͯ͠Δʁ

    View Slide

  8. 8
    ͜ΕΒΛGitHub্Ͱߏங͢Δ

    View Slide

  9. octocov
    9

    View Slide

  10. • https://github.com/k1LoW/octocov
    • ίʔυϝτϦΫεπʔϧ…ίʔυΧόϨοδɺCode to Test Ratioɺςετ࣮ߦ࣌ؒ
    • ଟ਺ͷϓϩάϥϛϯάݴޠͱɺओཁͳΧόϨοδϨϙʔτϑΥʔϚοτʹରԠ
    • GitHub Actionsͷactionͱͯ͠ίʔυϝτϦΫεΛूܭɺP/RίϝϯτɺϨϙʔτ
    • CLIπʔϧͱͯ͠ίʔυΧόϨοδΛूܭɻදࣔ
    • ΧόϨοδόοδੜ੒
    octocov
    10
    10
    octocov

    View Slide

  11. octocov
    • ༷ʑͳछྨͷΧόϨοδϑΥʔϚοτʹରԠ
    • Go coverage … Go
    • LCOV … ੲ͔Β͋ΔϑΥʔϚοτ
    • SimpleCov … Ruby
    • Clover … PHPUnit etc
    • Cobertura … Coverage.py etc
    • JaCoCo … Java Kotlin etc
    αϙʔτ͍ͯ͠ΔΧόϨοδϑΥʔϚοτ
    11

    View Slide

  12. octocov
    • ίʔυϝτϦΫεͷܭଌ݁ՌΛࢦఆͷσʔλετΞʹه࿥͢Δ͜ͱ͕Ͱ͖Δ
    • octocov͸ҟͳΔΧόϨοδϑΥʔϚοτ΋ڞ௨ͷߏ଄Խσʔλʹม׵͢Δ
    • ༷ʑͳछྨͷσʔλετΞʹରԠ
    • ϩʔΧϧϑΝΠϧγεςϜ
    • GitHubϦϙδτϦ
    • GitHub Artifact
    • Amazon S3
    • Google Cloud Storage(GCS)
    • BigQuery
    ίʔυϝτϦΫεσʔλͷه࿥
    12

    View Slide

  13. 13
    ίʔυϝτϦΫεͷܭଌ

    View Slide

  14. ίʔυϝτϦΫεͷܭଌ
    • ͨͬͨ3εςοϓ
    1. ςεςΟϯάϑϨʔϜϫʔΫͰΧόϨοδϨϙʔτΛग़ྗ͢ΔΑ͏ʹWorkflowΛमਖ਼
    • ֤ςεςΟϯάϑϨʔϜϫʔΫͷΦϓγϣϯΛมߋ͢Δ͚ͩ
    ɹɹgo test ./... -coverprofile=coverage.out -covermode=count
    2. .octocov.yml ΛϓϩδΣΫτϧʔτʹઃஔ
    octocov init
    3. k1LoW/octocov-action ΛΧόϨοδϨϙʔτग़ྗޙͷStepʹ௥Ճ
    octocovͷઃఆʢ୯ҰϦϙδτϦͰͷίʔυϝτϦΫεͷܭଌʣ
    14

    View Slide

  15. ίʔυϝτϦΫεͷܭଌ
    15
    .octocov.yml

    View Slide

  16. ίʔυϝτϦΫεͷܭଌ
    16
    .github/workflows/ci.yml

    View Slide

  17. ίʔυϝτϦΫεͷܭଌ
    17

    View Slide

  18. ίʔυϝτϦΫεͷܭଌ
    ✓ CIͷඪ४ग़ྗʹग़ྗ͍ͯ͠Δ
    ✓ ΧόϨοδʹᮢ஋ΛܾΊͯᮢ஋ΛԼճͬͨ৔߹͸CIΛFAIL͍ͤͯ͞Δ
    • શͯͷϝτϦΫεͰᮢ஋ΛઃఆՄೳ
    ✓ ΧόϨοδܭଌ݁ՌΛPull Requestʹίϝϯτ͍ͯ͠Δ
    ✓ લճͷܭଌ݁Ռͱൺֱͨ݁͠Ռ΋ද͍ࣔͯ͠Δ
    • ϦϙδτϦͷArtifactʹܭଌ݁ՌΛอଘ͢Δ͜ͱͰલճܭଌ݁ՌΛऔಘՄೳ
    ✓ ܭଌ݁ՌΛ஝ੵ͍ͯ͠Δ
    • Artifactʹܭଌ݁ՌΛ஝ੵͰ͖Δʢͨͩɺ࠶ར༻͸͠ʹ͍͘ʣ
    GitHubͷ୯ҰϦϙδτϦ಺ʹดͨ͡ܗͰίʔυϝτϦΫεͷܭଌ͕Մೳ
    18

    View Slide

  19. ίʔυϝτϦΫεͷܭଌ
    • monorepoʹରԠ
    • ͦΕͧΕͷίϯϙʔωϯτͰίʔυϝτϦΫεܭଌ͕Մೳ
    • ΧόϨοδϨϙʔτͷϚʔδ͕Մೳ
    • ରԠ͍ͯ͠ΔશͯͷΧόϨοδϑΥʔϚοτͷܭଌ݁ՌΛڞ௨ͷߏ଄Խσʔλʹม׵ՄೳͳͨΊ
    • ฒྻ࣮ߦͨ͠ςετΧόϨοδͷϚʔδ΋Մೳ
    • ҟͳΔϓϩάϥϛϯάݴޠͷςετΧόϨοδͷϚʔδ΋Մೳ
    • ಠཱͨ͠CLIπʔϧͱͯ͠ར༻Մೳ
    • ରԠ͍ͯ͠ΔΧόϨοδϑΥʔϚοτͰ͋Ε͹खݩͰίʔυΧόϨοδΛ֬ೝͰ͖Δ
    • ιʔείʔυͷΧόʔ͍ͯ͠Δߦͷ֬ೝ΋Մೳ
    • https://k1low.hatenablog.com/entry/2022/05/10/083000
    ଞʹ΋octocovͰ͖Δ͜ͱ
    19

    View Slide

  20. 20
    ίʔυϝτϦΫεͷ஝ੵ

    View Slide

  21. ίʔυϝτϦΫεͷ஝ੵ
    • ίʔυϝτϦΫε͸ͦͷίʔυͷϦϙδτϦʹϓϩμΫτͷϝτϦΫεͱͳΓ͏Δ
    • Four Keys΋ϓϩμΫτͷϝτϦΫε
    • Four KeysΛGitHub͔Βऔಘ͢ΔऔΓ૊Έʹ͍ͭͯ͸ https://tech.pepabo.com/
    2022/01/06/four-keys-dashboard/ Λੋඇ
    • ୯ҰϦϙδτϦͰͷܭଌͰ΋ྑ͍͕ɺϓϩδΣΫτ୯Ґɺ૊৫୯ҐͰΈΔͱݟ͑ͯ͘Δ
    ΋ͷ͕͋Δ
    • ࣌ܥྻʹϓϩοτ͢Δ͜ͱͰݟ͑ͯ͘Δ΋ͷ͕͋Δ
    ૊৫શମͰͷίʔυϝτϦΫεͷ஝ੵ
    21

    View Slide

  22. ίʔυϝτϦΫεͷ஝ੵ
    • ίʔυϝτϦΫεͷܭଌ݁ՌΛࢦఆͷσʔλετΞʹه࿥͢Δ͜ͱ͕Ͱ͖Δ
    • ରԠ͍ͯ͠ΔΧόϨοδϑΥʔϚοτͷܭଌ݁Ռ͸ڞ௨ͷߏ଄Խσʔλʹม׵
    ͢Δ
    • ༷ʑͳछྨͷσʔλετΞʹରԠ
    • ϩʔΧϧϑΝΠϧγεςϜ
    • GitHubϦϙδτϦ
    • GitHub Artifact
    • Amazon S3
    • Google Cloud Storage(GCS)
    • BigQuery
    ίʔυϝτϦΫεσʔλͷه࿥ʢ࠶ܝʣ
    22

    View Slide

  23. ίʔυϝτϦΫεͷ஝ੵ
    • octocov͸ .octocov.yml ͷઃఆΛมߋ͢Δ͜ͱͰίʔυϝτϦΫεΛܭଌ͢ΔϞʔυ͔Βɺίʔ
    υϝτϦΫεσʔλΛऩू͢ΔϞʔυʢCentral modeʣʹಈ࡞Λ੾Γସ͑Δ͜ͱ͕Ͱ͖Δɻ
    • GitHub ActionsͰಈ͔͢૝ఆͰ࡞ΒΕ͍ͯΔ
    • Central modeͰͰ͖Δ͜ͱ
    • σʔλετΞ͔Β֤ϦϙδτϦͷ࠷৽ͷίʔυϝτϦΫεσʔλΛऔಘͯ͠ϩʔΧϧʹอ
    ଘ͢Δ
    • ඞཁͰ͋Ε͹ git push ΋͢Δ
    • ίʔυϝτϦΫεσʔλ͔ΒόοδΛੜ੒͢Δ
    • ࠷৽ͷίʔυϝτϦΫεͷ஋΍όοδURLΛऔಘͰ͖ΔMarkdownϖʔδʢΠϯσοΫ
    εʣΛ࡞੒͢Δ
    Central mode
    23

    View Slide

  24. ίʔυϝτϦΫεͷ஝ੵ
    24

    View Slide

  25. ίʔυϝτϦΫεͷ஝ੵ
    • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹCentral RepoͷGitHubϦϙ
    δτϦ΋ࢦఆ
    • Central RepoͰ͸ϦϙδτϦ͔Βͷίϛοτͷon.pushΠϕϯτΛτϦΨʔʹόοδ
    ΍ΠϯσοΫεΛੜ੒
    ίʔυϝτϦΫεܭଌج൫ on GitHub
    25

    View Slide

  26. ίʔυϝτϦΫεͷ஝ੵ
    • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹS3όέοτΛࢦఆ
    • Central RepoͰ͸on.scheduleΛτϦΨʔʹS3όέοτͷϝτϦΫεσʔλ͔Βόο
    δ΍ΠϯσοΫεΛੜ੒
    ίʔυϝτϦΫεܭଌج൫ using Amazon S3 bucket
    26

    View Slide

  27. ίʔυϝτϦΫεͷ஝ੵ
    • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹGCSόέοτΛࢦఆ
    • Central RepoͰ͸on.scheduleΛτϦΨʔʹGCSόέοτͷϝτϦΫεσʔλ͔Β
    όοδ΍ΠϯσοΫεΛੜ੒
    ίʔυϝτϦΫεܭଌج൫ using Google Cloud Storage
    27

    View Slide

  28. ίʔυϝτϦΫεͷ஝ੵ
    • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹBigQueryςʔϒϧΛࢦఆ
    • ςʔϒϧ͸͋Β͔͡ΊࢦఆͷεΩʔϚͰ࡞੒͓ͯ͘͠
    octocov migrate-bq-table
    • Central RepoͰ͸on.scheduleΛτϦΨʔʹBigQueryͷςʔϒϧͷϝτϦΫεσʔλ
    ͔Βόοδ΍ΠϯσοΫεΛੜ੒
    ίʔυϝτϦΫεܭଌج൫ using BigQuery
    28

    View Slide

  29. ίʔυϝτϦΫεͷ஝ੵ
    • BigQueryʹ஝ੵ֤ͨ͠Ϧϙδτ
    ϦͷίʔυϝτϦΫεͷσʔλΛ
    ͔ͭͬͯϝτϦΫεμογϡϘʔ
    υΛ࡞੒͠ɺՄࢹԽ
    • ৄ͘͠͸ https://
    tech.pepabo.com/2022/04/25/
    code-metrics-dashboard/ Λੋඇ
    ίʔυϝτϦΫεܭଌج൫ at ϖύϘ
    29

    View Slide

  30. ίʔυϝτϦΫεͷ஝ੵ
    30

    View Slide

  31. ίʔυϝτϦΫεܭଌج൫Λ
    ࣮ݱ͢ΔͨΊʹ࢖༻ͨ͠
    GitHub΍GitHub Actionsͷػೳ
    31

    View Slide

  32. GitHub API
    32

    View Slide

  33. GitHub API
    • ਎΋֖΋ͳ͍Ͱ͕͢ɺGitHub্ʹίʔυϝτϦΫεܭଌج൫Λߏங͢Δͱ͍͏
    ʮGitHubΛ֦ு͢Δʯʹ͋ͨͬͯඞਢͰ͔ͭศརͳػೳ͸API
    • REST APIɺGraphQL APIͱ΋ʹ׆༻
    • ͔͜͜ΒʮGitHub্ʹίʔυϝτϦΫεܭଌج൫Λߏங͢Δʹ͋ͨͬͯศརͳAPI
    ΤϯυϙΠϯτʯΛ঺հ
    GitHub API
    33

    View Slide

  34. 34
    Deep Dive

    View Slide

  35. GitHub API
    • List repository languages
    • octocov init Ͱݴޠ༻ͷ .octocov.yml Λग़ྗ͢ΔͨΊͷϦϙδτϦͷओͱͳΔϓϩά
    ϥϛϯάݴޠಛఆͷͨΊػೳͷϑΥʔϧόοΫઌͱͯ͠࢖༻
    • ݴޠಛఆػೳࣗମ͸·ͩGoͷಛఆ͔͠࡞͍ͬͯͳ͍ͷͰɺ࣮࣭͜ͷAPI͕ୟ͔ΕΔ
    GET /repos/{owner}/{repo}/languages
    35

    View Slide

  36. GitHub API
    • List jobs for a workflow run
    • ςετ࣮ߦ࣌ؒͷऔಘͷͨΊʹ࢖༻
    • ΧόϨοδϨϙʔτϑΝΠϧͷλΠϜελϯϓ͕ൣғ಺ʹؚ·ΕΔStepͷܦա࣌ؒ=ςετ
    ࣮ߦ࣌ؒͱ͍ͯ͠Δ
    • StepΛಛఆ͢ΔͨΊʹɺ·ͣ͸ݱࡏͷJob ID͕ඞཁ
    • ͔͠͠APIͰ࢖͏Job ID͸؀ڥม਺GTIHUB_JOBʹؚ·Ε͍ͯΔจࣈྻͰ͸ͳ͍ɻ਺
    ࣈɻ
    • Job IDΛಛఆ͢ΔͨΊʹؤுͬͯୟ͍͍ͯΔAPI͕ίϨ
    • ؀ڥม਺GITHUB_JOB_ID͕ཉ͍͠!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    • https://github.com/actions/runner/issues/324
    GET /repos/{owner}/{repo}/actions/runs/{run_id}/jobs
    36

    View Slide

  37. GitHub API
    • Minimizes a comment on an Issue, Commit, Pull Request, or Gist.
    • ΧόϨοδϨϙʔτͷPull Request΁ͷίϝϯτ࣌ʹҎલͷίϝϯτΛӅͨ͢Ίʹ࢖

    • ࣮͸REST API (v3)ʹ͸ͳ͍ػೳ
    • GraphQL APIʹ͋Δ
    • https://docs.github.com/en/graphql/reference/mutations#minimizecomment
    minimizeComment
    37

    View Slide

  38. GitHub API
    • Download an artifact
    • σʔλετΞͱͯ͠ͷར༻
    • ͔͠͠Upload an artifactͷAPI͸ެࣜϦϑΝϨϯεʹ͸هࡌ͕ͳ͍
    • ͳͥʁ → Actions؀ڥҎ֎͔Β͸ୟ͚ͳ͍API͔ͩΒ
    1. Actions؀ڥͰ͔͠ಘΒΕͳ͍؀ڥม਺ACTIONS_RUNTIME_URL΍GITHUB_RUN_ID
    Λ࢖ͬͯߏஙͨ͠URLʹArtifactͷΞοϓϩʔυ༻Ұ࣌URLΛϦΫΤετͯ͠ɺ
    2. Actions؀ڥͰ͔͠ಘΒΕͳ͍؀ڥม਺ACTIONS_RUNTIME_TOKENΛೝূΩʔͱ͠
    ͯɺҰ࣌URLʹରͯ͠Ξοϓϩʔυ
    • https://github.com/actions/toolkit ͷίʔυΛݩʹGoͰ࣮૷
    • https://github.com/k1LoW/go-github-actions
    GET /repos/{owner}/{repo}/actions/artifacts/{artifact_id}/{archive_format}
    38

    View Slide

  39. ͓ΘΓʹ
    39

    View Slide

  40. • GitHub্Ͱߏங͢ΔίʔυϝτϦΫεܭଌج൫ͷίΞͱͳΔπʔϧͱͯ͠octocovΛ
    ঺հ
    • ૊৫ͱͯ͠ܭଌͨ͠ίʔυϝτϦΫεΛ׆༻͍ͯ͘͠࢓૊Έͱͯ͠octocovͷ
    Central mode΍ɺԠ༻ࣄྫͱͯ͠BigQueryʹऩूͨ͠ίʔυϝτϦΫεΛՄࢹԽ͠
    ͨϖύϘͷࣄྫΛ঺հ
    • GitHub API͸ศར
    40
    ͓ΘΓʹ
    ͓ΘΓʹ

    View Slide

  41. Thank You!
    Thank You!
    41

    View Slide

  42. 42
    GitHub΁Feature Request!!!!!!!!!

    View Slide

  43. GitHub API
    • ؀ڥม਺GITHUB_JOB_ID͕ཉ͍͠!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    • https://github.com/actions/runner/issues/324
    • ೖΕͯ͸͍͚ͳ͍ཧ༝͕ͳͦ͞͏ͳͷͰੋඇݕ౼ͯ͠ཉ͍͠
    GITHUB_JOB_ID͕ཉ͍͠!!!!!!!!!!!!!!
    43

    View Slide

  44. GitHub API
    • ྫ͑͹forkઌϦϙδτϦ͔ΒͷPull RequestΛτϦΨʔʹίʔυΧόϨοδΛܭଌ͠
    ͯίϝϯτ͍͕ͨ͠Actionsͷݖݶ͕଍Γͳͯ͘ͰͰ͖ͳ͍
    • 403 Resource not accessible by integration
    • ࣗ਎ʢಉҰuserʣͷଞͷϦϙδτϦ΁ͷwriteݖݶͱ͔…
    • Personal Access TokenΛSecretsʹొ࿥͢ΔҎ֎ͷํ๏Ͱ…
    GitHub Actions͕΋͍ͬͯΔ secrets.GITHUB_TOKEN ͷݖݶΛ͍͡Γ͍ͨ!!!!!!!!
    44

    View Slide