GitHub上で構築するコードメトリクス計測基盤 / TECH STAND #9 GitHub

https://standfm.connpass.com/event/256786/

GMOペパボではGitHub Enterprise Server を利用しています。
また、CI/CD基盤としてGitHub Actionsを活用してしています。
本発表ではGitHub上に構築したコードメトリクス計測基盤と、それを実現するために使用したGitHubやGitHub Actionsの機能について紹介します。

Links:
https://github.com/k1LoW/octocov
https://tech.pepabo.com/2022/01/06/four-keys-dashboard/
https://tech.pepabo.com/2022/04/25/code-metrics-dashboard/
https://recruit.pepabo.com/

Transcript

1. GitHub্Ͱߏங͢Δ
   ίʔυϝτϦΫεܭଌج൫
   খࢁ݈Ұ࿠ / GMO PEPABO inc.
   2022.8.31 TECH STAND #9 GitHub
   1
   

   View Slide

2. ࣗݾ঺հ
   ٕज़෦ɹٕज़ج൫νʔϜ
   2018೥ த్ೖࣾ
   খࢁ ݈Ұ࿠ Ken’ichiro Oyama
   গ࣮͠༻తͰখ͞ͳOSSΛॻ͘ͷ͕झຯɻ
   ● GitHub : k1LoW
   ● Twitter : @k1LoW
   2
   

   View Slide

3. GMOϖύϘʹ͍ͭͯ
   3
   https://pepabo.com
   

   View Slide

4. 4
   ίʔυϝτϦΫε
   

   View Slide

5. • ιϑτ΢ΣΞͷιʔείʔυʹ·ͭΘΔ͞·͟·ͳϝτϦΫε
   • ࢀߟʮίʔυϝτϦοΫ஋ʯ https://docs.microsoft.com/ja-jp/visualstudio/code-
   quality/code-metrics-values?view=vs-2022
   • ʢ͜ͷηογϣϯʹ͓͚ΔʮίʔυϝτϦΫεʯʣͷྫͱͯ͠
   • ίʔυΧόϨοδʢςετΧόϨοδʣ
   • Code to Test Ratio
   • ςετ࣮ߦ࣌ؒ
   • อक༰қੑࢦ਺
   • αΠΫϩϚςΟοΫෳࡶ౓
   5
   ίʔυϝτϦΫε
   ίʔυϝτϦΫεͱ͸
   

   View Slide

6. • ίʔυΧόϨοδ͸֤ϓϩάϥϛϯάݴޠͷςεςΟϯάϑϨʔϜϫʔΫ͸େ఍ܭ
   ଌɾग़ྗͰ͖Δ
   • ඪ४ग़ྗ
   • HTML
   • ͭ·ΓɺGitHub Actions্ͰCI؀ڥΛߏஙͯ͠ςετΛ࣮ߦ͍ͯ͠Διϑτ΢ΣΞͰ
   ͋Ε͹େମऔಘͰ͖Δ
   • ςετΛ࣮ߦ͍ͯ͠ΔςεςΟϯάϑϨʔϜϫʔΫͷΦϓγϣϯΛม͑Δ͚ͩ
   6
   ίʔυϝτϦΫε
   ྫ͑͹ίʔυΧόϨοδʢςετΧόϨοδʣ
   

   View Slide

7. ✓ CIͷඪ४ग़ྗʹग़ྗ͍ͯ͠Δ
   ✓ ΧόϨοδʹᮢ஋ΛܾΊͯᮢ஋ΛԼճͬͨ৔߹͸CIΛFAIL͍ͤͯ͞Δ
   ✓ ΧόϨοδܭଌ݁ՌΛPull Requestʹίϝϯτ͍ͯ͠Δ
   ✓ લճͷܭଌ݁Ռͱൺֱͨ݁͠Ռ΋ද͍ࣔͯ͠Δ
   ✓ ܭଌ݁ՌΛ஝ੵ͍ͯ͠Δ
   7
   ίʔυϝτϦΫε
   ܭଌͨ͠ΧόϨοδΛͲ͏͍ͯ͠Δʁ
   

   View Slide

8. 8
   ͜ΕΒΛGitHub্Ͱߏங͢Δ
   

   View Slide

9. octocov
   9
   

   View Slide

10. • https://github.com/k1LoW/octocov
    • ίʔυϝτϦΫεπʔϧ…ίʔυΧόϨοδɺCode to Test Ratioɺςετ࣮ߦ࣌ؒ
    • ଟ਺ͷϓϩάϥϛϯάݴޠͱɺओཁͳΧόϨοδϨϙʔτϑΥʔϚοτʹରԠ
    • GitHub Actionsͷactionͱͯ͠ίʔυϝτϦΫεΛूܭɺP/RίϝϯτɺϨϙʔτ
    • CLIπʔϧͱͯ͠ίʔυΧόϨοδΛूܭɻදࣔ
    • ΧόϨοδόοδੜ੒
    octocov
    10
    10
    octocov
    

    View Slide

11. octocov
    • ༷ʑͳछྨͷΧόϨοδϑΥʔϚοτʹରԠ
    • Go coverage … Go
    • LCOV … ੲ͔Β͋ΔϑΥʔϚοτ
    • SimpleCov … Ruby
    • Clover … PHPUnit etc
    • Cobertura … Coverage.py etc
    • JaCoCo … Java Kotlin etc
    αϙʔτ͍ͯ͠ΔΧόϨοδϑΥʔϚοτ
    11
    

    View Slide

12. octocov
    • ίʔυϝτϦΫεͷܭଌ݁ՌΛࢦఆͷσʔλετΞʹه࿥͢Δ͜ͱ͕Ͱ͖Δ
    • octocov͸ҟͳΔΧόϨοδϑΥʔϚοτ΋ڞ௨ͷߏ଄Խσʔλʹม׵͢Δ
    • ༷ʑͳछྨͷσʔλετΞʹରԠ
    • ϩʔΧϧϑΝΠϧγεςϜ
    • GitHubϦϙδτϦ
    • GitHub Artifact
    • Amazon S3
    • Google Cloud Storage(GCS)
    • BigQuery
    ίʔυϝτϦΫεσʔλͷه࿥
    12
    

    View Slide

13. 13
    ίʔυϝτϦΫεͷܭଌ
    

    View Slide

14. ίʔυϝτϦΫεͷܭଌ
    • ͨͬͨ3εςοϓ
    1. ςεςΟϯάϑϨʔϜϫʔΫͰΧόϨοδϨϙʔτΛग़ྗ͢ΔΑ͏ʹWorkflowΛमਖ਼
    • ֤ςεςΟϯάϑϨʔϜϫʔΫͷΦϓγϣϯΛมߋ͢Δ͚ͩ
    ɹɹgo test ./... -coverprofile=coverage.out -covermode=count
    2. .octocov.yml ΛϓϩδΣΫτϧʔτʹઃஔ
    octocov init
    3. k1LoW/octocov-action ΛΧόϨοδϨϙʔτग़ྗޙͷStepʹ௥Ճ
    octocovͷઃఆʢ୯ҰϦϙδτϦͰͷίʔυϝτϦΫεͷܭଌʣ
    14
    

    View Slide

15. ίʔυϝτϦΫεͷܭଌ
    15
    .octocov.yml
    

    View Slide

16. ίʔυϝτϦΫεͷܭଌ
    16
    .github/workflows/ci.yml
    

    View Slide

17. ίʔυϝτϦΫεͷܭଌ
    17
    

    View Slide

18. ίʔυϝτϦΫεͷܭଌ
    ✓ CIͷඪ४ग़ྗʹग़ྗ͍ͯ͠Δ
    ✓ ΧόϨοδʹᮢ஋ΛܾΊͯᮢ஋ΛԼճͬͨ৔߹͸CIΛFAIL͍ͤͯ͞Δ
    • શͯͷϝτϦΫεͰᮢ஋ΛઃఆՄೳ
    ✓ ΧόϨοδܭଌ݁ՌΛPull Requestʹίϝϯτ͍ͯ͠Δ
    ✓ લճͷܭଌ݁Ռͱൺֱͨ݁͠Ռ΋ද͍ࣔͯ͠Δ
    • ϦϙδτϦͷArtifactʹܭଌ݁ՌΛอଘ͢Δ͜ͱͰલճܭଌ݁ՌΛऔಘՄೳ
    ✓ ܭଌ݁ՌΛ஝ੵ͍ͯ͠Δ
    • Artifactʹܭଌ݁ՌΛ஝ੵͰ͖Δʢͨͩɺ࠶ར༻͸͠ʹ͍͘ʣ
    GitHubͷ୯ҰϦϙδτϦ಺ʹดͨ͡ܗͰίʔυϝτϦΫεͷܭଌ͕Մೳ
    18
    

    View Slide

19. ίʔυϝτϦΫεͷܭଌ
    • monorepoʹରԠ
    • ͦΕͧΕͷίϯϙʔωϯτͰίʔυϝτϦΫεܭଌ͕Մೳ
    • ΧόϨοδϨϙʔτͷϚʔδ͕Մೳ
    • ରԠ͍ͯ͠ΔશͯͷΧόϨοδϑΥʔϚοτͷܭଌ݁ՌΛڞ௨ͷߏ଄Խσʔλʹม׵ՄೳͳͨΊ
    • ฒྻ࣮ߦͨ͠ςετΧόϨοδͷϚʔδ΋Մೳ
    • ҟͳΔϓϩάϥϛϯάݴޠͷςετΧόϨοδͷϚʔδ΋Մೳ
    • ಠཱͨ͠CLIπʔϧͱͯ͠ར༻Մೳ
    • ରԠ͍ͯ͠ΔΧόϨοδϑΥʔϚοτͰ͋Ε͹खݩͰίʔυΧόϨοδΛ֬ೝͰ͖Δ
    • ιʔείʔυͷΧόʔ͍ͯ͠Δߦͷ֬ೝ΋Մೳ
    • https://k1low.hatenablog.com/entry/2022/05/10/083000
    ଞʹ΋octocovͰ͖Δ͜ͱ
    19
    

    View Slide

20. 20
    ίʔυϝτϦΫεͷ஝ੵ
    

    View Slide

21. ίʔυϝτϦΫεͷ஝ੵ
    • ίʔυϝτϦΫε͸ͦͷίʔυͷϦϙδτϦʹϓϩμΫτͷϝτϦΫεͱͳΓ͏Δ
    • Four Keys΋ϓϩμΫτͷϝτϦΫε
    • Four KeysΛGitHub͔Βऔಘ͢ΔऔΓ૊Έʹ͍ͭͯ͸ https://tech.pepabo.com/
    2022/01/06/four-keys-dashboard/ Λੋඇ
    • ୯ҰϦϙδτϦͰͷܭଌͰ΋ྑ͍͕ɺϓϩδΣΫτ୯Ґɺ૊৫୯ҐͰΈΔͱݟ͑ͯ͘Δ
    ΋ͷ͕͋Δ
    • ࣌ܥྻʹϓϩοτ͢Δ͜ͱͰݟ͑ͯ͘Δ΋ͷ͕͋Δ
    ૊৫શମͰͷίʔυϝτϦΫεͷ஝ੵ
    21
    

    View Slide

22. ίʔυϝτϦΫεͷ஝ੵ
    • ίʔυϝτϦΫεͷܭଌ݁ՌΛࢦఆͷσʔλετΞʹه࿥͢Δ͜ͱ͕Ͱ͖Δ
    • ରԠ͍ͯ͠ΔΧόϨοδϑΥʔϚοτͷܭଌ݁Ռ͸ڞ௨ͷߏ଄Խσʔλʹม׵
    ͢Δ
    • ༷ʑͳछྨͷσʔλετΞʹରԠ
    • ϩʔΧϧϑΝΠϧγεςϜ
    • GitHubϦϙδτϦ
    • GitHub Artifact
    • Amazon S3
    • Google Cloud Storage(GCS)
    • BigQuery
    ίʔυϝτϦΫεσʔλͷه࿥ʢ࠶ܝʣ
    22
    

    View Slide

23. ίʔυϝτϦΫεͷ஝ੵ
    • octocov͸ .octocov.yml ͷઃఆΛมߋ͢Δ͜ͱͰίʔυϝτϦΫεΛܭଌ͢ΔϞʔυ͔Βɺίʔ
    υϝτϦΫεσʔλΛऩू͢ΔϞʔυʢCentral modeʣʹಈ࡞Λ੾Γସ͑Δ͜ͱ͕Ͱ͖Δɻ
    • GitHub ActionsͰಈ͔͢૝ఆͰ࡞ΒΕ͍ͯΔ
    • Central modeͰͰ͖Δ͜ͱ
    • σʔλετΞ͔Β֤ϦϙδτϦͷ࠷৽ͷίʔυϝτϦΫεσʔλΛऔಘͯ͠ϩʔΧϧʹอ
    ଘ͢Δ
    • ඞཁͰ͋Ε͹ git push ΋͢Δ
    • ίʔυϝτϦΫεσʔλ͔ΒόοδΛੜ੒͢Δ
    • ࠷৽ͷίʔυϝτϦΫεͷ஋΍όοδURLΛऔಘͰ͖ΔMarkdownϖʔδʢΠϯσοΫ
    εʣΛ࡞੒͢Δ
    Central mode
    23
    

    View Slide

24. ίʔυϝτϦΫεͷ஝ੵ
    24
    

    View Slide

25. ίʔυϝτϦΫεͷ஝ੵ
    • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹCentral RepoͷGitHubϦϙ
    δτϦ΋ࢦఆ
    • Central RepoͰ͸ϦϙδτϦ͔Βͷίϛοτͷon.pushΠϕϯτΛτϦΨʔʹόοδ
    ΍ΠϯσοΫεΛੜ੒
    ίʔυϝτϦΫεܭଌج൫ on GitHub
    25
    

    View Slide

26. ίʔυϝτϦΫεͷ஝ੵ
    • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹS3όέοτΛࢦఆ
    • Central RepoͰ͸on.scheduleΛτϦΨʔʹS3όέοτͷϝτϦΫεσʔλ͔Βόο
    δ΍ΠϯσοΫεΛੜ੒
    ίʔυϝτϦΫεܭଌج൫ using Amazon S3 bucket
    26
    

    View Slide

27. ίʔυϝτϦΫεͷ஝ੵ
    • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹGCSόέοτΛࢦఆ
    • Central RepoͰ͸on.scheduleΛτϦΨʔʹGCSόέοτͷϝτϦΫεσʔλ͔Β
    όοδ΍ΠϯσοΫεΛੜ੒
    ίʔυϝτϦΫεܭଌج൫ using Google Cloud Storage
    27
    

    View Slide

28. ίʔυϝτϦΫεͷ஝ੵ
    • ֤ϦϙδτϦ͸σʔλετΞͱͯࣗ͠਎ͷArtifactͷଞʹBigQueryςʔϒϧΛࢦఆ
    • ςʔϒϧ͸͋Β͔͡ΊࢦఆͷεΩʔϚͰ࡞੒͓ͯ͘͠
    octocov migrate-bq-table
    • Central RepoͰ͸on.scheduleΛτϦΨʔʹBigQueryͷςʔϒϧͷϝτϦΫεσʔλ
    ͔Βόοδ΍ΠϯσοΫεΛੜ੒
    ίʔυϝτϦΫεܭଌج൫ using BigQuery
    28
    

    View Slide

29. ίʔυϝτϦΫεͷ஝ੵ
    • BigQueryʹ஝ੵ֤ͨ͠Ϧϙδτ
    ϦͷίʔυϝτϦΫεͷσʔλΛ
    ͔ͭͬͯϝτϦΫεμογϡϘʔ
    υΛ࡞੒͠ɺՄࢹԽ
    • ৄ͘͠͸ https://
    tech.pepabo.com/2022/04/25/
    code-metrics-dashboard/ Λੋඇ
    ίʔυϝτϦΫεܭଌج൫ at ϖύϘ
    29
    

    View Slide

30. ίʔυϝτϦΫεͷ஝ੵ
    30
    

    View Slide

31. ίʔυϝτϦΫεܭଌج൫Λ
    ࣮ݱ͢ΔͨΊʹ࢖༻ͨ͠
    GitHub΍GitHub Actionsͷػೳ
    31
    

    View Slide

32. GitHub API
    32
    

    View Slide

33. GitHub API
    • ਎΋֖΋ͳ͍Ͱ͕͢ɺGitHub্ʹίʔυϝτϦΫεܭଌج൫Λߏங͢Δͱ͍͏
    ʮGitHubΛ֦ு͢Δʯʹ͋ͨͬͯඞਢͰ͔ͭศརͳػೳ͸API
    • REST APIɺGraphQL APIͱ΋ʹ׆༻
    • ͔͜͜ΒʮGitHub্ʹίʔυϝτϦΫεܭଌج൫Λߏங͢Δʹ͋ͨͬͯศརͳAPI
    ΤϯυϙΠϯτʯΛ঺հ
    GitHub API
    33
    

    View Slide

34. 34
    Deep Dive
    

    View Slide

35. GitHub API
    • List repository languages
    • octocov init Ͱݴޠ༻ͷ .octocov.yml Λग़ྗ͢ΔͨΊͷϦϙδτϦͷओͱͳΔϓϩά
    ϥϛϯάݴޠಛఆͷͨΊػೳͷϑΥʔϧόοΫઌͱͯ͠࢖༻
    • ݴޠಛఆػೳࣗମ͸·ͩGoͷಛఆ͔͠࡞͍ͬͯͳ͍ͷͰɺ࣮࣭͜ͷAPI͕ୟ͔ΕΔ
    GET /repos/{owner}/{repo}/languages
    35
    

    View Slide

36. GitHub API
    • List jobs for a workflow run
    • ςετ࣮ߦ࣌ؒͷऔಘͷͨΊʹ࢖༻
    • ΧόϨοδϨϙʔτϑΝΠϧͷλΠϜελϯϓ͕ൣғ಺ʹؚ·ΕΔStepͷܦա࣌ؒ=ςετ
    ࣮ߦ࣌ؒͱ͍ͯ͠Δ
    • StepΛಛఆ͢ΔͨΊʹɺ·ͣ͸ݱࡏͷJob ID͕ඞཁ
    • ͔͠͠APIͰ࢖͏Job ID͸؀ڥม਺GTIHUB_JOBʹؚ·Ε͍ͯΔจࣈྻͰ͸ͳ͍ɻ਺
    ࣈɻ
    • Job IDΛಛఆ͢ΔͨΊʹؤுͬͯୟ͍͍ͯΔAPI͕ίϨ
    • ؀ڥม਺GITHUB_JOB_ID͕ཉ͍͠!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    • https://github.com/actions/runner/issues/324
    GET /repos/{owner}/{repo}/actions/runs/{run_id}/jobs
    36
    

    View Slide

37. GitHub API
    • Minimizes a comment on an Issue, Commit, Pull Request, or Gist.
    • ΧόϨοδϨϙʔτͷPull Request΁ͷίϝϯτ࣌ʹҎલͷίϝϯτΛӅͨ͢Ίʹ࢖
    ༻
    • ࣮͸REST API (v3)ʹ͸ͳ͍ػೳ
    • GraphQL APIʹ͋Δ
    • https://docs.github.com/en/graphql/reference/mutations#minimizecomment
    minimizeComment
    37
    

    View Slide

38. GitHub API
    • Download an artifact
    • σʔλετΞͱͯ͠ͷར༻
    • ͔͠͠Upload an artifactͷAPI͸ެࣜϦϑΝϨϯεʹ͸هࡌ͕ͳ͍
    • ͳͥʁ → Actions؀ڥҎ֎͔Β͸ୟ͚ͳ͍API͔ͩΒ
    1. Actions؀ڥͰ͔͠ಘΒΕͳ͍؀ڥม਺ACTIONS_RUNTIME_URL΍GITHUB_RUN_ID
    Λ࢖ͬͯߏஙͨ͠URLʹArtifactͷΞοϓϩʔυ༻Ұ࣌URLΛϦΫΤετͯ͠ɺ
    2. Actions؀ڥͰ͔͠ಘΒΕͳ͍؀ڥม਺ACTIONS_RUNTIME_TOKENΛೝূΩʔͱ͠
    ͯɺҰ࣌URLʹରͯ͠Ξοϓϩʔυ
    • https://github.com/actions/toolkit ͷίʔυΛݩʹGoͰ࣮૷
    • https://github.com/k1LoW/go-github-actions
    GET /repos/{owner}/{repo}/actions/artifacts/{artifact_id}/{archive_format}
    38
    

    View Slide

39. ͓ΘΓʹ
    39
    

    View Slide

40. • GitHub্Ͱߏங͢ΔίʔυϝτϦΫεܭଌج൫ͷίΞͱͳΔπʔϧͱͯ͠octocovΛ
    ঺հ
    • ૊৫ͱͯ͠ܭଌͨ͠ίʔυϝτϦΫεΛ׆༻͍ͯ͘͠࢓૊Έͱͯ͠octocovͷ
    Central mode΍ɺԠ༻ࣄྫͱͯ͠BigQueryʹऩूͨ͠ίʔυϝτϦΫεΛՄࢹԽ͠
    ͨϖύϘͷࣄྫΛ঺հ
    • GitHub API͸ศར
    40
    ͓ΘΓʹ
    ͓ΘΓʹ
    

    View Slide

41. Thank You!
    Thank You!
    41
    

    View Slide

42. 42
    GitHub΁Feature Request!!!!!!!!!
    

    View Slide

43. GitHub API
    • ؀ڥม਺GITHUB_JOB_ID͕ཉ͍͠!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    • https://github.com/actions/runner/issues/324
    • ೖΕͯ͸͍͚ͳ͍ཧ༝͕ͳͦ͞͏ͳͷͰੋඇݕ౼ͯ͠ཉ͍͠
    GITHUB_JOB_ID͕ཉ͍͠!!!!!!!!!!!!!!
    43
    

    View Slide

44. GitHub API
    • ྫ͑͹forkઌϦϙδτϦ͔ΒͷPull RequestΛτϦΨʔʹίʔυΧόϨοδΛܭଌ͠
    ͯίϝϯτ͍͕ͨ͠Actionsͷݖݶ͕଍Γͳͯ͘ͰͰ͖ͳ͍
    • 403 Resource not accessible by integration
    • ࣗ਎ʢಉҰuserʣͷଞͷϦϙδτϦ΁ͷwriteݖݶͱ͔…
    • Personal Access TokenΛSecretsʹొ࿥͢ΔҎ֎ͷํ๏Ͱ…
    GitHub Actions͕΋͍ͬͯΔ secrets.GITHUB_TOKEN ͷݖݶΛ͍͡Γ͍ͨ!!!!!!!!
    44
    

    View Slide