CI/CDのボトルネックを把握できていますか？BigQueryでビルド情報ダッシュボードを構築した話

by Kenta Kase

Slide 1

Slide 1 text

CI/CDͷϘτϧωοΫΛ೺ѲͰ͖͍ͯ·͔͢ʁ BigQueryͰϏϧυ৘ใμογϡϘʔυΛ   ߏஙͨ͠࿩ CI/CD Conference 2021ʢ2021/09/03ʣ Ճ੉݈ଠ @Kesin11

Slide 2

Slide 2 text

ࣗݾ঺հ • ॴଐɿDeNAͷSWETάϧʔϓʢSoftWare Engineer in Testʣ • Twitterɿ@Kesin11ɺGitHubɿKesin11 • CI/CDαʔϏεͷϚχΞͳͷͰ༗໊ͳαʔϏε͸େମνϣοτϫΧϧ • ͜͜2೥΄ͲओʹϞόΠϧήʔϜͷJenkinsͷߏஙɺӡ༻ɺύΠϓϥΠϯߏங • CEDEC2020ʮϞ バ Πϧ ゲ ʔϜ։ൃʹ͓͚ΔJenkins ʙΫϥ΢ ド ࣌୅ͷ Jenkinsߏஙͱ؅ཧςΫχοΫʙʯ

Slide 3

Slide 3 text

ൃද಺༰ • CI/CDΛ௕ظؒӡ༻͢Δͱݟ͑ͯ͘Δ໰୊ • ࣗ࡞ͷϏϧυσʔλऩूπʔϧͷ঺հ • μογϡϘʔυΛ࡞੒ͨ͜͠ͱͰՄೳʹͳͬͨ͜ͱ • ࣮຿ͰղܾͰ͖࣮ͨྫͷ঺հ

Slide 4

Slide 4 text

લఏ • ຊൃද͸ओʹiOS/AndroidΞϓϦΛCIαʔϏε্ͰϏϧυ͢Δߏ੒ • ಛʹJenkins + ࣗલͷmac + iOS/Android + Unity • k8s + ίϯςφϏϧυͷੈքͱ͸՝୊ײ͕ҟͳΔ͔΋͠Ε·ͤΜ

Slide 5

Slide 5 text

͜͜࠷ۙCI/CDք۾͕೤͍ • Github Actions, CircleCI, Bitrise, Azure Pipelines • k8sͱίϯςφք۾΋৽πʔϧͷ࿩͕ͲΜͲΜฉ͑ͯ͘͜Δ • ৽͍͠πʔϧͷ৽͍͠ػೳͰ࣮ݱͰ͖Δ͜ͱΛߟ͑Δͷ͸ͱͯ΋ָ͠ ͍

Slide 6

Slide 6 text

CI/CDΛ௕ظӡ༻͢Δͱݟ͑ͯ͘Δ໰୊ • Ϗϧυ͕͍࣌ؒͭͷؒʹ͔ͱͯ΋௕͘ͳ͍ͬͯΔ • ϥϯμϜʹͨ·ʹࣦഊ͢ΔϏϧυ • ϏϧυΩϡʔ͕٧·Γ͕ͪ • Ϗϧυɾςετͷࣦഊ͕์ஔ͞Ε͕ͪʹͳΔ

Slide 7

Slide 7 text

Azure Pipelines • Azure PipelinesͰ͸ਆμογϡϘʔυ͕σϑΥϧτͰ࢖͑Δʂ ը૾ɿmicrosoft/typescriptͷAzure Pieplines

Slide 8

Slide 8 text

ύΠϓϥΠϯͷ੒ޭ/ ࣦഊ෼ੳ ੒ޭ཰ ͍ࣦͭഊ͔ͨ͠ Ͳͷεςοϓ͕ࣦഊ͔ͨ͠ https://typescript.visualstudio.com/TypeScript/_pipeline/analytics/ stageawareoutcome?de fi nitionId=6&contextType=build

Slide 9

Slide 9 text

ςετ݁Ռ ςετͷ੒ޭ཰ ੒ޭ཰ͷਪҠ ࣦഊͨ͠ςετέʔεͷҰཡ ςετέʔε͝ͱͷฏۉ࣌ؒ https://typescript.visualstudio.com/TypeScript/_test/analytics? de fi nitionId=6&contextType=build

Slide 10

Slide 10 text

ύΠϓϥΠϯͷ࣌ؒ ࣌ؒͷਪҠ εςοϓ͝ͱʹ͔͔Δ࣌ؒ https://typescript.visualstudio.com/TypeScript/_pipeline/analytics/ duration?de fi nitionId=6&contextType=build

Slide 11

Slide 11 text

͜Εͩɾɾɾʂ

Slide 12

Slide 12 text

ଞͷCIαʔϏεͰ΋ಉ͡΋ͷ͕ཉ͍͠ɾɾɾʂ • ౰࣌͸ྨࣅαʔϏεΛݟ͚ͭΒΕͳ͔ͬͨ • ݱࡏͰ͸ଘࡏ͠·͢ʢޙड़ʣ • Github Actions΍CircleCIͷAPIΛࢼͨ͠ͱ͜ΖɺAzure Pipelinesͱಉ ౳ͷμογϡϘʔυΛ࡞Δͷʹඞཁͳσʔλ͸औಘͰ͖ͦ͏ • ͜͏ͳͬͨΒࣗ࡞͢Δ͔͠ͳ͍

Slide 13

Slide 13 text

CIAnalyzer • https://github.com/Kesin11/CIAnalyzer • ֤CIαʔϏεͷAPI͔Βσʔλऔಘɺ੔ܗɺ݁ՌͷΤΫεϙʔτ • Github Actions, CircleCI, Jenkins, BitriseʹରԠ • αʔόʔͷӡ༻ෆཁ • DBͷӡ༻ෆཁ • CIଆʹಛผͳπʔϧ΍εςοϓͷ௥Ճෆཁ • σʔλ͸֤ࣗͰ޷͖ͳ৔ॴʹΤΫεϙʔτʢBigQueryΛެࣜαϙʔτʣ

Slide 14

Slide 14 text

Ϗϧυ࣌ؒͱ੒ޭ཰

Slide 15

Slide 15 text

δϣϒͱεςοϓຖͷ࣌ؒ

Slide 16

Slide 16 text

ςετ݁ՌͱςετέʔεҰཡ

Slide 17

Slide 17 text

Ωϡʔ଴ͪ࣌ؒ

Slide 18

Slide 18 text

BigQuery + DataStudio • σʔλ͸BigQueryʹ஝ੵ • ࣗલͰDBӡ༻͕ෆཁ • σʔλྔʹΑΔύϑΥʔϚϯεͷݒ೦͕ͳ͍ • ਖ਼نԽͤͣʹJSONͷߏ଄Λͦͷ··౤ೖՄೳ • μογϡϘʔυ͸DataStudioʢDataPortalʣ • ಉ͡GoogleͷαʔϏεͳͷͰBigQueryͱ૬ੑൈ܈ • JSONͷߏ଄ͷ··͍͍ײ͡ʹάϥϑ͕࡞ΕΔ • μογϡϘʔυ༻ͷWebαʔόʔͷӡ༻͕ෆཁ • ݖݶ؅ཧ͕GDriveͱಉ͡UIͳͷͰ୭Ͱ΋ઃఆ͠΍͍͢

Slide 19

Slide 19 text

CIAnalyzerͷΞʔΩςΫνϟ • ֤αʔϏεͷAPIΛར༻ͯ͠σʔλΛऔಘ͢Δpullܕ • ύΠϓϥΠϯதͰಛผͳπʔϧΛಈ͔͢ઃఆͷ௥Ճ͸ෆཁ • ऩूର৅Λઃఆ༻ͷYAMLʹهࡌ͢Δ • Ͳ͔͜ͷ؀ڥͰఆظతʹىಈͰ͖Ε͹OK • ͓࢖͍ͷCIαʔϏεͷεέδϡʔϧ࣮ߦΛར༻͢Δͷ͕؆୯ • ࣗલͰαʔόʔͷӡ༻͕ෆཁ

Slide 20

Slide 20 text

શମతͳಛ௃ • ͱʹ͔͘௿ίετʹӡ༻Ͱ͖ͯۀ຿Ͱ࢖͏োน͕ۃྗͳ͍Α͏ʹઃܭ • ࣗલͰαʔόʔΛӡ༻͢Δඞཁ͕ͳ͍ • ύΠϓϥΠϯͷதͰಛผͳπʔϧΛಈ͔͢ඞཁ͕ͳ͍ • ֎෦αʔϏεʹσʔλΛ౉͞ͳ͍ͷͰۀ຿Ͱ΋࢖͍΍͍͢

Slide 21

Slide 21 text

μογϡϘʔυΛ࡞੒ͨ͠   ͜ͱͰՄೳʹͳͬͨ͜ͱ

Slide 22

Slide 22 text

աڈͷϏϧυه࿥Λ શͯḪΕΔ BigQueryͷྗʹΑͬͯաڈͷσʔ λશͯΛදࣔͤͯ͞΋࣌ؒ͸΄΅͔ ͔Βͳ͍ ΄ͱΜͲͷCIαʔϏεͰաڈͷه࿥ ͸࢒͍ͬͯΔ΋ͷͷɺḪͬͯूܭ͢ Δ͜ͱ͸ࠔ೉ͳͷͰେ͖ͳϝϦοτ ͱͳΔ iOS/Androidͷ1೥ؒͷφΠτϦʔϏϧυ ࣦഊͨ͠Ϗϧυɺࣦഊͨ͠εςοϓͷΈߜΓࠐΈ

Slide 23

Slide 23 text

ฏۉతͳϏϧυ࣌ؒ ͷ૿Ճ܏޲ͷਖ਼֬ͳ ೺Ѳ ҆ఆ͢Δϒϥϯνɺ੒ޭͨ͠Ϗϧυ͚ͩͰϓ ϩοτ masterϒϥϯνɺφΠτϦʔϏϧυΈ͍ͨ ͳ҆ఆ͍ͯ͠Δ৚͕݅ϕετ ମײͰ͸ͳ͘ɺ਺ϲ݄લͱൺֱͯ͠ͲΕ͚ͩ ͕࣌ؒ௕͘ͳ͍ͬͯΔ͔ਖ਼֬ʹ೺ѲͰ͖Δ ௚ۙͰϏϧυ͕࣌ؒڐ༰Ͱ͖ͳ͍଎౓Ͱ૿Ճ ܏޲Ͱ͋Ε͹ૣظʹରࡦΛݕ౼͢Δ iOS/Androidͷ1೥ؒͷϏϧυ࣌ؒͷਪҠ φΠτϦʔɿΩϟογϡONɺΫϦʔϯɿΩϟογϡOFF

Slide 24

Slide 24 text

CI/CDͷσʔλ஝ੵʹΑͬͯ ղܾͰ͖࣮ͨྫ঺հ

Slide 25

Slide 25 text

ಛఆͷεςοϓ͕ੲʹൺ΂ͯେ෯ʹ͕͔͔࣌ؒ ΔΑ͏ʹͳ͍ͬͯͨࣄྫ • iOS/AndroidΞϓϦͷϏϧυ࣌ؒվળͷͨΊɺ͕͔͔͍࣌ؒͬͯΔεςοϓΛ෼ੳ͍ͯͨ͠ͱ͜ΖGCS ʹΞοϓϩʔυ͍ͯ͠Δεςοϓʹ7෼΋͔͔͍ͬͯͨ • ੲ͸ͦΜͳʹ͕͔͔͍࣌ؒͬͯͨهԱ͕ͳ͍ͷͰ2ϲ݄·ͰσʔλΛḪͬͯൺֱͨ͠ͱ͜Ζɺ   2෼ˠ7෼ʹ৳ͼ͍ͯͨ͜ͱ͕൑໌ • 1ճͷϏϧυதʹGCS΁ͷΞοϓϩʔυ͸ෳ਺ճߦ͍ͬͯͨͷͰɺ͋Δδϣϒʹ͓͍ͯ͸શମͷϏϧυ ࣌ؒͷ໿20%ऑ΋઎Ίͯ͠·͍ͬͯͨ • gsutil rsyncͷࡍʹసૹʹඞཁͳϑΝΠϧΛಛఆ͢Δॲཧ͕࣌ؒϑΝΠϧ਺૿Ճʹ൐ͬͯ৳ͼ͍ͯͨ͜ͱ ͕ݪҼ • gsutil cp ʹஔ͖׵͑Δ͜ͱͰݩͷ7෼ˠ30ඵʹ·ͰվળͰ͖ͨ

Slide 26

Slide 26 text

CircleCIͰͷόοΫΤϯυαʔόʔͷϏϧυ͕ كʹࣦഊ͢Δࣄྫ • ͋Δͱ͖͔ΒCircleCIͷϏϧυδϣϒ͕كʹࣦഊɺϦτϥΠ͢Δͱ௚Δ͜ͱ΋͋Δͱ͍͏Ṗݱ৅͕ൃੜ • μογϡϘʔυ্ͰFailedͳδϣϒɺ͞ΒʹFailedʹͳͬͨݪҼͷεςοϓΛௐࠪͯ͠ҎԼΛݪҼΛಛఆ • docker buildதͷίϯύΠϧΤϥʔ • ڊେϦϙδτϦͷgit cloneʹࣦഊ͍ͯ͠Δ • ݪҼ͕2ͭ͋Δ͜ͱɺٯʹͦΕҎ֎ͷݪҼ͸ແ͍͜ͱΛಥ͖ࢭΊΒΕͨ͜ͱ͕େ͖͍ • લऀ͸ίʔυΛ௚ͯ͠΋Β͍ɺޙऀͷgit clone͸ݪҼΛಛఆͰ͖ͳ͔ͬͨͷͰCircleCIͷresource_class Λ1ஈ֊্͛ͯΈΔ͜ͱʹ • 1िؒ΄Ͳ༷ࢠݟΛͨ݁͠Ռ࠶ൃ͠ͳ͘ͳͬͨͷͰղܾ

Slide 27

Slide 27 text

ྨࣅαʔϏεͷ঺հ • CIAnalyzerͱࣅͨΑ͏ͳػೳΛఏڙ͍ͯ͠ΔαʔϏεͷ঺հ • ͨͩ͠ɺ͍ͣΕ΋CIAnalyzer΄Ͳৄࡉͳ෼ੳ͸Ͱ͖ͳ͍ • Azure Pipelinesʢ࠶ܝʣ • ࣗ෼ʹͱͬͯ͸ຊՈɻAzureΛ͓࢖͍ͷํ͸ͥͻ͓ࢼ͠Λ

Slide 28

Slide 28 text

CircleCIͷInsightsػೳ Ϗϧυ࣌ؒɺ੒ޭ཰͸ݟΒΕΔ தԝ஋ͱ95%λΠϧʹΑͬͯ֎Ε஋Λ আ֎Ͱ͖Δͷ͸ૉ੖Β͍͠ ςετέʔε͝ͱͷ੒ޭ཰΍ɺ࣌ؒ৘ใ ΋ݟΒΕΔʢ·ͩpreivewػೳʣ εςοϓ͝ͱͷ࣌ؒ΍੒ޭ཰ΛݟΒΕ ͳ͍ͷͰݸਓతʹ͸ػೳ͕଍Γͳ͍

Slide 29

Slide 29 text

meercode.io ෳ਺ͷCIαʔϏεʹରԠͨ͠μογϡ ϘʔυΛ࡞੒Ͱ͖ΔαʔϏε Github Actions΍Bitriseɺͦͷଞͷαʔ Ϗε΋αϙʔτ Ϗϧυ࣌ؒɺ੒ޭ཰ɺϦϙδτϦ͝ͱͷ CIαʔϏεͷ࢖༻࣌ؒͳͲ εςοϓ͝ͱͷ࣌ؒ΋෼ੳՄೳ

Slide 30

Slide 30 text

·ͱΊ • CI/CDΛ௕ظӡ༻͍ͯ͘͠ͱൃੜ͢Δ໰୊ͷௐࠪίετ͸ߴ͍ • Ϗϧυ࣌ؒͷ૿Ճ • ͨ·ʹࣦഊ͢ΔϏϧυ • ϏϧυσʔλΛ෼ੳՄೳʹ͢Δ • ࣗ࡞ͨ͠CIAnalyzerͷ঺հ • BigQuery + DataStudioʢDataPortalʣ • ࣮ࡍʹۀ຿Ͱศརʹ׆༻Ͱ͖͍ͯΔࣄྫͷ঺հ k8s + ίϯςφϏϧυͷCI/CDͷੈք͸ৄ͘͠ͳ͍ͷͰɺ   ͜ͷք۾Ͱ͸͜͏͍͏πʔϧ͕͋Γ·͢Έ͍ͨͳ৘ใΛڭ͑ͯ΋Β͑Δͱخ͍͠Ͱ͢ʂ

Slide 31

Slide 31 text

ࢀߟ৘ใ • CIAnalyzer https://github.com/Kesin11/CIAnalyzer • ࢖͍ํͷ঺հهࣄ https://zenn.dev/kesin11/articles/cf08579949b8b0 • Azure Pipelines • TypeScriptʢnode14ͷCIʣ https://typescript.visualstudio.com/TypeScript/ _build?de fi nitionId=6&view=runs • Windows Terminal https://dev.azure.com/ms/terminal/_build? de fi nitionId=136&view=runs • GithubͷmicrosoftͷϦϙδτϦ͸Azure PipelinesΛ࢖ͬͯΔ΋ͷ͕ଟ͍

Slide 32

Slide 32 text

ࢀߟ৘ใ • CircleCIʢInsightͷυΩϡϝϯτʣ https://circleci.com/docs/2.0/ insights/ • Meercode meercode.io

Slide 33

Slide 33 text

No content