Terraform Provider for TROCCO × AI で 半自動化する複数プロダクトの連携運用 / Semi-Automating Multi-Product Data Integration Ops with the Terraform Provider for TROCCO × AI

2026/07/17 開催の #p_UG 東京：星じゃなくてAIに願いを ～データ基盤における AI 活用紹介～ での登壇資料です。
登壇者 : 人材プラットフォーム本部 CTO 室データマネジメントグループ 山邉哲生
イベント URL : https://pug.connpass.com/event/396734/

Transcript

1. #p_UG ౦ژ 2026.07.07 / ੕͡Όͳͯ͘AIʹئ͍Λ ʙσʔλج൫ʹ͓͚Δ AI ׆༻঺հʙ Terraform Provider for TROCCO × AI Ͱ ൒ࣗಈԽ͢Δෳ਺ϓϩμΫτͷ࿈ܞӡ༻ Semi-automating multi-product data pipeline operations גࣜձࣾϝυϨʔ ࢁᬒ ఩ੜ @beniyama લճʮTROCCO × Terraform × AI Ͱ kintone ࿈ܞ΋ IaC Խʯͷޙ೔ஊ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED.
2. ΠϯτϩμΫγϣϯ ࣗݾ঺հ @beniyama ࢁᬒ ఩ੜ Tetsuo Yamabe גࣜձࣾϝυϨʔ ਓࡐϓϥοτϑΥʔϜຊ෦ σʔλϚωδϝϯτάϧʔϓ άϧʔϓϚωδϟʔ σʔλج൫ͷ։ൃӡ༻΍σʔλ૊৫ͷ্ཱͪ͛ͳͲΛܦݧޙɺ2025/04 ΑΓϝυϨʔʹࢀըɻ ਓࡐϓϥοτϑΥʔϜྖҬͰ TROCCO × dbt × AI Λத৺ͱͨ͠σʔλج൫ͷߏஙͱ૊৫ͮ͘Γʹฃಆதɻ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 02
3. ΠϯτϩμΫγϣϯ ࣄۀͱϓϩμΫτ ※ 2025೥12݄ݱࡏ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 03
4. ΠϯτϩμΫγϣϯ ຊ೔ͷ͓࿩ 01 લճͷ;Γ͔͑Γͱɺӡ༻Ͱ௚໘ͨ͠՝୊ 02 ൒ࣗಈԽͷ͘͠Έ ʙݕ஌͔Βຊ൪൓ө·Ͱʙ 03 1ͭͷج൫Ͱෳ਺ϓϩμΫτΛѻ͏ઃܭ 04 ·ͱΊ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 04
5. ;Γ͔͑Γ લճͷ͋Β͢͡ × Terraform × AI Ͱ kintone ࿈ܞ΋ IaC Խ ࿈ܞઃఆΛɺGUI Ͱ͸ͳ͘ ίʔυʢTerraformʣͰ؅ཧ GUI Ͱ࡞ͬͨઃఆΛ AI ʹղੳͤͯ͞ tf Λੜ੒ kintone ΁ͷసૹ΋ Git ؅ཧԼʹ → IaC (Infrastructure as Code) Խ·ͰͰ͖ͨ SpeakerDeck ͰલճࢿྉΛݟΔ ⾭ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 05
6. ;Γ͔͑Γ IaC ԽͰมΘͬͨ͜ͱ BEFORE / GUI Ͱϙνϙν × ઃఆ͢ΔਓʹΑͬͯ಺༰͕ϒϨΔ × Ͳ͜ͰԿ͕มΘ͔ͬͨ௥͑ͳ͍ × ํ਑Λม͑ͨΒશ෦Λ։͍ͯ௚͢ × సૹઃఆ͕ଟ͗ͯ͢άϥϑ͕Θ͔Βͳ͍ → AFTER / ίʔυͱ Git ✓ มߋΛϨϏϡʔͰ͖Δ ✓ ୭͕ԿΛม͔͑ͨཤྺ͕࢒Δ ✓ ํ਑มߋΛҰׅͰ൓өͰ͖Δ ✓ ίʔυϕʔεͰAI ؅ཧ΁ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 06
7. ;Γ͔͑Γ σʔλج൫ͱ TROCCO ͷ࢖͍Ͳ͜Ζ ϓϩμΫτຊମ A σʔλϕʔε X ֎෦γεςϜ kintone / App Store Connect / ޿ࠂഔମ औΓࠐΈ BigQuery GCP · prj-Aʢdbt Ͱม׵ʣ ϦόʔεETL ֎෦γεςϜʢॻ͖໭͠ʣ kintone ௚઀ࢀর → BIɾՄࢹԽ Metabase / εϓϨουγʔτ ϓϩμΫτຊମ B σʔλϕʔε Y ֎෦γεςϜ kintone / App Store Connect / ޿ࠂഔମ औΓࠐΈ BigQuery GCP · prj-Bʢdbt Ͱม׵ʣ ϦόʔεETL ֎෦γεςϜʢॻ͖໭͠ʣ kintone ௚઀ࢀর → BIɾՄࢹԽ Metabase / εϓϨουγʔτ ϓϩμΫτຊମ C σʔλϕʔε Z ֎෦γεςϜ kintone / App Store Connect / ޿ࠂഔମ औΓࠐΈ BigQuery GCP · prj-Cʢdbt Ͱม׵ʣ ϦόʔεETL ֎෦γεςϜʢॻ͖໭͠ʣ kintone ௚઀ࢀর → BIɾՄࢹԽ Metabase / εϓϨουγʔτ ⋯ → TROCCO Λத৺ͱͨ͠ɺऔΓࠐΈɾdbt δϣϒͷ࣮ߦɾϦόʔε ETL ͷ؅ཧɻಉ͡ߏ੒͕ϓϩμΫτͷ਺͚ͩฒͿ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 07
8. ӡ༻ϑΣʔζͷ՝୊ ࣮༻Խʹཱͪ͸͔ͩΔෳ਺ϓϩμΫτͷน TROCCO ͷར׆༻͕֦͕ΔʹͭΕͯ ࿈ܞ͢ΔϓϩμΫτ΋૿Ճ ϓϩμΫτ͝ͱʹ࠾༻͍ͯ͠Δ ϑϨʔϜϫʔΫ΍σʔλϕʔε͕ҟͳΔ ઃఆมߋͷස౓΍εΩʔϚͷϥΠϑαΠΫϧ΋ଟ༷ ։ൃ෦ॺ΋όϥόϥͰ ґཔϑϩʔͷ੔උɾӡ༻ίετ΋૿େ গਓ਺νʔϜͷ৔߹ɺਓྗͰؤுΔͷʹ͸ݶք → ؾ͚ͮ͹ζϨ͕ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 08
9. ӡ༻ϑΣʔζͷ՝୊ ؾ͚ͮ͹ੜ·ΕΔζϨ Axis 1 ᶃ ࣮ମͱίʔυͷζϨ TROCCO GUI ͔ΒͷมߋͰੜ·ΕΔဃ཭ Axis 2 ᶄ εΩʔϚͱίʔυͷζϨ ΞϓϦέʔγϣϯଆͷσʔλϕʔεεΩʔϚͷมߋͰੜ·ΕΔ ဃ཭ → ͜ͷ2ͭΛࣗಈݕ஌ͯ͠मਖ਼͍ͨ͠ ※ ίʔυʹTerraform ϑΝΠϧ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 09
10. ΍ͬͨ͜ͱ मਖ਼ҊΛࣗಈੜ੒͢ΔϑϩʔΛ৽ઃ STEP 1ɾࣗಈɾखಈ ݕ஌ɾґཔ ৽ن࿈ܞґཔ ਓ͕ґཔ͢Δ ઃఆဃ཭ͷࣗಈݕ஌ ζϨΛݟ͚ͭΔ → STEP 2 ɾࣗಈ PR ࡞੒ & plan GitHub Workflow ͕ PR Λ࡞੒͠ɺplan ·ͰࣗಈͰग़͢ → STEP 3 ɾਓ ϨϏϡʔ & ঝೝ plan ಺༰Λ֬ೝͯ͠ঝೝ → STEP 4 ൓ө ϦϦʔε PR ΁ͷϚʔδͰ apply → PR ͷ࡞੒ͱ plan ͷ࣮ߦ·ͰϫʔΫϑϩʔ͕ࣗಈͰ࣮ࢪɻਓ͸ planʢมߋ఺ʣͷ֬ೝʹूத → GitHub Issue / PR ʹͷͤΔ͜ͱͰه࿥Λ࢒ͭͭ͠ɺ೔ʑͷ։ൃϫʔΫϑϩʔͰѻ͑ΔΑ͏ʹ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 10
11. ΍ͬͨ͜ͱ ݕ஌ᶃ ࣮ମͱίʔυͷζϨ Axis 1 TROCCO ͷ࣮ମ ͍·࣮ࡍʹಈ͍͍ͯΔઃఆ νΣοΫ → ߹ΘͤΔ → ίʔυʢtfʣ Git Ͱ؅ཧ͍ͯ͠Δఆٛ ࠩ෼ΛνΣοΫ͠ɺζϨ͕͋Ε͹ Issue Ͱ௨஌ ௚͢޲͖͸ ࣮ମ → ίʔυʢ࣮ମʹ߹ΘͤͯίʔυΛ࠶ੜ੒ʣ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 11
12. ΍ͬͨ͜ͱ ݕ஌ᶄ εΩʔϚͱίʔυͷζϨ Axis 2 ΞϓϦͷεΩʔϚ ϓϩμΫτଆͷ߲໨ఆٛ νΣοΫ → ߹ΘͤΔ → ίʔυʢtfʣ ࿈ܞͷग़ྗΧϥϜఆٛ ߲໨͕ ૿ݮͨ͠Βݕ஌ʢRails / Drizzle / Prisma ΛಡΈ෼͚ʣ ௚͢޲͖͸ εΩʔϚ → ίʔυʢεΩʔϚʹ߹Θͤͯ࠶ੜ੒ʣ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 12
13. ΍ͬͨ͜ͱ PR ࡞੒ˍ planᶃ ૿͑ͨΧϥϜΛ Schema Drift Summary ͱͯࣗ͠ಈͰॻ͖ग़͠ɻgithub-actions ͕ Issue Λىථɻ • Open [drift] trocco/ϓϩμΫτB axis-2 drift detected #1868 github-actions bot opened 5h ago Schema Drift Summary — ϓϩμΫτB UC1: column-level drift ϓϩμΫτB_customers (trocco/job-definitions/ϓϩμΫτB/job_******.tf) column kind tf_type source_type mapped col_1 added_in_source - varchar string col_2 added_in_source - varchar string col_3 added_in_source - datetime long col_4 added_in_source - varchar string col_5 added_in_source - varchar string axis-2 by AI drift product/ϓϩμΫτB trocco ݕ஌ͱ΄΅ಉ࣌ʹ म෮ PR Λࣗಈ࡞੒ → ⑂ PR fix(trocco): ϓϩμΫτB UC1 ΧϥϜ drift ղফ #1869 · ci-bot bot → develop by AI drift trocco ݕ஌ͨ͠಺༰ʹରԠ͢Δ म෮ίʔυΛىͨ͜͠ PR Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 13
14. ΍ͬͨ͜ͱ PR ࡞੒ˍ planᶄ ⑂ Draft fix(trocco): ϓϩμΫτB UC1 ΧϥϜ drift ղফ (schema → tf) #1869 ci-bot bot → develop ← trocco-schema-sync/ϓϩμΫτB-update-******** Trocco Schema Sync (Axis 2) — ϓϩμΫτB / update ͜ͷ PR ͸ trocco-schema-sync.yml ʹΑΔࣗಈੜ੒Ͱ͢ɻ ํ޲ ΞϓϦεΩʔϚΛਖ਼ͱͯ͠ɺtf ͷ filter_columns / parquet_parser.columns Λ࠶ੜ੒ ର৅ςʔϒϧ ϓϩμΫτB_customers ⾠ v1 ͷط஌੍໿ طଘͷඇσϑΥϧτ஋͸ςϯϓϨʔτ࠶ੜ੒Ͱ্ॻ͖ / ܕ overrideɾtimestamp format ౳͸ߋ৽͞Εͳ͍ → Ϛʔδલʹ diff Λ֬ೝ ϨϏϡʔ؍఺ ᶃ ΧϥϜू߹ɾܕ͕ schema ͱҰக͍ͯ͠Δ͔ ᶄ ҙਤ͠ͳ͍ΧϥϜ࡟আ͕ແ͍͔ ᶅʢUC2ʣຊ౰ʹ࡟আͯ͠ྑ͍ςʔϒϧ͔ Triggered by: workflow_dispatch(product=ϓϩμΫτB, mode=update) github-actions bot commented Terraform Plan ͷ݁ՌͰ͢ʢmerge Ͱ applyʣ plan ର৅Ϧιʔε --target=module.job_definitions.module.ϓϩμΫτB.trocco_job_definition.job_****** ▼ Plan ݁Ռ ~ resource "trocco_job_definition" { ~ filter_columns = [ + { name = "col_1" type = "string" } + { name = "col_2" type = "string" } + { name = "col_3" type = "long" } + { name = "col_4" type = "string" } + { name = "col_5" type = "string" } # (54 unchanged hidden) ] } Plan: 0 to add, 1 to change, 0 to destroy. @reviewer ࠩ෼Λ֬ೝ͍ͯͩ͘͠͞ → ਓ͸ plan Λݟͯ /plan-ok Ͱঝೝ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 14
15. ΍ͬͨ͜ͱ GitHub ϫʔΫϑϩʔͷશମ૾ ݕ஌ drift-detect / schema-drift-detect › ࣮ମɾεΩʔϚͱίʔυͷʮζϨʯΛݟ͚ͭΔ ىථɾରԠ։࢝ drift-issue / dispatch › Issue Ͱ௨஌͠ɺղফϑϩʔΛࣗಈͰىಈ͢Δ ੜ੒ AI auto-pr › ৽ن࿈ܞґཔͷ৔߹͸ AI ͕ΞϓϦέʔγϣϯεΩʔϚΛ֬ೝͯ͠ tf ϑΝΠϧΛࣗಈੜ੒ ΠγϡʔͷղऍͱεΩʔϚߏ଄ͷཧղʹ AI Λ׆༻ ಉظ drift-sync / schema-sync › ဃ཭ͷ࣠ʹ߹Θͤͯ tf ϑΝΠϧΛ࠶ੜ੒ plan plan / plan-reviewed › ࣗಈͰ plan Λ࣮ߦ͠ɺPR ʹهࡌ ঝೝɾ൓ө apply › plan ֬ೝΛ໌ࣔ͠ɺϦϦʔεϒϥϯν΁ͷϚʔδͰຊ൪൓ө Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 15
16. ΍ͬͨ͜ͱ generator ʹΑΔந৅Խ generator ʹ config ΛಡΜͰ tf ϑΝΠϧΛࣗಈੜ੒͢Δ࢓૊ΈɻϓϩμΫτ͝ͱͷҧ͍͸ɺ͜ͷ generator ͕ٵऩ͢Δ ϓϩμΫτA ϓϩμΫτB ϓϩμΫτC ϓϩμΫτD → config/<product>.yaml ࿈ܞํࣜ S3→BQ / kintone→BQ / BQ→kintone ͳͲ Λ੾Γସ͑ ม׵ϙϦγʔɾϧʔϧ ΧϥϜϑΟϧλɾ໋໊نଇɾআ֎ϧʔϧΛ config Ͱએݴ ܕϚοϐϯά DB ༝དྷͷܕ → BQ ܕͷडཧදΛڞ௨Խ એݴ(config)ͱखଓ͖(ݕ஌ɾੜ੒)Λ෼཭ → ϓϩμΫτ͝ͱͷ࿈ܞઃఆͷܕΛఆٛ͠ɺϓϥΨϒϧʹѻ͑ΔΑ͏ʹ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 16
17. ΍ͬͨ͜ͱ generator ʹΑΔந৅Խ ᶃ એݴ͢Δ config # config/product_a.yaml schema_source: rails # εΩʔϚه๏ database: postgresql # DB ΋༷ʑ flavor: s3_to_bq # సૹͷ޲͖ column_denylist: - secret_memo → ᶄ AI ͕ੜ੒͢Δ tf ϑΝΠϧʢΠϝʔδʣ job "staff" { source = bigquery { query = "SELECT ..." } dest = kintone { app_id="***", mode="upsert" } columns = [ id, name, status ] } Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 17
18. ΍ͬͨ͜ͱ ݁Ռ શࣗಈʹ͸Ͱ͖ͳ͍͕ɺͦΕͰ΋ӡ༻͸͔ͳΓվળɾޮ཰Խͨ͠ ⾠ ܕԽ͞Ε͍ͯͳ͍ύλʔϯ͸ generator ͷ֦ு͕ඞཁ ⾠ ίʔυ্ͷεΩʔϚͷมߋ ≠ DB ΁ͷద༻ ͳͷͰɺϓϩμΫτଆͷσϓϩΠঢ়گ͸ཁ֬ೝ ⾠ ݸਓ৘ใͳͲͷػඍͳ߲໨ͷ൑ఆ͸ඞͣਓͷ໨Ͱ → ਓͷϨϏϡʔ࣌ͷ൑அࡐྉΛ੔ཧͯ͠ɺͳΔ΂͘ࣄલʹ AI ʹऩूɾ෇هͤ͞Δ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 18
19. ·ͱΊ ຊ೔͓࿩ͨ͜͠͠ͱ ࠓճͷऔΓ૊Έ ɾʮอͭʯΛ൒ࣗಈԽ͠ɺগਓ਺νʔϜͰͷෳ਺ϓϩμΫτͷσʔλ࿈ܞΛ࣮ݱ ɾݕ஌ → ఏҊʢAIʣ→ ঝೝʢਓʣ→ ൓ө·ͰΛ AI × ਓͷҰ࿈ͷϫʔΫϑϩʔʹ ɾεΩʔϚมߋΛࣗಈݕ஌ͯ͠ PR Խ → ϓϩμΫτ։ൃଆ͔ΒͷґཔΛ࠷খݶʹ ɾෳ਺ϓϩμΫτ × σʔλιʔεΛɺ֦ுੑͷߴ͍ config × generatorͷܗࣜͰରԠ AI ͕ޮ͍͍ͯΔͱ͜Ζ ɾܾఆ࿦తʹରԠͰ͖ΔྖҬΛ֦ॆ͢ΔͨΊͷύλʔϯநग़ ɾ࿈ܞઃఆґཔͷղऍͳͲɺϫʔΫϑϩʔ΁ͷೖྗ৘ใͷղੳ ɾਓ͕൑அ͢ΔͨΊͷ৘ใௐࠪͱ੔ཧʢΤϥʔରԠؚΉʣ ɾ͜ͷ࢓૊Έͦͷ΋ͷͷ։ൃ Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED. 19
20. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ גࣜձࣾϝυϨʔ ࢁᬒ ఩ੜ X : @beniyama Copyright© MEDLEY, Inc. ALL RIGHTS RESERVED.