Terraform Provider開発のノウハウ

2022年8月25日に開催のHashiTalks: Japanにて発表した内容になります。
https://www.youtube.com/watch?v=4A-c-ykc0ak

References
- https://github.com/hashicorp/terraform-provider-scaffolding-framework
- https://github.com/hashicorp/terraform-provider-aws/blob/main/ROADMAP.md
- https://www.terraform.io/cli/config/config-file
- https://www.terraform.io/cli/config/config-file#development-overrides-for-provider-developers

Transcript

1. Terraform Provider
 ։ൃͷϊ΢ϋ΢ Kanji Yomoda
2. ࣗݾ঺հ ࢛ํా؏ࣇ (Kanji Yomoda) / גࣜձࣾϝϧΧϦ(US@Tokyo) ݕࡧج൫νʔϜ @k_yomo_jp k-yomo
3. ࣗݾ঺հ ϑϥϯεͷDIYͷECͷձࣾ΍ຊՈAlgoliaͷࣾ಺Ͱ΋࢖༻ͯ͠௖͍͍ͯΔ
4. Agenda - Terraform Providerͷ໾ׂ - TerraformϓϩόΠμʔͷॳظߏங - खݩͰProviderΛಈ͔͢ - Ϧιʔεͷཻ౓ͷܾΊํ - ෳ਺Ϧιʔεͷڝ߹ճආ - σόοάϩάΛຒΊࠐΉ - ࡟আ࿙Εςετ༻ϦιʔεͷରԠ
5. Terraform Providerͷ໾ׂ - TerraformͷStateͱAPIͷreq/resͷڮ౉͠ - Core(terraformίϚϯυ)ͷϓϩηεͱ͸ผʹ1Provider 1ϓϩηεཱͭ - Coreͱ͸RPC(gRPC)Λհͯ͠௨৴
6. Terraform Providerͷ໾ׂ - TerraformͷStateͱAPIͷreq/resͷڮ౉͠ - Core(terraformίϚϯυ)ͷϓϩηεͱ͸ผʹ1Provider 1ϓϩηεཱͭ - Coreͱ͸RPC(gRPC)Λհͯ͠௨৴
7. TerraformϓϩόΠμʔͷॳظߏங - ެࣜυΩϡϝϯτ͕ͱͯ΋ॆ࣮͍ͯ͠Δ - Terraform Provider༻SDK - terraform-plugin-sdkͰ͸ͳ͘terraform-plugin-frameworkΛ࢖͏ - Template - ϘΠϥϓϨʔτɺCI(Test,Release)ɺυΩϡϝϯτੜ੒ɺ౳ʑશ෦ೖΓ https://github.com/hashicorp/terraform-provider-scaffolding-framework
8. terraform-plugin-framework https://github.com/hashicorp/terraform-provider-aws/blob/main/ROADMAP.md terraform-aws-provider Roadmap: August 2022 - October 2022 - Terraform-provider-aws΋terraform-plugin-frameworkʹҠߦத - PRνϟϯεʂʂ
9. खݩͰProviderΛಈ͔͢ - Terraform ProviderͷΠϯετʔϧͷྲྀΕ - tfϑΝΠϧΛಡΈඞཁͳProviderɺVersionΛ֬ೝ - Ωϟογϡʹ͋Ε͹ΩϟογϡΛ࢖͏ - CLIͷConfigϑΝΠϧͷઃఆ(Mac OSͷ৔߹͸~/.terraformrc)ΛಡΈΠϯετʔϧઌΛ֬ೝ - ໌ࣔ͞Ε͍ͯͳ͚Ε͹҉໧తʹԼهͷॱͰΠϯετʔϧ(Mac OSͷ৔߹) - $HOME/.terraform.d/plugins/ - ~/Library/Application Support/io.terraform/plugins - /Library/Application Support/io.terraform/plugins - ./terraform.d/plugins - ֎෦ϨδετϦ https://www.terraform.io/cli/config/config-file
10. खݩͰProviderΛಈ͔͢ - Terraform ProviderͷΠϯετʔϧͷྲྀΕ - tfϑΝΠϧΛಡΈඞཁͳProviderɺVersionΛ֬ೝ - Ωϟογϡʹ͋Ε͹ΩϟογϡΛ࢖͏ - CLIͷConfigϑΝΠϧͷઃఆ(Mac OSͷ৔߹͸~/.terraformrc)ΛಡΈΠϯετʔϧઌΛ֬ೝ - ໌ࣔ͞Ε͍ͯͳ͚Ε͹҉໧తʹԼهͷॱͰΠϯετʔϧ(Mac OSͷ৔߹) - $HOME/.terraform.d/plugins/ - ~/Library/Application Support/io.terraform/plugins - /Library/Application Support/io.terraform/plugins - ./terraform.d/plugins - ֎෦ϨδετϦ https://www.terraform.io/cli/config/config-file
11. खݩͰProviderΛಈ͔͢ - ԼهͷConfigϑΝΠϧͷઃఆͰ։ൃ༻ʹ࢖༻͢ΔProviderΛ೚ҙͷσΟϨΫτϦͷύεʹ্ॻ͖͕Մೳ - Windows: %APPDATA%ͷ௚Լͷterraform.rc - ͦΕҎ֎: ~/.terraformrc - Version΍Checksumͷݕূ͕εΩοϓ͞ΕΔ (= ศར but ҙਤ͠ͳ্͍ॻ͖ʹ஫ҙ) https://www.terraform.io/cli/config/config-file#development-overrides-for-provider-developers ※ dev_overrides͸v0.14Ҏ߱ͷΈ࢖༻Մೳ
12. खݩͰProviderΛಈ͔͢ - ~/.teraform.d/plugins/registry.terraform.io/(Max OSͷ৔߹)Լͷ
 ॴఆͷσΟϨΫτϦʹϏϧυͨ͠όΠφϦΛஔ͘ - όʔδϣϯࢦఆͰ͖ͨΓɺchecksumͷ֬ೝ͕͋ͬͨΓͱ҆શ 
 but checksumͷ஋͕buildͷ౓ʹ(ࠩ෼͕͋Ε͹)ຖճมΘΔͷͰগ͠໘౗
13. Ϧιʔεͷཻ౓ͷܾΊํ Ұ౓ʹෳ਺ૢ࡞͍ͨ͠έʔε͕͋Δ(e.g. ݖݶɺϗϫΠτϦετ౳) - 1ͭͷϦιʔεʹ഑ྻorϒϩοΫͰఆٛ͢Δ - Or Ϣʔβʔʹෳ਺Ϧιʔε࡞ͬͯ΋Β͏ ߟྀϙΠϯτ - Ϣʔεέʔε => ݸผ and/or ҰׅͰઃఆ͍ͨ͠έʔε͕͋Δ͔൱͔ - ઃఆ͞Ε͏Δྔ => ଟ͍৔߹͸ݸผͷϦιʔεͩͱplan/applyʹ͕͔͔࣌ؒΔ - API => Ұׅૢ࡞͢ΔAPI͕ఏڙ͞Ε͍ͯΔ͔ - Quota / Rate limit => େྔͷAPIίʔϧͰQuota / rate limitͷ্ݶʹୡ͠ͳ͍͔
14. Ϧιʔεͷཻ౓ͷܾΊํ - ྨࣅޠΛొ࿥͢ΔAPI༻ͷϦιʔεఆٛ - 1ͭͷindex(≒table)ʹର͠ɺෳ਺ͷྨࣅޠͷϖΞΛొ࿥Ͱ͖Δ(e.g. ϙςν<=>ϙςτνοϓε) ߟྀϙΠϯτ => ݁ՌɺindexຖʹҰׅૢ࡞͢ΔsynonymsϦιʔεΛఆٛ - Ϣʔεέʔε => ҰׅͰઃఆ͢Δέʔε͕େ൒ - ઃఆ͞Ε͏Δྔ => ~100k - API => Ұׅૢ࡞͢ΔAPI͋Γ - Quota / Rate limit => ্ݶ͸௿Ί(ແྉϓϥϯ͕ཧ༝ͳՄೳੑ΋) Algolia Providerͷྫ
15. ෳ਺Ϧιʔεͷڝ߹ճආ - ෳ਺ͷϦιʔε͔Βڝ߹͢ΔAPIϦιʔεΛมߋ͍ͨ͠৔߹ - ϨϓϦΧΠϯελϯε౳ɺϓϥΠϚϦʔͷઃఆ΋ߋ৽͢Δඞཁ͕͋Δ͕ɺϓϥΠϚϦʔͷߋ৽΍ଞͷϨϓϦΧ ͔Βͷߋ৽ͱڝ߹͢Δ resource “replica” “replica1” { primary_id = “primary” } resource “replica” “replica2” { primary_id = “primary” } Target API Primary instance Replica1 Replica2
16. ෳ਺Ϧιʔεͷڝ߹ճආ - ղܾࡦ => ΦϯϝϞϦͷLockΛऔΔ(sync.Mutex౳Ͱ) - ※ ಉ͡TerraformͷstateͰର৅ϦιʔεΛ؅ཧ & Terraformͷ࣮ߦ࣌ͷϩοΫ΋औ͍ͬͯΔલఏ - ԼهGoolge ProviderͷCloud SQLΠϯελϯεͷresourceͷྫ
17. σόοάϩάΛຒΊࠐΉ - HttpϦΫΤετɺϨεϙϯεͷϩάΛऔΕΔΑ ͏ʹ͓ͯ͘͠ - httputil.DumpRequestOut, httputil.DumpResponse͕ͱͯ΋ศར - http.TransportΛϥοϓͯ͠req/resͷμϯϓ Λϩάʹॻ͖ग़͢ - terraform-plugin-sdkʹ͸ؙͬͱ΍ͬͯ͘ΕΔ ϔϧύʔؔ਺ logging.NewLoggingHTTPTransport()͕͋Δ ͕ɺterraform-plugin-frameworkʹ͸ͳ͍ https://www.terraform.io/plugin/debugging
18. ࡟আ࿙Εςετ༻ϦιʔεͷରԠ - Acceptance Test => ࣮ࡍʹAPIϦιʔεΛ࡞੒ͯ͠ߦ͏E2Eςετ - ్தͰΤϥʔ͕ى͖ͨ৔߹ͳͲɺ࡞੒ͨ͠APIϦιʔε͕࢒ͬͯ͠·͏έʔε͕͋Δ
 (=> ͓͕͔͔ۚΔέʔεͳͲ΋͋ΔͨΊཁ஫ҙ) - ղܾࡦ => PrefixΛ͚ͭΔ & Prefix෇͖ͷAPIϦιʔεΛҰׅ࡟আ͢ΔεΫϦϓτΛ༻ҙ (Test༻ΞΧ΢ϯτ ΛผͰ෼͚͓ͯ͘ͳͲ΋ซͤͯߦ͏ͱ҆શ)
19. ·ͱΊ - Terraform Providerͷ໾ׂ => TerraformͷStateͱAPIͷreq/resͷڮ౉͠ - TerraformϓϩόΠμʔͷॳظߏங => terraform-plugin-framework & templateΛ࢖͏ - खݩͰProviderΛಈ͔͢ => ~/.terraformrcͷઃఆ or ॴఆͷύεʹόΠφϦΛஔ͘ - σόοάϩάΛຒΊࠐΉ => ࠷௿ݶHttpϦΫΤετɺϨεϙϯεͷϩάΛऔ͓ͬͯ͘ͱศར - Ϧιʔεͷཻ౓ͷܾΊํ => ϢʔεέʔεͱAPIͷ࢓༷͔ΒέʔεόΠέʔεͰߟ͑Δ - ෳ਺Ϧιʔεͷڝ߹ճආ => ΦϯϝϞϦͷLockΛऔΔ - ࡟আ࿙Εςετ༻APIϦιʔεͷରԠ => Ϧιʔε໊ʹprefixΛ͚ͭͯҰׅ࡟আ
20. References - https://github.com/hashicorp/terraform-provider-scaffolding-framework - https://github.com/hashicorp/terraform-provider-aws/blob/main/ROADMAP.md - https://www.terraform.io/cli/config/config-file - https://www.terraform.io/cli/config/config-file#development-overrides-for-provider-developers