Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to develop a custom Terraform provider@Merpay SRE Tech Talk

Keke
March 19, 2020

How to develop a custom Terraform provider@Merpay SRE Tech Talk

Keke

March 19, 2020
Tweet

More Decks by Keke

Other Decks in Technology

Transcript

  1. Merpay SRE
    keke
    “How to develop a
    custom terraform provider”

    View Slide

  2. 1. What’s custom Terraform provider
    2. How to develop your first Terraform provider
    3. Making it better
    “Content”

    View Slide

  3. Whoever listens this presentation can
    1. Understand how the Terraform is using Terraform
    provider to manage resources
    2. Develop custom Terraform provider
    “Goal”

    View Slide

  4. “What’s custom
    Terraform provider”

    View Slide

  5. When $ terraform apply…
    Terraform
    Deep dive !

    View Slide

  6. When $ terraform apply…
    Terraform
    Deep dive !

    View Slide

  7. Terraform
    Terraform Core Terraform Plugin
    When $ terraform apply…

    View Slide

  8. Terraform
    Terraform Core Terraform Provider
    When $ terraform apply…
    Terraform
    Provisioner

    View Slide

  9. Terraform
    Terraform Core Terraform Provider
    When $ terraform apply…
    Terraform
    Provisioner
    “Basic logic” “Logic between the service
    and how to manage”
    “Logic of
    how to
    provision”

    View Slide

  10. Example: Terraform Provider
    3 kinds of Terraform provider

    View Slide

  11. Example: Terraform Provider
    ɾNull resource

    ɾSecret

    ɾ…
    3 kinds of Terraform provider
    1. Built-in provider

    View Slide

  12. Example: Terraform Provider
    ɾNull resource

    ɾSecret

    ɾ…
    3 kinds of Terraform provider
    1. Built-in provider
    ɾterraform-provider-aws

    ɾterraform-provider-google

    ɾterraform-provider-azure

    ɾterraform-provider-datadog

    ɾ…
    2. Distributed provider by Hashicorp
    Needs review by Hashicorp

    View Slide

  13. Example: Terraform Provider
    ɾNull resource

    ɾSecret

    ɾ…
    3 kinds of Terraform provider
    1. Built-in provider
    ɾterraform-provider-aws

    ɾterraform-provider-google

    ɾterraform-provider-azure

    ɾterraform-provider-datadog

    ɾ…
    2. Distributed provider by Hashicorp
    3. Third party provider
    ɾterraform-provider-keke

    ɾ…

    ɾterraform-provider-xxxx
    Needs review by Hashicorp
    You can built your own one!!!
    = Custom Terraform Provider

    View Slide

  14. Example: Terraform Provisioner
    ɾLocal-exec: Execute command

    View Slide

  15. Example: Terraform Provisioner
    ɾchef: Install chef agent

    View Slide

  16. Example: Terraform Provisioner
    ɾMore!!!

    View Slide

  17. Example: Terraform Provisioner
    ɾMore!!!
    Note
    You can not build a custom Terraform Provisioner

    View Slide

  18. Terraform
    Terraform Core
    When $ terraform apply…
    Terraform
    Provisioner
    Terraform Provider

    View Slide

  19. Terraform
    Terraform Core
    When $ terraform apply…
    1.Request PRC
    2. Actual PRC
    Implemented in the Provider
    3. Result
    4. Store
    5. Retrieve
    6. Result
    tfstate
    Terraform
    Provisioner
    Terraform Provider

    View Slide

  20. When $ terraform apply…
    Any API
    Terraform
    Terraform Core Terraform Provider
    1.Request PRC
    2. Actual PRC
    Implemented in the Provider
    3. Result
    4. Store
    5. Retrieve
    6. Result
    tfstate
    Terraform
    Provisioner
    Custom

    View Slide

  21. Whoever listens this presentation can
    1. Understand how the Terraform is using Terraform
    provider to manage resources
    2. Develop custom Terraform provider
    “Goal”

    View Slide

  22. “How to develop
    your first
    terraform provider”

    View Slide

  23. “Understand the rules”

    View Slide

  24. “Understand the
    Provider’s principles”

    View Slide

  25. 1. One provider should focus on one service
    A custom provider
    (terraform-provider-god)
    API A
    API B
    API C

    View Slide

  26. A custom provider
    (terraform-provider-god)
    API A
    API B
    API C
    1. One provider should focus on one service

    View Slide

  27. A custom provider
    (terraform-provider-api-a)
    API A
    API B
    API C
    A custom provider
    (terraform-provider-api-b)
    A custom provider
    (terraform-provider-api-c)
    1. One provider should focus on one service

    View Slide

  28. 2. Seperate API Client package
    A custom provider
    (terraform-provider-god)
    API B
    API Client package
    No unit tests or PRC logic in the provider

    View Slide

  29. “Understand the
    Resource’s principles”

    View Slide

  30. 1. One resource should focus on one object in API
    A resource
    API for Object A
    API for Object B
    API for Object C
    .tf

    View Slide

  31. 1. One resource should focus on one object in API
    A resource
    API for Object A
    API for Object B
    API for Object C
    .tf

    View Slide

  32. 1. One resource should focus on one object in API
    resource_a
    .tf
    API for Object A
    API for Object B
    API for Object C
    resource_b
    .tf
    resource_c
    .tf

    View Slide

  33. 2. Follow schema to the actual API object
    API for xxxxxx
    {
    “name”: “xxxx”,
    “email”: “xxxx”
    }
    API field

    View Slide

  34. “Directory Style”

    View Slide

  35. “All providers are same directory structure”
    ← Define this custom Terraform provider
    so that Terraform core can import
    ← Define the provider
    ← Define the resource

    View Slide

  36. “Implement Provider”

    View Slide

  37. “Minimum provider”

    View Slide

  38. “Minimum provider: main.go”

    View Slide

  39. “Minimum provider: {NAME}/provider.go”

    View Slide

  40. “Minimum provider: build & use”
    1. Run $ go build
    2. Create tf file for your provider
    3. Run terraform init

    View Slide

  41. “Minimum provider: build & use”
    1. Run $ go build
    2. Create tf file for your provider
    3. Run terraform init

    View Slide

  42. “Implement Resource”

    View Slide

  43. “Implement Resouce: Define Schema”

    View Slide

  44. “Implement Resouce: Define Schema”

    View Slide

  45. “Implement Resouce: Define CRUD”

    View Slide

  46. “Implement Resouce: Define CRUD”
    CRUD syncs with tfstate by ID
    Every resource must have ID

    View Slide

  47. “Minimum provider: build & use”
    1. Run $ go build
    2. Create tf file for your provider
    3. Run terraform init

    View Slide

  48. “How can we create diffs?”

    View Slide

  49. “How can we create diffs?”
    Store in the tfstate by d.Set()

    View Slide

  50. “Minimum provider: build & use”
    1. Run $ go build
    2. Create tf file for your provider
    3. Run terraform init

    View Slide

  51. “Minimum provider: build & use”
    1. Run $ go build
    2. Create tf file for your provider
    3. Run terraform init

    View Slide

  52. Whoever listens this presentation can
    1. Understand how the Terraform is using Terraform
    provider to manage resources
    2. Develop custom Terraform provider
    “Goal”

    View Slide

  53. “Making it better”

    View Slide

  54. “Validating attributes”
    Add ValidateFunc

    View Slide

  55. “Sensitive data”
    Add Sensitive

    View Slide

  56. “Releasing”
    Use goreleaser

    View Slide

  57. “Thank you”

    View Slide