How to develop a custom Terraform provider@Merpay SRE Tech Talk

Transcript

1. Merpay SRE
   keke
   “How to develop a
   custom terraform provider”
   

   View Slide

2. 1. What’s custom Terraform provider
   2. How to develop your first Terraform provider
   3. Making it better
   “Content”
   

   View Slide

3. Whoever listens this presentation can
   1. Understand how the Terraform is using Terraform
   provider to manage resources
   2. Develop custom Terraform provider
   “Goal”
   

   View Slide

4. “What’s custom
   Terraform provider”
   

   View Slide

5. When $ terraform apply…
   Terraform
   Deep dive !
   

   View Slide

6. When $ terraform apply…
   Terraform
   Deep dive !
   

   View Slide

7. Terraform
   Terraform Core Terraform Plugin
   When $ terraform apply…
   

   View Slide

8. Terraform
   Terraform Core Terraform Provider
   When $ terraform apply…
   Terraform
   Provisioner
   

   View Slide

9. Terraform
   Terraform Core Terraform Provider
   When $ terraform apply…
   Terraform
   Provisioner
   “Basic logic” “Logic between the service
   and how to manage”
   “Logic of
   how to
   provision”
   

   View Slide

10. Example: Terraform Provider
    3 kinds of Terraform provider
    

    View Slide

11. Example: Terraform Provider
    ɾNull resource
    
    ɾSecret
    
    ɾ…
    3 kinds of Terraform provider
    1. Built-in provider
    

    View Slide

12. Example: Terraform Provider
    ɾNull resource
    
    ɾSecret
    
    ɾ…
    3 kinds of Terraform provider
    1. Built-in provider
    ɾterraform-provider-aws
    
    ɾterraform-provider-google
    
    ɾterraform-provider-azure
    
    ɾterraform-provider-datadog
    
    ɾ…
    2. Distributed provider by Hashicorp
    Needs review by Hashicorp
    

    View Slide

13. Example: Terraform Provider
    ɾNull resource
    
    ɾSecret
    
    ɾ…
    3 kinds of Terraform provider
    1. Built-in provider
    ɾterraform-provider-aws
    
    ɾterraform-provider-google
    
    ɾterraform-provider-azure
    
    ɾterraform-provider-datadog
    
    ɾ…
    2. Distributed provider by Hashicorp
    3. Third party provider
    ɾterraform-provider-keke
    
    ɾ…
    
    ɾterraform-provider-xxxx
    Needs review by Hashicorp
    You can built your own one!!!
    = Custom Terraform Provider
    

    View Slide

14. Example: Terraform Provisioner
    ɾLocal-exec: Execute command
    

    View Slide

15. Example: Terraform Provisioner
    ɾchef: Install chef agent
    

    View Slide

16. Example: Terraform Provisioner
    ɾMore!!!
    

    View Slide

17. Example: Terraform Provisioner
    ɾMore!!!
    Note
    You can not build a custom Terraform Provisioner
    

    View Slide

18. Terraform
    Terraform Core
    When $ terraform apply…
    Terraform
    Provisioner
    Terraform Provider
    

    View Slide

19. Terraform
    Terraform Core
    When $ terraform apply…
    1.Request PRC
    2. Actual PRC
    Implemented in the Provider
    3. Result
    4. Store
    5. Retrieve
    6. Result
    tfstate
    Terraform
    Provisioner
    Terraform Provider
    

    View Slide

20. When $ terraform apply…
    Any API
    Terraform
    Terraform Core Terraform Provider
    1.Request PRC
    2. Actual PRC
    Implemented in the Provider
    3. Result
    4. Store
    5. Retrieve
    6. Result
    tfstate
    Terraform
    Provisioner
    Custom
    

    View Slide

21. Whoever listens this presentation can
    1. Understand how the Terraform is using Terraform
    provider to manage resources
    2. Develop custom Terraform provider
    “Goal”
    

    View Slide

22. “How to develop
    your first
    terraform provider”
    

    View Slide

23. “Understand the rules”
    

    View Slide

24. “Understand the
    Provider’s principles”
    

    View Slide

25. 1. One provider should focus on one service
    A custom provider
    (terraform-provider-god)
    API A
    API B
    API C
    

    View Slide

26. A custom provider
    (terraform-provider-god)
    API A
    API B
    API C
    1. One provider should focus on one service
    

    View Slide

27. A custom provider
    (terraform-provider-api-a)
    API A
    API B
    API C
    A custom provider
    (terraform-provider-api-b)
    A custom provider
    (terraform-provider-api-c)
    1. One provider should focus on one service
    

    View Slide

28. 2. Seperate API Client package
    A custom provider
    (terraform-provider-god)
    API B
    API Client package
    No unit tests or PRC logic in the provider
    

    View Slide

29. “Understand the
    Resource’s principles”
    

    View Slide

30. 1. One resource should focus on one object in API
    A resource
    API for Object A
    API for Object B
    API for Object C
    .tf
    

    View Slide

31. 1. One resource should focus on one object in API
    A resource
    API for Object A
    API for Object B
    API for Object C
    .tf
    

    View Slide

32. 1. One resource should focus on one object in API
    resource_a
    .tf
    API for Object A
    API for Object B
    API for Object C
    resource_b
    .tf
    resource_c
    .tf
    

    View Slide

33. 2. Follow schema to the actual API object
    API for xxxxxx
    {
    “name”: “xxxx”,
    “email”: “xxxx”
    }
    API field
    

    View Slide

34. “Directory Style”
    

    View Slide

35. “All providers are same directory structure”
    ← Define this custom Terraform provider
    so that Terraform core can import
    ← Define the provider
    ← Define the resource
    

    View Slide

36. “Implement Provider”
    

    View Slide

37. “Minimum provider”
    

    View Slide

38. “Minimum provider: main.go”
    

    View Slide

39. “Minimum provider: {NAME}/provider.go”
    

    View Slide

40. “Minimum provider: build & use”
    1. Run $ go build
    2. Create tf file for your provider
    3. Run terraform init
    

    View Slide

41. “Minimum provider: build & use”
    1. Run $ go build
    2. Create tf file for your provider
    3. Run terraform init
    
    

    View Slide

42. “Implement Resource”
    

    View Slide

43. “Implement Resouce: Define Schema”
    

    View Slide

44. “Implement Resouce: Define Schema”
    

    View Slide

45. “Implement Resouce: Define CRUD”
    

    View Slide

46. “Implement Resouce: Define CRUD”
    CRUD syncs with tfstate by ID
    Every resource must have ID
    

    View Slide

47. “Minimum provider: build & use”
    1. Run $ go build
    2. Create tf file for your provider
    3. Run terraform init
    

    View Slide

48. “How can we create diffs?”
    

    View Slide

49. “How can we create diffs?”
    Store in the tfstate by d.Set()
    

    View Slide

50. “Minimum provider: build & use”
    1. Run $ go build
    2. Create tf file for your provider
    3. Run terraform init
    

    View Slide

51. “Minimum provider: build & use”
    1. Run $ go build
    2. Create tf file for your provider
    3. Run terraform init
    
    

    View Slide

52. Whoever listens this presentation can
    1. Understand how the Terraform is using Terraform
    provider to manage resources
    2. Develop custom Terraform provider
    “Goal”
    

    View Slide

53. “Making it better”
    

    View Slide

54. “Validating attributes”
    Add ValidateFunc
    

    View Slide

55. “Sensitive data”
    Add Sensitive
    

    View Slide

56. “Releasing”
    Use goreleaser
    

    View Slide

57. “Thank you”
    

    View Slide