Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Creating a Terraform Provider for Just About Anything

Creating a Terraform Provider for Just About Anything

Terrafom is an amazing tool that lets you define your infrastructure as code. Under the hood it's an incredibly powerful state machine that makes API requests and marshals resources. In this talk we'll dive into the inner workings of Terraform and examine all the elements of a provider — from the documentation to the test suite. You'll walk away with the knowledge of how to contribute to an existing provider or create your own from scratch. We'll also take a look at some of the things we encountered while working on the DigitalOcean provider and the lessons learned from the community.

https://youtu.be/noxwUVet5RE

3f8f51e1aa14cdd87cffeaa26bb0ce13?s=128

Eddie Zaneski

October 23, 2018
Tweet

Transcript

  1. Creating a Terraform Provider for Just about Anything Eddie Zaneski

    @eddiezane @DigitalOcean
  2. Terraform is?

  3. Getty Images/Shutterstock/NASA; illustration by Dave Mosher/Business Insider https://www.businessinsider.com/elon-musk-mars-colony-details-new-space-study-2017-6

  4. Terraform is?

  5. Terraform is a state machine

  6. Terraform Provider?

  7. None
  8. provider "digitalocean" { token = "DIGITALOCEAN_API_TOKEN" } resource "digitalocean_droplet" "web"

    { name = "web" region = "sfo1" size = "s-1vcpu-1gb" image = "ubuntu-18-04-x64" }
  9. provider "kubernetes" { config_context_auth_info = "ops" config_context_cluster = "mycluster" }

    resource "kubernetes_service" "dope_web_server" { metadata { name = "dope_web_server" labels = { app = "web" } } spec { type = "LoadBalancer" selector { app = "${kubernetes_pod.web.metadata.0.labels.app}" } port { port = 8080 target_port = 80 } } }
  10. provider "github" { token = "${var.github_token}" organization = "${var.github_organization}" }

    resource "github_repository" "codez" { name = "uber-for-tacos" description = "My new startup" private = true }
  11. provider "googlecalendar" {} resource "googlecalendar_event" "talk" { summary = "This

    Terraform talk" location = "Gold Room" start = "2018-10-23T14:35:00-08:00" end = "2018-10-23T15:10:00-08:00" attendee { email = "eddiezane@digitalocean.com" } } @sethvargo https://www.hashicorp.com/blog/managing-google-calendar-with-terraform
  12. provider "hue" { token = "YOUR_HUE_TOKEN" bridge_ip = "192.168.1.137" }

    data "hue_light" "kitchen" { light_id = "1" } resource "hue_light_state" "kitchen" { light_id = "${data.hue_light.kitchen.light_id}" brightness = 42 red = 246 blue = 74 green = 138 }
  13. provider "todoist" { api_key = "TODOIST_API_KEY" } resource "todoist_task" "pack"

    { content = "Pack to move to Denver" due_string = "Friday" responsible_uid = "my_wife" completed = false } https://github.com/eddiezane/terraform-provider-todoist https://github.com/eddiezane/todoist-rest-go
  14. None
  15. Build a Provider for?

  16. Your Company ◦ You're a cloud provider ◦ Your customers

    ask for one ◦ You have a resource in an API that makes sense to manage with Terraform ◦ Skip a front end ◦ Don't need to write duct tape apps ◦ Internal
  17. terraform-provider-dointernal ◦ Very shallow shim/fork ◦ Additional features/fields ◦ Hypervisor

    placement ◦ Register into Chef Server ◦ Expose beta features
  18. Contribute to a Provider

  19. Contribute ◦ Fix a bug ◦ Add a feature ◦

    Hacktoberfest!
  20. Maintain a Fork ◦ Org specific defaults or needs ◦

    Missing features ◦ Unresponsive maintainer
  21. Convention

  22. Where to Start?

  23. Go API Client ◦ Separate your API logic from your

    provider logic ◦ Error handling ◦ Logging
  24. Skeleton/Guide package todoist import ( todoistRest "github.com/eddiezane/todoist-rest-go" "github.com/hashicorp/terraform/helper/schema" "github.com/hashicorp/terraform/terraform" )

    func Provider() terraform.ResourceProvider { https://www.terraform.io/docs/extend/writing-custom-providers.html
  25. Read other Providers

  26. return &schema.Provider{ Schema: map[string]*schema.Schema{ "api_key": &schema.Schema{ Type: schema.TypeString, Description: "Your

    Todoist API key", Required: true, DefaultFunc: schema.EnvDefaultFunc("TODOIST_API_KEY", nil), }, }, ConfigureFunc: configureFunc(), ... } Schema o Used in all the things o Type enforcement
  27. Types ◦ TypeBool ◦ TypeInt ◦ TypeFloat ◦ TypeString ◦

    TypeList ◦ TypeMap ◦ TypeSet
  28. "Terraform stdlib" ◦ https://github.com/hashicorp/terraform/tree/master/helper ◦ Read it! ◦ schema ◦

    Provider ◦ Resource ◦ ResourceData ◦ Validators ◦ IntBetween ◦ Testing ◦ RandString(strlen int)
  29. func configureFunc() func(*schema.ResourceData) (interface{}, error) { return func(d *schema.ResourceData) (interface{},

    error) { client := todoistRest.NewClient(d.Get("api_key").(string)) return client, nil } } ConfigureFunc (meta) o Return the meta
  30. ResourcesMap: map[string]*schema.Resource{ "todoist_task": resourceTodoistTask(), "todoist_project": resourceTodoistProject(), }, Resources o One

    file per resource o resource_todoist_task.go
  31. func resourceTodoistTask() *schema.Resource { return &schema.Resource{ Schema: map[string]*schema.Schema{ "content": &schema.Schema{

    Type: schema.TypeString, Required: true, }, "completed": &schema.Schema{ Type: schema.TypeBool, Optional: true, Default: false, }, }, Create: resourceTodoistTaskCreate, Read: resourceTodoistTaskRead, Update: resourceTodoistTaskUpdate, Delete: resourceTodoistTaskDelete, } }
  32. func resourceTodoistTaskCreate(d *schema.ResourceData, meta interface{}) error { client := meta.(*todoistRest.Client)

    content := d.Get("content").(string) newTask := &todoistRest.NewTask{ Content: content, } task, err := client.CreateTask(newTask) if err != nil { return err } d.SetId(task.Id) return resourceTodoistTaskRead(d, meta) }
  33. func resourceTodoistTaskRead(d *schema.ResourceData, meta interface{}) error { client := meta.(*todoistRest.Client)

    id := d.Id() task, err := client.GetTask(id) if err != nil { return err } ... d.Set("content", t.Content) d.Set("completed", t.Completed) return nil }
  34. func resourceTodoistTaskUpdate(d *schema.ResourceData, meta interface{}) error { client := meta.(*todoistRest.Client)

    id := d.Id() if d.HasChange("content") { content := d.Get("content").(string) err := client.UpdateTask(... } return nil }
  35. func resourceTodoistTaskDelete(d *schema.ResourceData, meta interface{}) error { client := meta.(*todoistRest.Client)

    id := d.Id() err := client.DeleteTask(id) if err != nil { return err } return nil }
  36. ◦ Extremely robust to failures and fault tolerant ◦ Log

    all the things! ◦ TF_LOG=INFO terraform plan ◦ Quickly identify what's your API's fault vs Terraform vs random ◦ Partial state If the Create callback returns with or without an error and an ID has been set, the resource is assumed created and all state is saved with it. Error Handling https://www.terraform.io/docs/extend/writing-custom-providers.html#error-handling-amp-partial-state
  37. Testing

  38. func TestAccTodoistTask_basic(t *testing.T) { var task todoistRest.Task content := "test

    task content" resource.Test(t, resource.TestCase{ PreCheck: func() { testAccPreCheck(t) }, Providers: testAccProviders, CheckDestroy: testAccCheckTodoistTaskDestroy, Steps: []resource.TestStep{ { Config: testAccCheckTodoistTaskConfig_basic(content), Check: resource.ComposeTestCheckFunc( testAccCheckTodoistTaskExists("todoist_task.test", &task), testAccCheckTodoistTaskAttributes(&task, content), resource.TestCheckResourceAttr("todoist_task.test", "content", content), ), }, }, }) }
  39. func testAccCheckTodoistTaskConfig_basic(content string) string { return fmt.Sprintf(` resource "todoist_task" "test"

    { content = "%s" } `, content) }
  40. ◦ Precheck > Test Steps > Destroy > CheckDestroy ◦

    One config per step ◦ Use steps to simulate updates and deletes ◦ Reuse as much as possible—especially early on ◦ Makefile ◦ make testacc TESTARGS='-run=TestAccDigitalOceanDomain_Basic' Testing
  41. Docs ◦ Great place to get started ◦ Magic `website`

    folder
  42. Process Notes ◦ Engage HashiCorp early ◦ Travis CI for

    CI/CD ◦ Releases via Slack ◦ TeamCity for releasing https://www.terraform.io/guides/terraform-provider-development-program.html
  43. Tips ◦ Learn a bit of Go first ◦ Get

    familiar with `strconv` ◦ Custom `UnmarshalJSON` functions ◦ Have a solid Go API client library ◦ Fix design problems with your API if possible ◦ Understand Terraform Modules ◦ Don't be afraid to copy/pasta ◦ Focus on composability and reusable functions ◦ Sweepers https://www.terraform.io/guides/terraform-provider-development-program.html
  44. Resources ◦ Other providers ◦ Terraform docs, guides, and source

    code ◦ Videos ◦ How to Extend the Terraform Provider List - Paul Stack ▪ https://youtu.be/2BvpqmFpchI ◦ Terraform and The Extensible Provider Architecture - Clint Shryock ▪ https://youtu.be/TMmovxyo5sY ◦ OpenCredo
  45. Thanks! @eddiezane @DigitalOcean https://do.co/hashiconf-2018-terraform