Terraform at Honestbee - Speaker Deck

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Agenda 1. Who am I? 2. What is Honestbee? 3. Scaling Honestbee 4. Adoption of Terraform at Honestbee 5. Reproduce Environments using Workspaces 6. Empower engineers using Modules 7. Adopt Git-Ops with Atlantis

Slide 3

Slide 3 text

Who am I? Vincent De Smet DevOps Engineer @ Honestbee Singapore Co-organiser for Kubernetes, Cloud Native and GCP Singapore user groups in Singapore vincentdesmet so0k

Slide 4

Slide 4 text

What is Honestbee? Groceries, Food, Medical, Movie / Attraction Tickets, ...

Slide 5

Slide 5 text

What is Honestbee? … many more services

Slide 6

Slide 6 text

Scaling Honestbee

Slide 7

Slide 7 text

Initial Honestbee Set Up Born in the cloud (AWS)

Slide 8

Slide 8 text

Initial Honestbee Set Up - Small team, move fast - Monolithic code base Ruby on Rails - Platform as a Service AWS Elastic Beanstalk

Slide 9

Slide 9 text

Scaling your company in the cloud - Grow engineering headcount Monolithic code base - Stay agile - move towards - Loosely Coupled Services - Frequent / incremental updates - Elastic cloud resources - but… keep cloud resources and costs under control Reference: http://docker.com - Agile evolution

Slide 10

Slide 10 text

Scaling your company in the cloud How to adopt loosely coupled services … Immutable deployments, maximise resource utilisation, resilient architecture, ...

Slide 11

Slide 11 text

Scaling your company in the cloud Kubernetes on AWS in Singapore …. Not GA (yet)

Slide 12

Slide 12 text

Adoption of Terraform at Honestbee

Slide 13

Slide 13 text

Adopting Terraform Kubernetes (single-az) on AWS: https://engineers.sg/v/2225 https://speakerdeck.com/so0k/singapore-kubernetes-meetup-cluster-bootstrap

Slide 14

Slide 14 text

Adopting Terraform Hosted services on AWS:

Slide 15

Slide 15 text

Adopting Terraform High Availability:

Slide 16

Slide 16 text

Adopting Terraform Multiple / Repeatable environments: staging production

Slide 17

Slide 17 text

Adopting Terraform now iterate on and maintain this infrastructure … staging production

Slide 18

Slide 18 text

Adopting Terraform … and empower engineers to launch additional databases / caches / … staging production

Slide 19

Slide 19 text

Reproduce environments using Workspaces

Slide 20

Slide 20 text

Terraform config defines cloud resources (for example): Terraform Workspaces resource "aws_rds_cluster" "mysvc" { cluster_identifier = "mysvc-staging-db" ... } resource "aws_s3_bucket" "mysvc" { bucket = "mysvc-staging-bucket" acl = "private" ... } resource "aws_elasticache_cluster" "mysvc" { cluster_id = "mysvc-staging-cache" ... }

Slide 21

Slide 21 text

Terraform apply instantiates environments with state tracking changes: default Terraform Workspaces resource "aws_rds_cluster" "mysvc" { cluster_identifier = "mysvc-staging-db" ... } resource "aws_s3_bucket" "mysvc" { bucket = "mysvc-staging-bucket" acl = "private" ... } resource "aws_elasticache_cluster" "mysvc" { cluster_id = "mysvc-staging-cache" ... } state

Slide 22

Slide 22 text

Terraform workspaces allow us to instantiate multiple environments, keeping separate state: Terraform Workspaces resource "aws_rds_cluster" "mysvc" { cluster_identifier = "mysvc- ${terraform.workspace}-db" ... } resource "aws_s3_bucket" "mysvc" { bucket = "mysvc- ${terraform.workspace}-bucket" acl = "private" ... } resource "aws_elasticache_cluster" "mysvc" { cluster_id = "mysvc- ${terraform.workspace}-cache" ... } staging state prod state

Slide 23

Slide 23 text

Empower engineers using Modules

Slide 24

Slide 24 text

Modules Portable Terraform configurations (packages) Allow separation of concerns and responsibilities among teams Why? - Provide off-the-shelf functionality for Engineers - Enforce best practices and conventions for cloud infrastructure

Slide 25

Slide 25 text

Modules Spoiler Alert! Modules are just Terraform configurations inside a folder - there's nothing special about them.

Slide 26

Slide 26 text

$ tree my-module my-module └── main.tf Terminal

Slide 27

Slide 27 text

variable "ami_id" {} variable "instance_type" {} resource "aws_instance" "db" { # ... } output "address" { value = "${aws_instance.db.private_dns}" } my-instance-module/main.tf

Slide 28

Slide 28 text

module "special_instance" { source = "./my-instance-module" ami_id = "value" instance_type = "value" } main.tf

Slide 29

Slide 29 text

$ terraform init Downloading modules... Get: file:///home/training/terraform-workshop/example-module Initializing provider plugins... - Checking for available provider plugins on https://releases.hashicorp.com... - Downloading plugin for provider "aws" (1.2.0)... - Downloading plugin for provider "null" (1.0.0)... ... Terminal

Slide 30

Slide 30 text

Modules: Source Terraform supports module import from: ● Local file paths ● Git (GitHub / BitBucket / Generic Git ... ) ● HTTP URLs ● S3 buckets ● Terraform Registry

Slide 31

Slide 31 text

Module Source: Git Using git (ssh): source = "git@github.com:hashicorp/example.git//subdir" GitHub source URLs require that Git is installed on your system and that you have access to the repository

Slide 32

Slide 32 text

Module Source: Git Initial adoption: source = "git@github.com:hashicorp/example.git//subdir" Pros: - Get started using shared modules fast (fetch directly from git) Cons: - workflow becomes slow (full mono-repo is cloned per module import) - versioning requires the use of tags and careful release management

Slide 33

Slide 33 text

Module Source: HTTP Fetching modules over http: source = "https://modules.mycompany.com/module.tar.gz" Terraform will download and uncompress archives automatically

Slide 34

Slide 34 text

Module Source: HTTP Fetching modules over http: source = "https://modules.mycompany.com/module.tar.gz" Pros: - Modules are fetched much faster and take up less space Cons: - Requires proper CI/CD set up (linting / builds / file hosting) - Requires private network & VPN set up …

Slide 35

Slide 35 text

Module Source: HTTP Example CI / CD set up - using Drone.io & https://github.com/mbtproject/mbt

Slide 36

Slide 36 text

Module Source: HTTP Serving from s3 using jessfraz/s3server - https://github.com/honestbee/s3server

Slide 37

Slide 37 text

Module Source: GIT Fetching modules over HTTP vs GIT 60s

Slide 38

Slide 38 text

Module Source: HTTP Fetching modules over HTTP vs GIT 20s

Slide 39

Slide 39 text

Writing Modules: Balancing - big / generic modules VS small single purpose modules Note: HCL Conditionals are a HACK! this module provisions: - postgres instance - s3 bucket - iam policy - dns entries - ... postgres instance s3 bucket iam policy simple dns

Slide 40

Slide 40 text

Writing Modules: Balancing - individual variables VS config "maps" Note: HCL map support can be icky! variable "db_host" { } variable "db_user" { } variable "db_password" { } db_host = "" db_user = "" db_password = "" variable "db" { type = "map" } db = { host = "", user = "", password = "", }

Slide 41

Slide 41 text

Writing Modules: Balancing - Beware of lists / count(list) Note: Cascading state changes

Slide 42

Slide 42 text

Adopt Git-Ops with Atlantis

Slide 43

Slide 43 text

Atlantis Open source project originally out of Hootsuite https://www.runatlantis.io/ Enable engineers to run terraform directly from pull requests Why? - Better collaboration and visibility - Review and Approval Workflow - Engineers don't need full AWS Access

Slide 44

Slide 44 text

Demo Add cross cluster load balancing to http://apse1a-devops-sg-demo.honestbee.com http://apse1b-devops-sg-demo.honestbee.com

Slide 45

Slide 45 text

Demo We want http://devops-sg-demo.honestbee.com to use: - Weighted Route53 Records across apse1a and apse1b AZs - Route53 HealthChecks for automated failover on AZ failure https://github.com/honestbee/devops-singapore-example/pull/4/files

Slide 46

Slide 46 text

Conclusion Today: Terraform configurations essential for hosted services not managed by Kubernetes Long term: migrate towards control loops and declarative resource manifests: See https://kubernetes.io/docs/concepts/service-catalog/ Terraform will always remain as a way to define our core cloud infrastructure.