High Availability Vault Service on AWS Environment Gea-Suan Lin (DK) Director of SW Platform and Infrastructures

Links ● This slide: ○ https://bit.ly/3igUbgh ● AWS Summit Taiwan 2021: ○ https://aws.amazon.com/tw/events/taiwan/2021summit/ ● My wiki: ○ https://wiki.gslin.org/wiki/Vault/Install (in Chinese)

Explain “Migo” ● https://wiki.gslin.org/wiki/Migo

What is HashiCorp Vault? ● “Manage secrets and protect sensitive data” ● Usually: ○ Credentials ○ Tokens ○ … ● Sometimes: ○ Endpoint information ○ …

Why do people need Vault? ● Auditing. ● Credentials/tokens versioning. ● We don’t want to put credentials into Ansible and/or GitLab…

Today’s objectives ● High availability. ○ But I don’t want to manage HA by myself.

Technologies ● Amazon EC2 (Multi-AZ) ○ (or container-based services like ECS/EKS) ● Amazon DynamoDB ● AWS KMS ● AWS ELB ● AWS ACM (optional)

Setup DynamoDB ● Create a table called vault. ○ Primary key as Path. ○ Sort key as Key.

No content

Setup KMS ● Create a key with SYMMETRIC_DEFAULT.

No content

Setup EC2 ● Create two t3a.nano or t4g.nano instances. ● We choose Ubuntu 20.04.

No content

Install Vault curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -; sudo apt-add-repository "deb https://apt.releases.hashicorp.com $(lsb_release -cs) main"; sudo apt update && sudo apt install vault

Setup Vault api_addr = "http://10.10.10.10:8200" cluster_addr = "http://10.10.10.10:8201" log_level = "Info" ui = true listener "tcp" { address = "0.0.0.0:8200" cluster_address = "10.10.10.10:8201" tls_disable = "true" } seal "awskms" { region = "ap-southeast-1" access_key = "x" secret_key = "x" kms_key_id = "x" } storage "dynamodb" { ha_enabled = "true" region = "ap-southeast-1" table = "vault" access_key = "x" secret_key = "x" }

Setup EC2 IAM Role ● Create an EC2 role. ● Attach two inline policies. ● Attach to EC2 instances.

EC2 IAM Role - Policy-Vault-DynamoDB { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "dynamodb:BatchGetItem", "dynamodb:BatchWriteItem", "dynamodb:PutItem", "dynamodb:DescribeTable", "dynamodb:DeleteItem", "dynamodb:GetItem", "dynamodb:Scan", "dynamodb:ListTagsOfResource", "dynamodb:Query", "dynamodb:UpdateItem", "dynamodb:DescribeTimeToLive", "dynamodb:GetRecords" ], "Resource": [ "arn:aws:dynamodb:ap-southeast-1:123456789012:table/vault/stream/*", "arn:aws:dynamodb:ap-southeast-1:123456789012:table/vault/index/*", "arn:aws:dynamodb:ap-southeast-1:123456789012:table/vault" ] }, { "Sid": "VisualEditor1", "Effect": "Allow", "Action": [ "dynamodb:DescribeReservedCapacityOfferings", "dynamodb:ListTables", "dynamodb:DescribeReservedCapacity", "dynamodb:DescribeLimits" ], "Resource": "*" } ] }

EC2 IAM Role - Policy-Vault-KMS { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:Encrypt", "kms:DescribeKey" ], "Resource": "arn:aws:kms:ap-southeast-1:123456789012:key/01234567-89ab-cdef-0123-456789abcdef" } ] }

No content

Setup ELB ● Choose ALB ● /v1/sys/health as health check path. ● Backend in port 8200. ● Frontend in port 80. ○ Recommend to use ACM for HTTPS (port 443).

Start Vault sudo systemctl enable vault; sudo service vault start

Initialization # Remember to write down the root token vault operator init \ -recovery-shares=1 \ -recovery-threshold=1 \ -address=http://127.0.0.1:8200

Now it’s working ● http://vault.example.com/ ○ https://vault.example.com/ (HTTPS)

Monitoring ● Cloudwatch ○ ELB (ALB) ○ EC2 ○ DynamoDB ○ KMS

That’s it… ● Q&A after sessions. ● And we’re hiring!