HashiCorp Tools Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 1

HashiCorp Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 2

HashiCorp 1 • ։ൃɾӡ༻ʹޮՌతͳπʔϧΛ OSS Ͱల։͢Δձࣾ • ૑ઃऀ (2012 ೥૑ઃ) • Mitchell Hashimoto, Armon Dadgar • ୅දతͳπʔϧ • Vagrant, Packer, Serf, Consul, Terraform, Vault 1 https://hashicorp.com/ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 3

Tools • Vagrant • Packer • Serf • Consul • Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 4

Tools • Vagrant • Packer • Serf • Consul • Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 5

Vagrant Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 6

Vagrant • Ծ૝Խιϑτ΢ΣΞͷίϚϯυϥΠϯϥούʔ • جຊతʹ͸ VirtualBox ͱڞʹ༻͍ΒΕΔ͜ͱ͕ଟ͍ • Provider ͱͯ͠ VMware ΍ AWS ΍ KVM ౳΋બ΂Δ • ݸʑਓͷ(։ൃ|ݕূ)؀ڥͷηοτΞοϓʹΑ͘༻͍ΒΕΔ • Vagrantfile ʹԾ૝Ϛγϯͷߏ੒΍ϓϩϏδϣχϯάΛॻ͚Δ • ෳ਺୆ͷωοτϫʔΫߏ੒΍ CPU/Memory ·ͰؚΊͯશͯ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 7

Vagrantfile example Vagrant.configure(2) do |config| config.vm.define :web do |web| web.vm.box = "centos64" web.vm.network :forwarded_port, guest: 80, host: 8080 web.vm.network :private_network, ip: "192.0.2.1" web.vm.provision :shell, :inline => "yum -y install httpd" end config.vm.define :db do |db| db.vm.box = "centos64" db.vm.network :private_network, ip: "192.0.2.2" end end Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 8

Vagrant • ར༻ऀ͸ vagrant up, vagrant ssh ౳Λ࣮ߦ͢Δ͚ͩ • ઌఔͷྫͩͱ 2 ୆্ཱ͕ͪΔ • Ծ૝ϚγϯͷϕʔεΠϝʔδ͕ར༻Ͱ͖Δ(Box) • Πϯλʔωοτӽ͠ʹڞ༗ͨ͠Γ΋ग़དྷΔ 2 • ϛυϧ΢ΣΞ౳͕ "͙͢ࢼͤΔ" Box ͕ެ։͞Ε͍ͯͯศར • ։ൃ؀ڥ΍ςετ؀ڥΛ༻ҙ͢Δࡍ͸ੵۃతʹ࢖͏ͱྑ͍ 2 http://www.vagrantbox.es/ , https://atlas.hashicorp.com/boxes/search . Box ͕৴པͰ͖Δ͔ͳͲ͸஫ҙ͕ඞཁ. Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 9

Tools • Vagrant • Packer • Serf • Consul • Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 10

Packer Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 11

Packer • Ծ૝ϚγϯͷςϯϓϨʔτΛ࡞੒͢Δҝͷπʔϧ • VirtualBox, VMware, Amazon EC2, Docker • packer build -var-file=var.json config.json • ઃఆϑΝΠϧΛॻ͍࣮ͯߦ͢Ε͹ςϯϓϨʔτ͕࡞੒͞ΕΔ • Ұ౓ͷϏϧυͰ AMI ͱ VirtualBox ͷςϯϓϨʔτΛ࡞Δ౳΋Մೳ • ΰʔϧσϯΠϝʔδͷ࡞੒,ෳ਺ͷج൫޲͚ͷΠϝʔδ࡞੒͕؆୯ʹ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 12

Packer config exmaple { "variables": {"aws_access_key": "", "aws_secret_key": ""}, "builders": [{ "type": "amazon-ebs", "access_key": "{{user `aws_access_key`}}", "secret_key": "{{user `aws_secret_key`}}", "region": "ap-northeast-1", "source_ami": "ami-test-12345", "instance_type": "t2.small", "ami_name": "Web_{{isotime | clean_ami_name}}", "ssh_username": "ec2-user", "ssh_timeout": "5m" }], "provisioners": [ {"type": "shell", "inline": [ "sudo yum -y install python-setuptools", "sudo easy_install pip", "pip install ansible" ]}, {"type": "ansible-local", "playbook_file": "playbook.yml"} ] } Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 13

Tools • Vagrant • Packer • Serf • Consul • Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 14

Serf Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 15

Serf • αʔϏεσΟεΧόϦ/ΦʔέετϨʔγϣϯͷҝͷπʔϧ • ෳ਺αʔόͰΫϥελΛܗ੒͠Πϕϯτ఻೻Λߦ͏ • ൃՐͨ͠ΠϕϯτຖʹίϚϯυ౳Λఆٛग़དྷΔ • ϗετ͕Ϋϥελʹ join ͨ͠Β xx Λ࣮ߦ... • Ϣʔβ೚ҙͷΠϕϯτΛൃՐͤ͞Δ͜ͱ΋Մೳ • Ϋϥελ΍Πϕϯτͷ؅ཧ͸ Gossip ͱ͍͏ϓϩτίϧΛ࢖༻͍ͯ͠Δ • ֤ϊʔυ͔ΒͷϨεϙϯεΛड͚औΔΫΤϦͱ͍͏ػೳ΋͋Δ • Φʔτεέʔϧͷ؀ڥ΍େن໛ΦϖϨʔγϣϯ౳͕ඞཁͳ؀ڥͰ༗༻ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 16

Serf Cluster & Event Handler (at n1) [user@n1] $ serf agent -node=node1 -bind=192.0.2.1 -log-level=debug \ -event-handler -member-join="echo member joined!" ==> Starting Serf agent... : --- (at n2) [user@n2] $ serf agent -node=node2 -bind=192.0.2.2 -join=192.0.2.1 --- (at n1) 2015/05/25 14:15:01 [INFO] serf: EventMemberJoin: node2 192.0.2.2 2015/05/25 14:15:01 [DEBUG] agent: Event 'member-join' script output: member joined! [user@n1] $ serf members node1 192.0.2.1:7946 alive node2 192.0.2.2:7946 alive Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 17

Serf Query [user@n1] $ serf agent -node=node1 -bind=192.0.2.1 \ -event-handler query:uptime=uptime [user@n2] $ serf agent -node=node2 -bind=192.0.2.2 \ -event-handler query:uptime=uptime \ -join=192.0.2.1 --- [user@n2] $ serf query uptime Query 'uptime' dispatched Ack from 'node1' Response from 'node1': 15:29:29 up 23 days, 6:27, 2 users, load average: 0.13, 0.25, 0.30 Ack from 'node2' Response from 'node2': 15:29:29 up 15 days, 6:27, 1 users, load average: 0.01, 0.02, 0.02 Total Acks: 1 Total Responses: 1 Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 18

Tools • Vagrant • Packer • Serf • Consul • Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 19

Consul • Serf ͱಉ͘͡αʔϏεσΟεΧόϦ/ΦʔέετϨʔγϣϯͷҝͷπʔϧ • Serf ͱͷେ͖ͳҧ͍ • Serf ΑΓ Consistency ʹدͬͨπʔϧ(Consul: CP دΓ, Serf: AP دΓ) • Raft ͱ͍͏߹ҙϓϩτίϧΛ༻͍ͯ Consistency Λอূ • Key-Value Store ػೳ͕෇͍͍ͯΔ (HTTP API ܦ༝Ͱૢ࡞͢Δ) • σʔληϯλɾαʔϏε/λά౳ͷ֓೦͕͋Δ(ෳ਺ͷϊʔυΛଋͶΔ΋ͷ) • Health Check ػೳ͕෇͍͍ͯΔ • DNS ΠϯλϑΣʔε ͕෇͍͍ͯΔ • WebUI ͕෇͍͍ͯΔ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 20

Consul Health Check • HTTP POST Ͱొ࿥Ͱ͖Δ(΋ͪΖΜઃఆϑΝΠϧͰ΋) • health check ͕ࣦഊͨ͠ΒಛఆͷΠϕϯτ͕ൃՐ͢Δ • health check script ͷ࢓༷͸ Nagios Plugin ޓ׵ curl -vvv -X PUT \ -d ' { "ID": "check_swap", "Name": "Swap Utilization", "Notes": "Check swap space on local machine.", "Script": "/usr/lib64/nagios/plugins/check_swap -w 80% -c 30%", "Interval": "10s" } ' http://localhost:8500/v1/agent/check/register Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 21

Consul DNS Interface • DCɾϊʔυɾαʔϏεɾλά౳ͷ୯ҐͰ DNS ϨίʔυΛҾ͚Δ • dig @127.0.0.1 -p 8600 foo.node.consul ANY • foo ͱ͍͏ node ͷ IP ͕ฦͬͯ͘Δ • dig @127.0.0.1 -p 8600 redis.service.dc1.consul. ANY • dc1 ʹ͋Δ redis αʔϏεʹొ࿥͞Ε͍ͯΔϊʔυͷ IP શͯ • લड़ͷ Health Check ౳ͱซͤͯ৑௕ԽͷػߏΛ࣮૷Ͱ͖Δ • Bind ౳ͷϛυϧ΢ΣΞͱ૊Έ߹ΘͤΔࣄ΋Մೳ (DNS Forwarding) Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 22

Tools • Vagrant • Packer • Serf • Consul • Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 23

Terraform Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 24

Terraform • Ϋϥ΢υαʔϏε౳ͷԾ૝ϚγϯࣗମͷઃఆΛ؅ཧ͢Δ • ΠϯελϯελΠϓ, Ϧʔδϣϯ, IP, ϕʔεΠϝʔδ, ACL ... • AWS, DigitalOcean, GCE, Heroku, CloudFlare ... • ઃఆϑΝΠϧΛॻ͖ terraform apply ͢Ε͹ద༻͞ΕΔ • terraform plan Ͱมߋ಺༰ΛݟΔ͜ͱ͕ग़དྷΔ • Ұ౓࣮ߦͨ͠Β State Λอଘ͢ΔͨΊɺطଘͷΠϝʔδͷઃఆมߋ΋ग़དྷΔ • ઃఆϑΝΠϧΛมߋͯ͠ terraform plan, terraform apply • terraform destroy ͰઃఆϑΝΠϧͷ಺༰Λഁغ͢Δ͜ͱ΋ग़དྷΔ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 25

Terraform config example provider "aws" { region = "us-east-1" } resource "aws_elb" "web" { name = "terraform-example-elb" availability_zones = ["${aws_instance.web.*.availability_zone}"] listener { instance_port = 80 instance_protocol = "http" lb_port = 80 lb_protocol = "http" } instances = ["${aws_instance.web.*.id}"] } resource "aws_instance" "web" { count = 4 ami = "ami-12345" instance_type = "t1.micro" } Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 26

Tools • Vagrant • Packer • Serf • Consul • Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 27

Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 28

Vault • σʔλ҉߸Խɾػີ৘ใ؅ཧπʔϧ • Secret/Auth/Audit ͷػೳΛఏڙ͢Δ • Secret • ฏจΛ҉߸Խͯ͠؅ཧ͢Δ(AES-GCM 256bit Ͱ҉߸Խ͞ΕΔ) • AWS/MySQL/Postgres ౳ͷϢʔβ৘ใΛ؅ཧ͢Δ(ૢ࡞ݖݶ΍ظݶΛ෇͚ͨΓग़དྷΔ) • Auth • Secret ͷಡΈग़͠Λ؅ཧ͢Δ • Github ΍ LDAP ͱ࿈ܞͯ͠ Token ΛൃߦͰ͖Δ(ಛఆͷ૊৫ʹॴଐ͍ͯͨ͠Β Token ൃߦͳͲ) • Audit • ؂ࠪϩάΛϑΝΠϧ΍ syslog ʹग़ྗͰ͖Δ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 29

Vault example [server: 192.0.2.1] $ vault server $ vault init # ͜͜Ͱग़ྗ͞ΕΔ Key ΍ Token Λ߇͓͑ͯ͘ $ vault token-create policy="root" # ͜͜Ͱग़ྗ͞ΕΔ Token Λ Client ʹ༩͑Δ --- [client] $ export VAULT_ADDR="https://192.0.2.1:8200" $ vault auth ${CLIENT_TOKEN} $ vault write secret/name foo=buz $ vault read -format=json secret/name | grep -A2 data "data": { "foo": "buz" } Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 30

·ͱΊ • Hashicorp ͷπʔϧʹ͍ͭͯ֓ཁ(ຊ౰ʹͬ͘͟Γ)આ໌ • Vagrant, Packer, Serf, Consul, Terraform, Vault • ։ൃ؀ڥ΍ӡ༻Λշదʹग़དྷΔπʔϧ(ͱ๻͸ࢥ͍ͬͯ·͢) • ͥͻ৭ʑࢼͯ͠ΈͯԼ͍͞(Θ͔Βͳ͚Ε͹ฉ͍ͯԼ͍͞) • ಛʹ࣍ͷΑ͏ͳҊ݅Λ୲౰͞ΕͯΔํʹ͸ΦεεϝͰ͢ • ୆਺͕ଟ͍Ҋ݅΍૿ݮ͕ܹ͍͠Ҋ݅ • AWS ΍ GCE ౳ͷ֤छΫϥ΢υΛ࢖͍ͬͯΔҊ݅ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 31