Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
hashicorp-tools-overview
Search
rrreeeyyy
May 25, 2015
Technology
2
1.9k
hashicorp-tools-overview
プロダクト勉強会というものがあって hashicorp のツールそれぞれの概要を説明した時の資料です
rrreeeyyy
May 25, 2015
Tweet
Share
More Decks by rrreeeyyy
See All by rrreeeyyy
Incident Response Practices: Waroom's Features and Future Challenges
rrreeeyyy
0
200
An Efficient Incident Response Training with AI / SRE NEXT 2024 Sponsor Session
rrreeeyyy
1
4.2k
カンファレンスから見る SRE トレンド 2024 / SRE Trends from Conferences in 2024 #SRE_Findy
rrreeeyyy
4
2.3k
信頼性の育て方 / mackerel-meetup-15
rrreeeyyy
10
2.5k
SRE の歩き方・進め方 / sre-walk-through-procedure
rrreeeyyy
0
8.6k
「信頼性」を保ちつつ大規模サービスをリニューアルする / cookpad-tech-kitchen-service-embedded-sres
rrreeeyyy
11
12k
Cookpad and Prometheus
rrreeeyyy
6
20k
SRE-Lounge-8-Cookpad-Microservice-Architecture-Overview
rrreeeyyy
5
5.4k
A survey of anomaly detection methodologies for web system
rrreeeyyy
5
1.3k
Other Decks in Technology
See All in Technology
Helm , Kustomize に代わる !? 次世代 k8s パッケージマネージャー Glasskube 入門 / glasskube-entry
parupappa2929
0
290
ウォンテッドリーのデータパイプラインを支える ETL のための analytics, rds-exporter / analytics, rds-exporter for ETL to support Wantedly's data pipeline
unblee
0
110
データマネジメントのトレードオフに立ち向かう
ikkimiyazaki
6
1.2k
Apache Iceberg Case Study in LY Corporation
lycorptech_jp
PRO
0
260
Iceberg Meetup Japan #1 : Iceberg and Databricks
databricksjapan
0
290
【内製開発Summit 2025】イオンスマートテクノロジーの内製化組織の作り方/In-house-development-summit-AST
aeonpeople
1
500
Visualize, Visualize, Visualize and rclone
tomoaki0705
9
76k
次世代KYC活動報告 / 20250219-BizDay17-KYC-nextgen
oidfj
0
460
MIMEと文字コードの闇
hirachan
2
1.4k
【5分でわかる】セーフィー エンジニア向け会社紹介
safie_recruit
0
18k
OpenID BizDay#17 KYC WG活動報告(法人) / 20250219-BizDay17-KYC-legalidentity
oidfj
0
450
Reading Code Is Harder Than Writing It
trishagee
2
120
Featured
See All Featured
The Cult of Friendly URLs
andyhume
78
6.2k
Scaling GitHub
holman
459
140k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
A better future with KSS
kneath
238
17k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Raft: Consensus for Rubyists
vanstee
137
6.8k
We Have a Design System, Now What?
morganepeng
51
7.4k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
45
9.4k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
666
120k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
4
430
The Illustrated Children's Guide to Kubernetes
chrisshort
48
49k
YesSQL, Process and Tooling at Scale
rocio
172
14k
Transcript
HashiCorp Tools Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota (
@rrreeeyyy ) 1
HashiCorp Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 2
HashiCorp 1 • ։ൃɾӡ༻ʹޮՌతͳπʔϧΛ OSS Ͱల։͢Δձࣾ • ઃऀ (2012 ઃ)
• Mitchell Hashimoto, Armon Dadgar • දతͳπʔϧ • Vagrant, Packer, Serf, Consul, Terraform, Vault 1 https://hashicorp.com/ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 3
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 4
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 5
Vagrant Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 6
Vagrant • ԾԽιϑτΣΞͷίϚϯυϥΠϯϥούʔ • جຊతʹ VirtualBox ͱڞʹ༻͍ΒΕΔ͜ͱ͕ଟ͍ • Provider ͱͯ͠
VMware AWS KVM બΔ • ݸʑਓͷ(։ൃ|ݕূ)ڥͷηοτΞοϓʹΑ͘༻͍ΒΕΔ • Vagrantfile ʹԾϚγϯͷߏϓϩϏδϣχϯάΛॻ͚Δ • ෳͷωοτϫʔΫߏ CPU/Memory ·ͰؚΊͯશͯ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 7
Vagrantfile example Vagrant.configure(2) do |config| config.vm.define :web do |web| web.vm.box
= "centos64" web.vm.network :forwarded_port, guest: 80, host: 8080 web.vm.network :private_network, ip: "192.0.2.1" web.vm.provision :shell, :inline => "yum -y install httpd" end config.vm.define :db do |db| db.vm.box = "centos64" db.vm.network :private_network, ip: "192.0.2.2" end end Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 8
Vagrant • ར༻ऀ vagrant up, vagrant ssh Λ࣮ߦ͢Δ͚ͩ • ઌఔͷྫͩͱ
2 ্ཱ͕ͪΔ • ԾϚγϯͷϕʔεΠϝʔδ͕ར༻Ͱ͖Δ(Box) • Πϯλʔωοτӽ͠ʹڞ༗ͨ͠Γग़དྷΔ 2 • ϛυϧΣΞ͕ "͙͢ࢼͤΔ" Box ͕ެ։͞Ε͍ͯͯศར • ։ൃڥςετڥΛ༻ҙ͢Δࡍੵۃతʹ͏ͱྑ͍ 2 http://www.vagrantbox.es/ , https://atlas.hashicorp.com/boxes/search . Box ͕৴པͰ͖Δ͔ͳͲҙ͕ඞཁ. Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 9
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 10
Packer Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 11
Packer • ԾϚγϯͷςϯϓϨʔτΛ࡞͢Δҝͷπʔϧ • VirtualBox, VMware, Amazon EC2, Docker •
packer build -var-file=var.json config.json • ઃఆϑΝΠϧΛॻ͍࣮ͯߦ͢ΕςϯϓϨʔτ͕࡞͞ΕΔ • ҰͷϏϧυͰ AMI ͱ VirtualBox ͷςϯϓϨʔτΛ࡞ΔՄೳ • ΰʔϧσϯΠϝʔδͷ࡞,ෳͷج൫͚ͷΠϝʔδ࡞͕؆୯ʹ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 12
Packer config exmaple { "variables": {"aws_access_key": "", "aws_secret_key": ""}, "builders":
[{ "type": "amazon-ebs", "access_key": "{{user `aws_access_key`}}", "secret_key": "{{user `aws_secret_key`}}", "region": "ap-northeast-1", "source_ami": "ami-test-12345", "instance_type": "t2.small", "ami_name": "Web_{{isotime | clean_ami_name}}", "ssh_username": "ec2-user", "ssh_timeout": "5m" }], "provisioners": [ {"type": "shell", "inline": [ "sudo yum -y install python-setuptools", "sudo easy_install pip", "pip install ansible" ]}, {"type": "ansible-local", "playbook_file": "playbook.yml"} ] } Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 13
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 14
Serf Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 15
Serf • αʔϏεσΟεΧόϦ/ΦʔέετϨʔγϣϯͷҝͷπʔϧ • ෳαʔόͰΫϥελΛܗ͠ΠϕϯτΛߦ͏ • ൃՐͨ͠ΠϕϯτຖʹίϚϯυΛఆٛग़དྷΔ • ϗετ͕Ϋϥελʹ join
ͨ͠Β xx Λ࣮ߦ... • ϢʔβҙͷΠϕϯτΛൃՐͤ͞Δ͜ͱՄೳ • ΫϥελΠϕϯτͷཧ Gossip ͱ͍͏ϓϩτίϧΛ༻͍ͯ͠Δ • ֤ϊʔυ͔ΒͷϨεϙϯεΛड͚औΔΫΤϦͱ͍͏ػೳ͋Δ • ΦʔτεέʔϧͷڥେنΦϖϨʔγϣϯ͕ඞཁͳڥͰ༗༻ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 16
Serf Cluster & Event Handler (at n1) [user@n1] $ serf
agent -node=node1 -bind=192.0.2.1 -log-level=debug \ -event-handler -member-join="echo member joined!" ==> Starting Serf agent... : --- (at n2) [user@n2] $ serf agent -node=node2 -bind=192.0.2.2 -join=192.0.2.1 --- (at n1) 2015/05/25 14:15:01 [INFO] serf: EventMemberJoin: node2 192.0.2.2 2015/05/25 14:15:01 [DEBUG] agent: Event 'member-join' script output: member joined! [user@n1] $ serf members node1 192.0.2.1:7946 alive node2 192.0.2.2:7946 alive Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 17
Serf Query [user@n1] $ serf agent -node=node1 -bind=192.0.2.1 \ -event-handler
query:uptime=uptime [user@n2] $ serf agent -node=node2 -bind=192.0.2.2 \ -event-handler query:uptime=uptime \ -join=192.0.2.1 --- [user@n2] $ serf query uptime Query 'uptime' dispatched Ack from 'node1' Response from 'node1': 15:29:29 up 23 days, 6:27, 2 users, load average: 0.13, 0.25, 0.30 Ack from 'node2' Response from 'node2': 15:29:29 up 15 days, 6:27, 1 users, load average: 0.01, 0.02, 0.02 Total Acks: 1 Total Responses: 1 Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 18
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 19
Consul • Serf ͱಉ͘͡αʔϏεσΟεΧόϦ/ΦʔέετϨʔγϣϯͷҝͷπʔϧ • Serf ͱͷେ͖ͳҧ͍ • Serf ΑΓ
Consistency ʹدͬͨπʔϧ(Consul: CP دΓ, Serf: AP دΓ) • Raft ͱ͍͏߹ҙϓϩτίϧΛ༻͍ͯ Consistency Λอূ • Key-Value Store ػೳ͕͍͍ͯΔ (HTTP API ܦ༝Ͱૢ࡞͢Δ) • σʔληϯλɾαʔϏε/λάͷ֓೦͕͋Δ(ෳͷϊʔυΛଋͶΔͷ) • Health Check ػೳ͕͍͍ͯΔ • DNS ΠϯλϑΣʔε ͕͍͍ͯΔ • WebUI ͕͍͍ͯΔ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 20
Consul Health Check • HTTP POST ͰొͰ͖Δ(ͪΖΜઃఆϑΝΠϧͰ) • health check
͕ࣦഊͨ͠ΒಛఆͷΠϕϯτ͕ൃՐ͢Δ • health check script ͷ༷ Nagios Plugin ޓ curl -vvv -X PUT \ -d ' { "ID": "check_swap", "Name": "Swap Utilization", "Notes": "Check swap space on local machine.", "Script": "/usr/lib64/nagios/plugins/check_swap -w 80% -c 30%", "Interval": "10s" } ' http://localhost:8500/v1/agent/check/register Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 21
Consul DNS Interface • DCɾϊʔυɾαʔϏεɾλάͷ୯ҐͰ DNS ϨίʔυΛҾ͚Δ • dig @127.0.0.1
-p 8600 foo.node.consul ANY • foo ͱ͍͏ node ͷ IP ͕ฦͬͯ͘Δ • dig @127.0.0.1 -p 8600 redis.service.dc1.consul. ANY • dc1 ʹ͋Δ redis αʔϏεʹొ͞Ε͍ͯΔϊʔυͷ IP શͯ • લड़ͷ Health Check ͱซͤͯԽͷػߏΛ࣮Ͱ͖Δ • Bind ͷϛυϧΣΞͱΈ߹ΘͤΔࣄՄೳ (DNS Forwarding) Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 22
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 23
Terraform Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 24
Terraform • ΫϥυαʔϏεͷԾϚγϯࣗମͷઃఆΛཧ͢Δ • ΠϯελϯελΠϓ, Ϧʔδϣϯ, IP, ϕʔεΠϝʔδ, ACL ...
• AWS, DigitalOcean, GCE, Heroku, CloudFlare ... • ઃఆϑΝΠϧΛॻ͖ terraform apply ͢Εద༻͞ΕΔ • terraform plan Ͱมߋ༰ΛݟΔ͜ͱ͕ग़དྷΔ • Ұ࣮ߦͨ͠Β State Λอଘ͢ΔͨΊɺطଘͷΠϝʔδͷઃఆมߋग़དྷΔ • ઃఆϑΝΠϧΛมߋͯ͠ terraform plan, terraform apply • terraform destroy ͰઃఆϑΝΠϧͷ༰Λഁغ͢Δ͜ͱग़དྷΔ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 25
Terraform config example provider "aws" { region = "us-east-1" }
resource "aws_elb" "web" { name = "terraform-example-elb" availability_zones = ["${aws_instance.web.*.availability_zone}"] listener { instance_port = 80 instance_protocol = "http" lb_port = 80 lb_protocol = "http" } instances = ["${aws_instance.web.*.id}"] } resource "aws_instance" "web" { count = 4 ami = "ami-12345" instance_type = "t1.micro" } Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 26
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 27
Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 28
Vault • σʔλ҉߸Խɾػີใཧπʔϧ • Secret/Auth/Audit ͷػೳΛఏڙ͢Δ • Secret • ฏจΛ҉߸Խͯ͠ཧ͢Δ(AES-GCM
256bit Ͱ҉߸Խ͞ΕΔ) • AWS/MySQL/Postgres ͷϢʔβใΛཧ͢Δ(ૢ࡞ݖݶظݶΛ͚ͨΓग़དྷΔ) • Auth • Secret ͷಡΈग़͠Λཧ͢Δ • Github LDAP ͱ࿈ܞͯ͠ Token ΛൃߦͰ͖Δ(ಛఆͷ৫ʹॴଐ͍ͯͨ͠Β Token ൃߦͳͲ) • Audit • ࠪϩάΛϑΝΠϧ syslog ʹग़ྗͰ͖Δ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 29
Vault example [server: 192.0.2.1] $ vault server $ vault init
# ͜͜Ͱग़ྗ͞ΕΔ Key Token Λ߇͓͑ͯ͘ $ vault token-create policy="root" # ͜͜Ͱग़ྗ͞ΕΔ Token Λ Client ʹ༩͑Δ --- [client] $ export VAULT_ADDR="https://192.0.2.1:8200" $ vault auth ${CLIENT_TOKEN} $ vault write secret/name foo=buz $ vault read -format=json secret/name | grep -A2 data "data": { "foo": "buz" } Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 30
·ͱΊ • Hashicorp ͷπʔϧʹ͍ͭͯ֓ཁ(ຊʹͬ͘͟Γ)આ໌ • Vagrant, Packer, Serf, Consul, Terraform,
Vault • ։ൃڥӡ༻Λշదʹग़དྷΔπʔϧ(ͱࢥ͍ͬͯ·͢) • ͥͻ৭ʑࢼͯ͠ΈͯԼ͍͞(Θ͔Βͳ͚Εฉ͍ͯԼ͍͞) • ಛʹ࣍ͷΑ͏ͳҊ݅Λ୲͞ΕͯΔํʹΦεεϝͰ͢ • ͕ଟ͍Ҋ݅૿ݮ͕ܹ͍͠Ҋ݅ • AWS GCE ͷ֤छΫϥυΛ͍ͬͯΔҊ݅ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 31