Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
hashicorp-tools-overview
Search
rrreeeyyy
May 25, 2015
Technology
2
1.8k
hashicorp-tools-overview
プロダクト勉強会というものがあって hashicorp のツールそれぞれの概要を説明した時の資料です
rrreeeyyy
May 25, 2015
Tweet
Share
More Decks by rrreeeyyy
See All by rrreeeyyy
カンファレンスから見る SRE トレンド 2024 / SRE Trends from Conferences in 2024 #SRE_Findy
rrreeeyyy
3
1.7k
信頼性の育て方 / mackerel-meetup-15
rrreeeyyy
9
2k
SRE の歩き方・進め方 / sre-walk-through-procedure
rrreeeyyy
0
8.3k
「信頼性」を保ちつつ大規模サービスをリニューアルする / cookpad-tech-kitchen-service-embedded-sres
rrreeeyyy
11
11k
Cookpad and Prometheus
rrreeeyyy
6
20k
SRE-Lounge-8-Cookpad-Microservice-Architecture-Overview
rrreeeyyy
5
5k
A survey of anomaly detection methodologies for web system
rrreeeyyy
5
1.1k
エンジニアリングをちゃんとやる あるいは 人類の平和 について / wsa02-rrreeeyyy
rrreeeyyy
13
2.8k
「自立」したWebシステムを創る。自分の好きなことをする世界を目指して。/ ipsj-one-2018-rrreeeyyy
rrreeeyyy
3
1.7k
Other Decks in Technology
See All in Technology
The CloudCompare project by Dr. Daniel Girardeau-Montaut
kentaitakura
0
480
【SORACOM UG】(2024年度版) SIMってなんだ? ~セルラー通信がつながる仕組み、解説します~
soracom
PRO
0
120
プロダクト開発における ソフトウェアサプライチェーンセキュリティ: 実践的フレームワークとその活用 / Software Supply Chain Security in Product Development: Practical Framework and their applications
nttcom
1
310
反実仮想機械学習とは何か
usaito
PRO
3
450
クラウドサインにおけるプロダクトマネージャーの役割と開発プロセス / 20240410_cloudsign-PdM
bengo4com
1
660
メールセキュリティの DMARC周りの設定を少しずつ設定してみたお話
miu_crescent
1
260
ユーザーストーリーのレビューを自動化したみたの
bun913
1
230
ビジネスとエンジニアリングの接合点 そしてコード品質がそこに及ぼす影響 v1.1 / The Intersections of Business and Engineering, and The Impact of Code Quality There (v1.1)
mtx2s
11
2.4k
Let's get started with Ruby && Rails Tips
sinsoku
0
190
AWS を使う上で知っておきたいオンプレミス知識/aws-on-premise-essentials
emiki
1
4.1k
TiUG #1 HBaseからTiDBへの移行を選んだ理由
watanori
0
140
ログラスにおけるコード品質でビジネスに貢献する仕組み・カルチャー / A system and culture that contributes to business through code quality in Loglass
yoshikiiida
11
1.6k
Featured
See All Featured
Bash Introduction
62gerente
604
210k
Building an army of robots
kneath
300
41k
What's new in Ruby 2.0
geeforr
336
31k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
272
12k
Practical Orchestrator
shlominoach
180
9.7k
The Straight Up "How To Draw Better" Workshop
denniskardys
227
130k
Building a Modern Day E-commerce SEO Strategy
aleyda
16
6.3k
Being A Developer After 40
akosma
56
580k
Typedesign – Prime Four
hannesfritz
36
2k
The World Runs on Bad Software
bkeepers
PRO
61
6.7k
Raft: Consensus for Rubyists
vanstee
130
6.2k
Building Adaptive Systems
keathley
29
1.8k
Transcript
HashiCorp Tools Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota (
@rrreeeyyy ) 1
HashiCorp Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 2
HashiCorp 1 • ։ൃɾӡ༻ʹޮՌతͳπʔϧΛ OSS Ͱల։͢Δձࣾ • ઃऀ (2012 ઃ)
• Mitchell Hashimoto, Armon Dadgar • දతͳπʔϧ • Vagrant, Packer, Serf, Consul, Terraform, Vault 1 https://hashicorp.com/ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 3
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 4
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 5
Vagrant Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 6
Vagrant • ԾԽιϑτΣΞͷίϚϯυϥΠϯϥούʔ • جຊతʹ VirtualBox ͱڞʹ༻͍ΒΕΔ͜ͱ͕ଟ͍ • Provider ͱͯ͠
VMware AWS KVM બΔ • ݸʑਓͷ(։ൃ|ݕূ)ڥͷηοτΞοϓʹΑ͘༻͍ΒΕΔ • Vagrantfile ʹԾϚγϯͷߏϓϩϏδϣχϯάΛॻ͚Δ • ෳͷωοτϫʔΫߏ CPU/Memory ·ͰؚΊͯશͯ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 7
Vagrantfile example Vagrant.configure(2) do |config| config.vm.define :web do |web| web.vm.box
= "centos64" web.vm.network :forwarded_port, guest: 80, host: 8080 web.vm.network :private_network, ip: "192.0.2.1" web.vm.provision :shell, :inline => "yum -y install httpd" end config.vm.define :db do |db| db.vm.box = "centos64" db.vm.network :private_network, ip: "192.0.2.2" end end Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 8
Vagrant • ར༻ऀ vagrant up, vagrant ssh Λ࣮ߦ͢Δ͚ͩ • ઌఔͷྫͩͱ
2 ্ཱ͕ͪΔ • ԾϚγϯͷϕʔεΠϝʔδ͕ར༻Ͱ͖Δ(Box) • Πϯλʔωοτӽ͠ʹڞ༗ͨ͠Γग़དྷΔ 2 • ϛυϧΣΞ͕ "͙͢ࢼͤΔ" Box ͕ެ։͞Ε͍ͯͯศར • ։ൃڥςετڥΛ༻ҙ͢Δࡍੵۃతʹ͏ͱྑ͍ 2 http://www.vagrantbox.es/ , https://atlas.hashicorp.com/boxes/search . Box ͕৴པͰ͖Δ͔ͳͲҙ͕ඞཁ. Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 9
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 10
Packer Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 11
Packer • ԾϚγϯͷςϯϓϨʔτΛ࡞͢Δҝͷπʔϧ • VirtualBox, VMware, Amazon EC2, Docker •
packer build -var-file=var.json config.json • ઃఆϑΝΠϧΛॻ͍࣮ͯߦ͢ΕςϯϓϨʔτ͕࡞͞ΕΔ • ҰͷϏϧυͰ AMI ͱ VirtualBox ͷςϯϓϨʔτΛ࡞ΔՄೳ • ΰʔϧσϯΠϝʔδͷ࡞,ෳͷج൫͚ͷΠϝʔδ࡞͕؆୯ʹ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 12
Packer config exmaple { "variables": {"aws_access_key": "", "aws_secret_key": ""}, "builders":
[{ "type": "amazon-ebs", "access_key": "{{user `aws_access_key`}}", "secret_key": "{{user `aws_secret_key`}}", "region": "ap-northeast-1", "source_ami": "ami-test-12345", "instance_type": "t2.small", "ami_name": "Web_{{isotime | clean_ami_name}}", "ssh_username": "ec2-user", "ssh_timeout": "5m" }], "provisioners": [ {"type": "shell", "inline": [ "sudo yum -y install python-setuptools", "sudo easy_install pip", "pip install ansible" ]}, {"type": "ansible-local", "playbook_file": "playbook.yml"} ] } Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 13
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 14
Serf Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 15
Serf • αʔϏεσΟεΧόϦ/ΦʔέετϨʔγϣϯͷҝͷπʔϧ • ෳαʔόͰΫϥελΛܗ͠ΠϕϯτΛߦ͏ • ൃՐͨ͠ΠϕϯτຖʹίϚϯυΛఆٛग़དྷΔ • ϗετ͕Ϋϥελʹ join
ͨ͠Β xx Λ࣮ߦ... • ϢʔβҙͷΠϕϯτΛൃՐͤ͞Δ͜ͱՄೳ • ΫϥελΠϕϯτͷཧ Gossip ͱ͍͏ϓϩτίϧΛ༻͍ͯ͠Δ • ֤ϊʔυ͔ΒͷϨεϙϯεΛड͚औΔΫΤϦͱ͍͏ػೳ͋Δ • ΦʔτεέʔϧͷڥେنΦϖϨʔγϣϯ͕ඞཁͳڥͰ༗༻ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 16
Serf Cluster & Event Handler (at n1) [user@n1] $ serf
agent -node=node1 -bind=192.0.2.1 -log-level=debug \ -event-handler -member-join="echo member joined!" ==> Starting Serf agent... : --- (at n2) [user@n2] $ serf agent -node=node2 -bind=192.0.2.2 -join=192.0.2.1 --- (at n1) 2015/05/25 14:15:01 [INFO] serf: EventMemberJoin: node2 192.0.2.2 2015/05/25 14:15:01 [DEBUG] agent: Event 'member-join' script output: member joined! [user@n1] $ serf members node1 192.0.2.1:7946 alive node2 192.0.2.2:7946 alive Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 17
Serf Query [user@n1] $ serf agent -node=node1 -bind=192.0.2.1 \ -event-handler
query:uptime=uptime [user@n2] $ serf agent -node=node2 -bind=192.0.2.2 \ -event-handler query:uptime=uptime \ -join=192.0.2.1 --- [user@n2] $ serf query uptime Query 'uptime' dispatched Ack from 'node1' Response from 'node1': 15:29:29 up 23 days, 6:27, 2 users, load average: 0.13, 0.25, 0.30 Ack from 'node2' Response from 'node2': 15:29:29 up 15 days, 6:27, 1 users, load average: 0.01, 0.02, 0.02 Total Acks: 1 Total Responses: 1 Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 18
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 19
Consul • Serf ͱಉ͘͡αʔϏεσΟεΧόϦ/ΦʔέετϨʔγϣϯͷҝͷπʔϧ • Serf ͱͷେ͖ͳҧ͍ • Serf ΑΓ
Consistency ʹدͬͨπʔϧ(Consul: CP دΓ, Serf: AP دΓ) • Raft ͱ͍͏߹ҙϓϩτίϧΛ༻͍ͯ Consistency Λอূ • Key-Value Store ػೳ͕͍͍ͯΔ (HTTP API ܦ༝Ͱૢ࡞͢Δ) • σʔληϯλɾαʔϏε/λάͷ֓೦͕͋Δ(ෳͷϊʔυΛଋͶΔͷ) • Health Check ػೳ͕͍͍ͯΔ • DNS ΠϯλϑΣʔε ͕͍͍ͯΔ • WebUI ͕͍͍ͯΔ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 20
Consul Health Check • HTTP POST ͰొͰ͖Δ(ͪΖΜઃఆϑΝΠϧͰ) • health check
͕ࣦഊͨ͠ΒಛఆͷΠϕϯτ͕ൃՐ͢Δ • health check script ͷ༷ Nagios Plugin ޓ curl -vvv -X PUT \ -d ' { "ID": "check_swap", "Name": "Swap Utilization", "Notes": "Check swap space on local machine.", "Script": "/usr/lib64/nagios/plugins/check_swap -w 80% -c 30%", "Interval": "10s" } ' http://localhost:8500/v1/agent/check/register Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 21
Consul DNS Interface • DCɾϊʔυɾαʔϏεɾλάͷ୯ҐͰ DNS ϨίʔυΛҾ͚Δ • dig @127.0.0.1
-p 8600 foo.node.consul ANY • foo ͱ͍͏ node ͷ IP ͕ฦͬͯ͘Δ • dig @127.0.0.1 -p 8600 redis.service.dc1.consul. ANY • dc1 ʹ͋Δ redis αʔϏεʹొ͞Ε͍ͯΔϊʔυͷ IP શͯ • લड़ͷ Health Check ͱซͤͯԽͷػߏΛ࣮Ͱ͖Δ • Bind ͷϛυϧΣΞͱΈ߹ΘͤΔࣄՄೳ (DNS Forwarding) Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 22
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 23
Terraform Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 24
Terraform • ΫϥυαʔϏεͷԾϚγϯࣗମͷઃఆΛཧ͢Δ • ΠϯελϯελΠϓ, Ϧʔδϣϯ, IP, ϕʔεΠϝʔδ, ACL ...
• AWS, DigitalOcean, GCE, Heroku, CloudFlare ... • ઃఆϑΝΠϧΛॻ͖ terraform apply ͢Εద༻͞ΕΔ • terraform plan Ͱมߋ༰ΛݟΔ͜ͱ͕ग़དྷΔ • Ұ࣮ߦͨ͠Β State Λอଘ͢ΔͨΊɺطଘͷΠϝʔδͷઃఆมߋग़དྷΔ • ઃఆϑΝΠϧΛมߋͯ͠ terraform plan, terraform apply • terraform destroy ͰઃఆϑΝΠϧͷ༰Λഁغ͢Δ͜ͱग़དྷΔ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 25
Terraform config example provider "aws" { region = "us-east-1" }
resource "aws_elb" "web" { name = "terraform-example-elb" availability_zones = ["${aws_instance.web.*.availability_zone}"] listener { instance_port = 80 instance_protocol = "http" lb_port = 80 lb_protocol = "http" } instances = ["${aws_instance.web.*.id}"] } resource "aws_instance" "web" { count = 4 ami = "ami-12345" instance_type = "t1.micro" } Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 26
Tools • Vagrant • Packer • Serf • Consul •
Terraform • Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 27
Vault Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy
) 28
Vault • σʔλ҉߸Խɾػີใཧπʔϧ • Secret/Auth/Audit ͷػೳΛఏڙ͢Δ • Secret • ฏจΛ҉߸Խͯ͠ཧ͢Δ(AES-GCM
256bit Ͱ҉߸Խ͞ΕΔ) • AWS/MySQL/Postgres ͷϢʔβใΛཧ͢Δ(ૢ࡞ݖݶظݶΛ͚ͨΓग़དྷΔ) • Auth • Secret ͷಡΈग़͠Λཧ͢Δ • Github LDAP ͱ࿈ܞͯ͠ Token ΛൃߦͰ͖Δ(ಛఆͷ৫ʹॴଐ͍ͯͨ͠Β Token ൃߦͳͲ) • Audit • ࠪϩάΛϑΝΠϧ syslog ʹग़ྗͰ͖Δ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 29
Vault example [server: 192.0.2.1] $ vault server $ vault init
# ͜͜Ͱग़ྗ͞ΕΔ Key Token Λ߇͓͑ͯ͘ $ vault token-create policy="root" # ͜͜Ͱग़ྗ͞ΕΔ Token Λ Client ʹ༩͑Δ --- [client] $ export VAULT_ADDR="https://192.0.2.1:8200" $ vault auth ${CLIENT_TOKEN} $ vault write secret/name foo=buz $ vault read -format=json secret/name | grep -A2 data "data": { "foo": "buz" } Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 30
·ͱΊ • Hashicorp ͷπʔϧʹ͍ͭͯ֓ཁ(ຊʹͬ͘͟Γ)આ໌ • Vagrant, Packer, Serf, Consul, Terraform,
Vault • ։ൃڥӡ༻Λշదʹग़དྷΔπʔϧ(ͱࢥ͍ͬͯ·͢) • ͥͻ৭ʑࢼͯ͠ΈͯԼ͍͞(Θ͔Βͳ͚Εฉ͍ͯԼ͍͞) • ಛʹ࣍ͷΑ͏ͳҊ݅Λ୲͞ΕͯΔํʹΦεεϝͰ͢ • ͕ଟ͍Ҋ݅૿ݮ͕ܹ͍͠Ҋ݅ • AWS GCE ͷ֤छΫϥυΛ͍ͬͯΔҊ݅ Heartbeats ϓϩμΫτษڧձ Hashicorp (2015/05/25) - Yoshikawa Ryota ( @rrreeeyyy ) 31