Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
SRE at Cookpad
Search
rrreeeyyy
November 29, 2017
Technology
1
1.2k
SRE at Cookpad
#hbstudy の 79 回目で SRE の話をしました。
rrreeeyyy
November 29, 2017
Tweet
Share
More Decks by rrreeeyyy
See All by rrreeeyyy
An Efficient Incident Response Training with AI / SRE NEXT 2024 Sponsor Session
rrreeeyyy
1
3k
カンファレンスから見る SRE トレンド 2024 / SRE Trends from Conferences in 2024 #SRE_Findy
rrreeeyyy
4
2.1k
信頼性の育て方 / mackerel-meetup-15
rrreeeyyy
9
2.4k
SRE の歩き方・進め方 / sre-walk-through-procedure
rrreeeyyy
0
8.5k
「信頼性」を保ちつつ大規模サービスをリニューアルする / cookpad-tech-kitchen-service-embedded-sres
rrreeeyyy
11
12k
Cookpad and Prometheus
rrreeeyyy
6
20k
SRE-Lounge-8-Cookpad-Microservice-Architecture-Overview
rrreeeyyy
5
5.2k
A survey of anomaly detection methodologies for web system
rrreeeyyy
5
1.2k
エンジニアリングをちゃんとやる あるいは 人類の平和 について / wsa02-rrreeeyyy
rrreeeyyy
14
3k
Other Decks in Technology
See All in Technology
【インフラエンジニアbooks】30分でわかる「AWS継続的セキュリティ実践ガイド」
hssh2_bin
4
1.7k
【shownet.conf_】ShowNet x 宇宙ネットワーク
shownet
PRO
0
410
Databricks Appのご紹介
databricksjapan
0
250
入社半年(合計1年)でGoogle Cloud 認定を全冠した秘訣🤫
risatube
1
200
LINE-ChatGPT 倫理問題を整理する全力肯定彼氏くん [LuC4]に訪れたサービス開始以来の最大の危機
o_ob
1
130
リスクから学ぶKubernetesコンテナセキュリティ/k8s-risk-and-security
mochizuki875
1
320
How CERN serves 1EB of data via FUSE
ennael
PRO
0
16k
Oracle Database 23ai 新機能#4 Rolling Maintenance
oracle4engineer
PRO
0
130
Assisted reorganization of data structures
ennael
PRO
0
260
ゼロから実装まで!機械学習入門
natsuki0726
0
180
Semantic Kernel の Agent 機能試してみた!
okazuki
1
150
ADRを運用して3年経った僕らの現在地
onk
PRO
13
5.3k
Featured
See All Featured
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
1
290
The Language of Interfaces
destraynor
154
24k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
4
120
Thoughts on Productivity
jonyablonski
67
4.2k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
48k
From Idea to $5000 a Month in 5 Months
shpigford
381
46k
A Modern Web Designer's Workflow
chriscoyier
692
190k
How to Think Like a Performance Engineer
csswizardry
16
1k
Fireside Chat
paigeccino
32
3k
Build The Right Thing And Hit Your Dates
maggiecrowley
31
2.3k
How to train your dragon (web standard)
notwaldorf
87
5.6k
Making the Leap to Tech Lead
cromwellryan
131
8.8k
Transcript
SRE at Cookpad ΫοΫύουגࣜձࣾ ΠϯϑϥετϥΫνϟʔ෦ SRE άϧʔϓ ٢ ཽଠ (
@rrreeeyyy ) hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 1
Agenda • SRE ʹ͍ͭͯ(վΊͯ) • ΫοΫύουʹ͓͚ΔΠϯϑϥߏ • EC2 with spot
instances • hako + ECS ڥ • ΫοΫύουʹ͓͚Δ SRE ͷ׆ಈ • ࣄͷྲྀΕɾऔΓΈʹ͍ͭͯɹ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 2
!SSSFFFZZZ !SSSFFFZZZ IUUQTSSSFFFZZZDPN :PTIJLBXB3ZPUB Me • Yoshikawa Ryota ( @rrreeeyyy
[reɪ] ) • גࣜձࣾϋʔτϏʔπ (2010/11 ʙ 2016/12) • ΫοΫύουגࣜձࣾ (2017/01 ʙ) • ΠϯϑϥετϥΫνϟʔ෦ SRE άϧʔϓ • ڵຯྖҬ • ϞχλϦϯάɾ࣌ܥྻσʔλϕʔε • ࢄγεςϜɾϩʔυόϥϯα • झຯ • League of Legends, ΑΑΫϩχΫϧ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 3
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 4
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 5
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 6
(վΊͯ) Site Reliability Engineering • αΠτͷ৴པੑͷશͯʹൣғΛ࣋ͭ • ಛʹɺιϑτΣΞɾγεςϜΤϯδχΞϦϯάΛ༻͍ͯ՝Λղܾ͢Δ • ྫ͑ɺύϑΥʔϚϯε
• ύϑΥʔϚϯεʹ͕͋Δίʔυ SRE ͕ࣗίʔυΛॻ͍ͯमਖ਼͢Δ • ྫ͑ɺεέʔϥϏϦςΟɾΞϕΠϥϏϦςΟ • ਓؒʹΑΔखಈͷ࡞ۀΛίʔυʹΑΓࣗಈԽ͢Δ • ଐਓੑΛഉআͯ҆͠ఆͤ͞ΔɾਓؒΛ૿͞ͳͯ͘αʔϏεΛεέʔϧͤ͞Δ • ن͕૿େͯ҆͠ఆͯ͠αʔϏεΛఏڙͰ͖Δঢ়ଶʹ͢Δ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 7
SRE ͷ׆ಈ SRE ͷ׆ಈେ͖͚ͯ࣍͘ͷ 4 ͭʹྨͰ͖Δͱ͞Ε͍ͯΔ • So#ware Engineering •
Systems Engineering • Toil • Overhead hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 8
SRE ͷ׆ಈ • So#ware Engineering • ίʔυͷ࡞मਖ਼, ͦΕʹؔ࿈͢ΔઃܭυΩϡϝϯςʔγϣϯͷ࡞ۀͳͲ • ࣗಈԽεΫϦϓτͷ࡞,
πʔϧϑϨʔϜϫʔΫͷ࡞ ... • αʔϏεͷ৴པੑΛߴΊΔػೳͷՃ ... • Systems Engineering • 1 ճͷ࡞ۀͰӬଓతͳվળΛੜΉ࣍ͷΑ͏ͳ࡞ۀ • ຊ൪γεςϜͷઃఆมߋ ... • ։ൃνʔϜͷαʔϏεΞʔΩςΫνϟઃܭ ... hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 9
SRE ͷ׆ಈ • Toil • αʔϏεΛՔಇͤ͞ΔͨΊ͚ͩʹ܁Γฦ͠ߦΘΕΔखಈͰͷ࡞ۀ • ظతͳՁΛ࣋ͨͳ͍, αʔϏεͷʹରͯ͠ O(n)
Ͱ͋Δ ... • ͭ·Βͳ͍ࣄͰظతͳՁ͕͋Ε Toil Ͱͳ͍ • Overhead • αʔϏεΛՔಇͤ͞Δ͜ͱʹ݁͠ͳ͍ཧతͳ࡞ۀ • ࠾༻, ϛʔςΟϯά, ධՁ, τϨʔχϯά ... hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 10
ΫοΫύουʹ͓͚Δ SRE hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy )
11
ΫοΫύουʹ͓͚Δ SRE • ΠϯϑϥετϥΫνϟʔ෦ • ͷதʹ SRE άϧʔϓ͕͋Δ • DWH,
Security, ࣾIT ͕ಉ͡෦ॺʹ͋Δ • ϝϯόʔ SRE ͚ͩͰ 10 ਓఔ + ւ֎ʹ 2 ໊ఔ • ࠃ 7 ׂɾάϩʔόϧ 1 3 ׂ͙Β͍ͷൺ 1 h$ps:/ /speakerdeck.com/sorah/building-infrastructure-for-our-global-service hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 12
ΫοΫύουʹ͓͚ΔΠϯϑϥߏ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 13
ΫοΫύουʹ͓͚ΔΠϯϑϥߏ • ༷ʑ͋Δ͕ϝΠϯͰಈ͍͍ͯΔͷେ͖͚ͯ͘ 2 ͭ • EC2 + spotscaler ڥ
• EC2 ʹΞϓϦέʔγϣϯΛσϓϩΠ͍ͯ͠Δߏ • EC2 Πϯελϯεεϙοτʹͳ͍ͬͯΔ • ECS + hako ڥ • ECS Ϋϥελʹରͯ͠ίϯςφΛσϓϩΠ͍ͯ͠Δߏ • σϓϩΠϝϯτʹ eagletmt/hako Λར༻͍ͯ͠Δ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 14
EC2 + spotscaler ڥ • ΫοΫύουͷ͍ΘΏΔը໘ͷඳը෦ͳͲ͕͜Εʹ͋ͨΔ • PC ൛αΠτ, εϚϗ༻ͷ
API ... ͳͲ • Route53 + ALB + EC2 ͷ AWS ͷҰൠతͳߏ • EC2 spot instance Λ spotscaler 2 Λͬͯཧ͍ͯ͠Δ • deployment serf + sorah/mamiya 3 Λͬͯߦ͍ͬͯΔ 3 h$ps:/ /github.com/eagletmt/hako 2 h$ps:/ /speakerdeck.com/ryotarai/spot-instances-in-cookpad-number-cookpadtechconf-2017 hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 15
ECS + hako ڥ • ΫοΫύουͷ༷ʑͳϚΠΫϩαʔϏε͕͜Εʹ͋ͨΔ • λΠϜϥΠϯ, ࠂ, ೝূ...
ͳͲ • ػցֶशͷج൫͜͜ʹ͍ͬͯΔʢg2 Πϯελϯεͷ ECS Ϋϥελʣ • ECS ͷΫϥελʹରͯ͠ hako 3 Λར༻ͯ͠σϓϩΠ͍ͯ͠Δ • GHE ্ͷϦϙδτϦʹΞϓϦέʔγϣϯఆٛͷ PR Λग़ͯ͠Β͏ • σϓϩΠ Slack ্͔Β։ൃऀ͕جຊతʹࣗ༝ʹߦ͑Δ • PR ຖͷεςʔδϯάɾ౷߹ίϯιʔϧɾτϨʔγϯάͳͲ͕͋Δ 4 4 h$p:/ /techlife.cookpad.com/entry/2016/03/16/100043 3 h$ps:/ /github.com/eagletmt/hako hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 16
ECS + hako ڥ ʹ͓͚Δ YAML ઃఆ scheduler: <<: !include
schedulers/default.yml elb_v2: vpc_id: vpc-xxxxxx health_check_path: /healthcheck listeners: - port: 80 protocol: HTTP # : autoscaling: min_capacity: 2 max_capacity: 24 policies: # : policies for autoscaling hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 17
ECS + hako ڥ ʹ͓͚Δ YAML ઃఆ app: image: xxxxxxxxxx/service
# ECR Λࢦఆ͢Δ͜ͱ͕ଟ͍ cpu: 128 memory: 512 links: - fluentd env: $providers: - <<: !include env_providers/vault.yml directory: hako/group/service # vault hako/group/service ͷ - !include env_providers/vault_shared.yml # ֘ͷάϧʔϓʹॴଐ͢Δਓ͕ࣗ༝ʹઃఆͰ͖Δ RAILS_ENV: production WORKER_NUM: '2' additional_containers: front: !include containers/front.yml fluentd: !include containers/fluentd.yml volumes: td-agent: source_path: /var/lib/hako/td-agent/service hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 18
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 19
ΫοΫύουʹ͓͚ΔϚΠΫϩαʔϏεͷ࣮ફ • ৫ਓɾνʔϜɾαʔϏεͷʹͭΕͯݖݶͷҕৡ͕ඞཁʹͳͬͨ • SRE ج൫νʔϜ͕શͯΛཧ͢ΔͷෆՄೳͩ͠ෆ݈શ • ΞϓϦέʔγϣϯࣗମ͕େ͖͘ͳΓ༷͗ͯ͢ʑͳ͕ग़ΔΑ͏ʹͳͬͨ • ίʔυͷݟ௨͕͠ѱ͘ͳΔɾςετʹֻ͕͔࣌ؒΓ͗͢Δ
... • લ͔ΒঃʑʹඋΛਐΊ͍ͯΔ 4 4 h$p:/ /techlife.cookpad.com/entry/2016/03/16/100043 hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 20
ΫοΫύουʹ͓͚ΔϚΠΫϩαʔϏεͷ࣮ફ • جຊతʹϝΠϯͷΞϓϦέʔγϣϯΛ hako ڥʹΓग़͢ • ৽نʹ࡞͞ΕΔΞϓϦέʔγϣϯ hako ڥΛඪ४ʹ͢Δ •
ಉ࣌ʹɺmicroservice ԽʹΑΔ՝ʹ͍͔ͭ͘औΓΜͰ͍Δ • αʔϏεؒͷ Tracing ʹ AWS X-Ray Λར༻͍ͯ͠Δ • αʔϏεϝογϡͷ࣮ݱʹ envoyproxy/envoy Λར༻͍ͯ͠Δ • ͍͔ͭ͘՝͕͋Γ࣮ͭͭࡍʹಈ͔ͯ͠ΛਐΊ͍ͯΔ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 21
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 22
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 23
ΫοΫύουʹ͓͚Δ SRE ͷ׆ಈ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy
) 24
ΫοΫύουʹ͓͚Δ SRE ͷ׆ಈ • ΫοΫύουͷ SRE ͷ׆ಈΛ͍͔ͭ͘հ • αʔόߏͷཧɾมߋ •
৽نαʔϏεʹର͢Δ SRE ͷಈ͖ • ϞχλϦϯάͷվળ • ϩʔυόϥϯγϯάͷվળ • ৽ଔݚम • ... hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 25
αʔόߏͷมߋɾཧ • αʔόߏجຊతʹશͯίʔυԽ͞Ε͍ͯΔ • Codenize.tools 5 Itamae 6, Terraform
ͳͲ • IAM User Route53 GHE ্Ͱී௨ʹཧ͞Ε͍ͯΔ • ։ൃऀ͕ࣗ༝ʹ PR Λग़͢͜ͱ͕ग़དྷΔ • αʔόߏͷมߋίʔυͷ Pull request ͱͯ͠ग़͢ • ϨϏϡʔΛड͚ɺϚʔδ͔ͯ͠Βө • Ұ෦Ϛʔδͯࣗ͠ಈͰө͞ΕΔ 6 h$ps:/ /github.com/itamae-kitchen/itamae 5 h$ps:/ /codenize.tools/ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 26
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 27
৽نαʔϏεʹର͢Δ SRE ͷಈ͖ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy
) 28
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 29
৽نαʔϏεʹର͢Δ SRE ͷಈ͖ • େ͖ΊͳػೳαʔϏε୯ҐͰׂͯ͠։ൃ͢Δ • ߹ʹΑΔ͕ SRE ͕ 1
ਓΞαΠϯ͞ΕΔ • ֘ͷαʔϏεͷ৴པੑʹؔ͢Δ͜ͱجຊతʹશ෦Δ • ઃܭͷ૬ஊɾϨϏϡʔ • ϦϦʔεલͷෛՙࢼݧɾΩϟύγςΟϓϥϯχϯά • ύϑΥʔϚϯεʹର͢ΔΞϓϩʔν • ... hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 30
ઃܭͷ૬ஊɾϨϏϡʔ • ։ൃͷલʹҎԼͷΑ͏ͳ͜ͱΛڞ༗ͯ͠Β͍ٞ͢Δ • ͲͷΑ͏ͳΞϓϦέʔγϣϯΛ࡞Δ͔ • ͲͷΑ͏ͳίϯϙʔωϯτ͕ඞཁ͔ • ͲͷΑ͏ͳߏɾεέʔϧઓུʹ͢Δ͔ •
Design Document Λॻ͍ͯΒ͍ͦΕΛϕʔεʹΛͨ͠Γ͢Δ • େ͖ΊͷػೳͳͲΛ։ൃ͢Δ࣌ Design Document Λॻ͘͜ͱ͕ଟ͍ (SRE ) hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 31
ෛՙࢼݧ • ͋Δఔग़དྷ͖ͯͨลΓͰෛՙࢼݧͷΛ͢Δ • Ͳͷ͘Β͍ͷϦΫΤετ͕ఆ͞ΕΔ͔ • ͲͷΑ͏ͳϢʔβϦΫΤετ͕ఆ͞ΕΔ͔ • SRE ଆͰෛՙࢼݧͷγφϦΦڥΛ࡞Δ
• ෛՙࢼݧΛߦ͍ͳ͕Βຊ൪ڥͰͷΩϟύγςΟΛݟੵΔ • ΞϓϦέʔγϣϯ্ͷϘτϧωοΫ͕Ͳ͜ʹ͋Δ͔ΛݟΔ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 32
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 33
ύϑΥʔϚϯεʹର͢ΔΞϓϩʔν • ৴པੑʹؔ͢Δ͕͋Δ߹ࣗͰίʔυΛॻ͘ • ྫ͑ύϑΥʔϚϯε্ͷͳͲ • ΞϓϦέʔγϣϯͷ༷Λཧղͯ͠దͳΞϓϩʔνΛऔΔ • ྫ͑Ͳ͏͍͏ը໘ͷͲ͏͍͏ΫΤϦͳͷ͔ •
Ͳ͏͍͏ใ͕ཉͯ͘͠Ͳ͏͍͏੍͕͋Δͷ͔ • ͍ͬͯΔϛυϧΣΞͷ༷ཧղͯ͠దͳΞϓϩʔνΛऔΔ • ྫ͑ SQL ͷվળͳΒ MySQL ͷΠϯσοΫεͷߏ෦ॲཧͷ͕ࣝ͋Δͱྑ͍ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 34
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 35
৽نαʔϏεʹର͢Δ SRE ͷಈ͖ • ͜ͷลΓͨΓલͷࣄΛී௨ʹ͍ͬͯΔ͚ͩ • ͬͱվળग़དྷΔՕॴ͕ͨ͘͞Μ͋Δͱߟ͍͑ͯΔ • ྫ͑ෛՙࢼݧΛࣗಈͰߦ͏Α͏ʹ͢ΔɺͳͲ •
ΞϓϦέʔγϣϯΛཧղ͠ɺඞཁͳΒίʔυΛॻ͘ • SRE αΠτͷ৴པੑʹؔΘΔશͯʹΛෛ͏ • ΞϓϦέʔγϣϯͷಛੑઃܭΛཧղ͠ϛυϧΣΞΛΈ߹ΘͤΔ Systems Engineering • ༷ϛυϧΣΞΛཧղ࣮͠ࡍʹΞϓϦέʔγϣϯͷϩδοΫΛॻ͘ So.ware Engineering hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 36
ϞχλϦϯάͷվળ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 37
ϞχλϦϯάͷվળ • ݱࡏϝτϦΫεऔಘʹ Zabbix Λ༻͍ͯ͠Δ • ΑΓεέʔϧ͠ɺߴڃݴޠͰϝτϦΫεΞϥʔτΛѻ͑ΔΑ͏ʹ͍ͨ͠ • SRE ຊ
ͷ 10 ষʹॻ͔Ε͍ͯΔΑ͏ͳঢ়ଶʹͳΓ͍ͨ • Prometheus ͷಋೖΛঃʑʹ࣮ࢪ͍ͯ͠Δ • ϝτϦΫεͷղ૾ 15 ඵ୯ҐͳͲͰऔಘ͢ΔΑ͏ʹ͍ͯ͠Δ • ୯७ͳ HTTP ϨεϙϯεͰϝτϦΫεΛऩूग़དྷΔ • ΞϓϦέʔγϣϯࣗମͷϝτϦΫεͳͲ͕ൺֱత؆୯ʹऩूग़དྷΔ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 38
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 39
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 40
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 41
ϞχλϦϯάͷվળ • Ͳ͏͍ͬͨϛυϧΣΞΛબఆ͢Δ͔ • ϞχλϦϯάͰ͋Ε࣌ܥྻσʔλϕʔεͳͲʹؔ͢Δਂ͍ཧղΛ͔ͯ͠ΒબͿ • Ͳ͏͍ͬͨಛੑ͕͋Δ͔ʁࣗͨͪͷنʹ߹͍ͬͯΔ͔ʁεέʔϧ͢Δ͔ʁ • Γͳ͍ίϯϙʔωϯτͳ͍͔ʁແ͍ͳΒࣗ࡞Ͱ͖ͦ͏ͳن͔ʁ •
Prometheus ͳΒྫ͑ Long-Term Storage पΓͷίϯϙʔωϯτͳͲ • ͔ͬ͠Γͱͨ͠ Systems Engineering ͷݟΛར༻ͯ͠બఆ͢Δ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 42
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 43
ϩʔυόϥϯγϯάͷվળ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 44
ϩʔυόϥϯγϯάͷվળ • ैདྷΞϓϦέʔγϣϯαʔόͷϩʔυόϥϯγϯάʹ haproxy Λར༻͍ͯͨ͠ • Weighted Round-robin (wrr) Λར༻ͯ͠όϥϯγϯάΛߦ͍ͬͯͨ
• ෛՙͷ͍ΤϯυϙΠϯτͱߴ͍ΤϯυϙΠϯτ͕͋Δ߹ෛՙ͕Βͭ͘ • wrr ͩͱಛఆͷαʔόʹෛՙͷߴ͍ΤϯυϙΠϯτ͕ूத͢ΔՄೳੑ͕͋Δ • leastconn Λ͑ྑͦ͞͏͕ͩ HTTP keepalive ͍ͯ͠Δ߹ʹ͕͋Δ • ίωΫγϣϯ͕ΔͷͰ࣮ࡍʹॲཧ͍ͯ͠ΔϦΫΤετͱണ͢Δɹ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 45
ryotarai/simproxy • h#ps:/ /github.com/ryotarai/simproxy • Go Ͱॻ͔ΕͨγϯϓϧͳϦόʔεϓϩΩγ • ϔϧενΣοΫͳͲͷجຊతͳػೳ͋Δ •
Balancing Method ʹ leastreq Λ࣮͍ͯ͠Δ • ࢦඪͱͯ͠୯७ͳίωΫγϣϯͰͳ࣍͘ͷΑ͏ͳΛར༻͢Δ • όοΫΤϯυ͕ड͚औ͕ͬͨԠ͕ฦ͖͍ͬͯͯͳ͍ϦΫΤετͷ • ͜ΕʹΑΓ HTTP keepalive ͕͋ͬͯ leastconn ͷΑ͏ͳόϥϯγϯά͕Մೳ • ϩʔυόϥϯγϯάͷΞϧΰϦζϜʹؔ͢Δ Systems Engineering ͱ࣮͢Δ So.ware Engineering hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 46
before simproxy hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy )
47
a"er simproxy hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy )
48
৽ଔݚम • ৽ଔʹରͯ͠ΠϯϑϥͷݚमΛ 3 ؒߦ͍ͬͯΔ • MySQL ͷΠϯσοΫεͷߏΛཧղ͠ΫΤϦνϡʔχϯάΛߦ͏ ... •
AWS ͷ ALB RDS ͳͲΛ͍ࣗͷΞϓϦέʔγϣϯΛεέʔϧͤ͞Δ ... • ࣮ࡍͷΞϓϦέʔγϣϯΛॻ͘ϝϯόʔ͕Πϯϑϥͷཧղ͕͋Δํ͕ྑ͍ • શͯͷΞϓϦέʔγϣϯίʔυΛ SRE ͕νΣοΫ͍ͯ͠Δͱεέʔϧ͠ͳ͍ • ྨ্ Overhead ͔͠Εͳ͍͕αʔϏεΛεέʔϧͤ͞ΔͨΊʹඞཁ • ࣮ࡍʹ Systems Engineering ͷਂ͍ݟ͕ແ͍ͱڭ͑Δ͜ͱ͕ग़དྷͳ͍ hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 49
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 50
hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 51
PR hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 52
We are hiring! • ΫοΫύουͰຊհͨ͠Α͏ͳऔΓΈΛߦ͍ͬͯ·͢ • ·ͩ·ͩղܾ͍ͨ͠ɾऔΓΈ͍ͨ՝͕ͨ͘͞Μ͋Δ • ָ͠Έͳ͕ΒҰॹʹΕΔϝϯόʔΛืू͍ͯ͠·͢ →
h$ps:/ /cookpad.jobs hbstudy#79 (2017/11/20) | Yoshikawa Ryota ( @rrreeeyyy ) 53