Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
SRE大全 メルカリ編 前半 #hbstudy 75 / SRE Taizen Mercari 1 hbstudy#75
kazeburo
August 21, 2017
Technology
7
15k
SRE大全 メルカリ編 前半 #hbstudy 75 / SRE Taizen Mercari 1 hbstudy#75
SRE大全 メルカリ編 hbstudy#75
kazeburo
August 21, 2017
Tweet
Share
More Decks by kazeburo
See All by kazeburo
DNS権威サーバのクラウドサービス向けに行われた攻撃および対策 / DNS Pseudo-Random Subdomain Attack and mitigations
kazeburo
5
1.3k
sacloudns
kazeburo
2
220
「orchestratorとGTID運用を支える監視」の勉強 / Monitoring orchestrator and GTID operation
kazeburo
2
1k
最近の監視(仮)/Recent system monitoring with mackerel
kazeburo
3
3.8k
Mercari Item Search: Behind The Scenes (20min)
kazeburo
3
2.6k
成長し続けるインフラストラクチャとメルカリの挑戦/mercari infrastructure and software
kazeburo
23
4.7k
Microservices on Multi-Cloud
kazeburo
21
5.4k
logrotate殺プロセス事件 YAPC::Okinawa 2018 前夜祭 LT/Mystery of logrotate's death
kazeburo
9
11k
Perl in Mercari YAPC::Okinawa 2018 ONNASON
kazeburo
8
5k
Other Decks in Technology
See All in Technology
Oracle Transaction Manager for Microservices Free 22.3 製品概要
oracle4engineer
PRO
5
110
NGINXENG JP#2 - 4-NGINX-エンジニアリング勉強会
hiropo20
0
120
Oracle Cloud Infrastructure:2023年1月度サービス・アップデート
oracle4engineer
PRO
0
160
230125 モニターマウントLT ITガジェット翁(Ryu.Cyber)さん
comucal
PRO
0
4.7k
OpenShiftクラスターのアップグレード自動化への挑戦! / OpenShift Cluster Upgrade Automation
skitamura7446
0
160
S3とCloudWatch Logsの見直しから始めるコスト削減 / Cost saving S3 and CloudWatch Logs
shonansurvivors
0
260
IoT から見る AWS re:invent 2022 ― AWSのIoTの歴史を添えて/Point of view the AWS re:invent 2022 with IoT - with a history of IoT in AWS
ma2shita
0
280
メドレー エンジニア採用資料/ Medley Engineer Guide
medley
3
5.1k
PHPのimmutable arrayとは
hnw
1
160
AI Services 概要 / AI Services overview
oracle4engineer
PRO
0
170
Logbii(ログビー) 会社紹介
logbii
0
140
SPA・SSGでSSRのようなOGP対応!
simo123
2
160
Featured
See All Featured
How to Ace a Technical Interview
jacobian
270
21k
The World Runs on Bad Software
bkeepers
PRO
59
5.7k
JazzCon 2018 Closing Keynote - Leadership for the Reluctant Leader
reverentgeek
175
9.1k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
44
14k
Reflections from 52 weeks, 52 projects
jeffersonlam
338
18k
The Power of CSS Pseudo Elements
geoffreycrofte
52
4.3k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
500
130k
How GitHub Uses GitHub to Build GitHub
holman
465
280k
Support Driven Design
roundedbygravity
88
8.9k
The MySQL Ecosystem @ GitHub 2015
samlambert
240
11k
How GitHub (no longer) Works
holman
298
140k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
24
4.6k
Transcript
SREେશ: ϝϧΧϦฤ [લ] 2017/08/17 hbstudy#75 Masahiro Nagano @kazeburo
AGENDA • લ • ࣗݾհɺϝϧΧϦ͕SREΛ࠾༻ͨ͠ཧ༝ • ϝϧΧϦSREνʔϜհɺOnCallɺϝϧΧϦʹ͍ͭͯ • ઌͷCDNมߋͰͷΠϯγσϯτʹ͍ͭͯ
AGENDA • ޙ • PHP ΞϓϦέʔγϣϯͷ࠷దԽࣄྫ • ηΩϡϦςΟͷऔΓΈ(ύεϫʔυϦετ߈ܸࣄྫ) • ϝϧΧϦSREͷࠓޙ
• SREͷׂͷݱࡏͱະདྷɺϚΠΫϩαʔϏε
Me • Masahiro Nagano / խ • @kazeburo • Mercari,
Inc Principal Engineer Site Reliability Engineering (SRE) Team • BASE, Inc Technical Advisor
Me • ~ 2006: ژͰελʔτΞοϓࢀՃ • ։ൃΛ͠ͳ͕ΒΠϯϑϥͷ໘ΛݟΔɻDC࡞ۀͬͨ • ΞϓϦέʔγϣϯͷνϡʔχϯάɺۭ͍ͨϦιʔεͰ৽ػೳͷՃͱ͍͏αΠΫϧ •
mod_perlɺSquidʹΑΔReverse Proxy • 2006 ~: mixi • ʮΞϓϦέʔγϣϯӡ༻νʔϜʯ / DCʹߦ͔ͳ͘ͳͬͨ • େنͳը૾৴/memcached/Q4M
Me • 2010 ~: livedoor (NHN Japan => LINE) •
livedoorLINEϑΝϛϦʔͷαʔϏεΛԣஅͯ͠Πϯϑϥύ ϑΥʔϚϯεͷվળ • livedoor blog ͷMySQLνϡʔχϯά • GrowthForecast/HRForecast/Plack Optimization/MHA • 2015/02: mercari
࠷ۙͷ׆ಈ • ొஃ • AWS Dev Day Tokyo 2017 •
YAPC::Fukuoka 2017, YAPC::Hokkaido 2016 • هࣄ • WEB+DB PRESS Vol.88, Vol.92-97 ࿈ࡌ • ܦSYSTEMS 2017 7݄߸, ITPro
WEB+DB PRESS Vol.100 8&# %#13&44߸㊗ 8&# %#13&447PMͷग़൛ֶੜ࣌ͷ ࠷ॳʹखʹऔͬͨ࣌ʹ7PMʹهࣄΛॻ͘ͳΜͯ͜ͱ શ͘૾͍ͯ͠·ͤΜͰͨ͠ɻͱͯޫӫͰ͢
ϝϧΧϦ͕SREΛ࠾༻ͨ͠ཧ༝
ࣗশ `!ΠϯϑϥΤϯδχΞ` • (ݸਓతʹ) ΠϯϑϥΤϯδχΞͷٙ • 2006ҎདྷDC࡞ۀ͍ͯ͠ͳ͍ɻDC࡞ۀΛߦ͏νʔϜଞʹ͍Δ • σʔληϯλʔνʔϜ͕༻ҙͨ͠αʔόͷೳྗΛҾ͖ग़͠ɺΞϓϦέʔγϣ ϯΤϯδχΞ͕࡞ͨ͠ίʔυΛ࠷ߴͷܗͰಈ͔͢ͷ͕ࣗΒ(νʔϜ)ͷׂ
• αʔϏεͷՄ༻ੑϋʔυΣΞͷνʔϜͰͳ͘ɺιϑτΣΞΛѻ͏ νʔϜͷ
ΦϖϨʔγϣϯΤϯδχΞ • 2010ग़൛ʮWeb Operationsʯ • ʮΣϒΦϖϨʔγϣϯٕܳͰ͋ΓՊֶͰͳ͍ʯ • ܧଓతσϓϩΠɺDevOpsɺࣗಈԽɺࢹͳͲΦϖ Ϩʔγϣϯʹؔ͢ΔΤοηΠ •
ͨͩ͠ɺΦϖϨʔγϣϯ=ӡ༻ΛϧʔνϯϫʔΫͱଊ ͑Δਓଟ͍
SREͱͷग़ձ͍ • 2012/7 ༑ਓͱͷIRCͰͷձ͔Β • ΠϯϑϥͱαʔϏεͷՔಇɺ҆ఆੑΛ୲͢ΔνʔϜ͕SRE • https://research.googleblog.com/2012/07/site-reliability-engineers-solving-most.html ͜ͷهࣄ͕ެ։͞Εͨࠒ •
twitter ͷbioൃදεϥΠυʹʮSite ReliabilityʯΛՃͯ͠ҙࣝ • https://www.slideshare.net/kazeburo/yapc2102mysql/2 (2012/9) • 2015/11 ϝϧΧϦʹͯνʔϜ໊ͱͯ͠࠾༻
None
ϝϧΧϦͰSREΛ࠾༻ͨ͠ཧ༝ • ϝϧΧϦΛ͓٬͞·ʹͬͯ͘Β͏ʹʮ͍ͭͰշదʹ҆શʹ ͑Δʯ৴པੑ͕ॏཁ • Πϯϑϥ=ϞϊɺΦϖϨʔγϣϯ=ࣄͰͳ͘ɺ৴པੑʹΛ࣋ͭ͜ ͱΛ໌Β͔ʹ͢Δ • ւ֎Ͱ௨͡Δ໊শ •
JP/US/UKͰͷల։ɻάϩʔόϧͰͷ࠾༻ৗʹҙࣝ • ઌਐతͳऔΓΈͱͯ͠
ϝϧΧϦSREʹ͍ͭͯ
None
Mercari SRE • ͍ͭͰշద͔ͭ҆શʹར༻Ͱ͖Δʮ৴པੑͷߴ͍ʯαʔϏεͷ࣮ݱ • ʮ৽نαʔϏεͷ։ൃҎ֎ͷΤϯδχΞϦϯάશ෦Δʯ • ݱࡏϝϯόʔʮ9ਓʯ • શһ౦ژۈ
• ࣾπʔϧͷӡ༻վળɺAIܥͷۀʹܞΘΔϝϯόʔ͍Δ
Mercari SRE ͷۀൣғ Operations Software Eng. ج൫ߏங OnCall (োରԠ) Automation
εέʔϥϏϦςΟɾՄ༻ੑվળ DBAɺϛυϧΣΞߏங ΞϓϦέʔγϣϯͷઃܭϨϏϡʔ ϩάऩूɾੳج൫ͷߏஙɺӡ༻ αʔόϓϩϏδϣχϯάɾσϓϩΠɺϚΠΫϩαʔϏεج൫ͷඋ ηΩϡϦςΟʗෆਖ਼ར༻ݕग़
SRE൪/OnCall • ΈΜͳେ͖োରԠ • ΞϥʔτରԠ൪ͱి൪ • 4໊ͰOnCallͷScheduling • (2໊τϨʔχϯάத)
SRE൪/OnCall • ༵0͔࣌Β༵24࣌·Ͱ1िؒͰަ • Ξϥʔτͷड͚औΓͱҰ࣌ରԠ • ฏνʔϜϝϯόʔ͕ग़ࣾ͢Δ·Ͱࣗػ • 9͔࣌Βࣗػ͠ɺUS͔ΒͷґཔͳͲʹରԠ •
UK͔ΒͷରԠͰؒ࡞ۀ͋Δ • ٳ15-20ҎʹରԠ։࢝Ͱ͖Δ͜ͱ͕·͘͠ɺߦಈʹ੍ݶ͋Δ • ਂٳͷରԠͳͲɺՈͷڠྗඞཁ
൪/OnCall Λࢧ͑Δٕज़ • ࢹ • Mackerel, slacklog • ௨/Scheduling •
Slack, PagerDuty, Twilio
mackerel Worker Batch App App MySQL cron mackerel-agent fluent-plugin-mackerel mkr
ՄࢹԽ/ᮢͷઃఆ Metricsऩू ௨
[ઢ] ࠷ۙ࡞ͬͨmackerel-plugin #!/usr/bin/perl use HTTP::Date; my $NUM_LOG_WATCH = 1000; my
$CHECK_RANGE = 300; #5min my $exceptions = 0; my $now = time; open( my $messages_tail, "-|", "tail","-$NUM_LOG_WATCH","/var/log/messages") or die $!; while (<$messages_tail>) { if ( $_ !~ m!Machine Check Exception! ) { next; } if ( my ($time) = ($_ =~ m!^(\w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2})\s!) ) { $time = str2time($time); if ( $now - $time < $CHECK_RANGE ) { $exceptions++; } } } if ( $exceptions > 0 ) { print "CRITICAL: Machine Check Exception Found in this 5 minutes\n"; exit 2; } print "OK: No Machine Check Exception found\n"; exit 0; % dmesg | tail sbridge: HANDLING MCE MEMORY ERROR CPU 0: Machine Check Exception: 0 Bank 8: cc0427c000010090 TSC 0 ADDR 37805ac0 MISC 45048ce86 PROCESSOR 0:406f1 TIME 1495654896 SOCKET 0 APIC 0 [Hardware Error]: Machine check events logged EDAC MC1: CE row 0, channel 0, label "CPU_SrcID#0_Ha#0_Channel#0_DIMM": 4255 Unknown error(s): memory read on FATAL area OVERFLOW: cpu=0 Err=0001:0090 (ch=0), addr = 0x37805ac0 => socket=0, ha=1, Channel=0(mask=1), rank=0 DIFDLNBDIJOFFYDFQUJPOT ϝϞϦʔΤϥʔΛݟ͚ͭΔ
[ઢ] ࠷ۙ࡞ͬͨmackerel-plugin #!/bin/sh set -e if [ ! -f /opt/MegaRAID/MegaCli/MegaCli64
]; then exit fi if ( /opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL|grep 'Firmware state'|grep -v "Online, Spun Up" > /dev/null 2>&1 ); then /opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL|grep 'Firmware state' exit 2 fi /opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL|grep 'Firmware state' exit 0 DIFDLSBJEEJTL .FHB$MJΛ͔ͭͬͯ3"*%ͷঢ়ଶΛࢹ
slacklog ίϚϯυ $ slacklog -t alert-information --notify -- perl -e
'die "TEST!"' !LB[VIP͞ΜͷDSPOMPHΛࢀߟʹ࡞ CBUDICBDLVQͷࣦഊΛݕ slackboard ௨ͷू IUUQTHJUIVCDPNDVCJDEBJZBTMBDLCPBSE
slackͰͷΞϥʔτͷ • ؾ͘ͷ͕͍͠ • ௨ԻͷΧελϚΠζͰ͖ͳ͍ • 1ճ͔͠௨Ͱ͖ͳ͍ɻྲྀΕ͕ͪ • Push͜ͳ͍߹ •
ετϨε⤴ / ӡ༻ʹͳΓ͕ͪ
PagerDuty 4DIFEVMFϩʔςʔγϣϯཧ &TDBMBUJPOϧʔϧ
PagerDuty • ༷ʑͳखஈͰ௨Λߦ͏͜ͱ͕Ͱ͖Δ • mail • SMS • App •
ి • 12ʹҰిΛೖΕΔϧʔϧͰӡ༻ • Appͷ௨͕ศར
Emergency Call ☎ US/UKؚΊɺؾܰʹͬͯΒ͑ΔΑ͏ࣾࠂ ࠷ۙͰSREνʔϜͰΤεΧϨʔγϣϯతͰར༻
None
ϝϧΧϦʹ͍ͭͯ
Mercari • ࠃ࠷େڃͷϑϦϚΞϓϦ • 3Ͱ؆୯ʹग़ • ҆৺҆શͳܾࡁ
US/UK ͷల։ JP 2016/08 US AppStore 3Ґ US UK 2017/03/15
ϦϦʔε
Mercari KPI μϯϩʔυ GMV(૯औҾֹ) 7500ສDL(JP+US) ݄ؒ100ԯԁҎ্ ग़ 1100ສҎ্
ຊ࠷େͷϑϦϚΞϓϦ 1,200 1ؒͷग़(peek࣌ؒଳ) Ҏ্
ग़͔Β͙͢ʹചΕΔ 24 ࣌ؒҎ ചΕͨͷ50%͕ 24࣌ؒҎʹऔҾཱ
γεςϜ͔ΒΈͨϝϧΧϦ ©2011 Amazon Web Services LLC or its affiliates. All
rights reserved. Client Multimedia Corporate data center Traditional server Mobile Client IAM Add-on Example: IAM Add-on ence ) Assignment/ Task Requester Workers ग़! DB Search 5-දࣔ ݕࡧө ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific େྔͷϦΫΤετ ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corp data c Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ϦΫΤετԠ DB Search ߪೖ! ඵʙ30ඵ ඵʙ ߴʹେྔͷτϥϯβΫγϣϯΛѻ͏ ը૾ ܾࡁ AI
ΠϯϑϥετϥΫνϟ ੴङDC ઐ༻αʔό JP Cloud US Cloud UK
ΠϯϑϥετϥΫνϟ JP US UK DNS: Amazon Route53 CDN: Akamai, Fastly,
ImageFlux Storage: Amazon S3 Analysis: Google BigQuery ܾࡁ/ྲྀαʔϏε ܾࡁ/ྲྀαʔϏε ܾࡁ/ྲྀαʔϏε
ΞʔΩςΫνϟ • ࡾ+ΞϧϑΝͳΞʔΩςΫνϟ • Reverse Proxy(nginx) Application(Apache+mod_php) Database(MySQL) Cache(memcached) Search(Solr)
• ଟ͘Λʮઐ༻αʔόʯʹͯߏ • εέʔϧΞτͱεέʔϧΞοϓΛಉ࣌ʹߦ͏Diagonal Scale • ओʹ24ίΞ~56ίΞ·ͰͷαʔόΛར༻ • Databaseʹ ioMemory NVMe Λࡌͨ͠αʔόΛ࠾༻ nginx nginx nginx ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. Users Client Multimedia Corporate data center Traditional server Mobile Client WS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers ic DNS-RR App App App App App App MySQL MySQL memcached memcached util util cloud cloud JP Solr Solr
None
CDNΓସ͑ʹ͏ Web൛ϝϧΧϦʹ͓͚Δݸਓใྲྀग़ Πϯγσϯτʹ͍ͭͯ
ଟ͘ͷօ༷ʹ͝৺ɺ͝໎Λֻ͓͚͍ͨ͠·ͨ͠ ਂ͓͘ͼਃ͋͛͠·͢
͜͜Ͱൃੜͨ͠ࣄͱݪҼٴͼ ݱঢ়ͷରࡦʹ͍ͭͯհ͍ͨ͠·͢ɻ
ൃੜͨ͠ࣄ • ϝϧΧϦWeb൛ͷΞΫηε্ͱɺηΩϡϦςΟ্ͷ ͨΊɺCDNͷΓସ͑Λߦ͍·ͨ͠ɻͦͷࡍʹΓସ͑ઌͷ CDNʹ͓͚Δಈ࡞ʹ͍ͭͯɺΓସ͑Λ୲ͨ͠ΤϯδχΞͷ Ѳ͕ෆ͓ͯ͠Γɺ͓٬͞·ͷϨεϙϯε͕ผͷ͓٬͞· ʹҙਤͤͣදࣔ͞Εɺ݁Ռͱͯ͠ݸਓΛಛఆͰ͖ΔใΛؚΉ ༰͕ຊਓҎ֎ʹӾཡ͞ΕΔঢ়ଶͱͳΓ·ͨ͠
Timeline • 6/22 • 9:41ɹ CDNͷΓସ͑Λ࣮ࢪʢൃੜʣ • 14:41ɹΧελϚʔαϙʔτʹ͓ͯ٬͞·͔Βͷ͍߹ΘͤΛ֬ೝ͠ɺࣾใࠂ • 15:05ɹCDNͷΓସ͑Λதࢭ͠ɺैདྷͷCDN͢
• 15:16ɹWeb൛ͷϝϧΧϦΛϝϯςφϯεϞʔυΓସ͑ • 15:38ɹΓସ͑ઌCDNͷઃఆΛdeactivate͠ɺΞΫηεΛःஅ • 15:47ɹWeb൛ͷϝϧΧϦϝϯςφϯεϞʔυΛऴྃ • 17:55 ίʔϙϨʔταΠτʹ͓ΒͤΛܝࡌ • 20:45 Tech blogʹͯৄࡉެ։
Timeline • 6/28 • Γସ͑ઌCDNͷઃఆΛ࠶Activate • CacheΛແޮԽ͢ΔઃఆΛߦ͍ɺΓସ͑ઌCDNͷΤϯδχΞʹϨϏϡʔΛґཔ • nginxͷઃఆΛߋ৽͠ɺcacheʹؔ͢ΔϔομΛมߋ •
ࣾͰݕূ • 6/29 • Γସ͑ઌCDNͷΤϯδχΞػͷͱɺΓସ͑Λ࣮ࢪ
Γସ͑ઌCDNʹ͓͚Δcacheͷಈ࡞ • CacheΛແޮԽ͢ΔͨΊʹ “Cache-Control: private” ͘͠ "Set-Cookie" ͕ඞཁ • ”Cache-Control:
no-cache” “no-store” ແࢹ͞ΕΔ • Expiresϔομར༻͞ΕΔ͕ɺͷղऍʹࣦഊ͋Δ͍աڈͷ߹ “0ඵ” ͱͯ͠ѻΘΕΔ • ʮ0ඵͷΩϟογϡ͕ଘࡏ͢Δʯ • (্هઃఆʹΑΓΧελϚΠζ͕Մೳ)
0ඵͷcache • CDN͔ΒΦϦδϯͷϦΫΤετͷॲཧதʹɺಉ ͡URLʹରͯ͠ϦΫΤετ͕ൃੜ͢Δͱɺ࠷ॳͷ ϨεϙϯεΛͬͯɺ2ͭҎ߱ͷϦΫΤετʹ ಉ͡Ϩεϙϯε͕ฦ͞ΕΔ • ੩తίϯςϯπͰඇৗʹ༗ޮͰ͋Δ • ͜ͷ༷ʹؔ͢ΔѲ͕Ͱ͖ͯͳ͔ͬͨ
©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All r User Users Client Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Workers Amazon Mechanical Turk Non-Service Specific origin CDN (1) (2) (3) (4) (5) (5)
࠶ସͷࡍʹߦͬͨରࡦ • CDNͷઃఆΛมߋ͠ɺURLɺΦϦδϯͷϔομʹؔΘΒͣΩϟογϡΛ ͠ͳ͍ɺ·ͨ0ඵͷΩϟογϡͷͨΊʹϨεϙϯεΛͨͳ͍Α͏ʹ ͠ɺCDNఏڙࣾͷΤϯδχΞͷϨϏϡʔΛड͚ͨ • CDNͷϩάΛϦΞϧλΠϜʹS3ʹૹ৴͠ɺlambdaʹΑΓϩάΛղੳɺ mackerelͰՄࢹԽɻcache͕ΘΕ͍ͯͳ͍͔Λࢹ • nginx/ApacheʹͯΩϟογϡ͞ΕͮΒ͘͢ΔϔομΛՃ
• ϔομCDNͷมߋΛࢹ͢ΔscriptΛ࡞͠ɺslacklogܦ༝Ͱఆظ࣮ߦ
cache aware nginx configuration • ExpiresϔομΘͳ͍ • ݹ͍ϒϥβ͚ʹPragmaϔομͰରԠ • Cache-ControlҎ֎ʹΩϟογϡΛආ͚ΔͨΊ͚ͩͷ
Set-Cookieૹ৴ more_clear_headers 'Expires'; more_set_headers "Cache-Control: private, no-cache, no-store, must-revalidate" "Pragma: no-cache"; add_header Set-Cookie "merCtx=\"\"; HttpOnly" always; OHJOYDPOG
ࠓޙ՝ • css/js/fontͷassetsͷΩϟογϡ • ݱঢ়ɺશʹΩϟογϡແޮ • CDNͷઃఆͷܧଓతΠϯςΫάϨʔγϣϯ(CI) • Web൛Ҏ֎ͰͷCDNͷར༻ •
DDoSੑɺηΩϡϦςΟରࡦ
લऴྃ