SREେશ: ϝϧΧϦฤ [લ൒] 2017/08/17 hbstudy#75 Masahiro Nagano @kazeburo

AGENDA • લ൒ • ࣗݾ঺հɺϝϧΧϦ͕SREΛ࠾༻ͨ͠ཧ༝ • ϝϧΧϦSREνʔϜ঺հɺOnCallɺϝϧΧϦʹ͍ͭͯ • ઌ೔ͷCDNมߋͰͷΠϯγσϯτʹ͍ͭͯ

AGENDA • ޙ൒ • PHP ΞϓϦέʔγϣϯͷ࠷దԽࣄྫ • ηΩϡϦςΟͷऔΓ૊Έ(ύεϫʔυϦετ߈ܸࣄྫ) • ϝϧΧϦSREͷࠓޙ • SREͷ໾ׂͷݱࡏͱະདྷɺϚΠΫϩαʔϏε

Me • Masahiro Nagano / ௕໺խ޿ • @kazeburo • Mercari, Inc
 Principal Engineer
 Site Reliability Engineering (SRE) Team • BASE, Inc Technical Advisor

Me • ~ 2006: ژ౎ͰελʔτΞοϓࢀՃ • ։ൃΛ͠ͳ͕ΒΠϯϑϥͷ໘౗ΛݟΔɻDC࡞ۀ΋΍ͬͨ • ΞϓϦέʔγϣϯͷνϡʔχϯάɺۭ͍ͨϦιʔεͰ৽ػೳͷ௥Ճͱ͍͏αΠΫϧ • mod_perlɺSquidʹΑΔReverse Proxy • 2006 ~: mixi • ʮΞϓϦέʔγϣϯӡ༻νʔϜʯ / DCʹߦ͔ͳ͘ͳͬͨ • େن໛ͳը૾഑৴/memcached/Q4M

Me • 2010 ~: livedoor (NHN Japan => LINE) • livedoor΍LINEϑΝϛϦʔͷαʔϏεΛԣஅͯ͠Πϯϑϥ΍ύ ϑΥʔϚϯεͷվળ • livedoor blog ͷMySQLνϡʔχϯά • GrowthForecast/HRForecast/Plack Optimization/MHA • 2015/02: mercari

࠷ۙͷ׆ಈ • ొஃ • AWS Dev Day Tokyo 2017 • YAPC::Fukuoka 2017, YAPC::Hokkaido 2016 • هࣄ • WEB+DB PRESS Vol.88, Vol.92-97 ࿈ࡌ • ೔ܦSYSTEMS 2017 7݄߸, ITPro

WEB+DB PRESS Vol.100 8%#13&44߸㊗ 8%#13&447PMͷग़൛͸ֶੜ࣌୅ͷ೥ ࠷ॳʹखʹऔͬͨ࣌ʹ7PMʹهࣄΛॻ͘ͳΜͯ͜ͱ͸ શ͘૝૾͍ͯ͠·ͤΜͰͨ͠ɻͱͯ΋ޫӫͰ͢

ϝϧΧϦ͕SREΛ࠾༻ͨ͠ཧ༝

ࣗশ `!ΠϯϑϥΤϯδχΞ` • (ݸਓతʹ) ΠϯϑϥΤϯδχΞ΁ͷٙ໰ • 2006೥ҎདྷDC࡞ۀ͸͍ͯ͠ͳ͍ɻDC࡞ۀΛߦ͏νʔϜ͸ଞʹ͍Δ • σʔληϯλʔνʔϜ͕༻ҙͨ͠αʔόͷೳྗΛҾ͖ग़͠ɺΞϓϦέʔγϣ ϯΤϯδχΞ͕࡞੒ͨ͠ίʔυΛ࠷ߴͷܗͰಈ͔͢ͷ͕ࣗΒ(νʔϜ)ͷ໾ׂ • αʔϏεͷՄ༻ੑ͸ϋʔυ΢ΣΞͷνʔϜͰ͸ͳ͘ɺιϑτ΢ΣΞΛѻ͏ νʔϜͷ੹೚

ΦϖϨʔγϣϯΤϯδχΞ • 2010೥ग़൛ʮWeb Operationsʯ • ʮ΢ΣϒΦϖϨʔγϣϯ͸ٕܳͰ͋ΓՊֶͰ͸ͳ͍ʯ • ܧଓతσϓϩΠɺDevOpsɺࣗಈԽɺ؂ࢹͳͲΦϖ Ϩʔγϣϯʹؔ͢ΔΤοηΠ • ͨͩ͠ɺΦϖϨʔγϣϯ=ӡ༻ΛϧʔνϯϫʔΫͱଊ ͑Δਓ΋ଟ͍

SREͱͷग़ձ͍ • 2012/7 ༑ਓͱͷIRCͰͷձ࿩͔Β • ΠϯϑϥͱαʔϏεͷՔಇɺ҆ఆੑΛ୲౰͢ΔνʔϜ͕SRE • https://research.googleblog.com/2012/07/site-reliability-engineers-solving-most.html
 ͜ͷهࣄ͕ެ։͞Εͨࠒ • twitter ͷbio΍ൃදεϥΠυʹʮSite ReliabilityʯΛ௥Ճͯ͠ҙࣝ • https://www.slideshare.net/kazeburo/yapc2102mysql/2 (2012/9) • 2015/11 ϝϧΧϦʹͯνʔϜ໊ͱͯ͠࠾༻

No content

ϝϧΧϦͰSREΛ࠾༻ͨ͠ཧ༝ • ϝϧΧϦΛ͓٬͞·ʹ௕͘࢖ͬͯ΋Β͏ʹ͸ʮ͍ͭͰ΋շదʹ҆શʹ ࢖͑Δʯ৴པੑ͕ॏཁ • Πϯϑϥ=ϞϊɺΦϖϨʔγϣϯ=ࣄͰ͸ͳ͘ɺ৴པੑʹ੹೚Λ࣋ͭ͜ ͱΛ໌Β͔ʹ͢Δ • ւ֎Ͱ΋௨͡Δ໊শ • JP/US/UKͰͷల։ɻάϩʔόϧͰͷ࠾༻͸ৗʹҙࣝ • ઌਐతͳऔΓ૊Έͱͯ͠

ϝϧΧϦSREʹ͍ͭͯ

No content

Mercari SRE • ͍ͭͰ΋շద͔ͭ҆શʹར༻Ͱ͖Δʮ৴པੑͷߴ͍ʯαʔϏεͷ࣮ݱ • ʮ৽نαʔϏεͷ։ൃҎ֎ͷΤϯδχΞϦϯά͸શ෦΍Δʯ • ݱࡏϝϯόʔ͸ʮ9ਓʯ • શһ౦ژۈ຿ • ࣾ಺πʔϧͷӡ༻վળɺAIܥͷۀ຿ʹܞΘΔϝϯόʔ΋͍Δ

Mercari SRE ͷۀ຿ൣғ Operations Software Eng. ج൫ߏங OnCall (ো֐ରԠ) Automation εέʔϥϏϦςΟɾՄ༻ੑվળ DBAɺϛυϧ΢ΣΞߏங ΞϓϦέʔγϣϯͷઃܭϨϏϡʔ ϩάऩूɾ෼ੳج൫ͷߏஙɺӡ༻ αʔόϓϩϏδϣχϯάɾσϓϩΠɺϚΠΫϩαʔϏεج൫ͷ੔උ ηΩϡϦςΟʗෆਖ਼ར༻ݕग़

SRE౰൪/OnCall • ΈΜͳେ޷͖ো֐ରԠ • ΞϥʔτରԠ౰൪ͱి࿩౰൪ • 4໊ͰOnCallͷScheduling • (2໊τϨʔχϯάத)

SRE౰൪/OnCall • ೔༵೔0͔࣌Β౔༵೔24࣌·Ͱ1िؒͰަ୅ • Ξϥʔτͷड͚औΓͱҰ࣌ରԠ • ฏ೔͸νʔϜϝϯόʔ͕ग़ࣾ͢Δ·Ͱࣗ୐଴ػ • 9͔࣌Βࣗ୐଴ػ͠ɺUS͔ΒͷґཔͳͲʹରԠ • UK͔ΒͷରԠͰ໷ؒ࡞ۀ΋͋Δ • ٳ೔΋15-20෼Ҏ಺ʹରԠ։࢝Ͱ͖Δ͜ͱ͕๬·͘͠ɺߦಈʹ੍ݶ΋͋Δ • ਂ໷ٳ೔ͷରԠͳͲɺՈ଒ͷڠྗ΋ඞཁ

౰൪/OnCall Λࢧ͑Δٕज़ • ؂ࢹ • Mackerel, slacklog • ௨஌/Scheduling • Slack, PagerDuty, Twilio

mackerel Worker Batch App App MySQL cron mackerel-agent fluent-plugin-mackerel mkr ՄࢹԽ/ᮢ஋ͷઃఆ Metricsऩू ௨஌

[୤ઢ] ࠷ۙ࡞ͬͨmackerel-plugin #!/usr/bin/perl use HTTP::Date; my $NUM_LOG_WATCH = 1000; my $CHECK_RANGE = 300; #5min my $exceptions = 0; my $now = time; open( my $messages_tail, "-|", "tail","-$NUM_LOG_WATCH","/var/log/messages") or die $!; while (<$messages_tail>) { if ( $_ !~ m!Machine Check Exception! ) { next; } if ( my ($time) = ($_ =~ m!^(\w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2})\s!) ) { $time = str2time($time); if ( $now - $time < $CHECK_RANGE ) { $exceptions++; } } } if ( $exceptions > 0 ) { print "CRITICAL: Machine Check Exception Found in this 5 minutes\n"; exit 2; } print "OK: No Machine Check Exception found\n"; exit 0; % dmesg | tail 
 sbridge: HANDLING MCE MEMORY ERROR CPU 0: Machine Check Exception: 0 Bank 8: cc0427c000010090 TSC 0 ADDR 37805ac0 MISC 45048ce86 PROCESSOR 0:406f1 TIME 1495654896 SOCKET 0 APIC 0 [Hardware Error]: Machine check events logged EDAC MC1: CE row 0, channel 0, label "CPU_SrcID#0_Ha#0_Channel#0_DIMM": 4255 Unknown error(s): memory read on FATAL area OVERFLOW: cpu=0 Err=0001:0090 (ch=0), addr = 0x37805ac0 => socket=0, ha=1, Channel=0(mask=1), rank=0 DIFDLNBDIJOFFYDFQUJPOT ϝϞϦʔΤϥʔΛݟ͚ͭΔ

[୤ઢ] ࠷ۙ࡞ͬͨmackerel-plugin #!/bin/sh set -e if [ ! -f /opt/MegaRAID/MegaCli/MegaCli64 ]; then exit fi if ( /opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL|grep 'Firmware state'|grep -v "Online, Spun Up" > /dev/null 2>&1 ); then /opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL|grep 'Firmware state' exit 2 fi /opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL|grep 'Firmware state' exit 0 DIFDLSBJEEJTL .FHB$MJΛ͔ͭͬͯ3"*%ͷঢ়ଶΛ؂ࢹ

slacklog ίϚϯυ $ slacklog -t alert-information --notify -- perl -e 'die "TEST!"' !LB[VIP͞ΜͷDSPOMPHΛࢀߟʹ࡞੒ CBUDICBDLVQͷࣦഊΛݕ஌ slackboard ௨஌ͷू໿ IUUQTHJUIVCDPNDVCJDEBJZBTMBDLCPBSE

slackͰͷΞϥʔτͷ໰୊఺ • ؾ෇͘ͷ͕೉͍͠ • ௨஌ԻͷΧελϚΠζͰ͖ͳ͍ • 1ճ͔͠௨஌Ͱ͖ͳ͍ɻྲྀΕ͕ͪ • Push͜ͳ͍৔߹΋ • ετϨε⤴ / ӡ༻ʹͳΓ͕ͪ

PagerDuty 4DIFEVMFϩʔςʔγϣϯ؅ཧ &TDBMBUJPOϧʔϧ

PagerDuty • ༷ʑͳखஈͰ௨஌Λߦ͏͜ͱ͕Ͱ͖Δ • mail • SMS • App • ి࿩ • 12෼ʹҰ౓ి࿩ΛೖΕΔϧʔϧͰӡ༻ • Appͷ௨஌͕ศར

Emergency Call ☎ US/UKؚΊɺؾܰʹ࢖ͬͯ΋Β͑ΔΑ͏ࣾ಺ࠂ஌
 ࠷ۙͰ͸SREνʔϜ಺Ͱ΋ΤεΧϨʔγϣϯ໨తͰར༻

No content

ϝϧΧϦʹ͍ͭͯ

Mercari • ࠃ಺࠷େڃͷϑϦϚΞϓϦ • 3෼Ͱ؆୯ʹग़඼ • ҆৺҆શͳܾࡁ

US/UK ΁ͷల։ JP 2016/08 US AppStore 3Ґ US UK 2017/03/15 ϦϦʔε

Mercari KPI μ΢ϯϩʔυ਺ GMV(૯औҾֹ) 7500ສDL(JP+US) ݄ؒ100ԯԁҎ্ ग़඼਺ 1೔100ສ඼Ҏ্

೔ຊ࠷େͷϑϦϚΞϓϦ 1,200 1෼ؒͷग़඼਺(peek࣌ؒଳ) ඼Ҏ্

ग़඼͔Β͙͢ʹചΕΔ 24 ࣌ؒҎ಺ ചΕͨ঎඼ͷ໿50%͕ 24࣌ؒҎ಺ʹऔҾ੒ཱ

γεςϜ͔ΒΈͨϝϧΧϦ ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. Client Multimedia Corporate data center Traditional server Mobile Client IAM Add-on Example: IAM Add-on ence ) Assignment/ Task Requester Workers ग़඼! DB Search 5-දࣔ ݕࡧ൓ө ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific େྔͷϦΫΤετ ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corp data c Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ϦΫΤετԠ౴ DB Search ߪೖ! ਺ඵʙ30ඵ ਺ඵʙ ߴ଎ʹେྔͷτϥϯβΫγϣϯΛѻ͏ ը૾ ܾࡁ AI

ΠϯϑϥετϥΫνϟ ੴङDC ઐ༻αʔό JP Cloud US Cloud UK

ΠϯϑϥετϥΫνϟ JP US UK DNS: Amazon Route53 CDN: Akamai, Fastly, ImageFlux Storage: Amazon S3 Analysis: Google BigQuery ܾࡁ/෺ྲྀαʔϏε ܾࡁ/෺ྲྀαʔϏε ܾࡁ/෺ྲྀαʔϏε

ΞʔΩςΫνϟ • ࡾ૚+ΞϧϑΝͳΞʔΩςΫνϟ • Reverse Proxy(nginx)
 Application(Apache+mod_php)
 Database(MySQL)
 Cache(memcached)
 Search(Solr) • ଟ͘Λʮઐ༻αʔόʯʹͯߏ੒ • εέʔϧΞ΢τͱεέʔϧΞοϓΛಉ࣌ʹߦ͏Diagonal Scale • ओʹ24ίΞ~56ίΞ·ͰͷαʔόΛར༻ • Databaseʹ͸ ioMemory ΍ NVMe Λ౥ࡌͨ͠αʔόΛ࠾༻ nginx nginx nginx ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. Users Client Multimedia Corporate data center Traditional server Mobile Client WS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers ic DNS-RR App App App App App App MySQL MySQL memcached memcached util util cloud cloud JP Solr Solr

No content

CDN੾Γସ͑ʹ൐͏ Web൛ϝϧΧϦʹ͓͚Δݸਓ৘ใྲྀग़
 Πϯγσϯτʹ͍ͭͯ

ଟ͘ͷօ༷ʹ͝৺഑ɺ͝໎࿭Λֻ͓͚͍ͨ͠·ͨ͠ ਂ͓͘࿳ͼਃ͋͛͠·͢

͜͜Ͱ͸ൃੜͨ͠ࣄ৅ͱݪҼٴͼ
 ݱঢ়ͷରࡦʹ͍ͭͯ঺հ͍ͨ͠·͢ɻ

ൃੜͨ͠ࣄ৅ • ϝϧΧϦWeb൛΁ͷΞΫηε଎౓޲্ͱɺηΩϡϦςΟ޲্ͷ ͨΊɺCDNͷ੾Γସ͑Λߦ͍·ͨ͠ɻͦͷࡍʹ੾Γସ͑ઌͷ CDNʹ͓͚Δಈ࡞ʹ͍ͭͯɺ੾Γସ͑Λ୲౰ͨ͠ΤϯδχΞͷ ೺Ѳ͕ෆ଍͓ͯ͠Γɺ͓٬͞·΁ͷϨεϙϯε͕ผͷ͓٬͞· ʹҙਤͤͣදࣔ͞Εɺ݁Ռͱͯ͠ݸਓΛಛఆͰ͖Δ৘ใΛؚΉ ಺༰͕ຊਓҎ֎ʹӾཡ͞ΕΔঢ়ଶͱͳΓ·ͨ͠

Timeline • 6/22 • 9:41ɹ CDNͷ੾Γସ͑Λ࣮ࢪʢ໰୊ൃੜʣ • 14:41ɹΧελϚʔαϙʔτʹ͓ͯ٬͞·͔Βͷ໰͍߹ΘͤΛ֬ೝ͠ɺࣾ಺΁ใࠂ • 15:05ɹCDNͷ੾Γସ͑Λதࢭ͠ɺैདྷͷCDN΁໭͢ • 15:16ɹWeb൛ͷϝϧΧϦΛϝϯςφϯεϞʔυ΁੾Γସ͑ • 15:38ɹ੾Γସ͑ઌCDNͷઃఆΛdeactivate͠ɺΞΫηεΛःஅ • 15:47ɹWeb൛ͷϝϧΧϦϝϯςφϯεϞʔυΛऴྃ • 17:55 ίʔϙϨʔταΠτʹ͓஌ΒͤΛܝࡌ • 20:45 Tech blogʹͯৄࡉެ։

Timeline • 6/28 • ੾Γସ͑ઌCDNͷઃఆΛ࠶Activate • CacheΛແޮԽ͢ΔઃఆΛߦ͍ɺ੾Γସ͑ઌCDNͷΤϯδχΞʹϨϏϡʔΛґཔ • nginxͷઃఆΛߋ৽͠ɺcacheʹؔ͢ΔϔομΛมߋ • ࣾ಺Ͱݕূ • 6/29 • ੾Γସ͑ઌCDNͷΤϯδχΞ଴ػͷ΋ͱɺ੾Γସ͑Λ࣮ࢪ

੾Γସ͑ઌCDNʹ͓͚Δcacheͷಈ࡞ • CacheΛແޮԽ͢ΔͨΊʹ͸ “Cache-Control: private” ΋͘͠͸ "Set-Cookie" ͕ඞཁ • ”Cache-Control: no-cache” ΍ “no-store” ͸ແࢹ͞ΕΔ • Expiresϔομ΋ར༻͞ΕΔ͕ɺ೔෇ͷղऍʹࣦഊ͋Δ͍͸աڈ೔෇ͷ৔߹ ͸ “0ඵ” ͱͯ͠ѻΘΕΔ • ʮ0ඵͷΩϟογϡ͕ଘࡏ͢Δʯ • (্ه͸ઃఆʹΑΓΧελϚΠζ͕Մೳ)

0ඵͷcache • CDN͔ΒΦϦδϯ΁ͷϦΫΤετͷॲཧதʹɺಉ ͡URLʹରͯ͠ϦΫΤετ͕ൃੜ͢Δͱɺ࠷ॳͷ ϨεϙϯεΛ଴ͬͯɺ2ͭ໨Ҏ߱ͷϦΫΤετʹ ΋ಉ͡Ϩεϙϯε͕ฦ͞ΕΔ • ੩తίϯςϯπͰ͸ඇৗʹ༗ޮͰ͋Δ • ͜ͷ࢓༷ʹؔ͢Δ೺Ѳ͕Ͱ͖ͯͳ͔ͬͨ
 
 ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All r User Users Client Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Workers Amazon Mechanical Turk Non-Service Specific origin CDN (1) (2) (3) (4) (5) (5)

࠶੾ସͷࡍʹߦͬͨରࡦ • CDNͷઃఆΛมߋ͠ɺURLɺΦϦδϯͷϔομʹؔΘΒͣΩϟογϡΛ ͠ͳ͍ɺ·ͨ0ඵͷΩϟογϡͷͨΊʹϨεϙϯεΛ଴ͨͳ͍Α͏ʹ ͠ɺCDNఏڙࣾͷΤϯδχΞͷϨϏϡʔΛड͚ͨ • CDNͷϩάΛϦΞϧλΠϜʹS3ʹૹ৴͠ɺlambdaʹΑΓϩάΛղੳɺ mackerelͰՄࢹԽɻcache͕࢖ΘΕ͍ͯͳ͍͔Λ؂ࢹ • nginx/ApacheʹͯΩϟογϡ͞ΕͮΒ͘͢ΔϔομΛ௥Ճ • ϔομ΍CDNͷมߋΛ؂ࢹ͢ΔscriptΛ࡞੒͠ɺslacklogܦ༝Ͱఆظ࣮ߦ

cache aware nginx configuration • Expiresϔομ͸࢖Θͳ͍ • ݹ͍ϒϥ΢β޲͚ʹ͸PragmaϔομͰରԠ • Cache-ControlҎ֎ʹΩϟογϡΛආ͚ΔͨΊ͚ͩͷ Set-Cookie΋ૹ৴ more_clear_headers 'Expires'; more_set_headers "Cache-Control: private, no-cache, no-store, must-revalidate" "Pragma: no-cache"; add_header Set-Cookie "merCtx=\"\"; HttpOnly" always; OHJOYDPOG

ࠓޙ՝୊ • css/js/font౳ͷassetsͷΩϟογϡ • ݱঢ়ɺ׬શʹΩϟογϡແޮ • CDNͷઃఆͷܧଓతΠϯςΫάϨʔγϣϯ(CI) • Web൛Ҏ֎ͰͷCDNͷར༻ • DDoS଱ੑɺηΩϡϦςΟରࡦ

લ൒ऴྃ