Tweet
Tweet

Slide 1

Slide 1 text

CDN in Mercari Fastly Yamagoya Meetup 2017 Masahiro Nagano @kazeburo

Slide 2

Slide 2 text

Me • Masahiro Nagano / ௕໺խ޿ • @kazeburo • Mercari, Inc
 Principal Engineer
 Site Reliability Engineering (SRE) Team

Slide 3

Slide 3 text

Agenda • ϝϧΧϦͷ঺հ • ઌ೔ͷݸਓ৘ใྲྀग़Πϯγσϯτʹ͍ͭͯ • ϝϧΧϦͷΠϯϑϥετϥΫνϟͱCDN • ࠓޙͷ՝୊ɾऔΓ૊Έ

Slide 4

Slide 4 text

Mercari • ϑϦϚΞϓϦ • εϚϗͰࣸਅΛͱͬͯ؆୯ʹग़඼ • ҆৺ɾ҆શͳܾࡁ • ศརͳ഑ૹ

Slide 5

Slide 5 text

Mercari μ΢ϯϩʔυ਺ ྲྀ௨ֹ 7500ສDL(JP+US) ݄ؒ100ԯԁҎ্ ग़඼਺ 1೔100ສ඼Ҏ্

Slide 6

Slide 6 text

γεςϜ͔ΒΈͨϝϧΧϦ ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. Client Multimedia Corporate data center Traditional server Mobile Client IAM Add-on Example: IAM Add-on ence ) Assignment/ Task Requester Workers ग़඼! DB Search 5-දࣔ ݕࡧ൓ө ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific େྔͷϦΫΤετ ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corp data c Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ϦΫΤετԠ౴ DB Search ߪೖ! ϛϦඵʙ30ඵ ਺ඵʙ ߴ଎ʹେྔͷτϥϯβΫγϣϯΛѻ͏ ը૾ ܾࡁ AI

Slide 7

Slide 7 text

Mercari JP US UK JP/US/UKͰల։த

Slide 8

Slide 8 text

CDN੾Γସ͑ʹ൐͏ Web൛ϝϧΧϦʹ͓͚Δݸਓ৘ใྲྀग़
 Πϯγσϯτʹ͍ͭͯ

Slide 9

Slide 9 text

ଟ͘ͷօ༷ʹ͝৺഑ɺ͝໎࿭Λֻ͓͚͍ͨ͠·ͨ͠ ਂ͓͘࿳ͼਃ͋͛͠·͢ ଟେͳΔڠྗΛFastly༷ʹ͍͖ͨͩ·ͨ͠ɻվΊ͓ͯྱਃ্͛͠·͢

Slide 10

Slide 10 text

ൃੜͨ͠ࣄ৅ • ϝϧΧϦWeb൛΁ͷΞΫηε଎౓޲্ͱɺηΩϡϦςΟ޲্ͷͨΊɺCDN ͷ੾Γସ͑Λߦ͍·ͨ͠ɻͦͷࡍʹ੾Γସ͑ઌͷCDN(Fastly)ʹ͓͚Δ Ωϟογϡͷಈ࡞ʹ͍ͭͯͷ೺Ѳ͕ෆ଍͓ͯ͠Γɺ͓٬͞·΁ͷϨεϙϯ ε͕ผͷ͓٬͞·ʹҙਤͤͣදࣔ͞Εɺ݁Ռͱͯ͠ݸਓΛಛఆͰ͖Δ৘ใ ΛؚΉ಺༰͕ຊਓҎ֎ʹӾཡ͞ΕΔঢ়ଶͱͳΓ·ͨ͠

Slide 11

Slide 11 text

Timeline • 6/22 • 9:41ɹ CDNͷ੾Γସ͑Λ࣮ࢪʢ໰୊ൃੜʣ • 14:41ɹΧελϚʔαϙʔτʹ͓ͯ٬͞·͔Βͷ໰͍߹ΘͤΛ֬ೝ͠ɺࣾ಺΁ใࠂ • 15:05ɹCDNͷ੾Γସ͑Λதࢭ͠ɺैདྷͷCDN΁໭͢ • 15:16ɹWeb൛ͷϝϧΧϦΛϝϯςφϯεϞʔυ΁੾Γସ͑ • 15:38ɹCDNͷ഑৴ઃఆΛdeactivate͠ɺΞΫηεΛःஅ • 15:47ɹWeb൛ͷϝϧΧϦϝϯςφϯεϞʔυΛऴྃ • 17:55 ίʔϙϨʔταΠτʹ͓஌ΒͤΛܝࡌ • 20:45 Tech blogʹͯৄࡉެ։

Slide 12

Slide 12 text

Timeline • 6/28 • ഑৴ઃఆΛ࠶Activate • Cacheͷ׬શແޮԽઃఆΛߦ͍ɺFastlyࣾͷΤϯδχΞʹϨϏϡʔΛґཔ • nginxͷઃఆΛߋ৽͠ɺcacheʹؔ͢ΔϔομΛมߋ • ࣾ಺Ͱݕূ • 6/29 • FastlyࣾΤϯδχΞ଴ػͷ΋ͱɺ੾Γସ͑Λ࣮ࢪ

Slide 13

Slide 13 text

CacheແޮԽ • CacheΛແޮԽ͢ΔͨΊʹ͸ “Cache-Control: private” ΋͘͠͸ "Set-Cookie" ͕ඞཁ • ”Cache-Control: no-cache” ΍ “no-store” ͸ແࢹ͞ΕΔ • Expiresϔομ΋ར༻͞ΕΔ͕ɺ೔෇ͷղऍʹࣦഊ͋Δ͍͸աڈ೔෇ͷ৔߹ ͸ “0ඵ” ͱͯ͠ѻΘΕΔ • 0ඵ͸cacheແޮͰ͸ͳ͘ʮ0ඵͷcache͕ଘࡏ͢Δʯ

Slide 14

Slide 14 text

0ඵͷcache • CDN͔ΒΦϦδϯ΁ͷϦΫΤετͷॲཧதʹɺಉ͡ URLʹରͯ͠ϦΫΤετ͕ൃੜ͢Δͱɺ࠷ॳͷϨεϙ ϯεΛ଴ͬͯɺ2ͭ໨Ҏ߱ͷϦΫΤετʹ΋ಉ͡Ϩε ϙϯε͕ฦ͞ΕΔ • `Request collapsing` ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All r User Users Client Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Workers Amazon Mechanical Turk Non-Service Specific origin CDN (1) (2) (3) (4) (5) (5)

Slide 15

Slide 15 text

ରࡦ • VCLͷઃఆมߋ • ෳ਺ͷbackendʹରԠ͢ΔͨΊɺPriorityΛ௿͘ઃఆ # ͜͜ʹbackend੾Γସ͑ͷvcl͕ඳ͍ͯ͋Δ # Request Condition: always Prio: 100 if( req.url ) { ## ඞͣtrueʹͳΔ if (!req.http.Fastly-FF) { if (req.http.X-Forwarded-For) { set req.http.Fastly-Temp-XFF = req.http.X-Forwarded-For ", " client.ip; } else { set req.http.Fastly-Temp-XFF = client.ip } } else { set req.http.Fastly-Temp-XFF = req.http.X-Forwarded-For; } return(pass); }

Slide 16

Slide 16 text

ରࡦ • ϩάΛS3ʹૹ৴͠ɺlambdaʹΑΓϦΞϧλΠϜʹղੳɺmackerelͰՄࢹԽɻ ؂ࢹ • nginx/ApacheʹͯheaderΛ௥Ճ • header΍VCLͷมߋͷ؂ࢹ more_clear_headers 'Expires'; more_set_headers "Cache-Control: private, no-cache, no-store, must-revalidate" "Pragma: no-cache"; add_header Set-Cookie "merCtx=\"\"; HttpOnly" always;

Slide 17

Slide 17 text

https://docs.fastly.com/ja/guides/tutorials/cache-control-tutorial

Slide 18

Slide 18 text

ϝϧΧϦͷΠϯϑϥετϥΫνϟͱCDN

Slide 19

Slide 19 text

CDN in Mercari • ΑΓଟ͘ͷτϥϑΟοΫΛॲཧ͢ΔͨΊ • Πϯϑϥӡ༻ίετͷ࡟ݮͱͯ͠CDNΛར༻ • ը૾഑৴/ಈը഑৴ • UX/ηΩϡϦςΟͷվળͷͨΊ • Clientͱͷ௨৴؀ڥ࠷దԽ • αΠτ/APIΛؙ͝ͱCDNԽ

Slide 20

Slide 20 text

Infrastructure US UK ͦΕͧΕͷҬ಺ͷDCΛར༻ S3 S3 S3 ঎඼ը૾ ঎඼ը૾ ঎඼ը૾ Frontend for Backend
 microservices Backend API JP

Slide 21

Slide 21 text

Frontend for Backend/microservices API Gateway ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Non-Service Specific search personalization offer gRPC JSON over HTTPs

Slide 22

Slide 22 text

App Backend Infrastructure JP US UK API͸ͦΕͧΕผυϝΠϯ / CDN͸࢖༻ͯ͠ͳ͍(2017/10ݱࡏ) େن໛ͳτϥϑΟοΫͱͳΔ঎඼ը૾͸CDNΛར༻ͯ͠഑৴ Client Multimedia Corporate data center Traditional server Mobile Client Assignment/ Task Requester Workers User Users Client Multimedia Corporate data center Traditio serve Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific User Users Client Multimedia Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requ Workers Amazon Mechanical Turk Non-Service Specific api.example.jp api.example.com api.example.uk S3 S3 S3 KQ DEOJ[FOFU VTDEOJ[FOFU VLDEOJ[FOFU CDN CDN CDN *N BHF'MVY

Slide 23

Slide 23 text

ImageFlux • ͘͞ΒΠϯλʔωοτͱpixivͷڠۀ • ը૾ม׵ + Cache Storage • WebP΁ͷରԠ • CDNͷCache Hit Rateվળ

Slide 24

Slide 24 text

Web Backend Infrastructure JP US UK Web͸γϯάϧυϝΠϯɻURIͰࢀর͢ΔBackendΛ੾Γସ͑Δ ͓٬༷͔Β࠷΋͍ۙEdgeͰ൑அ͢Δ͜ͱͰϨεϙϯελΠϜ/UX޲্ S3 S3 S3 https://www.mercari.com/ /jp/ / /uk/

Slide 25

Slide 25 text

ࠓޙͷ՝୊ɾऔΓ૊Έ

Slide 26

Slide 26 text

ࠓޙͷ՝୊ɾऔΓ૊Έ • Security΁ͷϑΥʔΧε • L3-L7·Ͱେن໛ͳDoS߈ܸ͔Βͷ๷ޚ • WAF ͷݕূ • Bot੍ޚ • ߴ౓Խ͢ΔEdge Computing • Microservices Λ࣮ݱ͢Δखஈͱͯ͠ͷCDN • CDNͷઃఆ/VCLͷContinuous Integration • ίετ࠷దԽ

Slide 27

Slide 27 text

Ҏ্ Fastlyͷօ༷ɺࠓޙͱ΋ΑΖ͓͘͠ئ͍͠·͢ => www.mercari.com/jp/jobs/