Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CDNの使い方 in Mercari/CDN in Mercari

kazeburo
October 18, 2017

CDNの使い方 in Mercari/CDN in Mercari

CDN in Mercari/CDNの使い方 in Mercari
Fastly Yamagoya Meetup 2017

kazeburo

October 18, 2017
Tweet

More Decks by kazeburo

Other Decks in Technology

Transcript

  1. Me • Masahiro Nagano / ௕໺խ޿ • @kazeburo • Mercari,

    Inc
 Principal Engineer
 Site Reliability Engineering (SRE) Team
  2. γεςϜ͔ΒΈͨϝϧΧϦ ©2011 Amazon Web Services LLC or its affiliates. All

    rights reserved. Client Multimedia Corporate data center Traditional server Mobile Client IAM Add-on Example: IAM Add-on ence ) Assignment/ Task Requester Workers ग़඼! DB Search 5-දࣔ ݕࡧ൓ө ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific େྔͷϦΫΤετ ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corp data c Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ϦΫΤετԠ౴ DB Search ߪೖ! ϛϦඵʙ30ඵ ਺ඵʙ ߴ଎ʹେྔͷτϥϯβΫγϣϯΛѻ͏ ը૾ ܾࡁ AI
  3. Timeline • 6/22 • 9:41ɹ CDNͷ੾Γସ͑Λ࣮ࢪʢ໰୊ൃੜʣ • 14:41ɹΧελϚʔαϙʔτʹ͓ͯ٬͞·͔Βͷ໰͍߹ΘͤΛ֬ೝ͠ɺࣾ಺΁ใࠂ • 15:05ɹCDNͷ੾Γସ͑Λதࢭ͠ɺैདྷͷCDN΁໭͢

    • 15:16ɹWeb൛ͷϝϧΧϦΛϝϯςφϯεϞʔυ΁੾Γସ͑ • 15:38ɹCDNͷ഑৴ઃఆΛdeactivate͠ɺΞΫηεΛःஅ • 15:47ɹWeb൛ͷϝϧΧϦϝϯςφϯεϞʔυΛऴྃ • 17:55 ίʔϙϨʔταΠτʹ͓஌ΒͤΛܝࡌ • 20:45 Tech blogʹͯৄࡉެ։
  4. CacheແޮԽ • CacheΛແޮԽ͢ΔͨΊʹ͸ “Cache-Control: private” ΋͘͠͸ "Set-Cookie" ͕ඞཁ • ”Cache-Control:

    no-cache” ΍ “no-store” ͸ແࢹ͞ΕΔ • Expiresϔομ΋ར༻͞ΕΔ͕ɺ೔෇ͷղऍʹࣦഊ͋Δ͍͸աڈ೔෇ͷ৔߹ ͸ “0ඵ” ͱͯ͠ѻΘΕΔ • 0ඵ͸cacheແޮͰ͸ͳ͘ʮ0ඵͷcache͕ଘࡏ͢Δʯ
  5. 0ඵͷcache • CDN͔ΒΦϦδϯ΁ͷϦΫΤετͷॲཧதʹɺಉ͡ URLʹରͯ͠ϦΫΤετ͕ൃੜ͢Δͱɺ࠷ॳͷϨεϙ ϯεΛ଴ͬͯɺ2ͭ໨Ҏ߱ͷϦΫΤετʹ΋ಉ͡Ϩε ϙϯε͕ฦ͞ΕΔ • `Request collapsing` ©2011

    Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All r User Users Client Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Workers Amazon Mechanical Turk Non-Service Specific origin CDN (1) (2) (3) (4) (5) (5)
  6. ରࡦ • VCLͷઃఆมߋ • ෳ਺ͷbackendʹରԠ͢ΔͨΊɺPriorityΛ௿͘ઃఆ # ͜͜ʹbackend੾Γସ͑ͷvcl͕ඳ͍ͯ͋Δ # Request Condition:

    always Prio: 100 if( req.url ) { ## ඞͣtrueʹͳΔ if (!req.http.Fastly-FF) { if (req.http.X-Forwarded-For) { set req.http.Fastly-Temp-XFF = req.http.X-Forwarded-For ", " client.ip; } else { set req.http.Fastly-Temp-XFF = client.ip } } else { set req.http.Fastly-Temp-XFF = req.http.X-Forwarded-For; } return(pass); }
  7. ରࡦ • ϩάΛS3ʹૹ৴͠ɺlambdaʹΑΓϦΞϧλΠϜʹղੳɺmackerelͰՄࢹԽɻ ؂ࢹ • nginx/ApacheʹͯheaderΛ௥Ճ • header΍VCLͷมߋͷ؂ࢹ more_clear_headers 'Expires';

    more_set_headers "Cache-Control: private, no-cache, no-store, must-revalidate" "Pragma: no-cache"; add_header Set-Cookie "merCtx=\"\"; HttpOnly" always;
  8. CDN in Mercari • ΑΓଟ͘ͷτϥϑΟοΫΛॲཧ͢ΔͨΊ • Πϯϑϥӡ༻ίετͷ࡟ݮͱͯ͠CDNΛར༻ • ը૾഑৴/ಈը഑৴ •

    UX/ηΩϡϦςΟͷվળͷͨΊ • Clientͱͷ௨৴؀ڥ࠷దԽ • αΠτ/APIΛؙ͝ͱCDNԽ
  9. Frontend for Backend/microservices API Gateway ©2011 Amazon Web Services LLC

    or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Non-Service Specific search personalization offer gRPC JSON over HTTPs
  10. App Backend Infrastructure JP US UK API͸ͦΕͧΕผυϝΠϯ / CDN͸࢖༻ͯ͠ͳ͍(2017/10ݱࡏ) େن໛ͳτϥϑΟοΫͱͳΔ঎඼ը૾͸CDNΛར༻ͯ͠഑৴

    Client Multimedia Corporate data center Traditional server Mobile Client Assignment/ Task Requester Workers User Users Client Multimedia Corporate data center Traditio serve Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific User Users Client Multimedia Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requ Workers Amazon Mechanical Turk Non-Service Specific api.example.jp api.example.com api.example.uk S3 S3 S3 KQ DEOJ[FOFU VTDEOJ[FOFU VLDEOJ[FOFU CDN CDN CDN *N BHF'MVY
  11. ࠓޙͷ՝୊ɾऔΓ૊Έ • Security΁ͷϑΥʔΧε • L3-L7·Ͱେن໛ͳDoS߈ܸ͔Βͷ๷ޚ • WAF ͷݕূ • Bot੍ޚ

    • ߴ౓Խ͢ΔEdge Computing • Microservices Λ࣮ݱ͢Δखஈͱͯ͠ͷCDN • CDNͷઃఆ/VCLͷContinuous Integration • ίετ࠷దԽ